StORe: Phase One Summative Evaluation Final Report

Abstract

This document presents the final results of the phase one summative evaluation of the StORe project’s pilot demonstrator prototype system. As stated in the ‘Summative Evaluation Plan v1.3’ for WP 5 (see Appendix A), the “evaluation will assess the technical structure, functionality, design and quality of the demonstrator system, and the appropriateness of the ‘common model’ approach, using workshops to test the system with representative repository users”. In the recently completed two year StORe project a social science pilot demonstrator system has been built up and refined according to the stages of work (see http://jiscstore.jot.com/PilotMiddleware) at the UK Data Archive (UKDA). It federates a UKDA source repository and the Research Articles Online institutional output repository (ePrints) of the London School of Economics (LSE), based on the common/generic model, as outlined in the ‘WP 3: Business Analysis’ report. This business analysis report is one base document for the summative evaluation and itself based on data gathered in the ‘WP 2: Survey of Researchers’ (see http://jiscstore.jot.com/SurveyPhase). The summative evaluation consists of a workshop topic guide and a review of the WP 3 business analysis report described in the chapters 2 and 3, both part of deliverable one already presented in two separate reports end of April 2007 (for more details on deliverables see Appendix A). It turned out to be very difficult to commit users to participate in the planned two workshops. In the end one workshop could be conducted with only three users taking part, who not even had been participants in the StORe project before. Due to these circumstances this final report is called phase one, as it is probable that a second complementary evaluation phase will be conducted in the future also based on the approach introduced here. This process and its repercussions are described in detail under ‘Change Management’ in chapter 4. Chapter 5 shows how the workshop session occurred eventually (part of deliverable one) and chapter 6 finally presents the results of the evaluation (deliverables one and two). During the evaluation a close collaboration with the UK Data Archive in Essex was invaluable concerning the appropriate dates for the workshops, the technical prerequisites and demands for the user testing as well as respecting the knowledge and experience towards the pilot and its ongoing development and use.