Understanding requirements work in e-science projects
Ho, Ka Lai
The e-science vision is to create infrastructures to enable faster, better and more collaborative science to be carried out in the 21st Century. The goal is for these infrastructures to allow scientists to collaborate routinely, scaling geographical and disciplinary boundaries; to create ad hoc arrangements datasets, equipment or computational power to solve larger, more complex scientific problems; to federate remote datasets, hence, aiding scientists in data discovery and even data re-use. The work to turn the e-science vision into reality has been the subject of major research programmes in the UK (UK E-Science Programme) and the US (National Science Foundation’s Cyberinfrastructures Programme). Inevitably, e-science technologies and scientific practices will co-evolve as collaborative work becomes more prevalent, and cross-disciplinary work becomes routinised. Thus, the design and development of e-science technologies will play a critical part in the above process; there is a clear need to develop technologies which will accurately reflect end-users’ needs as well as account for the wider social structures of future scientific work. In e-science, there has been an on-going debate about whether new requirements techniques are needed to deal with the ‘unique’ characteristics of e-science projects, and the ambitious aims of e-science software. Some argue that e-science presents sufficiently novel combinations of challenges that new techniques are needed, whilst others argue that e-science practitioners should ‘borrow’ from pre-existing requirements engineering techniques. However, one barrier in settling this question (insomuch as it can be ‘settled’) is that there is currently a lack of empirical data on the requirements work and activities carried out in existing e-science projects. Studying requirements activities in e-science projects by examining the actual problems encountered would yield insights regarding the challenges in working out requirements for e-science technologies, and more generally, better inform the structure of requirements work in future projects. The research in this thesis examines the requirements activities of three UK e-science projects drawn from the astronomy, molecular simulation and translational science disciplines. Detailing the experience of project team members, the research explores issues and challenges encountered over the course of working out requirements for escience technologies. In particular, this research takes a slightly different approach from similar studies, with the unit of analysis being at the project level, reflecting shifts in emphasis on requirements work as projects evolve. Three aspects of requirements work in e-science projects are explored closely. Firstly, the temporal patterns in development work and how requirements activities fitted into such rhythms, phases and trajectories; secondly, the challenges of making a prototyping approach work; thirdly, the challenges of stabilising the ‘missing middle’ – a term to describe the gap between the high-level visionary description of the system from the project proposal, into fine-grained, detailed requirements. Then all three themes are drawn together, in order to make more general observations regarding the challenges of working out requirements for e-science technologies, as well as some observations regarding the shape of requirements work over the course of an e-science project. The thesis concludes that working out requirements for e-science technologies is challenging due to the complexities of supporting multi-disciplinary and multi-organisational work and the novelty of the technology to be developed. Team members have to grapple with multiple domains of knowledge, where there is little pre-existing expertise on how these aspects could be combined. Evidence from the data suggests that multiple strategies are employed to manage this complexity; where the selection of techniques is done on a contingent basis. Thus, one of the major implications of this thesis is to suggest more systematic and explicit capture of ‘lessons learnt’ from developers of previous e-science technologies.