Crowdsourcing and global health: strengthening current applications and identification of future uses

Abstract

INTRODUCTION: Despite the method existing for centuries, uses of crowdsourcing have been rising rapidly since the term was coined a decade ago. Crowdsourcing refers to ‘outsourcing’ a problem or task to a large group of people (i.e., a crowd) rapidly and cheaply. Researchers debate over definitions of crowdsourcing, and it is often conflated with mHealth, web 2.0, or data mining. Due to the inexpensive and rapid nature of crowdsourcing, it may be particularly amenable to health research and practice, especially in a global health context, where health systems, human resources, and finances are often scarce. Indeed, one of the dominant methods of health research prioritization uses crowdsourcing, and in particular, wisdom of the crowds. This method, called the Child Health and Nutrition Research Initiative (CHNRI) method, employs researchers to generate and rank research options which are scored against pre-set criteria. Their scores are combined with weights for each criterion, set by a larger, diverse group of stakeholders, to create a ranked list of research options. Unfortunately, due to difficulties in defining and assembling a group of stakeholders that would be appropriate to each exercise, 75% of CHNRI exercises to-date did not involve stakeholders, and therefore presented unweighted ranks.

METHODS: First, a crowdsourcing was defined through a literature review. Benefits and challenges of crowdsourcing were explored, in addition to ethical issues with crowdsourcing. A second literature review was conducted to explore ways in which crowdsourcing has been already used in health and global health. As crowdsourcing could be a potential solution to data scarcity or act as a platform for intervention in global health settings, but its potential has never been systematically assessed, a CHNRI exercise was conducted to explore potential uses of crowdsourcing in global health and conflict. Experts from both global health and crowdsourcing participated in generation and scoring ideas. This CHNRI exercise was conducted in-line with previously described steps of the CHNRI method for setting health research priorities. As three quarters of CHNRI exercises have not utilized a larger reference group (LRG) of stakeholders, and the public was cited as the most difficult stakeholder group to involve, we conducted a survey using Amazon Mechanical Turk, an online crowdsourcing platform, that involved an international group of predominantly laypersons who, in essence, formed a public stakeholder group, scoring the most common CHNRI criteria using a 5-point Likert scale. The resulting means were converted to weights that can be used in upcoming exercises. Differences in geographic location, and whether the respondents were health stakeholders were assessed through the Fisher exact test and Wilcoxon rank-sum test, respectively. The influence of other demographic characteristics was explored through random-intercept modelling and logistic regression. Finally, an example of a national-level CHNRI exercise, which is the largest CHNRI conducted to-date, exploring research priorities in child health in India is described.

RESULTS: A comprehensive definition of crowdsourcing is given, along with its benefits, challenges, and ethical considerations for using crowdsourcing, based on a literature review.

An overview of uses of crowdsourcing in health are discussed, and potential challenges and techniques for improving accuracy, such as introducing thresholds, qualifiers, introducing modular tasks and gamification. Crowdsourcing was frequently used as a diagnostics or surveillance tool. The CHNRI method was not identified in the second literature review. In re-weighting the CHNRI criteria using a public stakeholder group, we identified differences in relative importance of the criteria driven by geographic location and health status. When using random-intercept modelling to control for geographic location, we found differences due to health status in many criteria (n = 11), followed by gender (n = 10), ethnicity (n = 9), and religion (n = 8). We used the CHNRI method to explore potential uses of crowdsourcing in global health, and found that the majority of ideas were problem solving or data generation in nature. The top-ranked idea was to use crowdsourcing to generate more timely reports of future epidemics (such as in the case of Ebola), and other ideas relating to using crowdsourcing for the surveillance or control of communicable disease scored highly. Many ideas were related to the United Nations’ Sustainable Development Goals (SDGs). Finally, a national-level exercise to set research priorities in child health in India identified differential priorities for three regions (Empowered Action Group and North Eastern States, Northern States and Union Territories, and the Southern and Western States). The results will be very useful in developing targeted programmes for each region, enabling India to make progress towards SDG 3.2.

CONCLUSION: Crowdsourcing has grown exponentially in the past decade. Integrating gamification, machine learning, simplifying tasks and introducing thresholds or trustworthiness scores increases accuracy of results. This research provides recommendations for improvements in the CHNRI method itself, and for crowdsourcing, generally. Crowdsourcing is a rapid, inexpensive tool for research, and thus, is a promising data collection method or intervention for health and global health.