Crowdsourcing Research Questions? Leveraging the Crowd’s Experiential Knowledge for Problem Finding


Paper by Tiare-Maria Brasseur, Susanne Beck, Henry Sauermann, Marion Poetz: “Recently, both researchers and policy makers have become increasingly interested in involving the general public (i.e., the crowd) in the discovery of new science-based knowledge. There has been a boom of citizen science/crowd science projects (e.g., Foldit or Galaxy Zoo) and global policy aspirations for greater public engagement in science (e.g., Horizon Europe). At the same time, however, there are also criticisms or doubts about this approach. Science is complex and laypeople often do not have the appropriate knowledge base for scientific judgments, so they rely on specialized experts (i.e., scientists) (Scharrer, Rupieper, Stadtler & Bromme, 2017). Given these two perspectives, there is no consensus on what the crowd can do and what only researchers should do in scientific processes yet (Franzoni & Sauermann, 2014). Previous research demonstrates that crowds can be efficiently and effectively used in late stages of the scientific research process (i.e., data collection and analysis). We are interested in finding out what crowds can actually contribute to research processes that goes beyond data collection and analysis. Specifically, this paper aims at providing first empirical insights on how to leverage not only the sheer number of crowd contributors, but also their diversity in experience for early phases of the research process (i.e., problem finding). In an online and field experiment, we develop and test suitable mechanisms for facilitating the transfer of the crowd’s experience into scientific research questions. In doing so, we address the following two research questions: 1. What factors influence crowd contributors’ ability to generate research questions? 2. How do research questions generated by crowd members differ from research questions generated by scientists in terms of quality? There are strong claims about the significant potential of people with experiential knowledge, i.e., sticky problem knowledge derived from one’s own practical experience and practices (Collins & Evans, 2002), to enhance the novelty and relevance of scientific research (e.g., Pols, 2014). Previous evidence that crowds with experiential knowledge (e.g., users in Poetz & Schreier, 2012) or ?outsiders?/nonobvious individuals (Jeppesen & Lakhani, 2010) can outperform experts under certain conditions by having novel perspectives, support the assumption that the participation of non-scientists (i.e., crowd members) in scientific problem-finding might complement scientists’ lack of experiential knowledge. Furthermore, by bringing in exactly these new perspectives, they might help overcome problems of fixation/inflexibility in cognitive-search processes among scientists (Acar & van den Ende, 2016). Thus, crowd members with (higher levels of) experiential knowledge are expected to be superior in identifying very novel and out-of-the-box research problems with high practical relevance, as compared to scientists. However, there are clear reasons to be skeptical: despite their advantage to possess important experiential knowledge, the crowd lacks the scientific knowledge we assume to be required to formulate meaningful research questions. To study exactly how the transfer of crowd members’ experiential knowledge into science can be facilitated, we conducted two experimental studies in context of traumatology (i.e., research on accidental injuries). First, we conducted a large-scale online experiment (N=704) in collaboration with an international crowdsourcing platform to test the effect of two facilitating treatments on crowd members’ ability to formulate real research questions (study 1). We used a 2 (structuring knowledge/no structuring knowledge) x 2 (science knowledge/no science knowledge) between-subject experimental design. Second, we tested the same treatments in the field (study 2), i.e., in a crowdsourcing project in collaboration with LBG Open Innovation in Science Center. We invited patients, care takers and medical professionals (e.g., surgeons, physical therapists or nurses) concerned with accidental injuries to submit research questions using a customized online platform (https://tell-us.online/) to investigate the causal relationship between our treatments and different types and levels of experiential knowledge (N=118). An international jury of experts (i.e., journal editors in the field of traumatology) then assesses the quality of submitted questions (from the online and field experiment) along several quality dimensions (i.e., clarity, novelty, scientific impact, practical impact, feasibility) in an online evaluation process. To assess the net effect of our treatments, we further include a random sample of research questions obtained from early-stage research papers (i.e., conference papers) into the expert evaluation (blind to the source) and compare them with the baseline groups of our experiments. We are currently finalizing the data collection…(More)”.

Characterizing the cultural niches of North American birds


Justin G. Schuetz and Alison Johnston at PNAS: “Efforts to mitigate the current biodiversity crisis require a better understanding of how and why humans value other species. We use Internet query data and citizen science data to characterize public interest in 621 bird species across the United States. We estimate the relative popularity of different birds by quantifying how frequently people use Google to search for species, relative to the rates at which they are encountered in the environment.

In intraspecific analyses, we also quantify the degree to which Google searches are limited to, or extend beyond, the places in which people encounter each species. The resulting metrics of popularity and geographic specificity of interest allow us to define aspects of relationships between people and birds within a cultural niche space. We then estimate the influence of species traits and socially constructed labels on niche positions to assess the importance of observations and ideas in shaping public interest in birds.

Our analyses show clear effects of migratory strategy, color, degree of association with bird feeders, and, especially, body size on niche position. They also indicate that cultural labels, including “endangered,” “introduced,” and, especially, “team mascot,” are strongly associated with the magnitude and geographic specificity of public interest in birds. Our results provide a framework for exploring complex relationships between humans and other species and enable more informed decision-making across diverse bird conservation strategies and goals….(More)”.

Innovation Meets Citizen Science


Caroline Nickerson at SciStarter: “Citizen science has been around as long as science, but innovative approaches are opening doors to more and deeper forms of public participation.

Below, our editors spotlight a few projects that feature new approaches, novel research, or low-cost instruments. …

Colony B: Unravel the secrets of microscopic life! Colony B is a mobile gaming app developed at McGill University that enables you to contribute to research on microbes. Collect microbes and grow your colony in a fast-paced puzzle game that advances important scientific research.

AirCasting: AirCasting is an open-source, end-to-end solution for collecting, displaying, and sharing health and environmental data using your smartphone. The platform consists of wearable sensors, including a palm-sized air quality monitor called the AirBeam, that detect and report changes in your environment. (Android only.)

LingoBoingo: Getting computers to understand language requires large amounts of linguistic data and “correct” answers to language tasks (what researchers call “gold standard annotations”). Simply by playing language games online, you can help archive languages and create the linguistic data used by researchers to improve language technologies. These games are in English, French, and a new “multi-lingual” category.

TreeSnap: Help our nation’s trees and protect human health in the process. Invasive diseases and pests threaten the health of America’s forests. With the TreeSnap app, you can record the location and health of particular tree species–those unharmed by diseases that have wiped out other species. Scientists then use the collected information to locate candidates for genetic sequencing and breeding programs. Tag trees you find in your community, on your property, or out in the wild to help scientists understand forest health….(More)”.

The tools of citizen science: An evaluation of map-based crowdsourcing platforms


Paper by Zachary Lamoureux and Victoria Fast: “There seems to be a persistent yet inaccurate sentiment that collecting vast amounts of data via citizen science is virtually free, especially compared to the cost of privatized scientific endeavors (Bonney et al., 2009; Cooper, Hochachka & Dhondt, 2011). However, performing scientific procedures with the assistance of the public is often far more complex than traditional scientific enquiry (Bonter & Cooper, 2012).

Citizen science promotes the participation of the public in scientific endeavors (Hecker et al., 2018). While citizen science is not synonymous with volunteered geographic information (VGI)— broadly defined as the creation of geographic information by citizens (Goodchild, 2007)—it often produces geographic information. Similar to VGI, citizen science projects tend to follow specific protocols to ensure the crowdsourced geographic data serves as an input for (scientific) research (Haklay, 2013). Also similar to VGI, citizen science projects often require software applications and specialized training to facilitate citizen data collection. Notably, citizen science projects are increasingly requiring a webbased participatory mapping platform—i.e., Geoweb (Leszczynski & Wilson, 2013)—to coordinate the proliferation of citizen contributions. ...

In this research, we investigate publicly available commercial and opensource map-based tools that enable citizen science projects. Building on a comprehensive comparative framework, we conduct a systematic evaluation and overview of five map-based crowdsourcing platforms: Ushahidi, Maptionnaire, Survey123 (ArcGIS Online), Open Data Kit, and GIS Cloud. These tools have additional uses that extend beyond the field of citizen science; however, the scope of the investigation was narrowed to focus on aspects most suitable for citizen science endeavors, such as the collection, management, visualization and dissemination of crowdsourced data. It is our intention to provide information on how these publicly available crowdsourcing platforms suit generic geographic citizen science crowdsourcing needs….(More)”.

A Review of Citizen Science and Crowdsourcing in Applications of Pluvial Flooding


Jonathan D. Paul in Frontiers in Earth Science: “Pluvial flooding can have devastating effects, both in terms of loss of life and damage. Predicting pluvial floods is difficult and many cities do not have a hydrodynamic model or an early warning system in place. Citizen science and crowdsourcing have the potential for contributing to early warning systems and can also provide data for validating flood forecasting models. Although there are increasing applications of citizen science and crowdsourcing in fluvial hydrology, less is known about activities related to pluvial flooding. Hence the aim of this paper is to review current activities in citizen science and crowdsourcing with respect to applications of pluvial flooding.

Based on a search in Scopus, the papers were first filtered for relevant content and then classified into four main themes. The first two themes were divided into (i) applications relevant during a flood event, which includes automated street flooding detection using crowdsourced photographs and sensors, analysis of social media, and online and mobile applications for flood reporting; and (ii) applications related to post-flood events. The use of citizen science and crowdsourcing for model development and validation is the third theme while the development of integrated systems is theme four. All four main areas of research have the potential to contribute to early warning systems and build community resilience. Moreover, developments in one will benefit others, e.g., further developments in flood reporting applications and automated flood detection systems will yield data useful for model validation….(More)”.

Citizen science for environmental policy: Development of an EU-wide inventory and analysis of selected practices


EU Science Hub: “Citizen science is the non-professional involvement of volunteers in the scientific process, whether in the data collection phase or in other phases of the research.

It can be a powerful tool for environmental management that has the potential to inform an increasingly complex environmental policy landscape and to meet the growing demands from society for more participatory decision-making.

While there is growing interest from international bodies and national governments in citizen science, the evidence that it can successfully contribute to environmental policy development, implementation, evaluation or compliance remains scant.

Central to elucidating this question is a better understanding of the benefits delivered by citizen science, that is to determine to what extent these benefits can contribute to environmental policy, and to establish whether projects that provide policy support also co-benefit science and encourage meaningful citizen engagement.

EU-wide inventory 

In order to get an evidence base of citizen science activities that can support environmental policies in the European Union (EU), the European Commission (DG ENV, with the support of DG JRC) contracted Bio Innovation Service (FR), in association with Fundacion Ibercivis (ES) and The Natural History Museum (UK), to perform a “Study on an inventory of citizen science activities for environmental policies”.

The first objective was to develop an inventory of citizen science projects relevant for environmental policy and assess how these projects contribute to the Sustainable Development Goals (SDGs) set by the United Nations (UN) General Assembly.

To this end, a desk-research and an EU-wide survey were used to identify 503 citizen science projects of relevance to environmental policy.

The study demonstrates the breadth of citizen science that can be of relevance to environmental policy....Three salient features were found:

  • Government support, not only in the funding, but also through active participation in the design and implementation of the project appears to be a key factor for the successful uptake of citizen science in environmental policy.
  • When there is easy engagement process for the citizens, that is, with projects requiring limited efforts and a priori skills, this facilitates their policy uptake.
  • Scientific aspects on the other hand did not appear to affect the policy uptake of the analysed projects, but they were a strong determinant of how well the project could serve policy: projects with high scientific standards and endorsed by scientists served more phases of the environmental policy cycle.

In conclusion, this study demonstrates that citizen science has the potential to be a cost-effective way to contribute to policy and highlights the importance of fostering a diversity of citizen science activities and their innovativeness….(More)”.

Learning Through Citizen Science: Enhancing Opportunities by Design


National Academies: “Scientific research that involves nonscientists contributing to research processes – also known as ‘citizen science’ – supports participants’ learning, engages the public in science, contributes to community scientific literacy, and can serve as a valuable tool to facilitate larger scale research, says a new report from the National Academies of Sciences, Engineering, and Medicine.  If one of the goals of a citizen science project is to advance learning, designers should plan for it by defining intended learning outcomes and using evidence-based strategies to reach those outcomes.

“This report affirms that citizen science projects can help participants learn scientific practices and content, but most likely only if the projects are designed to support learning,” says Rajul Pandya, chair of the committee that wrote the report and director, Thriving Earth Exchange, AGU.  

The term “citizen science” can be applied to a wide variety of projects that invite nonscientists to engage in doing science with the intended goal of advancing scientific knowledge or application. For example, a citizen science project might engage community members in collecting data to monitor the health of a local stream. As another example, among the oldest continuous organized datasets in the United States are records kept by farmers and agricultural organizations that document the timing of important events, such as sowing, harvests, and pest outbreaks.

Citizen science can support science learning in several ways, the report says. It offers people the opportunity to participate in authentic scientific endeavors, encourages learning through projects conducted in real-world contexts, supports rich social interaction that deepens learning, and engages participants with real data. Citizen science also includes projects that grow out of a community’s desire to address an inequity or advance a priority. For example, the West-Oakland Indicators Project, a community group in Oakland, Calif., self-organizes to collect and analyze air quality data and uses that data to address trucking in and around schools to reduce local children’s exposure to air pollution. When communities can work alongside scientists to advance their priorities, enhanced community science literacy is one possible outcome….

In order to maximize learning outcomes, the report recommends that designers and practitioners of citizen science projects should intentionally build them for learning. This involves knowing the audience; intentionally designing for diversity; engaging stakeholders in the design; supporting multiple kinds of participant engagement; encouraging social interaction; building learning supports into the project; and iteratively improving projects through evaluation and refinement.  Engaging stakeholders and participants in design and implementation results in more learning for all participants, which can support other project goals. 

The report also lays out a research agenda that can help to build the field of citizen science by filling gaps in the current understanding of how citizen science can support science learning and enhance science education. Researchers should consider three important factors: citizen science extends beyond academia and therefore, evidence for practices that advance learning can be found outside of peer-reviewed literature; research should include attention to practice and link theory to application; and attention must be given to diversity in all research, including ensuring broad participation in the design and implementation of the research. Pursuing new lines of inquiry can help add value to the existing research, make future research more productive, and provide support for effective project implementation….(More)”.


Citizen science, public policy


Paper by Christi J. GuerriniMary A. Majumder,  Meaganne J. Lewellyn, and Amy L. McGuire in Science: “Citizen science initiatives that support collaborations between researchers and the public are flourishing. As a result of this enhanced role of the public, citizen science demonstrates more diversity and flexibility than traditional science and can encompass efforts that have no institutional affiliation, are funded entirely by participants, or continuously or suddenly change their scientific aims.

But these structural differences have regulatory implications that could undermine the integrity, safety, or participatory goals of particular citizen science projects. Thus far, citizen science appears to be addressing regulatory gaps and mismatches through voluntary actions of thoughtful and well-intentioned practitioners.

But as citizen science continues to surge in popularity and increasingly engage divergent interests, vulnerable populations, and sensitive data, it is important to consider the long-term effectiveness of these private actions and whether public policies should be adjusted to complement or improve on them. Here, we focus on three policy domains that are relevant to most citizen science projects: intellectual property (IP), scientific integrity, and participant protections….(More)”.

Information to Action: Strengthening EPA Citizen Science Partnerships for Environmental Protection


Report by the National Advisory Council for Environmental Policy and Technology: “Citizen science is catalyzing collaboration; new data and information brought about by greater public participation in environmental research are helping to drive a new era of environmental protection. As the body of citizen-generated data and information in the public realm continues to grow, EPA must develop a clear strategy to lead change and encourage action beyond the collection of data. EPA should recognize the variety of opportunities that it has to act as a conduit between the public and key partners, including state, territorial, tribal and local governments; nongovernmental organizations; and leading technology groups in the private sector. The Agency should build collaborations with new partners, identify opportunities to integrate equity into all relationships, and ensure that grassroots and community-based organizations are well supported and fairly resourced in funding strategies.

Key recommendations under this theme:

  • Recommendation 1. Catalyze action from citizen science data and information by providing guidance and leveraging collaboration.
  • Recommendation 2. Build inclusive and equitable partnerships by understanding partners’ diverse concerns and needs, including prioritizing better support for grassroots and community-based partnerships in EPA grantfunding strategies.

Increase state, territorial, tribal and local government engagement with citizen science

The Agency should reach out to tribes, states, territories and local governments throughout the country to understand the best practices and strategies for encouraging and incorporating citizen science in environmental protection. For states and territories looking for ways to engage in citizen science, EPA can help design strategies that recognize the community perspectives while building capacity in state and territorial governments. Recognizing the direct Executive Summary Information to Action: Strengthening EPA Citizen Science Partnerships for Environmental Protection connection between EPA and tribes, the Agency should seek tribal input and support tribes in using citizen science for environmental priorities. EPA should help to increase awareness for citizen science and where jurisdictional efforts already exist, assist in making citizen science accessible through local government agencies. EPA should more proactively listen to the voices of local stakeholders and encourage partners to embrace a vision for citizen science to accelerate the achievement of environmental goals. As part of this approach, EPA should find ways to define and communicate the Agency’s role as a resource in helping communities achieve environmental outcomes.

Key recommendations under this theme:

  • Recommendation 3. Provide EPA support and engage states and territories to better integrate citizen science into program goals.
  • Recommendation 4. Build on the unique strengths of EPA-tribal relationships.
  • Recommendation 5. Align EPA citizen science work to the priorities of local governments.

Leverage external organizations for expertise and project level support

Collaborations between communities and other external organizations—including educational institutions, civic organizations, and community-based organizations— are accelerating the growth of citizen science. Because EPA’s direct connection with members of the public often is limited, the Agency could benefit significantly by consulting with key external organizations to leverage citizen science efforts to provide the greatest benefit for the protection of human health and the environment. EPA should look to external organizations as vital connections to communities engaged in collaboratively led scientific investigation to address community-defined questions, referred to as community citizen science. External organizations can help EPA in assessing gaps in community-driven research and help the Agency to design effective support tools and best management practices for facilitating effective environmental citizen science programs….(More)”.

Help NASA create the world’s largest landslide database


EarthSky: “Landslides cause thousands of deaths and billions of dollars in property damage each year. Surprisingly, very few centralized global landslide databases exist, especially those that are publicly available.

Now NASA scientists are working to fill the gap—and they want your help collecting information. In March 2018, NASA scientist Dalia Kirschbaum and several colleagues launched a citizen science project that will make it possible to report landslides you have witnessed, heard about in the news, or found on an online database. All you need to do is log into the Landslide Reporter portal and report the time, location, and date of the landslide – as well as your source of information. You are also encouraged to submit additional details, such as the size of the landslide and what triggered it. And if you have photos, you can upload them.

Kirschbaum’s team will review each entry and submit credible reports to the Cooperative Open Online Landslide Repository (COOLR) — which they hope will eventually be the largest global online landslide catalog available.

Landslide Reporter is designed to improve the quantity and quality of data in COOLR. Currently, COOLR contains NASA’s Global Landslide Catalog, which includes more than 11,000 reports on landslides, debris flows, and rock avalanches. Since the current catalog is based mainly on information from English-language news reports and journalists tend to cover only large and deadly landslides in densely populated areas, many landslides never make it into the database….(More)”.