New study improves ‘crowd wisdom’ estimates

Curated on Posted on by Stefaan Verhulst

Santa Fe Institute: “In 1907, a statistician named Francis Galton recorded the entries from a weight-judging competition as people guessed the weight of an ox. Galton analyzed hundreds of estimates and found that while individual guesses varied wildly, the median of the entries was surprisingly accurate and within one percent of the ox’s real weight. When Galton published his results, he ushered the theory of collective intelligence, or the “wisdom of crowds,” into the public conscience.

Collective wisdom has its limits, though. In a new study published in the Journal of the Royal Society Interface, researchers Albert Kao (Harvard University), Andrew Berdahl (Santa Fe Institute), and their colleagues examined just how accurate our collective intelligence is and how individual bias and information sharing skew aggregate estimates. Using their findings, they developed a mathematical correction that takes into account bias and social information to generate an improved crowd estimate. In the study, their corrected measures were more accurate than the mean, median, and other traditional statistics.

“There is growing evidence that the wisdom of crowds can be really powerful,” Kao says. “A lot of studies show that you can calculate the average of estimates and that average can be surprisingly good.”

“However,” adds Berdahl, “there is a great deal of evidence that people have strong biases in estimation and decision tasks.”

The researchers recruited over 800 volunteers to participate in the study and asked each participant to guess the number of gumballs in a jar, which ranged over several orders of magnitude from 54 to more than 27,000. Additionally, they quantified how individuals incorporate social information into their own opinion. To do so, the researchers offered participants fake details about other people’s guesses and allowed them to change their estimate in light of that information.

Kao’s team found that while estimates varied considerably, they were highly predictable. People tended to guess numbers smaller than the actual value and guessed a wider range of numbers for larger jars. Social information also plays a role in collective wisdom. For example, the simulated social information revealed that peer advice more strongly influenced an individual if the knowledge suggested the actual number of items was higher than the guesser’s initial estimate. Smaller guesses, even if more accurate, appear to be more frequently discounted…(More)”

See: “Counteracting estimation bias and social influence to improve the wisdom of crowds” in the Journal of the Royal Society Interface (April 18, 2018)

A survey of incentive engineering for crowdsourcing

Curated on Posted on by Stefaan Verhulst

Conor Muldoon, Michael J. O’Grady and Gregory M. P. O’Hare in the Knowledge Engineering Review: “With the growth of the Internet, crowdsourcing has become a popular way to perform intelligence tasks that hitherto would be either performed internally within an organization or not undertaken due to prohibitive costs and the lack of an appropriate communications infrastructure.

In crowdsourcing systems, whereby multiple agents are not under the direct control of a system designer, it cannot be assumed that agents will act in a manner that is consistent with the objectives of the system designer or principal agent. In situations whereby agents’ goals are to maximize their return in crowdsourcing systems that offer financial or other rewards, strategies will be adopted by agents to game the system if appropriate mitigating measures are not put in place.

The motivational and incentivization research space is quite large; it incorporates diverse techniques from a variety of different disciplines including behavioural economics, incentive theory, and game theory. This paper specifically focusses on game theoretic approaches to the problem in the crowdsourcing domain and places it in the context of the wider research landscape. It provides a survey of incentive engineering techniques that enable the creation of apt incentive structures in a range of different scenarios….(More)”.

Literature review on collective intelligence: a crowd science perspective

Curated on Posted on by Stefaan Verhulst

Chao Yu in the International Journal of Crowd Science: “A group can be of more power and better wisdom than the sum of the individuals. Foreign scholars have noticed that for a long time and called it collective intelligence. It has emerged from the communication, collaboration, competition and brain storming, etc. Collective intelligence appears in many fields such as public decisions, voting activities, social networks and crowdsourcing.

Crowd science mainly focuses on the basic principles and laws of the intelligent activities of groups under the new interconnection model. It explores how to give full play to the intelligence agents and groups, dig their potential to solve the problems that are difficult for a single agent.

In this paper, we present a literature review on collective intelligence in a crowd science perspective. We focus on researchers’ related work, especially that under which circumstance can group show their wisdom, how to measure it, how to optimize it and its modern or future applications in the digital world. That is exactly what the crowd science pays close attention to….(More)”.

Use of data & technology for promoting waste sector accountability in Nepal

Curated on Posted on by Stefaan Verhulst

Saroj Bista at YoungInnovations: “All the Nepalese people are saddened to see waste abandoned in the Capital, Kathmandu. Among them, many are concerned to find solutions to such a problem, including Kathmandu City. A 2015 report stated that Kathmandu Metropolitan City (KMC) alone receives 525 tonnes of waste in a day while it manages to collect 516 tonnes out if it, meaning that 8 tonnes of waste are left/abandoned….

Despite many stakeholders including the government sector, non-governmental organizations, private sectors have been working to address the problem associated with solid waste mapping in urban sector, the problem continued to exist.

YoungInnovations and Clean Up Nepal came together to discuss if we could tackle this problem. We discussed if keeping track of everybody’s efforts as well as noticing every piece of waste in the city raises accountability of stakeholders adds a value. YoungInnovations has over a decade of experience in developing data and evidence-based tech solutions to problem. Clean Up Nepal is a civil society organization working to provide an enabling environment to improve solid waste management and water, sanitation and hygiene in Nepal by working closely with local communities and relevant stakeholders. In this, both the organizations agreed to work mixing the expertise of each other to offer the government with an technology that avails stakeholders with proper data related to solid waste and its management.

Also, the preliminary idea was tested with some ongoing initiatives of such kind (Waste Atlas, Letsdoitworld etc) while consultations were held with some of the organizations like The GovLab, ICIMOD learn from their expertise on open data as well as environmental aspects. A remarkable example of smart waste management being carried out in Ulaanbaatar, Capital of Mongolia did motivate us to test the idea in Nepal….

Nepal Waste Map Web App

Nepal Waste Map web is a composite of several features primarily focused at the following:

  1. Display of key stats and information about solid waste
  2. Admin panel to interact with the data for taking possible actions (update, edit and delete)…

Nepal Waste Map Mobile

A Mobile App primarily reflects Nepal Waste Map in the mobile phones. Most of the features resemble with the Nepal Waste Map Web App.

However, some functionalities in the app are key in terms of data aspects:

Crowdsourcing Functionality

Any public (users) who use the app can report issues related to illegal waste dumping and waste esp. Plastic burning. Example: if I saw somebody burning plastic wastes, I can use the app for reporting such an incident along with the photo as evidence as well as coordinates. The admin of the web app can view the report in a real time and take action (not limited to defined as acknowledge and marking resolved)…(More)”.

From Crowdsourcing to Extreme Citizen Science: Participatory Research for Environmental Health

Curated on Posted on by Stefaan Verhulst

P.B. English, M.J. Richardson, and C. Garzón-Galvis in the Annual Review of Public Health: “Environmental health issues are becoming more challenging, and addressing them requires new approaches to research design and decision-making processes. Participatory research approaches, in which researchers and communities are involved in all aspects of a research study, can improve study outcomes and foster greater data accessibility and utility as well as increase public transparency. Here we review varied concepts of participatory research, describe how it complements and overlaps with community engagement and environmental justice, examine its intersection with emerging environmental sensor technologies, and discuss the strengths and limitations of participatory research. Although participatory research includes methodological challenges, such as biases in data collection and data quality, it has been found to increase the relevance of research questions, result in better knowledge production, and impact health policies. Improved research partnerships among government agencies, academia, and communities can increase scientific rigor, build community capacity, and produce sustainable outcomes….(More)”

Citizen Sensing: A Toolkit

Curated on Posted on by Stefaan Verhulst

Book from Making Sense: “Collaboration using open-source technologies makes it possible to create new and powerful forms of community action, social learning and citizenship. There are now widely accessible platforms through which we can come together to make sense of urgent challenges, and discover ways to address these. Together we can shape our streets, neighbourhoods, cities and countries – and in turn, shape our future. You can join with others to become the solution to challenges in our environment, in our communities and in the way we live together.

In this book, there are ideas and ways of working that can help you build collective understanding and inspire others to take action. By coming together with others on issues you identify and define yourselves, and by designing and using the right tools collaboratively, both your awareness and ability to act will be improved. In the process, everyone involved will have better insights, better arguments and better discussions; sometimes to astonishing effect!

We hope this book will help you engage people to learn more about an issue that concerns you, support you to take action, and change the world for the better. This resource will teach you how to scope your questions, identify and nurture relevant communities, and plan an effective campaign. It will then help you gather data and evidence, interpret your findings, build awareness and achieve tangible outcomes. Finally, it will show you how to reflect on these outcomes, and offers suggestions on how you can leave a lasting legacy.

This book is intended to help community activists who are curious or concerned about one or more issues, whether local or global, and are motivated to take action. This resource can also be of value to professionals in organisations which support community actions and activists. Finally, this book will be of interest to researchers in the fields of citizen science, community activism and participatory sensing, government officials and other public policy actors who wish to include citizens’ voices in the decision-making process…(More)”.

Can we solve wicked problems?

Curated on Posted on by Stefaan Verhulst

Paper by Gianluca Elia and Alessandro Margherita describing “A conceptual framework and a collective intelligence system to support problem analysis and solution design for complex social issues…Wicked problems are complex and multifaceted issues that have no single solution, and are perceived by different stakeholders through contrasting views. Examples in the social context include climate change, poverty, energy production, sanitation, sustainable cities, pollution and homeland security.

Extant research has been addressed to support open discussion and collaborative decision making in wicked scenarios, but complexities derive from the difficulty to leverage multiple contributions, coming from both experts and non-experts, through a structured approach.

In such view, we present a conceptual framework for the study of wicked problem solving as a complex and multi-stakeholder process. Afterwards, we describe an integrated system of tools and associated operational guidelines aimed to support collective problem analysis and solution design. The main value of the article is to highlight the relevance of collective approaches in the endeavor of wicked problem resolution, and to provide an integrated framework of activities, actors and purposeful tools….(More)”.

 

Coastal research increasingly depends on citizen scientists

Curated on Posted on by Stefaan Verhulst

Brenna Visser at CS Monitor: “…This monthly ritual is a part of the COASST survey, a program that relies on data taken by volunteers to study large-scale patterns in seabird populations on the West Coast. The Haystack Rock Awareness Program conducts similar surveys for sea stars and marine debris throughout the year.

Surveys like these play a small part in a growing trend in the science community to use citizen scientists as a way to gather massive amounts of data. Over the weekend, marine scientists and conservationists came to Cannon Beach for an annual Coast Conference, a region wide event to discuss coastal science and stewardship.

Whether the presentation was about ocean debris, marine mammals, seabirds, or ocean jellies, many relied on the data collection work of volunteers throughout the state. A database for citizen science programs called Citsci.org, which recorded only a few dozen groups 10 years ago, now has more than 500 groups registered across the country, with new ones registering every day….

Part of the rise has to do with technology, she said. Apps that help identify species and allow unprecedented access to information have driven interest up and removed barriers that would have otherwise made it harder to collect data without formal training. Another is the science community slowly coming around to accept citizen science.

“I think there’s a lot of reticence in the science community to use citizen science. There’s some doubt the data collected is of the precision or accuracy that is needed to document phenomena,” Parrish said. “But as it grows, the more standardized it becomes. What we’re seeing right now is a lot of discussion in citizen science programs asking what they need to do to get to that level.”…While a general decline in federal funding for scientific research could play a factor in the science community’s acceptance of using volunteer-collected data, Parrish said, regardless of funding, there are some projects only citizen scientists can accomplish….(More)”

Say goodbye to the information age: it’s all about reputation now

Curated on Posted on by Stefaan Verhulst

Gloria Origgi at Aeon: “There is an underappreciated paradox of knowledge that plays a pivotal role in our advanced hyper-connected liberal democracies: the greater the amount of information that circulates, the more we rely on so-called reputational devices to evaluate it. What makes this paradoxical is that the vastly increased access to information and knowledge we have today does not empower us or make us more cognitively autonomous. Rather, it renders us more dependent on other people’s judgments and evaluations of the information with which we are faced.

We are experiencing a fundamental paradigm shift in our relationship to knowledge. From the ‘information age’, we are moving towards the ‘reputation age’, in which information will have value only if it is already filtered, evaluated and commented upon by others. Seen in this light, reputation has become a central pillar of collective intelligence today. It is the gatekeeper to knowledge, and the keys to the gate are held by others. The way in which the authority of knowledge is now constructed makes us reliant on what are the inevitably biased judgments of other people, most of whom we do not know.

Let me give some examples of this paradox. If you are asked why you believe that big changes in the climate are occurring and can dramatically harm future life on Earth, the most reasonable answer you’re likely to provide is that you trust the reputation of the sources of information to which you usually turn for acquiring information about the state of the planet. In the best-case scenario, you trust the reputation of scientific research and believe that peer-review is a reasonable way of sifting out ‘truths’ from false hypotheses and complete ‘bullshit’ about nature. In the average-case scenario, you trust newspapers, magazines or TV channels that endorse a political view which supports scientific research to summarise its findings for you. In this latter case, you are twice-removed from the sources: you trust other people’s trust in reputable science….(More)”.

The Promise of Community Citizen Science

Curated on Posted on by Stefaan Verhulst

Report by Ramya Chari, Luke J. Matthews, Marjory S. Blumenthal, Amanda F. Edelman, and Therese Jones: “Citizen science is public participation in research and scientific endeavors. Citizens volunteer as data collectors in science projects; collaborate with scientific experts on research design; and actively lead and carry out research, exerting a high degree of control and ownership over scientific activities. The last type — what we refer to as community citizen science — tends to involve action-oriented research to support interventional activities or policy change. This type of citizen science can be of particular importance to those working at the nexus of science and decisionmaking.

The authors examine the transformative potential of community citizen science for communities, science, and decisionmaking. The Perspective is based on the authors’ experiences working in collaboration with community groups, extensive readings of the scientific literature, and numerous interviews with leading scholars and practitioners in the fields of citizen science and participatory research. It first discusses models of citizen science in general, including community citizen science, and presents a brief history of its rise. It then looks at possible factors motivating the development of community citizen science, drawing from an exploration of the relationships among citizens, science, and decisionmaking. The final section examines areas in which community citizen science may exhibit promise in terms of outcomes and impacts, discusses concerns that may hinder its overall potential, and assesses the roles different stakeholders may play to continue to develop community citizen science into a positive force for science and society.

Key Findings

At Its Core, Citizen Science Is Public Participation in Research and Scientific Endeavors

  • Citizens volunteer as data collectors in science projects, collaborate with scientific experts on research design, and actively lead and carry out research.
  • It is part of a long tradition of rebirth of inventors, scientists, do-it-yourselfers, and makers at all levels of expertise.
  • Instead of working alone, today’s community citizen scientists take advantage of new technologies for networking and coordination to work collaboratively; learn from each other; and share knowledge, insights, and findings.

The Democratization of Science and the Increasingly Distributed Nature of Expertise Are Not Without Concern

  • There is some tension and conflict between current standards of practice and the changes required for citizen science to achieve its promising future.
  • There is also some concern about the potential for bias, given that some efforts begin as a form of activism.

Yet the Efforts of Community Citizen Science Can Be Transformative

  • Success will require an engaged citizenry, promote more open and democratic decisionmaking processes, and generate new solutions for intractable problems.
  • If its promise holds true, the relationship between science and society will be profoundly transformed for the betterment of all…(More)”.