MIT Crowdsources the Next Great (free) IQ Test

Curated on Posted on by Stefaan Verhulst

ThePsychReport: “Raven’s Matrices have long been a gold standard for psychologists needing to measure general intelligence. But the good ones, the ones scientists like to use, are too expensive for most research projects.

Christopher Chabris, associate professor of psychology at Union College, and David Engel, postdoctoral associate at MIT Sloan School of Management, think the public can help. They recently launched a campaign to crowdsource “the next great IQ test.” The Matrix Reasoning Challenge, created through MIT’s Center for Collective Intelligence with Anita Woolley and Tom Malone,  calls on the public to design and submit matrix puzzles – 3×3 grids that asks subjects to complete a pattern by filling in a missing square.

Chabris says they aren’t trying to compete with commercially available tests used for diagnostic or clinical purposes, but rather want to provide a trustworthy and free alternative for scientists. Because these types of puzzles are nonverbal, culturally neutral, and objective, they have wide-ranging applications and are particularly useful when conducting research across various demographics. If this project is successful, a lot more scientists could do a lot more research.

A simple example of a matrix puzzle. Source: Matrix Reasoning Challenge

“Researchers typically don’t have that much money,” Chabris said. “They can’t afford pay per use tests. Sometimes they have no research budgets, or if they do, they’re not large enough for that kind of thing. Our real goal is to create something useful for researchers.”

Through the Matrix Reasoning Challenge, Chabris and Engel also hope to better understand how crowdsourcing can be used to problem-solve in social and cognitive sciences.

Social scientists already widely use crowdsourcing sites like Amazon’s Mechanical Turk to recruit participants for their studies, but the matrix project is different in that it seeks to tap into the public’s expertise to help solve scientific problems. Scientists in computer science and bioinformatics have been able to harness this expertise to yield some incredible results. Using TopCoder.com, NASA was able to find a more efficient way to deploy solar panels on the International Space Station. Harvard Medical School was able to develop better software for analyzing immune-system genes. With The Matrix Reasoning Challenge, Chabris and Engel are beginning to explore crowdsourcing’s potential in the social sciences.”

Needed: A New Generation of Game Changers to Solve Public Problems

Curated on Posted on by Stefaan Verhulst

Beth Noveck: “In order to change the way we govern, it is important to train and nurture a new generation of problem solvers who possess the multidisciplinary skills to become effective agents of change. That’s why we at the GovLab have launched The GovLab Academy with the support of the Knight Foundation.
In an effort to help people in their own communities become more effective at developing and implementing creative solutions to compelling challenges, The Gov Lab Academy is offering two new training programs:
1) An online platform with an unbundled and evolving set of topics, modules and instructors on innovations in governance, including themes such as big and open data and crowdsourcing and forthcoming topics on behavioral economics, prizes and challenges, open contracting and performance management for governance;
2) Gov 3.0: A curated and sequenced, 14-week mentoring and training program.
While the online-platform is always freely available, Gov 3.0 begins on January 29, 2014 and we invite you to to participate. Please forward this email to your networks and help us spread the word about the opportunity to participate.
Please consider applying (individuals or teams may apply), if you are:

  • an expert in communications, public policy, law, computer science, engineering, business or design who wants to expand your ability to bring about social change;

  • a public servant who wants to bring innovation to your job;

  • someone with an important idea for positive change but who lacks key skills or resources to realize the vision;

  • interested in joining a network of like-minded, purpose-driven individuals across the country; or

  • someone who is passionate about using technology to solve public problems.

The program includes live instruction and conversation every Wednesday from 5:00– 6:30 PM EST for 14 weeks starting Jan 29, 2014. You will be able to participate remotely via Google Hangout.

Gov 3.0 will allow you to apply evolving technology to the design and implementation of effective solutions to public interest challenges. It will give you an overview of the most current approaches to smarter governance and help you improve your skills in collaboration, communication, and developing and presenting innovative ideas.

Over 14 weeks, you will develop a project and a plan for its implementation, including a long and short description, a presentation deck, a persuasive video and a project blog. Last term’s projects covered such diverse issues as post-Fukushima food safety, science literacy for high schoolers and prison reform for the elderly. In every case, the goal was to identify realistic strategies for making a difference quickly.  You can read the entire Gov 3.0 syllabus here.

The program will include national experts and instructors in technology and governance both as guests and as mentors to help you design your project. Last term’s mentors included current and former officials from the White House and various state, local and international governments, academics from a variety of fields, and prominent philanthropists.

People who complete the program will have the opportunity to apply for a special fellowship to pursue their projects further.

Previously taught only on campus, we are offering Gov 3.0 in beta as an online program. This is not a MOOC. It is a mentoring-intensive coaching experience. To maximize the quality of the experience, enrollment is limited.

Please submit your application by January 22, 2014. Accepted applicants (individuals and teams) will be notified on January 24, 2014. We hope to expand the program in the future so please use the same form to let us know if you would like to be kept informed about future opportunities.”

Innovation by Competition: How Challenges and Competition Get the Most Out of the Crowd

Curated on Posted on by Stefaan Verhulst

Innocentive: “Crowdsourcing has become the 21st century’s alternative to problem solving in place of traditional employee-based strategies. It has become the modern solution to provide for needed services, content, and ideas. Crowdsourced ideas are paving the way for today’s organizations to tackle innovation challenges that confront them in today’s competitive global marketplace. To put it all in perspective, crowds used to be thought of as angry mobs. Today, crowds are more like friendly and helpful contributors. What an interesting juxtaposition, eh?
Case studies proving the effectiveness of crowdsourcing to conquer innovation challenge, particularly in the fields of science and engineering abound. Despite this fact that success stories involving crowdsourcing are plentiful, very few firms are really putting its full potential to use. Advances in ALS and AIDS research have both made huge advances thanks to crowdsourcing, just to name a couple.
Biologists at the University of Washington were able to map the structure of an AIDS related virus thanks to the collaboration involved with crowdsourcing. How did they do this?  With the help of gamers playing a game designed to help get the information the University of Washington needed. It was a solution that remained unattainable for over a decade until enough top notch scientific minds were expertly probed from around the world with effective crowdsourcing techniques.
Dr. Seward Rutkove discovered an ALS biomarker to accurately measure the progression of the disease in patients through the crowdsourcing tactics utilized in a prize contest by an organization named Prize4Life, who utilized our Challenge Driven Innovation approach to engage the crowd.
The truth is, the concept of crowdsourcing to innovate has been around for centuries. But, with the growing connectedness of the world due to sheer Internet access, the power and ability to effectively crowdsource has increased exponentially. It’s time for corporations to realize this, and stop relying on stale sources of innovation. ..”

Prospects for Online Crowdsourcing of Social Science Research Tasks: A Case Study Using Amazon Mechanical Turk

Curated on Posted on by Stefaan Verhulst

New paper by Catherine E. Schmitt-Sands and Richard J. Smith: “While the internet has created new opportunities for research, managing the increased complexity of relationships and knowledge also creates challenges. Amazon.com has a Mechanical Turk service that allows people to crowdsource simple tasks for a nominal fee. The online workers may be anywhere in North America or India and range in ability. Social science researchers are only beginning to use this service. While researchers have used crowdsourcing to find research subjects or classify texts, we used Mechanical Turk to conduct a policy scan of local government websites. This article describes the process used to train and ensure quality of the policy scan. It also examines choices in the context of research ethics.”

Crowdsourcing forecasts on science and technology events and innovations

Curated on Posted on by Stefaan Verhulst

Kurzweil News: “George Mason University launched today, Jan. 10, the largest and most advanced science and technology prediction market in the world: SciCast.
The federally funded research project aims to improve the accuracy of science and technology forecasts. George Mason research assistant professor Charles Twardy is the principal investigator of the project.
SciCast crowdsources forecasts on science and technology events and innovations from aerospace to zoology.
For example, will Amazon use drones for commercial package delivery by the end of 2017? Today, SciCast estimates the chance at slightly more than 50 percent. If you think that is too low, you can estimate a higher chance. SciCast will use your estimate to adjust the combined forecast.
Forecasters can update their forecasts at any time; in the above example, perhaps after the Federal Aviation Administration (FAA) releases its new guidelines for drones. The continually updated and reshaped information helps both the public and private sectors better monitor developments in a variety of industries. SciCast is a real-time indicator of what participants think is going to happen in the future.
“Combinatorial” prediction market better than simple average


How SciCast works (Credit: George Mason University)
The idea is that collective wisdom from diverse, informed opinions can provide more accurate predictions than individual forecasters, a notion borne out by other crowdsourcing projects. Simply taking an average is almost always better than going with the “best” expert. But in a two-year test on geopolitical questions, the SciCast method did 40 percent better than the simple average.
SciCast uses the first general “combinatorial” prediction market. In a prediction market, forecasters spend points to adjust the group forecast. Significant changes “cost” more — but “pay” more if they turn out to be right. So better forecasters gain more points and therefore more influence, improving the accuracy of the system.
In a combinatorial market like SciCast, forecasts can influence each other. For example, forecasters might have linked cherry production to honeybee populations. Then, if forecasters increase the estimated percentage of honeybee colonies lost this winter, SciCast automatically reduces the estimated 2014 cherry production. This connectivity among questions makes SciCast more sophisticated than other prediction markets.
SciCast topics include agriculture, biology and medicine, chemistry, computational sciences, energy, engineered technologies, global change, information systems, mathematics, physics, science and technology business, social sciences, space sciences and transportation….

Crowdsourcing forecasts on science and technology events and innovations

George Mason University’s just-launched SciCast is largest and most advanced science and technology prediction market in the world
January 10, 2014


Example of SciCast crowdsourced forecast (credit: George Mason University)
George Mason University launched today, Jan. 10, the largest and most advanced science and technology prediction market in the world: SciCast.
The federally funded research project aims to improve the accuracy of science and technology forecasts. George Mason research assistant professor Charles Twardy is the principal investigator of the project.
SciCast crowdsources forecasts on science and technology events and innovations from aerospace to zoology.
For example, will Amazon use drones for commercial package delivery by the end of 2017? Today, SciCast estimates the chance at slightly more than 50 percent. If you think that is too low, you can estimate a higher chance. SciCast will use your estimate to adjust the combined forecast.
Forecasters can update their forecasts at any time; in the above example, perhaps after the Federal Aviation Administration (FAA) releases its new guidelines for drones. The continually updated and reshaped information helps both the public and private sectors better monitor developments in a variety of industries. SciCast is a real-time indicator of what participants think is going to happen in the future.
“Combinatorial” prediction market better than simple average


How SciCast works (Credit: George Mason University)
The idea is that collective wisdom from diverse, informed opinions can provide more accurate predictions than individual forecasters, a notion borne out by other crowdsourcing projects. Simply taking an average is almost always better than going with the “best” expert. But in a two-year test on geopolitical questions, the SciCast method did 40 percent better than the simple average.
SciCast uses the first general “combinatorial” prediction market. In a prediction market, forecasters spend points to adjust the group forecast. Significant changes “cost” more — but “pay” more if they turn out to be right. So better forecasters gain more points and therefore more influence, improving the accuracy of the system.
In a combinatorial market like SciCast, forecasts can influence each other. For example, forecasters might have linked cherry production to honeybee populations. Then, if forecasters increase the estimated percentage of honeybee colonies lost this winter, SciCast automatically reduces the estimated 2014 cherry production. This connectivity among questions makes SciCast more sophisticated than other prediction markets.
SciCast topics include agriculture, biology and medicine, chemistry, computational sciences, energy, engineered technologies, global change, information systems, mathematics, physics, science and technology business, social sciences, space sciences and transportation.
Seeking futurists to improve forecasts, pose questions


(Credit: George Mason University)
“With so many science and technology questions, there are many niches,” says Twardy, a researcher in the Center of Excellence in Command, Control, Communications, Computing and Intelligence (C4I), based in Mason’s Volgenau School of Engineering.
“We seek scientists, statisticians, engineers, entrepreneurs, policymakers, technical traders, and futurists of all stripes to improve our forecasts, link questions together and pose new questions.”
Forecasters discuss the questions, and that discussion can lead to new, related questions. For example, someone asked,Will Amazon deliver its first package using an unmanned aerial vehicle by Dec. 31, 2017?
An early forecaster suggested that this technology is likely to first be used in a mid-sized town with fewer obstructions or local regulatory issues. Another replied that Amazon is more likely to use robots to deliver packages within a short radius of a conventional delivery vehicle. A third offered information about an FAA report related to the subject.
Any forecaster could then write a question about upcoming FAA rulings, and link that question to the Amazon drones question. Forecasters could then adjust the strength of the link.
“George Mason University has succeeded in launching the world’s largest forecasting tournament for science and technology,” says Jason Matheny, program manager of Forecasting Science and Technology at the Intelligence Advanced Research Projects Activity, based in Washington, D.C. “SciCast can help the public and private sectors to better understand a range of scientific and technological trends.”
Collaborative but Competitive
More than 1,000 experts and enthusiasts from science and tech-related associations, universities and interest groups preregistered to participate in SciCast. The group is collaborative in spirit but also competitive. Participants are rewarded for accurate predictions by moving up on the site leaderboard, receiving more points to spend influencing subsequent prognostications. Participants can (and should) continually update their predictions as new information is presented.
SciCast has partnered with the American Association for the Advancement of Science, the Institute of Electrical and Electronics Engineers, and multiple other science and technology professional societies.
Mason members of the SciCast project team include Twardy; Kathryn Laskey, associate director for the C4I and a professor in the Department of Systems Engineering and Operations Research; associate professor of economics Robin Hanson; C4I research professor Tod Levitt; and C4I research assistant professors Anamaria Berea, Kenneth Olson and Wei Sun.
To register for SciCast, visit www.SciCast.org, or for more information, e-mail [email protected]. SciCast is open to anyone age 18 or older.”

Walgreens Taps Crowdsourcing to Deliver Cold Medicine to Shut-Ins

Curated on Posted on by Stefaan Verhulst

Mashable: “Walgreens is reaching out to consumers who are so walloped with a cold or flu that a trip to the corner drugstore seems an insurmountable obstacle.
The national drug chain is partnering with TaskRabbit, the online mobile marketplace, to allow deliveries of over-the-counter cold medicine in any of the 19 cities in which TaskRabbit is available. Such deliveries can be made via TaskRabbit’s iOS app or on its website. Standard TaskRabbit rates apply including a 20% service charge and a runner’s fee. So if a runner’s fee is $10, you would pay an additional $12 plus the cost of your cold medicine, to get the delivery.
The partnership, arranged by OMD’s Ignition Factory, runs this week through Feb. 18, typically the weeks in which cold and flu complaints have the sharpest increases. During that time, the Walgreens option will appear in TaskRabbit’s iOS app’s Task Wheel and on the website. Though TaskRabbit has partnered with other national brands, including Pepsi, this is its first with a retailer.
However, the deal is more of a pr exercise than anything else: Consumers have had the ability arrange a TaskRabbit to shop and buy cold medicine at Walgreens prior to the agreement. The chain is hoping to raise awareness about this option, though.
“We just wanted to make it as easy as possible,” says Wilson Standish, project manager at Ignition Factory. “When you’re sick, you don’t even want to get out of bed.”

Crowdsourcing Social Problems

Curated on Posted on by Stefaan Verhulst

Article by   in Reason: “reCAPTCHA and Duolingo both represent a distinctly 21st-century form of distributed problem solving. These Internet-enabled approaches tend to be faster, far less expensive, and far more resilient than the heavyweight industrial-age methods of solving big social problems that we’ve grown accustomed to over the past century. They typically involve highly diverse resources-volunteer time, crowdfunding, the capabilities of multinational corporations, entrepreneurial capital, philanthropic funding-aligned around common objectives such as reducing congestion, providing safe drinking water, or promoting healthy living. Crowdsourcing offers not just a better way of doing things, but a radical challenge to the bureaucratic status quo.
Here are several ways public, private, and nonprofit organizations can use lightweight, distributed approaches to solve societal problems faster and cheaper than the existing sclerotic models.
Chunk the Problem
The genius of reCAPTCHA and Duolingo is that they divide labor into small increments, performed for free, often by people who are unaware of the project they’re helping to complete. This strategy has wide public-policy applications, even in dealing with potholes….
Meanwhile, Finland’s DigitalKoot project enlisted volunteers to digitize their own libraries by playing a computer game that challenged them to transcribe scans of antique manuscripts.
Governments can set up a microtasking platform, not just for citizen engagement but as a way to harness the knowledge and skills of public employees across multiple departments and agencies. If microtasking can work to connect people outside the “four walls” of an organization, think of its potential as a platform to connect people and conduct work inside an organization-even an organization as bureaucratic as government.
…
Decentralize Service to the Self
A young woman slices her finger on a knife. As she compresses the bleeding with gauze, she needs to know if her wound warrants stitches. So she calls up Blue Cross’ 24-hour nurse hotline, where patients call to learn if they should see a doctor. The nurse asks her to describe the depth of the cut. He explains she should compress it with gauze and skip the ER. In aggregate, savings like this amount to millions of dollars of avoided emergency room visits.
Since 2003, Blue Cross has been shifting the work of basic triage and risk mitigation to customers. Britain’s National Health Service (NHS) implemented a similar program, NHS Direct, in 1998. NHS estimates that the innovation has saved it £44 million a year….
Gamify Drudgery
Finland’s national library houses an enormous archive of antique texts, which officials hoped to scan and digitize into ordinary, searchable text documents. Rather than simply hire people for the tedium of correcting garbled OCR scans, the library invited the public to play a game. An online program called DigitalKoot lets people transcribe scanned words, and by typing accurately, usher a series of cartoon moles safely across a bridge….
Build a Two-Sided Market
Road infrastructure costs government five cents per driver per mile, according to the Victoria Transport Policy Institute. “That’s a dollar the government paid for the paving of that road and the maintaining of that infrastructure…just for you, not the other 3,000 people that travelled that same segment of highway in that same hour that you did,” says Sean O’Sullivan, founder of Carma, a ridesharing application.
Ridesharing companies such as Carma, Lyft, and Zimride are attempting to recruit private cars for the public transit network, by letting riders pay a small fee to carpool. A passenger waits at a designated stop, and the app alerts drivers, who can scan a profile of their potential rider. It’s a prime example of a potent new business model…
Remove the Middleman
John McNair dropped out of high school at age 16. By his thirties, he became an entrepreneur, producing and selling handmade guitars, but carpentry alone wouldn’t grow his business. So the founder of Red Dog Guitars enrolled in a $20 class on Skillshare.com, taught by the illustrator John Contino, to learn to brand his work with hand lettered product labels. Soon, a fellow businessman was asking McNair for labels to market guitar pickups.
Traditionally, the U.S. government might invest in retraining someone like John. Instead, peer-to-peer technology has allowed a community of designers to help John develop his skills. Peer-to-peer strategies enable citizens to meet each other’s needs, cheaply. Peer-to-peer solutions can help fix problems, deliver services, and supplement traditional approaches.
Peer-to-peer can lessen our dependence on big finance. Kickstarter lets companies skip the energy of convincing a banker that their product is viable. They just need to convince customers…”

Crowdsourcing drug discovery: Antitumour compound identified

Curated on Posted on by Stefaan Verhulst

David Bradley in Spectroscopy.now: “American researchers have used “crowdsourcing” – the cooperation of a large number of interested non-scientists via the internet – to help them identify a new fungus. The species contains unusual metabolites, isolated and characterized, with the help of vibrational circular dichroism (VCD). One compound reveals itself to have potential antitumour activity.
So far, a mere 7 percent of the more than 1.5 million species of fungi thought to exist have been identified and an even smaller fraction of these have been the subject of research seeking bioactive natural products. …Robert Cichewicz of the University of Oklahoma, USA, and his colleagues hoped to remedy this situation by working with a collection of several thousand fungal isolates from three regions: Arctic Alaska, tropical Hawaii, and subtropical to semiarid Oklahoma. Collaborator Susan Mooberry of the University of Texas at San Antonio carried out biological assays on many fungal isolates looking for antitumor activity among the metabolites in Cichewicz’s collection. A number of interesting substances were identified…
However, the researchers realized quickly enough that the efforts of a single research team were inadequate if samples representing the immense diversity of the thousands of fungi they hoped to test were to be obtained and tested. They thus turned to the help of citizen scientists in a “crowdsourcing” initiative. In this approach, lay people with an interest in science, and even fellow scientists in other fields, were recruited to collect and submit soil from their gardens.
As the samples began to arrive, the team quickly found among them a previously unknown fungal strain – a Tolypocladium species – growing in a soil sample from Alaska. Colleague Andrew Miller of the University of Illinois did the identification of this new fungus, which was found to be highly responsive to making new compounds based on changes in its laboratory growth conditions. Moreover, extraction of the active chemicals from the isolate revealed a unique metabolite which was shown to have significant antitumour activity in laboratory tests. The team suggests that this novel substance may represent a valuable new approach to cancer treatment because it precludes certain biochemical mechanisms that lead to the emergence of drug resistance in cancer with conventional drugs…
The researchers point out the essential roles that citizen scientists can play. “Many of the groundbreaking discoveries, theories, and applied research during the last two centuries were made by scientists operating from their own homes,” Cichewicz says. “Although much has changed, the idea that citizen scientists can still participate in research is a powerful means for reinvigorating the public’s interest in science and making important discoveries,” he adds.”

The Effective Use of Crowdsourcing in E-Governance

Curated on Posted on by Stefaan Verhulst
Paper by Jayakumar Sowmya and Hussain Shafiq Pyarali: “The rise of Web 2.0 paradigm has empowered the Internet users to share information and generate content on social networking and media sharing platforms such as wikis and blogs. The trend of harnessing the wisdom of public using Web 2.0 distributed networks through open calls is termed as ‘Crowdsourcing’. In addition to businesses, this powerful idea of using collective intelligence or the ‘wisdom of crowd’ applies to different situations, such as in governments and non-profit organizations which have started utilizing crowdsourcing as an essential problem -solving tool. In addition, the widespread and easy access to technologies such as the Internet, mobile phones and other communication devices has resulted in an exponential growth in the use of crowdsourcing for government policy advocacy, e-democracy and e-governance during the past decade. However, utilizing collective intelligence and efforts of public to find solutions to real life problems using web 2.0 tools does come with its share of associated challenges and limitations. This paper aims at identifying and examining the value-adding strategies which contribute to the success of crowdsourcing in e-governance. The qualitative case study analysis and emphatic design methodology are employed to evaluate the effectiveness of the identified strategic and functional components, by analyzing the characteristics of some of the notable cases of crowdsourcing in e-governance and the findings are tabulated and discussed. The paper concludes with the limitations and the implications for future research”.