Prospects for Online Crowdsourcing of Social Science Research Tasks: A Case Study Using Amazon Mechanical Turk


New paper by Catherine E. Schmitt-Sands and Richard J. Smith: “While the internet has created new opportunities for research, managing the increased complexity of relationships and knowledge also creates challenges. Amazon.com has a Mechanical Turk service that allows people to crowdsource simple tasks for a nominal fee. The online workers may be anywhere in North America or India and range in ability. Social science researchers are only beginning to use this service. While researchers have used crowdsourcing to find research subjects or classify texts, we used Mechanical Turk to conduct a policy scan of local government websites. This article describes the process used to train and ensure quality of the policy scan. It also examines choices in the context of research ethics.”

Supporting open government in New Europe


Google Europe Blog: “The “New Europe” countries that joined the European Union over the past decade are moving ahead fast to use the Internet to improve transparency and open government. We recently partnered with Techsoup Global to support online projects driving forward good governance in Romania, the Czech Republic, and most recently, in Slovakia.
Techsoup Global, in partnership with the Slovak Center for Philanthropy, recently held an exciting social-startups awards ceremony Restart Slovakia 2013 in Bratislava. Slovakia’s Deputy Minister of Finance and Digital Champion Peter Pellegrini delivered keynote promoting Internet and Open Data and announced the winners of this year contest. Ambassadors from U.S., Israel and Romania and several distinguished Slovak NGOs also attended the ceremony.
Winning projects included:

  • Vzdy a vsade – Always and Everywhere – a volunteer portal offering online and anonymous psychological advice to internet users via chat.
  • Nemlcme.sk – a portal providing counsel for victims of sexual assaults.
  • Co robim – an educational online library of job careers advising young people how to choose their career paths and dream jobs.
  • Mapa zlocinu – an online map displaying various rates of criminality in different neighbourhoods.
  • Demagog.sk – a platform focused on analyzing public statements of politicians and releasing information about politicians and truthfulness of their speeches in a user-friendly format.”

From Faith-Based to Evidence-Based: The Open Data 500 and Understanding How Open Data Helps the American Economy


Beth Noveck in Forbes: “Public funds have, after all, paid for their collection, and the law says that federal government data are not protected by copyright. By the end of 2009, the US and the UK had the only two open data one-stop websites where agencies could post and citizens could find open data. Now there are over 300 such portals for government data around the world with over 1 million available datasets. This kind of Open Data — including weather, safety and public health information as well as information about government spending — can serve the country by increasing government efficiency, shedding light on regulated industries, and driving innovation and job creation.

It’s becoming clear that open data has the potential to improve people’s lives. With huge advances in data science, we can take this data and turn it into tools that help people choose a safer hospital, pick a better place to live, improve the performance of their farm or business by having better climate models, and know more about the companies with whom they are doing business. Done right, people can even contribute data back, giving everyone a better understanding, for example of nuclear contamination in post-Fukushima Japan or incidences of price gouging in America’s inner cities.

The promise of open data is limitless. (see the GovLab index for stats on open data) But it’s important to back up our faith with real evidence of what works. Last September the GovLab began the Open Data 500 project, funded by the John S. and James L. Knight Foundation, to study the economic value of government Open Data extensively and rigorously.  A recent McKinsey study pegged the annual global value of Open Data (including free data from sources other than government), at $3 trillion a year or more. We’re digging in and talking to those companies that use Open Data as a key part of their business model. We want to understand whether and how open data is contributing to the creation of new jobs, the development of scientific and other innovations, and adding to the economy. We also want to know what government can do better to help industries that want high quality, reliable, up-to-date information that government can supply. Of those 1 million datasets, for example, 96% are not updated on a regular basis.

The GovLab just published an initial working list of 500 American companies that we believe to be using open government data extensively.  We’ve also posted in-depth profiles of 50 of them — a sample of the kind of information that will be available when the first annual Open Data 500 study is published in early 2014. We are also starting a similar study for the UK and Europe.

Even at this early stage, we are learning that Open Data is a valuable resource. As my colleague Joel Gurin, author of Open Data Now: the Secret to Hot Start-Ups, Smart Investing, Savvy Marketing and Fast Innovation, who directs the project, put it, “Open Data is a versatile and powerful economic driver in the U.S. for new and existing businesses around the country, in a variety of ways, and across many sectors. The diversity of these companies in the kinds of data they use, the way they use it, their locations, and their business models is one of the most striking things about our findings so far.” Companies are paradoxically building value-added businesses on top of public data that anyone can access for free….”

FULL article can be found here.

Protecting personal data in E-government: A cross-country study


Paper by Yuehua Wu in Government Information Quarterly: “This paper presents the findings of a comparative study of laws and policies employed to protect personal data processed in the context of e-government in three countries (the United States, Germany, and China) with rather different approaches. Drawing on governance theory, the paper seeks to document the mechanisms utilized and to understand the factors that shape the governance modes adopted. The cases reveal that national government regulations have not kept pace with technological change and with the current information practices of the public sector. Nonetheless, traditional government regulation remains the major governance mode for the issue under discussion. Self-regulation and code-based regulation serve supplementary roles to traditional government regulation. National context is found to impact the form and level of data protection and the choice of governance modes.”

People Powered Social Innovation: The Need for Citizen Engagement


Paper for the Lien Centre for Social Innovation (Singapore): “the Citizen engagement is widely regarded as critical to the development and implementation of social innovation. What is citizen engagement? What does it mean in the context of social innovation? Julie Simon and Anna Davies discuss the importance as well as the implications of engaging the ground…”

Open data and transparency: a look back at 2013


Zoe Smith in the Guardian on the open data and development in 2013: “The clarion call for a “data revolution” made in the post-2015 high level panel report is a sign of a growing commitment to see freely flowing data become a tool for social change.

Web-based technology continued to offer increasing numbers of people the ability to share standardised data and statistics to demand better governance and strengthen accountability. 2013 seemed to herald the moment that the open data/transparency movement entered the mainstream.
Yet for those who have long campaigned on the issue, the call was more than just a catchphrase, it was a unique opportunity. “If we do get a global drive towards open data in relation to development or anything else, that would be really transformative and it’s quite rare to see such bold statements at such an early stage of the process. I think it set the tone for a year in which transparency was front and centre of many people’s agendas,” says David Hall Matthews, of Publish What You Fund.
This year saw high level discussions translated into commitments at the policy level. David Cameron used the UK’s presidency of the G8 to trigger international action on the three Ts (tax, trade and transparency) through the IF campaign. The pledge at Lough Erne, in Scotland, reaffirmed the commitment to the Busan open data standard as well as the specific undertaking that all G8 members would implement International Aid Transparency Index (IATI) standards by the end of 2015.
2013 was a particularly good year for the US Millenium Challenge Corporation (MCC) which topped the aid transparency index. While at the very top MCC and UK’s DfID were examples of best practice, there was still much room for improvement. “There is a really long tail of agencies who are not really taking transparency at all, yet. This includes important donors, the whole of France and the whole of Japan who are not doing anything credible,” says Hall-Matthews.
Yet given the increasing number of emerging and ‘frontier‘ markets whose growth is driven in large part by wealth derived from natural resources, 2013 saw a growing sense of urgency for transparency to be applied to revenues from oil, gas and mineral resources that may far outstrip aid. In May, the new Extractive Industries Transparency Initiative standard (EITI) was adopted, which is said to be far broader and deeper than its previous incarnation.
Several countries have done much to ensure that transparency leads to accountability in their extractive industries. In Nigeria, for example, EITI reports are playing an important role in the debate about how resources should be managed in the country. “In countries such as Nigeria they’re taking their commitment to transparency and EITI seriously, and are going beyond disclosing information but also ensuring that those findings are acted upon and lead to accountability. For example, the tax collection agency has started to collect more of the revenues that were previously missing,” says Jonas Moberg, head of the EITI International Secretariat.
But just the extent to which transparency and open data can actually deliver on its revolutionary potential has also been called into question. Governments and donors agencies can release data but if the power structures within which this data is consumed and acted upon do not shift is there really any chance of significant social change?
The complexity of the challenge is illustrated by the case of Mexico which, in 2014, will succeed Indonesia as chair of the Open Government Partnership. At this year’s London summit, Mexico’s acting civil service minister, spoke of the great strides his country has made in opening up the public procurement process, which accounts for around 10% of GDP and is a key area in which transparency and accountability can help tackle corruption.
There is, however, a certain paradox. As SOAS professor, Leandro Vergara Camus, who has written extensively on peasant movements in Mexico, explains: “The NGO sector in Mexico has more of a positive view of these kinds of processes than the working class or peasant organisations. The process of transparency and accountability have gone further in urban areas then they have in rural areas.”…
With increasing numbers of organisations likely to jump on the transparency bandwagon in the coming year the greatest challenge is using it effectively and adequately addressing the underlying issues of power and politics.

Top 2013 transparency publications

Open data, transparency and international development, The North South Institute
Data for development: The new conflict resource?, Privacy International
The fix-rate: a key metric for transparency and accountability, Integrity Action
Making UK aid more open and transparent, DfID
Getting a seat at the table: Civil Society advocacy for budget transparency in “untransparent” countries, International Budget Partnership

The dates that mattered

23-24 May: New Extractive Industries Transparency Index standard adopted
30 May: Post 2015 high level report calling for a ‘data revolution’ is published
17-18 June: UK premier, David Cameron, campaigns for tax, trade and transparency during the G8
24 October: US Millenium Challenge Corporation tops the aid transparency index”
30 October – 1 November: Open Government Partnership in London gathers civil society, governments and data experts

Lessons in the crowdsourced verification of news from Storyful and Reddit’s Syria forum


at GigaOm: “One of the most powerful trends in media over the past year is the crowdsourced verification of news, whether it’s the work of a blogger like Brown Moses or former NPR journalist Andy Carvin. Two other interesting efforts in this area are the “open newsroom” approach taken by Storyful — which specializes in verifying social-media reports for mainstream news entities — and a Reddit forum devoted to crowdsourcing news coverage of the civil war in Syria.
Storyful journalist Joe Galvin recently looked at some of the incidents that the company has helped either debunk or verify over the past year — including a fake tweet from the official account of the Associated Press about explosions at the White House (which sent the Dow Jones index plummeting before it was corrected), a claim from Russian authorities that a chemical attack in Syria had been pre-meditated, and a report from investigative journalist Seymour Hersh about the same attack that questioned whether the government had been involved….
Reddit, meanwhile, has been conducting some “open newsroom”-style experiments of its own around a number of news events, including the Syrian civil war. The site has come under fire in the past for some of those efforts — including the attempt to identify the bombers in the Boston bombings case, which went badly awry — but the Syrian thread in particular is a good example of how a smart aggregator can make sense of an ongoing news event. In a recent post at a site called Dissected News, one of the moderators behind the /r/SyrianCivilWar sub-Reddit — a 22-year-old law student named Christopher Kingdon (or “uptodatepronto” as he is known on the site) — wrote about his experiences with the forum, which is trying to be a broadly objective source for breaking news and information about the conflict….
Some of what the moderators do in the forum is similar to the kind of verification that Storyful or the BBC’s “user-generated content desk” do — checking photos and video for obvious signs of fakery and hoaxes. But Kingdon also describes how much effort his team of volunteers puts into ensuring that the sub-Reddit doesn’t degenerate into trolling or flame-wars. Strict rules are enforced “to prevent personal attacks, offensive and violent language and racism” and the moderators favor posts that “utilize sources, background information and a dash of common sense.”

Why This Company Is Crowdsourcing, Gamifying The World's Most Difficult Problems


FastCompany: “The biggest consultancy firms–the McKinseys and Janeses of the world–make many millions of dollars predicting the future and writing what-if reports for clients. This model is built on the idea that those companies know best–and that information and ideas should be handed down from on high.
But one consulting house, Wikistrat, is upending the model: Instead of using a stable of in-house analysts, the company crowdsources content and pays the crowd for its time. Wikistrat’s hundreds of analysts–primarily consultants, academics, journalists, and retired military personnel–are compensated for participating in what they call “crowdsourced simulations.” In other words, make money for brainstorming.

According to Joel Zamel, Wikistrat’s founder, approximately 850 experts in various fields rotate in and out of different simulations and project exercises for the company. While participating in a crowdsourced simulation, consultants are are paid a flat fee plus performance bonuses based on a gamification engine where experts compete to win extra cash. The company declined revealing what the fee scale is, but as of 2011 bonus money appears to be in the $10,000 range.
Zamel characterizes the company’s clients as a mix of government agencies worldwide and multinational corporations. The simulations are semi-anonymous for players; consultants don’t know who their paper is being written for or who the end consumer is, but clients know which of Wikistrat’s contestants are participating in the brainstorm exercise. Once an exercise is over, the discussions from the exercise are taken by full-time employees at Wikistrat and converted into proper reports for clients.
“We’ve developed a quite significant crowd network and a lot of functionality into the platform,” Zamel tells Fast Company. “It uses a gamification engine we created that incentivizes analysts by ranking them at different levels for the work they do on the platform. They are immediately rewarded through the engine, and we also track granular changes made in real time. This allows us to track analyst activity and encourages them to put time and energy into Wiki analysis.” Zamel says projects typically run between three and four weeks, with between 50 and 100 analysts working on a project for generally between five and 12 hours per week. Most of the analysts, he says, view this as a side income on top of their regular work at day jobs but some do much more: Zamel cited one PhD candidate in Australia working 70 hours a week on one project instead of 10 to 15 hours.
Much of Wikistrat’s output is related to current events. Although Zamel says the bulk of their reports are written for clients and not available for public consumption, Wikistrat does run frequent public simulations as a way of attracting publicity and recruiting talent for the organization. Their most recent crowdsourced project is called Myanmar Moving Forward and runs from November 25 to December 9. According to Wikistrat, they are asking their “Strategic community to map out Myanmar’s current political risk factor and possible futures (positive, negative, or mixed) for the new democracy in 2015. The simulation is designed to explore the current social, political, economic, and geopolitical threats to stability–i.e. its political risk–and to determine where the country is heading in terms of its social, political, economic, and geopolitical future.”…

Phone Apps Help Government, Others Counter Violence Against Women


NextGov: “Smart and mobile phones have helped authorities solve crimes from beatings that occurred during the London riots to the Boston Marathon bombing. A panel of experts gathered on Monday said the devices can also help reduce and combat rapes and other gender-based violence.
Smartphone apps and text messaging services proliferated in India following a sharp rise in reported gang rapes, including the brutal 2012 rape and murder of a 23-year-old medical student in Delhi, according to panelists at the Wilson Center event on gender-based violence and innovative technologies.
The apps fall into four main categories, said Alex Dehgan, chief data scientist at the United States Agency for International Development: apps that aid sexual assault and domestic violence victims, apps that empower women to fight back against gender-based violence, apps focused on advocacy and apps that crowdsource and map cases of sexual assault.
The final category of apps is largely built on the Ushahidi platform, which was developed to track reports of missing people following the 2010 Haiti earthquake.
One of the apps, Safecity, offers real-time alerts about sexual assaults across India to help women identify unsafe areas.
Similar apps have been launched in Egypt and Syria, Dehgan said. In lower-tech countries the systems often operate using text messages rather than smartphone apps so they’re more widely accessible.
One of the greatest impediments to using mobile technology to reduce gender violence is third world nations in which women often don’t have access to their own mobile or smartphones and rural areas in the U.S. and abroad in which there is limited service or broadband, Christopher Burns, USAID’s team leader for mobile access, said.
Burns suggested international policymakers should align plans for expanding broadband and mobile service with crowdsourced reports of gender violence.
“One suggestion for policy makers to focus on is to take a look at the crowd maps we’ve talked about today and see where there are greater incidences of gender-based violence and violence against women,” he said. “In all likelihood, those pockets probably don’t have the connectivity, don’t have the infrastructure [and] don’t have the capacity in place for survivors to benefit from those tools.”
One tool that’s been used in the U.S. is Circle of 6, an app for women on college campuses to automatically draw on friends when they think they’re in danger. The app allows women to pick six friends they can automatically text if they think they’re in a dangerous situation, asking them to call with an excuse for them to leave.
The app is designed to look like a game so it isn’t clear women are using their phones to seek help, said Nancy Schwartzman, executive director of Tech 4 Good, which developed the app.
Schwartzman has heard reports of gay men on college campuses using the app as well, she said. The military has been in contact with Tech 4 Good about developing a version of the app to combat sexual assault on military bases, she said.”

The Age of Democracy


Xavier Marquez at Abandoned Footnotes: “This is the age of democracy, ideologically speaking. As I noted in an earlier post, almost every state in the world mentions the word “democracy” or “democratic” in its constitutional documents today. But the public acknowledgment of the idea of democracy is not something that began just a few years ago; in fact, it goes back much further, all the way back to the nineteenth century in a surprising number of cases.
Here is a figure I’ve been wanting to make for a while that makes this point nicely (based on data graciously made available by the Comparative Constitutions Project). The figure shows all countries that have ever had some kind of identifiable constitutional document (broadly defined) that mentions the word “democracy” or “democratic” (in any context – new constitution, amendment, interim constitution, bill of rights, etc.), arranged from earliest to latest mention. Each symbol represents a “constitutional event” – a new constitution adopted, an amendment passed, a constitution suspended, etc. – and colored symbols indicate that the text associated with the constitutional event in question mentions the word “democracy” or “democratic”…
The earliest mentions of the word “democracy” or “democratic” in a constitutional document occurred in Switzerland and France in 1848, as far as I can tell.[1] Participatory Switzerland and revolutionary France look like obvious candidates for being the first countries to embrace the “democratic” self-description; yet the next set of countries to embrace this self-description (until the outbreak of WWI) might seem more surprising: they are all Latin American or Caribbean (Haiti), followed by countries in Eastern Europe (various bits and pieces of the Austro-Hungarian empire), Southern Europe (Portugal, Spain), Russia, and Cuba. Indeed, most “core” countries in the global system did not mention democracy in their constitutions until much later, if at all, despite many of them having long constitutional histories; even French constitutions after the fall of the Second Republic in 1851 did not mention “democracy” until after WWII. In other words, the idea of democracy as a value to be publicly affirmed seems to have caught on first not in the metropolis but in the periphery. Democracy is the post-imperial and post-revolutionary public value par excellence, asserted after national liberation (as in most of the countries that became independent after WWII) or revolutions against hated monarchs (e.g., Egypt 1956, Iran 1979, both of them the first mentions of democracy in these countries but not their first constitutions).

Today only 16 countries have ever failed to mention their “democratic” character in their constitutional documents (Australia, Brunei, Denmark, Japan, Jordan, Malaysia, Monaco, Nauru, Oman, Samoa, Saudi Arabia, Singapore, Tonga, the United Kingdom, the USA, and Vatican City).[2] And no country that has ever mentioned “democracy” in an earlier constitutional document fails to mention it in its current constitutional documents (though some countries in the 19th and early 20th centuries went back and forth – mentioning democracy in one constitution, not mentioning it in the next). Indeed, after WWII the first mention of democracy in constitutions tended to be contemporaneous with the first post-independence constitution of the country; and with time, even countries with old and settled constitutional traditions seem to be more and more likely to mention “democracy” or “democratic” in some form as amendments or bills of rights accumulate (e.g., Belgium in 2013, New Zealand in 1990, Canada in 1982, Finland in 1995). The probability of a new constitution mentioning “democracy” appears to be asymptotically approaching 1. To use the language of biology, the democratic “meme” has nearly achieved “fixation” in the population, despite short-term fluctuations, and despite the fact that there appears to be no particular correlation between a state calling itself democratic and actually being democratic, either today or in the past.[3]
Though the actual measured level of democracy around the world has trended upwards (with some ups and downs) over the last two centuries, I don’t think this is the reason why the idea of democracy has achieved near-universal recognition in public documents. Countries do not first become democratic and then call themselves democracies; if anything, most mentions of democracy seem to be rather aspirational, if not entirely cynical. (Though many constitutions that mention democracy were also produced by people who seem to have been genuinely committed to some such ideal, even if the regimes that eventually developed under these constitutions were not particularly democratic). What we see, instead, is a broad process in which earlier normative claims about the basis of authority – monarchical, imperial, etc. – get almost completely replaced, regardless of the country’s cultural context, by democratic claims, regardless of the latter’s effectiveness as an actual basis for authority or the existence of working mechanisms for participation or vertical accountability. (These democratic claims to authority also sometimes coexist in uneasy tension with other claims to authority based on divine revelation, ideological knowledge, or tradition, invented or otherwise; consider the Chinese constitution‘s claims about the “people’s democratic dictatorship” led by the CCP).
I thus suspect the conquest of ideological space by “democratic” language did not happen just because democratic claims to authority (especially in the absence of actual democracy) have proved more persuasive than other claims to authority. Rather, I think the same processes that resulted in the emergence of modern national communities – e.g. the rituals associated with nationalism, which tended to “sacralize” a particular kind of imagined community – led to the symbolic production of the nation not only as the proper object of government but also as its proper active agent (the people, actively ruling itself), regardless of whether or not “the people” had any ability to rule or even to exercise minimal control over the rulers.[4] There thus seems to have been a kind of co-evolution of symbols of nationality and symbols of democracy, helped along by the practice/ritual of drafting constitutions and approving them through plebiscites or other forms of mass politics, a ritual that already makes democratic assumptions about “social contracts.” The question is whether the symbolic politics of democracy eventually has any sort of impact on actual institutions. But more on this later….”