Why governments need guinea pigs for policies

Curated on Posted on by Stefaan Verhulst

Jonathan Breckon in the Guardian:”People are unlikely to react positively to the idea of using citizens as guinea pigs; many will be downright disgusted. But there are times when government must experiment on us in the search for knowledge and better policy….

Though history calls into question the ethics of experimentation, unless we try things out, we will never learn. The National Audit Office says that £66bn worth of government projects have no plans to evaluate their impact. It is unethical to roll out policies in this arbitrary way. We have to experiment on a small scale to have a better understanding of how things work before rolling out policies across the UK. This is just as relevant to social policy, as it is to science and medicine, as set out in a new report by the Alliance for Useful Evidence.

Whether it’s the best ways to teach our kids to read, designing programmes to get unemployed people back to work, or encouraging organ donation – if the old ways don’t work, we have to test new ones. And that testing can’t always be done by a committee in Whitehall or in a university lab.

Experimentation can’t happen in isolation. What works in Lewisham or Londonnery, might not work in Lincoln – or indeed across the UK. For instance, there is a huge amount debate around the current practice of teaching children to read and spell using phonics, which was based on a small-scale study in Clackmannanshire, as well as evidence from the US. A government-commissioned review on the evidence for phonics led professor Carole Torgerson, then at York University, to warn against making national policy off the back of just one small Scottish trial.

One way round this problem is to do larger experiments. The increasing use of the internet in public services allows for more and faster experimentation, on a larger scale for lower cost – the randomised controlled trial on voter mobilisation that went to 61 million users in the 2010 US midterm elections, for example. However, the use of the internet doesn’t get us off the ethical hook. Facebook had to apologise after a global backlash to secret psychological tests on their 689,000 users.

Contentious experiments should be approved by ethics committees – normal practice for trials in hospitals and universities.

We are also not interested in freewheeling trial-and-error; robust and appropriate research techniques to learn from experiments are vital. It’s best to see experimentation as a continuum, ranging from the messiness of attempts to try something new to experiments using the best available social science, such as randomised controlled trials.

Experimental government means avoiding an approach where everything is fixed from the outset. What we need is “a spirit of experimentation, unburdened by promises of success”, as recommended by the late professor Roger Jowell, author of the 2003 Cabinet Office report, Trying it out [pdf]….(More)”

Big Data for Social Good

Curated on Posted on by Stefaan Verhulst

Introduction to a Special Issue of the Journal “Big Data” by Catlett Charlie and Ghani Rayid: “…organizations focused on social good are realizing the potential as well but face several challenges as they seek to become more data-driven. The biggest challenge they face is a paucity of examples and case studies on how data can be used for social good. This special issue of Big Data is targeted at tackling that challenge and focuses on highlighting some exciting and impactful examples of work that uses data for social good. The special issue is just one example of the recent surge in such efforts by the data science community. …

This special issue solicited case studies and problem statements that would either highlight (1) the use of data to solve a social problem or (2) social challenges that need data-driven solutions. From roughly 20 submissions, we selected 5 articles that exemplify this type of work. These cover five broad application areas: international development, healthcare, democracy and government, human rights, and crime prevention.

“Understanding Democracy and Development Traps Using a Data-Driven Approach” (Ranganathan et al.) details a data-driven model between democracy, cultural values, and socioeconomic indicators to identify a model of two types of “traps” that hinder the development of democracy. They use historical data to detect causal factors and make predictions about the time expected for a given country to overcome these traps.

“Targeting Villages for Rural Development Using Satellite Image Analysis” (Varshney et al.) discusses two case studies that use data and machine learning techniques for international economic development—solar-powered microgrids in rural India and targeting financial aid to villages in sub-Saharan Africa. In the process, the authors stress the importance of understanding the characteristics and provenance of the data and the criticality of incorporating local “on the ground” expertise.

In “Human Rights Event Detection from Heterogeneous Social Media Graphs,” Chen and Neil describe efficient and scalable techniques to use social media in order to detect emerging patterns in human rights events. They test their approach on recent events in Mexico and show that they can accurately detect relevant human rights–related tweets prior to international news sources, and in some cases, prior to local news reports, which could potentially lead to more timely, targeted, and effective advocacy by relevant human rights groups.

“Finding Patterns with a Rotten Core: Data Mining for Crime Series with Core Sets” (Wang et al.) describes a case study with the Cambridge Police Department, using a subspace clustering method to analyze the department’s full housebreak database, which contains detailed information from thousands of crimes from over a decade. They find that the method allows human crime analysts to handle vast amounts of data and provides new insights into true patterns of crime committed in Cambridge…..(More)

Our New Three Rs: Rigor, Relevance, and Readability

Curated on Posted on by Stefaan Verhulst

Article by Stephen J. Del Rosso in Governance: “…Because of the dizzying complexity of the contemporary world, the quest for a direct relationship between academic scholarship and its policy utility is both quixotic and unnecessary. The 2013 U.S. Senate’s vote to prohibit funding for political science projects through the National Science Foundation, except for those certified “as promoting national security or the economic interests of the United States,” revealed a fundamental misreading of the nonlinear path between idea and policy. Rather than providing a clear blueprint for addressing emergent or long-standing challenges, a more feasible role for academic scholarship is what political scientist Roland Paris describes as helping to “order the world in which officials operate.” Scholarly works can “influence practitioners’ understandings of what is possible or desirable in a particular policy field or set of circumstances,” he believes, by “creating operational frameworks for … identifying options and implementing policies.”

It is sometimes claimed that think tanks should play the main role in conveying scholarly insights to policymakers. But, however they may have mastered the sound bite, the putative role of think tanks as effective transmission belts for policy-relevant ideas is limited by their lack of academic rigor and systematic peer review. There is also a tendency, particularly among some “Inside the Beltway” experts, to trim their sails to the prevailing political winds and engage in self-censorship to keep employment options open in current or future presidential administrations. Scholarship’s comparative advantage in the marketplace of ideas is also evident in terms of its anticipatory function—the ability to loosen the intellectual bolts for promising policies not quite ready for implementation. A classic example is Swedish Nobel laureate Gunner Myrdal’s 1944 study of race relations, The American Dilemma, which was largely ignored and even disavowed by its sponsors for over a decade until it proved essential to the landmark Supreme Court decision in Brown v. Board of Education. Moreover, it should also be noted, rather than providing a detailed game plan for addressing the problem of race in the country, Myrdal’s work was a quintessential example of the power of scholarship to frame critically important issues.

To bridge the scholarship–policy gap, academics must balance rigor and relevance with a third “R”—readability. There is no shortage of important scholarly work that goes unnoticed or unread because of its presentation. Scholars interested in having influence beyond the ivory tower need to combine their pursuit of disciplinary requirements with efforts to make their work more intelligible and accessible to a broader audience. For example, new forms of dissemination, such as blogs and other social media innovations, provide policy-relevant scholars with ample opportunities to supplement more traditional academic outlets. The recent pushback from the editors of the International Studies Association’s journals to the announced prohibition on their blogging is one indication that the cracks in the old system are already appearing.

At the risk of oversimplification, there are three basic tribes populating the political science field. One tribe comprises those who “get it” when it comes to the importance of policy relevance, a second eschews such engagement with the real world in favor of knowledge for knowledge’s sake, and a third is made up of anxious untenured assistant professors who seek to follow the path that will best provide them with secure employment. If war, as was famously said, is too important to be left to the generals, then the future of the political science field is too important to be left to the intellectual ostriches who bury their heads in self-referential esoterica. However, the first tribe needs to be supported, and the third tribe needs to be shown that there is professional value in engaging with the world, both to enlighten and, perhaps more importantly, to provoke—a sentiment the policy-relevant scholar and inveterate provocateur, Huntington, would surely have endorsed…(More)”

Data democracy – increased supply of geospatial information and expanded participatory processes in the production of data

Curated on Posted on by Stefaan Verhulst

Paper by Max Craglia & Lea Shanley: “The global landscape in the supply, co-creation and use of geospatial data is changing very rapidly with new satellites, sensors and mobile devices reconfiguring the traditional lines of demand and supply and the number of actors involved. In this paper we chart some of these technology-led developments and then focus on the opportunities they have created for the increased participation of the public in generating and contributing information for a wide range of uses, scientific and non. Not all this information is open or geospatial, but sufficiently large portions of it are to make it one of the most significant phenomena of the last decade. In fact, we argue that while satellite and sensors have exponentially increased the volumes of geospatial information available, the participation of the public is transformative because it expands the range of participants and stakeholders in society using and producing geospatial information, with opportunities for more direct participation in science, politics and social action…(View full text)”

Data scientists rejoice! There’s an online marketplace selling algorithms from academics

Curated on Posted on by Stefaan Verhulst

SiliconRepublic: “Algorithmia, an online marketplace that connects computer science researchers’ algorithms with developers who may have uses for them, has exited its private beta.

Algorithms are essential to our online experience. Google uses them to determine which search results are the most relevant. Facebook uses them to decide what should appear in your news feed. Netflix uses them to make movie recommendations.

Founded in 2013, Algorithmia could be described as an app store for algorithms, with over 800 of them available in its library. These algorithms provide the means of completing various tasks in the fields of machine learning, audio and visual processing, and computer vision.

Algorithmia found a way to monetise algorithms by creating a platform where academics can share their creations and charge a royalty fee per use, while developers and data scientists can request specific algorithms in return for a monetary reward. One such suggestion is for ‘punctuation prediction’, which would insert correct punctuation and capitalisation in speech-to-text translation.

While it’s not the first algorithm marketplace online, Algorithmia will accept and sell any type of algorithm and host them on its servers. What this means is that developers need only add a simple piece of code to their software in order to send a query to Algorithmia’s servers, so the algorithm itself doesn’t have to be integrated in its entirety….

Computer science researchers can spend years developing algorithms, only for them to be published in a scientific journal never to be read by software engineers.

Algorithmia intends to create a community space where academics and engineers can meet to discuss and refine these algorithms for practical use. A voting and commenting system on the site will allow users to engage and even share insights on how contributions can be improved.

To that end, Algorithmia’s ultimate goal is to advance the development of algorithms as well as their discovery and use….(More)”

Turning smartphones into personal, real-time pollution-location monitors

Curated on Posted on by Stefaan Verhulst

Kurzweil Newsletter: “Scientists reporting in the ACS journal Environmental Science & Technology have used smartphone and sensing technology to better pinpoint times and locations of the worst air pollution, which is associated with respiratory and cardiovascular problems.

Most such studies create a picture of exposure based on air pollution levels outside people’s homes. This approach ignores big differences in air quality in school and work environments. It also ignores spikes in pollution that happen over the course of the day such as during rush hour.

To fill in these gaps, Mark J. Nieuwenhuijsen and colleagues in Spain, The Netherlands, and the U.S. equipped 54 school children from from 29 different schools around Barcelona with smartphones that could track their location and physical activity. The children also received sensors that continuously measured the ambient levels of black carbon, a component of soot. Although most children spent less than 4 percent of their day traveling to and from school, this exposure contributed 13 percent of their total potential black carbon exposure.

The study was associated with BREATHE, an epidemiological study of the relation between air pollution and brain development.

The researchers conclude that mobile technologies could contribute valuable new insights into air pollution exposure….

More: Mark J. Nieuwenhuijsen, David Donaire-Gonzalez, Ioar Rivas, Montserrat de Castro, Marta Cirach, Gerard Hoek, Edmund Seto, Michael Jerrett, Jordi Sunyer. Variability in and Agreement between Modeled and Personal Continuously Measured Black Carbon Levels Using Novel Smartphone and Sensor Technologies. Environmental Science & Technology, 2015; 150209104136008 DOI: 10.1021/es505362x

Index: Prizes and Challenges

Curated on Posted on by Shruti Sannon

The Living Library Index – inspired by the Harper’s Index – provides important statistics and highlights global trends in governance innovation. This installment focuses on prizes and challenges and was originally published in 2015.

This index highlights recent findings about two key techniques in shifting innovation from institutions to the general public:

  • Prize-Induced Contests – using monetary rewards to incentivize individuals and other entities to develop solutions to public problems; and
  • Grand Challenges – posing large, audacious goals to the public to spur collaborative, non-governmental efforts to solve them.

You can read more about Governing through Prizes and Challenges here. You can also watch Alph Bingham, co-founder of Innocentive, answer the GovLab’s questions about challenge authoring and defining the problem here.

Previous installments of the Index include Measuring Impact with Evidence, The Data Universe, Participation and Civic Engagement and Trust in Institutions. Please share any additional statistics and research findings on the intersection of technology in governance with us by emailing shruti at thegovlab.org.

Prize-Induced Contests

  • Year the British Government introduced the Longitude Prize, one of the first instances of prizes by government to spur innovation: 1714
  • President Obama calls on “all agencies to increase their use of prizes to address some of our Nation’s most pressing challenges” in his Strategy for American Innovation: September 2009
  • The US Office of Management and Budget issues “a policy framework to guide agencies in using prizes to mobilize American ingenuity and advance their respective core missions”:  March 2010
  • Launch of Challenge.gov, “a one-stop shop where entrepreneurs and citizen solvers can find public-sector prize competitions”: September 2010
    • Number of competitions currently live on Challenge.gov in February 2015: 22 of 399 total
    • How many competitions on Challenge.gov are for $1 million or above: 23
  • The America COMPETES Reauthorization Act is introduced, which grants “all Federal agencies authority to conduct prize competitions to spur innovation, solve tough problems, and advance their core missions”: 2010
  • Value of prizes authorized by COMPETES: prizes up to $50 million
  • Fact Sheet and Frequently Asked Questions memorandum issued by the Office of Science and Technology Policy and the Office of Management and Budget to aid agencies to take advantage of authorities in COMPETES: August 2011
  • Number of prize competitions run by the Federal government from 2010 to April 2012: 150
  • How many Federal agencies have run prize competitions by 2012: 40
  • Prior to 1991, the percentage of prize money that recognized prior achievements according to an analysis by McKinsey and Company: 97%
    • Since 1991, percentage of new prize money that “has been dedicated to inducement-style prizes that focus on achieving a specific, future goal”: 78%
  • Value of the prize sector as estimated by McKinsey in 2009: $1-2 billion
  • Growth rate of the total value of new prizes: 18% annually
  • Growth rate in charitable giving in the US: 2.5% annually
  • Value of the first Horizon Prize awarded in 2014 by the European Commission to German biopharmaceutical company CureVac GmbH “for progress towards a novel technology to bring life-saving vaccines to people across the planet in safe and affordable ways”: €2 million
  • Number of solvers registered on InnoCentive, a crowdsourcing company: 355,000+ from nearly 200 countries
    • Total Challenges Posted: 2,000+ External Challenges
    • Total Solution Submissions: 40,000+
    • Value of the awards: $5,000 to $1+ million
    • Success Rate for premium challenges: 85%

Grand Challenges

  • Value of the Progressive Insurance Automotive X Prize, sponsored in part by DOE to develop production-capable super fuel-efficient vehicles: $10 million
    • Number of teams around the world who took part in the challenge “to develop a new generation of technologies” for production-capable super fuel-efficient vehicles: 111 teams
  • Time it took for the Air Force Research Laboratory to receive a workable solution on “a problem that had vexed military security forces and civilian police for years” by opening the challenge to the world: 60 days
  • Value of the HHS Investing in Innovation initiative to spur innovation in Health IT, launched under the new COMPETES act: $5 million program
  • Number of responses received by NASA for its Asteroid Grand Challenge RFI which seeks to identify and address all asteroid threats to the human population: over 400
  • The decreased cost of sequencing a single human genome as a result of the Human Genome Project Grand Challenge: $7000 from $100 million
  • Amount the Human Genome Project Grand Challenge has contributed to the US economy for every $1 invested by the US federal government: $141 for every $1 invested
  • The amount of funding for research available for the “Brain Initiative,” a collaboration between the National Institute of Health, DARPA and the National Science Foundation, which seeks to uncover new prevention and treatment methods for brain disorders like Alzheimer’s, autism and schizophrenia: $100 million
  • Total amount offered in cash awards by the Department of Energy’s “SunShot Grand Challenge,” which seeks to eliminate the cost disparity between solar energy and coal by the end of the decade: $10 million

Sources

Encyclopedia of Social Network Analysis and Mining

Curated on Posted on by Stefaan Verhulst

“The Encyclopedia of Social Network Analysis and Mining (ESNAM) is the first major reference work to integrate fundamental concepts and research directions in the areas of social networks and  applications to data mining. While ESNAM  reflects the state-of-the-art in  social network research, the field  had its start in the 1930s when fundamental issues in social network research were broadly defined. These communities were limited to relatively small numbers of nodes (actors) and links. More recently the advent of electronic communication, and in particular on-line communities, have created social networks of hitherto unimaginable sizes. People around the world are directly or indirectly connected by popular social networks established using web-based platforms rather than by physical proximity.

Reflecting the interdisciplinary nature of this unique field, the essential contributions of diverse disciplines, from computer science, mathematics, and statistics to sociology and behavioral science, are described among the 300 authoritative yet highly readable entries. Students will find a world of information and insight behind the familiar façade of the social networks in which they participate. Researchers and practitioners will benefit from a comprehensive perspective on the methodologies for analysis of constructed networks, and the data mining and machine learning techniques that have proved attractive for sophisticated knowledge discovery in complex applications. Also addressed is the application of social network methodologies to other domains, such as web networks and biological networks….(More)”

Evaluating Complex Social Initiatives

Curated on Posted on by Stefaan Verhulst

Srik Gopal at Stanford Social Innovation Review: “…the California Endowment (TCE) .. and ..The Grand Rapids Community Foundation (GRCF) …are just two funders who are actively “shifting the lens” in terms of how they are looking at evaluation in the light of complexity. They are building on the recognition that a traditional approach to evaluation—assessing specific effects of a defined program according to a set of pre-determined outcomes, often in a way that connects those outcomes back to the initiative—is increasingly falling short. There is a clear message emerging that evaluation needs to accommodate complexity, not “assume it away.”

My colleagues at FSG and I have, based on our work with TCE, GRCF, and numerous other clients, articulated a set of nine “propositions” in a recent practice brief that are helpful in guiding how we conceptualize, design, and implement evaluations of complex initiatives. We derived these propositions from what we now know (based on the emerging field of complexity science) as distinctive characteristics of complex systems. We know, for example, that complex systems are always changing, often in unpredictable ways; they are never static. Hence, we need to design evaluations so that they are adaptive, flexible, and iterative, not rigid and cookie-cutter.

Below are three of the propositions in more detail, along with tools and methods that can help apply the proposition in practice.

It is important to note that many of the traditional tools and methods that form the backbone of sound evaluations—such as interviews, focus groups, and surveys—are still relevant. We would, however, suggest that organizations adapt those methods to reflect a complexity orientation. For example, interviews should explore the role of context; we should not confine them to the initiative’s boundaries. Focus groups should seek to understand local adaptation, not just adherence. And surveys should probe for relationships and interdependencies, not just perceived outcomes. In addition to traditional methods, we suggest incorporating newer, innovative techniques that provide a richer narrative, including:

  • Systems mapping—an iterative, often participatory process of graphically representing a system, including its components and connections
  • Appreciative inquiry—a group process that inquires into, identifies, and further develops the best of “what is” in organizations
  • Design thinking—a user-centered approach to developing new solutions to abstract, ill-defined, or complex problems… (More)”

If Data Sharing is the Answer, What is the Question?

Curated on Posted on by Stefaan Verhulst

Christine L. Borgman at ERCIM News: “Data sharing has become policy enforced by governments, funding agencies, journals, and other stakeholders. Arguments in favor include leveraging investments in research, reducing the need to collect new data, addressing new research questions by reusing or combining extant data, and reproducing research, which would lead to greater accountability, transparency, and less fraud. Arguments against data sharing rarely are expressed in public fora, so popular is the idea. Much of the scholarship on data practices attempts to understand the socio-technical barriers to sharing, with goals to design infrastructures, policies, and cultural interventions that will overcome these barriers.
However, data sharing and reuse are common practice in only a few fields. Astronomy and genomics in the sciences, survey research in the social sciences, and archaeology in the humanities are the typical exemplars, which remain the exceptions rather than the rule. The lack of success of data sharing policies, despite accelerating enforcement over the last decade, indicates the need not just for a much deeper understanding of the roles of data in contemporary science but also for developing new models of scientific practice. Science progressed for centuries without data sharing policies. Why is data sharing deemed so important to scientific progress now? How might scientific practice be different if these policies were in place several generations ago?
Enthusiasm for “big data” and for data sharing are obscuring the complexity of data in scholarship and the challenges for stewardship. Data practices are local, varying from field to field, individual to individual, and country to country. Studying data is a means to observe how rapidly the landscape of scholarly work in the sciences, social sciences, and the humanities is changing. Inside the black box of data is a plethora of research, technology, and policy issues. Data are best understood as representations of observations, objects, or other entities used as evidence of phenomena for the purposes of research or scholarship. Rarely do they stand alone, separable from software, protocols, lab and field conditions, and other context. The lack of agreement on what constitutes data underlies the difficulties in sharing, releasing, or reusing research data.
Concerns for data sharing and open access raise broader questions about what data to keep, what to share, when, how, and with whom. Open data is sometimes viewed simply as releasing data without payment of fees. In research contexts, open data may pose complex issues of licensing, ownership, responsibility, standards, interoperability, and legal harmonization. To scholars, data can be assets, liabilities, or both. Data have utilitarian value as evidence, but they also serve social and symbolic purposes for control, barter, credit, and prestige. Incentives for scientific advancement often run counter to those for sharing data.
….
Rather than assume that data sharing is almost always a “good thing” and that doing so will promote the progress of science, more critical questions should be asked: What are the data? What is the utility of sharing or releasing data, and to whom? Who invests the resources in releasing those data and in making them useful to others? When, how, why, and how often are those data reused? Who benefits from what kinds of data transfer, when, and how? What resources must potential re-users invest in discovering, interpreting, processing, and analyzing data to make them reusable? Which data are most important to release, when, by what criteria, to whom, and why? What investments must be made in knowledge infrastructures, including people, institutions, technologies, and repositories, to sustain access to data that are released? Who will make those investments, and for whose benefit?
Only when these questions are addressed by scientists, scholars, data professionals, librarians, archivists, funding agencies, repositories, publishers, policy makers, and other stakeholders in research will satisfactory answers arise to the problems of data sharing…(More)”.