A new approach to measuring the impact of open data

Curated on Posted on by Stefaan Verhulst

 at SunLight Foundation: “Strong evidence on the long-term impact of open data initiatives is incredibly scarce. The lack of compelling proof is partly due to the relative novelty of the open government field, but also to the inherent difficulties in measuring good governance and social change. We know that much of the impact of policy advocacy, for instance, occurs even before a new law or policy is introduced, and is thus incredibly difficult to evaluate. At the same time, it is also very hard to detect the causality between a direct change in the legal environment and the specific activities of a policy advocacy group. Attribution is equally challenging when it comes to assessing behavioral changes – who gets to take credit for increased political engagement and greater participation in democratic processes?

Open government projects tend to operate in an environment where the contribution of other stakeholders and initiatives is essential to achieving sustainable change, making it even more difficult to show the causality between a project’s activities and the impact it strives to achieve. Therefore, these initiatives cannot be described through simple “cause and effect” relationships, as they mostly achieve changes through their contribution to outcomes produced by a complex ecosystem of stakeholders — including journalists, think tanks, civil society organizations, public officials and many more — making it even more challenging to measure their direct impact.

We at the Sunlight Foundation wanted to tackle some of the methodological challenges of the field through building an evidence base that can empower further generalizations and advocacy efforts, as well as developing a methodological framework to unpack theories of change and to evaluate the impact of open data and digital transparency initiatives. A few weeks ago, we presented our research at the Cartagena Data Festival, and today we are happy to launch the first edition of our paper, which you can read below or on Scribd.

The outputs of this research include:

  • A searchable repository of more than 100 examples on the outputs, outcomes and impacts of open data and digital technology projects;
  • Three distinctive theories of change for open data and digital transparency initiatives from the Global South;
  • A methodological framework to help develop more robust indicators of social and political change for the ecosystem of open data initiatives, by applying and revising the Outcome Mapping approach of IDRC to the field…(You can read the study at :The Social Impact of Open Data by juliakeseru)

Nepal Aid Workers Helped by Drones, Crowdsourcing

Curated on Posted on by Stefaan Verhulst
Shirley Wang et al in the Wall Street Journal: “….It is too early to gauge the exact impact of the technology in Nepal relief efforts, which have just begun amid chaos on the ground. Aid organizations have reported hospitals are overstretched, a shortage of capacity at Katmandu’s airport is crippling aid distribution and damaged roads and the mountainous country’s difficult terrain make reaching villages difficult.

Still, technology is playing an increasing role in the global response to humanitarian crises. Within hours of Saturday’s 7.8-magnitude temblor, U.S. giants such as Google Inc. and Facebook Inc. were offering their networks for use in verifying survivors and helping worried friends and relatives locate their loved ones.

Advances in online mapping—long used to calculate distances and plot driving routes—and the ability of camera-equipped drones are playing an increasingly important role in coordinating emergency responses at ground zero of any disaster.

A community of nonprofit groups uses satellite images, private images and open-source mapping technology to remap areas affected by the earthquake. They mark damaged buildings and roads so rescuers can identify the worst-hit areas and assess how accessible different areas are. The technology complements more traditional intelligence from aircraft.

Such crowdsourced real-time mapping technologies were first used in the 2010 Haiti earthquake, according to Chris Grundy, a professor in Geographical Information Systems at the London School of Hygiene and Tropical Medicine. The technology “has been advancing a little bit every time [every situation where it is used] as we start to see what works,” said Prof. Grundy.

The American Red Cross supplied its relief team on the Wednesday night flight to Nepal from Washington, D.C. with 50 digital maps and an inch-thick pile of paper maps that help identify where the needs are. The charity has a mapping project with the British Red Cross, Doctors Without Borders and the Humanitarian OpenStreetMap Team, a crowdsourced data-sharing group.

Almost a week after the Nepal earthquake, two more people have been pulled from the rubble in Katmandu by teams of international rescuers. But hope for finding more survivors is waning. Photo: Sean McLain/The Wall Street Journal.

Mapping efforts have grown substantially since Haiti, according to Dale Kunce, head of the geographic information systems team at the American Red Cross. In the two months after the Haiti temblor, 600 mapping contributors made 1.5 million edits, while in the first 48 hours after the Nepal earthquake, 2,000 mappers had already made three million edits, Mr. Kunce said.

Some 3,400 volunteers from around the world are now inspecting images of Nepal online to identify road networks and conditions, to assess the extent of damage and pinpoint open spaces where displaced persons tend to congregate, according to Nama Budhathoki, executive director of a nonprofit technology company called Katmandu Living Labs.

His group is operating from a cramped but largely undamaged meeting room in a central-Katmandu office building to help coordinate the global effort of various mapping organizations with the needs of agencies like Doctors Without Borders and the international Red Cross community.

In recent days the Nepal Red Cross and Nepalese army have requested and been supplied with updated maps of severely damaged districts, said Dr. Budhathoki….(More)”

The road to better data

Curated on Posted on by Stefaan Verhulst

Johannes Jütting at OECDInsights: Tradition tells us that more than 3,000 years ago, Moses went to the top of Mount Sinai and came back down with 10 commandments. When the world’s presidents and prime ministers go to the top of the Sustainable Development Goals (SDGs) mountain in New York late this summer they will come down with not 10 commandments but 169. Too many?

Some people certainly think so. “Stupid development goals,” The Economist said recently. It argued that the 17 SDGs and roughly 169 targets should “honour Moses and be pruned to ten goals”. Others disagree. In a report for the Overseas Development Institute, May Miller-Dawkins, warned of the dangers of letting practicality “blunt ambition”. She backed SDGs with “high ambition”.

The debate over the “right” number of goals and targets is interesting, important even. But it misses a key point: No matter how many goals and targets are finally agreed, if we can’t measure their real impact on people’s lives, on our societies and on the environment, then they risk becoming irrelevant.

Unfortunately, we already know that many developing countries have problems compiling even basic social and economic statistics, never mind the complex web of data that will be needed to monitor the SDGs. A few examples: In 2013, about 35% of all live births were not officially registered worldwide, rising to two-thirds in developing countries. In Africa, just seven countries have data on their total number of landholders and women landholders, and none have data from before 2004. Last but not least, fast-changing economies and associated measurement challenges mean we are not sure today if we have worldwide a billion people living in extreme poverty, half a billion or more than a billion.

Why does this matter? Without adequate data, we cannot identify the problems that planning and policymaking need to address. We also cannot judge if governments and others are meeting their commitments. As a report from the Centre for Global Development notes, “Data […] serve as a ‘currency’ for accountability among and within governments, citizens, and civil society at large, and they can be used to hold development agencies accountable.”…(More)”

What makes some federal agencies better than others at innovation

Curated on Posted on by Stefaan Verhulst

Tom Fox at the Washington Post: “Given the complexity and difficulty of the challenges that government leaders face, encouraging innovation among their workers can pay dividends. Government-wide employee survey data, however, suggest that much more needs to be done to foster this type of culture at many federal organizations.

According to that data, nearly 90 percent of federal employees are looking for ways to be more innovative and effective, but only 54 percent feel encouraged by their leaders to come up with new ways of doing work. To make matters worse, fewer than a third say they believe creativity and innovation are rewarded in their agencies.

It’s worth pausing to examine what sets apart those agencies that do. They tend to have developed innovative cultures by providing forums for employees to share and test new ideas, by encouraging responsible risk-taking, and by occasionally bringing in outside talent for rotational assignments to infuse new thinking into the workplace.

The Department of Health and Human Services (HHS) is one example of an agency working at this. In 2010 it created the Idea Lab, with the goal to “remove barriers HHS employees face and promote better ways of working in government.”

It launched an awards program as part of Idea Lab called HHS Innovates to identify promising, new ideas likely to improve effectiveness. And to directly support implementing these ideas, the lab launched HHS Ignites, which provides teams with seed funding of $5,000 and a three-month timeframe to work on approved action plans. When the agency needs a shot of outside inspiration, it has its Entrepreneurs-in-Residence program, which enlists experts from the private and nonprofit sectors to join HHS for one or two years to develop new approaches and improve practices….

While the HHS Idea Lab program is a good concept, it’s the agency’s implementation that distinguishes it from other government efforts. Federal leaders elsewhere would be wise to borrow a few of their tactics.

As a starting point, federal leaders should issue a clear call for innovation that demands a measurable result. Too often, leaders ask for changes without any specificity as to the result they are looking to achieve. If you want your employees to be more innovative, you need to set a concrete, data-driven goal — whether that’s to reduce process steps or process times, improve customer satisfaction or reduce costs.

Secondly, you should help your employees take their ideas to implementation by playing equal parts cheerleader and drill sergeant. That is, you need to boost their confidence while at the same time pushing them to develop concrete action plans, experiments and measurements to show their ideas deliver results….(More)”

The International Handbook Of Public Administration And Governance

Curated on Posted on by Stefaan Verhulst

New book edited by Andrew Massey and Karen Johnston: “…Handbook explores key questions around the ways in which public administration and governance challenges can be addressed by governments in an increasingly globalized world. World-leading experts explore contemporary issues of government and governance, as well as the relationship between civil society and the political class. The insights offered will allow policy makers and officials to explore options for policy making in a new and informed way.

Adopting global perspectives of governance and public sector management, the Handbook includes scrutiny of current issues such as: public policy capacity, wicked policy problems, public sector reforms, the challenges of globalization and complexity management. Practitioners and scholars of public administration deliver a range of perspectives on the abiding wicked issues and challenges to delivering public services, and the way that delivery is structured. The Handbook uniquely provides international coverage of perspectives from Africa, Asia, North and South America, Europe and Australia.

Practitioners and scholars of public administration, public policy, public sector management and international relations will learn a great deal from this Handbook about the issues and structures of government and governance in an increasingly complex world. (Full table of contents)… (More).”

Rebooting Democracy

Curated on Posted on by Stefaan Verhulst

 John Boik, Lorenzo Fioramonti, and Gary Milante at Foreign Policy: “….The next generation of political and economic systems may look very different from the ones we know today.

Some changes along these lines are already happening. Civil society groups, cities, organizations, and government agencies have begun to experiment with a host of innovations that promote decentralization, redundancy, inclusion, and diversity. These include participatory budgeting, where residents of a city democratically choose how public monies are spent. They also include local currency systems, open-source development, open-design, open-data and open-government, public banking, “buy local” campaigns, crowdfunding, and socially responsible business models.

Such innovations are a type of churning on the edges of current systems. But in complex systems, changes at the periphery can cascade to changes at the core. Further, the speed of change is increasing. Consider the telephone, first introduced by Bell in 1876. It took about 75 years to reach adoption by 50 percent of the market. A century later the Internet did the same in about 35 years. We can expect that the next major innovations will be adopted even faster.

Following the examples of the telephone and Internet, it appears likely that the technology of new economic and political decision-making systems will first be adopted by small groups, then spread virally. Indeed, small groups, such as neighborhoods and cities, are among today’s leaders in innovation. The influence of larger bodies, such as big corporations and non-governmental organizations, is also growing steadily as nation states increasingly share their powers, willingly or not.

Changes are evident even within large corporations. Open-source software development has become the norm, for example, and companies as large as Toyota have announced plans to freely share their intellectual property.

While these innovations represent potentially important parts of new political and economic systems, they are only the tip of the iceberg. Systems engineering design could eventually integrate these and other innovations into efficient, user-friendly, scalable, and resilient whole systems. But the need for this kind of innovation is not yet universally acknowledged. In its list of 14 grand challenges for the 21st century, the U.S. National Academy of Engineering addresses many of the problems caused by poor decision making, such as climate change, but not the decision-making systems themselves. The work has only just begun.

…

The development of new options will dramatically alter how democracy is used, adjusted, and exported. Attention will shift toward groups, perhaps at the city/regional level, who wish to apply the flexible tools freely available on the Internet. Future practitioners of democracy will invest more time and resources to understand what communities want and need — helping them adapt designs to make them fit for their purpose — and to build networked systems that beneficially connect diverse groups into larger political and economic structures. In time, when the updates to next-generation political and economic near completion, we might find ourselves more fully embracing the notion “engage local, think global.”…(More)

Eight ways to make government more experimental

Curated on Posted on by Stefaan Verhulst
Jonathan Breckon et al at NESTA: “When the banners and bunting have been tidied away after the May election, and a new bunch of ministers sit at their Whitehall desks, could they embrace a more experimental approach to government?

Such an approach requires a degree of humility.  Facing up to the fact that we don’t have all the answers for the next five years.  We need to test things out, evaluate new ways of doing things with the best of social science, and grow what works.  And drop policies that fail.

But how best to go about it?  Here are our 8 ways to make it a reality:

  1. Make failure OK. A more benign attitude to risk is central to experimentation.  As a 2003 Cabinet Office review entitled Trying it Out said, a pilot that reveals a policy to be flawed should be ‘viewed as a success rather than a failure, having potentially helped to avert a potentially larger political and/or financial embarrassment’. Pilots are particularly important in fast moving areas such as technology to try promising fresh ideas in real-time. Our ‘Visible Classroom’ pilot tried an innovative approach to teacher CPD developed from technology for television subtitling.
  2. Avoid making policies that are set in stone.  Allowing policy to be more project–based, flexible and time-limited could encourage room for manoeuvre, according to a previous Nesta report State of Uncertainty; Innovation policy through experimentation.  The Department for Work and Pensions’ Employment Retention and Advancement pilot scheme to help people back to work was designed to influence the shape of legislation. It allowed for amendments and learning as it was rolled out.  We need more policy experiments like this.
  3. Work with the grain of current policy environment. Experimenters need to be opportunists. We need to be nimble and flexible. Ready to seize windows of opportunity to  experiment. Some services have to be rolled out in stages due to budget constraints. This offers opportunities to try things out before going national. For instance, The Mexican Oportunidades anti-poverty experiments which eventually reached 5.8 million households in all Mexican states, had to be trialled first in a handful of areas. Greater devolution is creating a patchwork of different policy priorities, funding and delivery models – so-called ‘natural experiments’. Let’s seize the opportunity to deliberately test and compare across different jurisdictions. What about a trial of basic income in Northern Ireland, for example, along the lines of recent Finnish proposals, or universal free childcare in Scotland?
  4. Experiments need the most robust and appropriate evaluation methods such as, if appropriate, Randomised Controlled Trials. Other methods, such as qualitative research may be needed to pry open the ‘black box’ of policies – to learn about why and how things are working. Civil servants should use the government trial advice panel as a source of expertise when setting up experiments.
  5. Grow the public debate about the importance of experimentation. Facebook had to apologise after a global backlash to psychological experiments on their 689,000 users web-users. Approval by ethics committees – normal practice for trials in hospitals and universities – is essential, but we can’t just rely on experts. We need a dedicated public understanding of experimentation programmes, perhaps run by Evidence Matters or Ask for Evidence campaigns at Sense about Science. Taking part in an experiment in itself can be a learning opportunity creating  an appetite amongt the public, something we have found from running an RCT with schools.
  6. Create ‘Skunkworks’ institutions. New or improved institutional structures within government can also help with experimentation.   The Behavioural Insights Team, located in Nesta,  operates a classic ‘skunkworks’ model, semi-detached from day-to-day bureaucracy. The nine UK What Works Centres help try things out semi-detached from central power, such as the The Education Endowment Foundation who source innovations widely from across the public and private sectors- including Nesta-  rather than generating ideas exclusively in house or in government.
  7. Find low-cost ways to experiment. People sometimes worry that trials are expensive and complicated.  This does not have to be the case. Experiments to encourage organ donation by the Government Digital Service and Behavioural Insights Team involved an estimated cost of £20,000.  This was because the digital experiments didn’t involve setting up expensive new interventions – just changing messages on  web pages for existing services. Some programmes do, however, need significant funding to evaluate and budgets need to be found for it. A memo from the White House Office for Management and Budget has asked for new Government schemes seeking funding to allocate a proportion of their budgets to ‘randomized controlled trials or carefully designed quasi-experimental techniques’.
  8. Be bold. A criticism of some experiments is that they only deal with the margins of policy and delivery. Government officials and researchers should set up more ambitious experiments on nationally important big-ticket issues, from counter-terrorism to innovation in jobs and housing….(More)

World Justice Project (WJP) Open Government Index

Curated on Posted on by Stefaan Verhulst

“The World Justice Project (WJP) Open Government Index™ provides scores and rankings on four dimensions of government openness: publicized laws and government data, right to information, civic participation, and complaint mechanisms (full descriptions below). Scores are based on responses to household surveys and in-country expert questionnaires collected for the WJP Rule of Law Index. The WJP Open Government Index 2015 covers a total of 102 countries and jurisdictions.

This index is the product of two years of development, consultation, and vetting with policy makers, civil society groups, and academics from several countries. It is our hope that over time this diagnostic tool will help identify strengths and weaknesses in countries under review and encourage policy choices that enhance openness, promote effective public oversight, and increase collaboration amongst public and private sectors.

Secrecy versus openness: Internet security and the limits of open source and peer production

Curated on Posted on by Stefaan Verhulst

Dissertation by Andreas Schmidt:” Open source and peer production have been praised as organisational models that could change the world for the better. It is commonly asserted that almost any societal activity could benefit from distributed, bottom-up collaboration — by making societal interaction more open, more social, and more democratic. However, we also need to be mindful of the limits of these models. How could they function in environments hostile to openness? Security is a societal domain more prone to secrecy than any other, except perhaps for romantic love. In light of the destructive capacity of contemporary cyber attacks, how has the Internet survived without a comprehensive security infrastructure? Secrecy vs. openness describes the realities of Internet security production through the lenses of open source and peer production theories. The study offers a glimpse into the fascinating communities of technical experts, who played a pivotal role when the chips were down for the Internet after large-scale attacks. After an initial flirtation with openness in the early years, operational Internet security communities have put in place institutional mechanisms that have resulted in less open forms of social production…(More)”

UNESCO demonstrates global impact through new transparency portal

Curated on Posted on by Stefaan Verhulst

“Opendata.UNESCO.org  is intended to present comprehensive, quality and timely information about UNESCO’s projects, enabling users to find information by country/region, funding source, and sector and providing comprehensive project data, including budget, expenditure, completion status, implementing organization, project documents, and more. It publishes program and financial information that are in line with UN system-experience of the IATI (International Aid Transparency Initiative) standards and other relevant transparency initiatives. UNESCO is now part of more than 230 organizations that have published to the IATI Registry, which brings together donor and developing countries, civil society organizations and other experts in aid information who are committed to working together to increase the transparency of aid.

Since its creation 70 years ago, UNESCO has tirelessly championed the causes of education, culture, natural sciences, social and human sciences, communication and information, globally. For instance – started in March 2010, the program for the Enhancement of Literacy in Afghanistan (ELA) benefited from a $19.5 million contribution by Japan. It aimed to improve the level of literacy, numeracy and vocational skills of the adult population in 70 districts of 15 provinces of Afghanistan. Over the next three years, until April 2013, the ELA programme helped some 360,000 adult learners in General Literacy compotency. An interactive map allows for an easy identification of UNESCO’s high-impact programs, and up-to-date information of current and future aid allocations within and across countries.

Public participation and interactivity are key to the success of any open data project. http://Opendata.UNESCO.org will evolve as Member States and partners will get involved, by displaying data on their own websites and sharing data among different networks, building and sharing applications, providing feedback, comments, and recommendations. …(More)”