Open data and transparency: a look back at 2013

Curated on Posted on by Stefaan Verhulst
Zoe Smith in the Guardian on the open data and development in 2013: “The clarion call for a “data revolution” made in the post-2015 high level panel report is a sign of a growing commitment to see freely flowing data become a tool for social change.

Web-based technology continued to offer increasing numbers of people the ability to share standardised data and statistics to demand better governance and strengthen accountability. 2013 seemed to herald the moment that the open data/transparency movement entered the mainstream.
Yet for those who have long campaigned on the issue, the call was more than just a catchphrase, it was a unique opportunity. “If we do get a global drive towards open data in relation to development or anything else, that would be really transformative and it’s quite rare to see such bold statements at such an early stage of the process. I think it set the tone for a year in which transparency was front and centre of many people’s agendas,” says David Hall Matthews, of Publish What You Fund.
This year saw high level discussions translated into commitments at the policy level. David Cameron used the UK’s presidency of the G8 to trigger international action on the three Ts (tax, trade and transparency) through the IF campaign. The pledge at Lough Erne, in Scotland, reaffirmed the commitment to the Busan open data standard as well as the specific undertaking that all G8 members would implement International Aid Transparency Index (IATI) standards by the end of 2015.
2013 was a particularly good year for the US Millenium Challenge Corporation (MCC) which topped the aid transparency index. While at the very top MCC and UK’s DfID were examples of best practice, there was still much room for improvement. “There is a really long tail of agencies who are not really taking transparency at all, yet. This includes important donors, the whole of France and the whole of Japan who are not doing anything credible,” says Hall-Matthews.
Yet given the increasing number of emerging and ‘frontier‘ markets whose growth is driven in large part by wealth derived from natural resources, 2013 saw a growing sense of urgency for transparency to be applied to revenues from oil, gas and mineral resources that may far outstrip aid. In May, the new Extractive Industries Transparency Initiative standard (EITI) was adopted, which is said to be far broader and deeper than its previous incarnation.
Several countries have done much to ensure that transparency leads to accountability in their extractive industries. In Nigeria, for example, EITI reports are playing an important role in the debate about how resources should be managed in the country. “In countries such as Nigeria they’re taking their commitment to transparency and EITI seriously, and are going beyond disclosing information but also ensuring that those findings are acted upon and lead to accountability. For example, the tax collection agency has started to collect more of the revenues that were previously missing,” says Jonas Moberg, head of the EITI International Secretariat.
But just the extent to which transparency and open data can actually deliver on its revolutionary potential has also been called into question. Governments and donors agencies can release data but if the power structures within which this data is consumed and acted upon do not shift is there really any chance of significant social change?
The complexity of the challenge is illustrated by the case of Mexico which, in 2014, will succeed Indonesia as chair of the Open Government Partnership. At this year’s London summit, Mexico’s acting civil service minister, spoke of the great strides his country has made in opening up the public procurement process, which accounts for around 10% of GDP and is a key area in which transparency and accountability can help tackle corruption.
There is, however, a certain paradox. As SOAS professor, Leandro Vergara Camus, who has written extensively on peasant movements in Mexico, explains: “The NGO sector in Mexico has more of a positive view of these kinds of processes than the working class or peasant organisations. The process of transparency and accountability have gone further in urban areas then they have in rural areas.”…
With increasing numbers of organisations likely to jump on the transparency bandwagon in the coming year the greatest challenge is using it effectively and adequately addressing the underlying issues of power and politics.

Top 2013 transparency publications

Open data, transparency and international development, The North South Institute
Data for development: The new conflict resource?, Privacy International
The fix-rate: a key metric for transparency and accountability, Integrity Action
Making UK aid more open and transparent, DfID
Getting a seat at the table: Civil Society advocacy for budget transparency in “untransparent” countries, International Budget Partnership

The dates that mattered

23-24 May: New Extractive Industries Transparency Index standard adopted
30 May: Post 2015 high level report calling for a ‘data revolution’ is published
17-18 June: UK premier, David Cameron, campaigns for tax, trade and transparency during the G8
24 October: US Millenium Challenge Corporation tops the aid transparency index”
30 October – 1 November: Open Government Partnership in London gathers civil society, governments and data experts

Open Budgets Portal

Curated on Posted on by Stefaan Verhulst

About: “The Open Budgets Portal is the first effort to create a one-stop shop for budget data worldwide with the hope of bringing visibility to countries’ efforts in this field, facilitating access and promoting use of spending data, and motivating other countries into action.

The portal offers the opportunity to showcase a subset of developing countries and subnational entities (identified by blue markers in the map) that have excelled in the exploration of new frontiers of fiscal transparency by choosing to disseminate their entire public spending datasets in accessible formats (i.e., soft copy), with the expectation that these efforts could motivate other countries into action . Users will be able to download the entire public expenditure landscape of the members of the portal in consolidated files, all of which were rigorously collected, cleaned and verified through the BOOST Initiative.

For each of these countries, the site also includes links to their original open data portals, which provide additional important information (i.e., higher frequencies other than annual, links to output data and other socio economic indicators, etc.). While every effort has been done to certify the quality of these databases according to BOOST approach and methodology, users are encouraged to refer back to the country-owned open data portals to ensure complete consistency of data with published official figures, as well as consult accompanying user manuals for potential caveats on uses of the data.

This portal represents a starting point to build momentum within the growing interest around fiscal transparency and the importance of data for enhanced decision-making processes and improved budget outcomes and accountability. While most initiatives on open budgets rightfully center on availability of key documents, little focus has been given to the quality of data dissemination and to the importance of its analytical use for incorporation into evidence-based decision-making processes.

This Open Budgets Portal aims to fill this gap by providing access to budget data worldwide and particularly to the most disaggregated and comprehensive data collected through the BOOST Initiative. The portal combines this information with a variety of tools, manuals, reports and best practices aimed at stimulating use by intermediaries, as well as easier to interpret visualization for non-experts. Our objective is to encourage all potential uses of this data to unbind the analytical power of such data.

The Open Budgets Portal was launched at the event “Boosting Fiscal Transparency for Better Policy Outcomes,” held on December 17, 2013 in Washington, DC. The following presentations were shown at the event:

Presentation of the Open Budgets Portal by Massimo Mastruzzi, Senior Economist, Open Goverment, World Bank.

Building a Citizen’s Budget Understanding – BudgetStories.md by Victoria Vlad, Economist of “Expert-Grup” from the Republic of Moldova.”

Selected Readings on Data Visualization

Curated on Posted on by Andrew Young

The Living Library’s Selected Readings series seeks to build a knowledge base on innovative approaches for improving the effectiveness and legitimacy of governance. This curated and annotated collection of recommended works on the topic of data visualization was originally published in 2013.

Data visualization is a response to the ever-increasing amount of  information in the world. With big data, informatics and predictive analytics, we have an unprecedented opportunity to revolutionize policy-making. Yet data by itself can be overwhelming. New tools and techniques for visualizing information can help policymakers clearly articulate insights drawn from data. Moreover, the rise of open data is enabling those outside of government to create informative and visually arresting representations of public information that can be used to support decision-making by those inside or outside governing institutions.

Selected Reading List (in alphabetical order)

Annotated Selected Reading List (in alphabetical order)

Duke, D.J., K.W. Brodlie, D.A. Duce and I. Herman. “Do You See What I Mean? [Data Visualization].” IEEE Computer Graphics and Applications 25, no. 3 (2005): 6–9. http://bit.ly/1aeU6yA.

  • In this paper, the authors argue that a more systematic ontology for data visualization to ensure the successful communication of meaning. “Visualization begins when someone has data that they wish to explore and interpret; the data are encoded as input to a visualization system, which may in its turn interact with other systems to produce a representation. This is communicated back to the user(s), who have to assess this against their goals and knowledge, possibly leading to further cycles of activity. Each phase of this process involves communication between two parties. For this to succeed, those parties must share a common language with an agreed meaning.”
  • That authors “believe that now is the right time to consider an ontology for visualization,” and “as visualization move from just a private enterprise involving data and tools owned by a research team into a public activity using shared data repositories, computational grids, and distributed collaboration…[m]eaning becomes a shared responsibility and resource. Through the Semantic Web, there is both the means and motivation to develop a shared picture of what we see when we turn and look within our own field.”

Friendly, Michael. “A Brief History of Data Visualization.” In Handbook of Data Visualization, 15–56. Springer Handbooks Comp.Statistics. Springer Berlin Heidelberg, 2008. http://bit.ly/17fM1e9.

  • In this paper, Friendly explores the “deep roots” of modern data visualization. “These roots reach into the histories of the earliest map making and visual depiction, and later into thematic cartography, statistics and statistical graphics, medicine and other fields. Along the way, developments in technologies (printing, reproduction), mathematical theory and practice, and empirical observation and recording enabled the wider use of graphics and new advances in form and content.”
  • Just as the general the visualization of data is far from a new practice, Friendly shows that the graphical representation of government information has a similarly long history. “The collection, organization and dissemination of official government statistics on population, trade and commerce, social, moral and political issues became widespread in most of the countries of Europe from about 1825 to 1870. Reports containing data graphics were published with some regularity in France, Germany, Hungary and Finland, and with tabular displays in Sweden, Holland, Italy and elsewhere.”

Graves, Alvaro and James Hendler. “Visualization Tools for Open Government Data.” In Proceedings of the 14th Annual International Conference on Digital Government Research, 136–145. Dg.o ’13. New York, NY, USA: ACM, 2013. http://bit.ly/1eNSoXQ.

  • In this paper, the authors argue that, “there is a gap between current Open Data initiatives and an important part of the stakeholders of the Open Government Data Ecosystem.” As it stands, “there is an important portion of the population who could benefit from the use of OGD but who cannot do so because they cannot perform the essential operations needed to collect, process, merge, and make sense of the data. The reasons behind these problems are multiple, the most critical one being a fundamental lack of expertise and technical knowledge. We propose the use of visualizations to alleviate this situation. Visualizations provide a simple mechanism to understand and communicate large amounts of data.”
  • The authors also describe a prototype of a tool to create visualizations based on OGD with the following capabilities:
    • Facilitate visualization creation
    • Exploratory mechanisms
    • Viralization and sharing
    • Repurpose of visualizations

Hidalgo, César A. “Graphical Statistical Methods for the Representation of the Human Development Index and Its Components.” United Nations Development Programme Human Development Reports, September 2010. http://bit.ly/166TKur.

  • In this paper for the United Nations Human Development Programme, Hidalgo argues that “graphical statistical methods could be used to help communicate complex data and concepts through universal cognitive channels that are heretofore underused in the development literature.”
  • To support his argument, representations are provided that “show how graphical methods can be used to (i) compare changes in the level of development experienced by countries (ii) make it easier to understand how these changes are tied to each one of the components of the Human Development Index (iii) understand the evolution of the distribution of countries according to HDI and its components and (iv) teach and create awareness about human development by using iconographic representations that can be used to graphically narrate the story of countries and regions.”

Stowers, Genie. “The Use of Data Visualization in Government.” IBM Center for The Business of Government, Using Technology Series, 2013. http://bit.ly/1aame9K.

  • This report seeks “to help public sector managers understand one of the more important areas of data analysis today — data visualization. Data visualizations are more sophisticated, fuller graphic designs than the traditional spreadsheet charts, usually with more than two variables and, typically, incorporating interactive features.”
  • Stowers also offers numerous examples of “visualizations that include geographical and health data, or population and time data, or financial data represented in both absolute and relative terms — and each communicates more than simply the data that underpin it. In addition to these many examples of visualizations, the report discusses the history of this technique, and describes tools that can be used to create visualizations from many different kinds of data sets.”

Lessons in the crowdsourced verification of news from Storyful and Reddit’s Syria forum

Curated on Posted on by Stefaan Verhulst

at GigaOm: “One of the most powerful trends in media over the past year is the crowdsourced verification of news, whether it’s the work of a blogger like Brown Moses or former NPR journalist Andy Carvin. Two other interesting efforts in this area are the “open newsroom” approach taken by Storyful — which specializes in verifying social-media reports for mainstream news entities — and a Reddit forum devoted to crowdsourcing news coverage of the civil war in Syria.
Storyful journalist Joe Galvin recently looked at some of the incidents that the company has helped either debunk or verify over the past year — including a fake tweet from the official account of the Associated Press about explosions at the White House (which sent the Dow Jones index plummeting before it was corrected), a claim from Russian authorities that a chemical attack in Syria had been pre-meditated, and a report from investigative journalist Seymour Hersh about the same attack that questioned whether the government had been involved….
Reddit, meanwhile, has been conducting some “open newsroom”-style experiments of its own around a number of news events, including the Syrian civil war. The site has come under fire in the past for some of those efforts — including the attempt to identify the bombers in the Boston bombings case, which went badly awry — but the Syrian thread in particular is a good example of how a smart aggregator can make sense of an ongoing news event. In a recent post at a site called Dissected News, one of the moderators behind the /r/SyrianCivilWar sub-Reddit — a 22-year-old law student named Christopher Kingdon (or “uptodatepronto” as he is known on the site) — wrote about his experiences with the forum, which is trying to be a broadly objective source for breaking news and information about the conflict….
Some of what the moderators do in the forum is similar to the kind of verification that Storyful or the BBC’s “user-generated content desk” do — checking photos and video for obvious signs of fakery and hoaxes. But Kingdon also describes how much effort his team of volunteers puts into ensuring that the sub-Reddit doesn’t degenerate into trolling or flame-wars. Strict rules are enforced “to prevent personal attacks, offensive and violent language and racism” and the moderators favor posts that “utilize sources, background information and a dash of common sense.”

The Impact of Innovation Inducement Prizes

Curated on Posted on by Stefaan Verhulst

From the Compendium of Evidence on Innovation Policy/NESTA: “Innovation inducement prizes are one of the oldest types of innovation policy measure.  The popularity of innovation inducement prizes has gradually decreased during the early 20th century. However, innovation inducement prizes have regained some of their popularity since the 1990s with new prizes awarded by the US X Prize Foundation and with the current USA Administration’s efforts to use them in various government departments as an innovation policy instrument. Innovation Prizes are also becoming an important innovation policy instrument in the UK.  A recent report by McKinsey & Company (2009) estimates the value of prizes awarded to be between £600 million and £1.2million. Despite the growing popularity of innovation inducement prizes, the impact of this innovation policy measure is still not understood. This report brings together the existing evidence on the effects of innovation inducement prizes by drawing on a number of ex-ante and ex-post evaluations as well as limited academic literature. This report focuses on ex-ante innovation inducement prizes where the aim is to induce investment or attention to a specific goal or technology. This report does not discuss the impact of ex-post recognition prizes where the prize is given as a recognition after the intended outcome happens (e.g. Nobel Prize).
Innovation inducement prizes have a wide range of rationales and there is no agreed on dominant rationale in the literature. Traditionally, prizes have been seen as an innovation policy instrument that can overcome market failure by creating an incentive for the development of a particular technology or technology application. A second rationale is that the implementation demonstration projects in which not only creation of a specific technology is intended but also demonstration of the feasible application of this technology is targeted. A third rationale is related to the creation of a technology that will later be put in the public domain to attract subsequent research. Prizes are also increasingly organised for community and leadership building. As prizes probably allow more flexibility than most of the other innovation policy instruments, there is a large number of different prize characteristics and thus a vast number of prize typologies based on these characteristics.
Evidence on the effectiveness of prizes is scarce. There are only a few evaluations or academic works that deal with the creation of innovation output and even those which deal with the innovation output only rarely deals with the additionality. Only a very limited number of studies looked at if innovation inducement prizes led to more innovation itself or innovation outputs. As well as developing the particular technology that the innovation inducement prizes produce, they create prestige for both the prize sponsor and entrants. Prizes might also increase the public and sectoral awareness on specific technology issues. A related issue to the prestige gained from the prizes is the motivations of participants as a conditioning factor for innovation performance. Design issues are the main concern of the prizes literature. This reflects the importance of a careful design for the achievement of desired effects (and the limitation of undesired effects). There are a relatively large number of studies that investigated the influence of the design of prize objective on the innovation performance. A number of studies points out that sometimes prizes should be accompanied with or followed by other demand side initiatives to fulfil their objectives, mostly on the basis of ex-ante evaluations. Finally, prizes are also seen as a valuable opportunity for experimentation in innovation policy.
It is evident from the literature we analysed that the evidence on the impact of innovation inducement prizes is scarce. There is also a consensus that innovation inducement prizes are not a substitute for other innovation policy measures but are complementary under certain conditions. Prizes can be effective in creating innovation through more intense competition, engagement of wide variety of actors, distributing risks to many participants and by exploiting more flexible solutions through a less prescriptive nature of the definition of the problem in prizes. They can overcome some of the inherent barriers to other instruments, but if prizes are poorly designed, managed and awarded, they may be ineffective or even harmful.”

Buenos Aires, A Pocket of Civic Innovation in Argentina

Curated on Posted on by Stefaan Verhulst

Rebecca Chao in TechPresident: “…In only a few years, the government, civil society and media in Buenos Aires have actively embraced open data. The Buenos Aires city government has been publishing data under a creative commons license and encouraging civic innovation through hackathons. NGOs have launched a number of tech-driven tools and Argentina’s second largest newspaper, La Nación, has published several hard-hitting data journalism projects. The result is a fledgling but flourishing open data culture in Buenos Aires, in a country that has not yet adopted a freedom of information law.

A Wikipedia for Open Government Data

In late August of this year, the Buenos Aires government declared a creative commons license for all of its digital content, which allows it be used for free, like Wikipedia content, with proper attribution. This applies to their new open data catalog that allows users to visualize the data, examine apps that have been created using the data and even includes a design lab for posting app ideas. Launched only in March, the government has already published fairly substantial data sets, including the salaries of city officials. The website also embodies the principals of openness in its design; it is built with open-source software and its code is available for reuse via GitHub.
“We were the first city in Argentina doing open government,” Rudi Borrmann tells techPresident over Skype. Borrmann is the Director of Buenos Aires’ Open Government Initiative. Previously, he was the social media editor at the city’s New Media Office but he also worked for many years in digital media…
While the civil society and media sectors have forged ahead in using open data, Borrmann tells techPresident that up in the ivory tower, openness to open data has been lagging. “Only technical schools are starting to create areas focused on working on open data,” he says.
In an interview with NYU’s govlab, Borrmann explained the significance of academia in using and pushing for more open data. “They have the means, the resources, the methodology to analyze…because in government you don’t have that time to analyze,” he said.
Another issue with open data is getting other branches of the government to modernize. Borrmann says that a lot of the Open Government’s work is done behind the scenes. “In general, you have very poor IT infrastructure all over Latin America” that interferes with the gathering and publishing of data, he says. “So in some cases it’s not about publishing or not publishing,” but about “having robust infrastructure for the information.”
It seems that the behind the scenes work is bearing some fruit. Just last week, on Dec. 6, the team behind the Buenos Aires open data website launched an impressive, interactive timeline, based on a similar timelapse map developed by a 2013 Knight-Mozilla Fellow, Noah Veltman. Against faded black and white photos depicting the subway from different decades over the last century, colorful pops of the Subterráneo lines emerge alongside factoids that go all the way back to 1910.”

20 Innovations that Mattered in 2013

Curated on Posted on by Stefaan Verhulst

New Guide from GovLoop: “The end of the year means two things: setting unrealistic New Year’s resolutions and endless retrospectives.  While we can’t force you to put down the cake and pick up a carrot, we can help you to do your job better by highlighting some of the biggest and best innovations to come out of government in the last 365 days.
The past year brought us the Interior Department’s Instagram feed and Colorado’s redesigned website. It also brought us St. Louis’ optimized data analytics that make their city safer and North Carolina’s iCenter that adopted a “try before you buy” policy.
All of these new technologies and tactics saved time and resources, critical outcomes in the current government landscape where budget cuts are making each new purchase risky.
But these were not the only buzzworthy projects for government technology in 2013. In this end-of-year issue, GovLoop analyzed the 20 best innovations in government in four different categories:

  • Mobile Apps Movers and Shakers?
  • Big Data Dynamos
  • Social Media Mavericks
  •  Website Wonders

We also asked two of the most innovative Chief Information Officers in the country to don some Google Glass’ In a year where the government shutdown and sequestration brought progress to a screeching halt, many agencies were able to rise above the inauspicious environment and produce groundbreaking and innovative programs.
For instance, when the horrible bombings brought terror to the finish line of the Boston Marathon, the local police department sprang into action. They immediately mobilized their forces on the ground. But then they did something else, too. The Boston Police Department took to Twitter. The social media team was informative, timely and, accurate. The BPD flipped the script on emergency media management. They innovated in a time of crisis and got people the information they needed in a timely and appropriate manner.
We know that oftentimes it is hard to see through the budget cuts and government shutdowns, but government innovation is all around us. Our goal with this guide is to showcase programs that are not only making a difference but demonstrate risk and reward….”

Map of Government Innovation Labs

Curated on Posted on by Stefaan Verhulst

New publication by Parsons DESIS Lab: “There is a growing perception that the state’s ability to provide various forms of public service can be enhanced through public-private partnerships, the use of new technologies, as well as various strategies for increased public participation. Such new approaches to provide better services for individuals and communities require breaking the established routines and traditional structures of government, and rethinking the opposition between government bureaucracies and community actors. The good news is that new kinds of collaboration are emerging. Such partnerships are characterized by horizontal practices such as co-governance, co-design, or co-production where citizens, experts, and governments work closely, together, to provide better public services. In the United States and across the world we can observe efforts to promote radical innovations in the public sector followed by the creation of different types of “authorizing environments” that foster these experiments….
GovLabsConstellation

…A Government Innovation Lab is a specific type of Public Innovation Place characterized by a direct connection with the public sector and created to tackle complex challenges that more traditional governmental structures seek to resolve. Government Innovation Labs experiment with and propose innovative public services and policies, while at the same time, try to reform and change the way government operates.”

Government's Crowdsourcing Revolution

Curated on Posted on by Stefaan Verhulst

John M. Kamensky  in Governing: “In a recent report for the IBM Center for the Business of Government, Brabham says that an important distinction between crowdsourcing and other forms of online participation is that crowdsourcing “entails a mix of top-down, traditional, hierarchical process and a bottom-up, open process involving an online community.”
Crowdsourcing in the public sector can be done within government, among employees as a way to surface ideas — such as the New York City government’s “Simplicity” initiative — or it can be done by nonprofit groups in ways that influence government operations. For example, a transportation advocacy group in New York City has created a site where citizens can report “near miss” accidents, which are then mapped to determine patterns. The idea is that, while the city government already maps accidents that have happened, hazardous traffic zones can be detected and resolved faster by mapping near-misses without waiting for a large number of actual accidents.
Brabham offers a strategic view of crowdsourcing and when it is useful to address public problems. His report also identifies four specific approaches, describing which is most useful for a given category of problem:
• Knowledge discovery and management. This approach is best for information-gathering and cataloguing problems through an online community, such as the reporting of earth tremors or potholes to a central source. This approach could also be used to report conditions of parks or hiking trails or for cataloging public art projects as have been done in several cities across the country.
• Distributed human-intelligence tasking: This approach is most useful when human intelligence is more effective than computer analysis. It involves distributing “micro-tasks” that require human intelligence to solve, such as transcribing handwritten historical documents into electronic files. For example, when the handwritten 1940 census records were publicly released in 2012, the National Archives catalyzed the electronic tagging of more than 130 million records so they could be searchable online. More than 150,000 people volunteered.
• Broadcast search: This approach is most useful when an agency is attempting to find creative solutions to problems. It involves broadcasting a problem-solving challenge widely on the Internet and offering an award for the best solution. NASA, for example, offered a prize for an algorithm to predict solar flares. The federal government sponsors a contest and awards Web platform, Challenge.gov, that various federal agencies can use to post their challenges. To date, hundreds of diverse challenges have been posted, with thousands of people proposing solutions.
• Peer-vetted creative production: This approach is most useful when an agency is looking for innovative ideas that must meet a test of taste or market support. It involves an online community that both proposes possible solutions and is empowered to collectively choose among them. For example, the Utah Transit Authority sponsored the Next Stop Design project, allowing citizens to design and vote on an ideal bus-stop shelter. Nearly 3,200 people participated, submitting 260 high-quality architectural renderings, and there were more than 10,000 votes leading to a final selection….”