The Global Open Data Index 2014

Curated on Posted on by Stefaan Verhulst
Open Knowledge Foundation: “The Global Open Data Index ranks countries based on the availability and accessibility of information in ten key areas, including government spending, election results, transport timetables, and pollution levels.
The UK tops the 2014 Index retaining its pole position with an overall score of 96%, closely followed by Denmark and then France at number 3 up from 12th last year. Finland comes in 4th while Australia and New Zealand share the 5th place. Impressive results were seen from India at #10 (up from #27) and Latin American countries like Colombia and Uruguay who came in joint 12th .
Sierra Leone, Mali, Haiti and Guinea rank lowest of the countries assessed, but there are many countries where the governments are less open but that were not assessed because of lack of openness or a sufficiently engaged civil society.
Overall, whilst there is meaningful improvement in the number of open datasets (from 87 to 105), the percentage of open datasets across all the surveyed countries remained low at only 11%.
Even amongst the leaders on open government data there is still room for improvement: the US and Germany, for example, do not provide a consolidated, open register of corporations. There was also a disappointing degree of openness around the details of government spending with most countries either failing to provide information at all or limiting the information available – only two countries out of 97 (the UK and Greece) got full marks here. This is noteworthy as in a period of sluggish growth and continuing austerity in many countries, giving citizens and businesses free and open access to this sort of data would seem to be an effective means of saving money and improving government efficiency.
Explore the Global Open Data Index 2014 for yourself!”

How do we improve open data for police accountability?

Curated on Posted on by Stefaan Verhulst

at the SunLight Foundation: “This is a challenging time for people who worry about the fairness of American governmental institutions. In quick succession, grand juries declined to indict two police officers accused of killing black men. In the case of Ferguson, Mo. officer Darren Wilson’s killing of Michael Brown, the grand jury’s decision appeared to center on uncertainty about whether Wilson’s action was legal and whether he killed under threat. In the case of New York City police officer Daniel Pantaleo’s killing of Eric Garner, however, a bystander recorded and made public a video of the police officer causing Garner’s death through an illegal chokehold. In Pantaleo’s case, the availability of video data has made the question about institutional fairness even more urgent, as people can see for themselves the context in which the officer exercised power. The data has given us a common set of facts to use in judging police behavior.
We grant law enforcement and corrections departments the right to exercise more physical power over the public than we do to any other part of our government. But do we generally have the data we need to evaluate how they’re using it?….
The time to find good solutions to these problems is now. Responding to widespread frustration, President Obama has just announced a three-part initiative to “strengthen community policing”: an increased focus on transparency and oversight for federal-to-local transfers of military equipment, a proposal to provide matching funding to local police departments to buy body cameras, and a “Task Force on 21st Century Policing” that will make recommendations for how to implement community-oriented policing practices.
While each element of Obama’s initiative corresponds to a distinct set of concerns about policing, one element they share in common is the need to increase access to information about police work. Each of the three approaches will rely on mechanisms to increase the flow of public information about what police officers are doing in their official roles and how they are doing it. How are police officers going about fulfilling their responsibility to ensure public safety? Are they working in ways that appropriately respect individual rights? Are they responsive to public concerns, when concerns are raised?
By encouraging the collection and publication of more data about how government is working, Obama’s initiative has the potential to support precisely the kind of increase in data availability that can transform public outcomes. When applied with the intent to improve transparency and accountability and to increase public engagement, open data — and the civic tech that uses this data — can bridge the often too-large gap between the public and government.
However, because Obama’s initiatives depend on the effective collection, publication, and communication of information, open data advocates have a particular contribution to make. It’s important to think about what lessons we can apply from our experiences with open data — and with data collected and used for police accountability — in order to ensure that this initiative has the greatest possible impact. As an open data and open government community, can we make recommendations that can help improve the data we’re collecting for police transparency and accountability?
I’m going to begin a list, but it’s just a beginning – I am certain that you have many more recommendations to make. I’ll categorize them first by Obama’s “Strengthening Community Policing” initiatives and then keep thinking about what additional data is needed. Please think along with me about what kind of datasets we will need, what potential issues with data availability and quality we’re likely to see, what kind of laws may need to be changed to improve access to the data necessary for police accountability, then make your recommendations in the Google Doc embedded at the end of this post. If you’ve seen any great projects you’ve seen which improve police transparency and accountability, be sure to share those as well….”

The Year of Data-Driven Government Accountability

Curated on Posted on by Stefaan Verhulst

Robert Runge in Pacific Standard: “Indeed, 2014 could be called the Year of Government Accountability, as voters on just about every continent have demanded that public officials govern with relentless efficiency, fiscal responsibility, and transparency….
The bottom line, in my view, is that facts must be the fundamental basis for critical and strategic decision-making at every level of government around the world today.
This belief—the foundation of massive technology and social movements, such as open data, big data, and data-driven government—is currently shared by a number of global government leaders. Just recently, for example, President Obama declared that “We must respond based on facts, not fear” when confronting the global Ebola crisis.
To be sure, presenting facts to decision-makers where and when they are needed is one of the most urgent technology priorities of our time. The good news is that we’re seeing progress on this front each and every day as civic organizations around the world rush to open their vast troves of data on the Internet and usher in a new era in data-driven government that will produce facts at the speed of light, and deliver them in context to political leaders, everyday citizens, professional academicians, scientists, journalists, and software developers wherever they are connected to the Web.
Data-driven government, which capitalizes on data, one of the most valuable natural resources of the 21st century, is a breakthrough opportunity of truly significant proportions. And it will be absolutely critical if governments everywhere are to achieve their ultimate mission. Without it, I worry that we just won’t be able to provide citizens with a higher quality of life and with greater opportunities to achieve their full potential.

Forget the FOIA Request: Cities, States Open Data Portals

Curated on Posted on by Stefaan Verhulst

in MediaShift (PBS): “In almost any city you can read your local leaders’ emails if you formally ask for them. In Gainesville, Fla., all you have to do is go here.
In most states you can find out how tax dollars are being spent if you officially request expenditure records. In Wisconsin, you just click here.
For the last 50 years, governments have given up public records in response to Freedom of Information requests. But a number of public agencies are learning the value of proactively providing information before anyone has to ask for it.
The trend is part of the open-data movement that most large cities and the federal government have already begun to embrace. The information itself can range from simple emails to complex datasets, but the general idea is the same: Deliver information directly to the public using digital tools that can save money and serve the goal of government transparency.
…And there’s the added benefit of helping the bottom line. Users don’t have to request information if it’s already posted, saving agencies time and money, and a centralized FOIA tracking system can further streamline processing.
Sean Moulton of the Center for Effective Government testified before Congress that full participation in FOIAonline could save federal agencies an estimated $40 million per year in processing costs.
And Reinvent Albany, a nonprofit that pushes for transparency in New York, estimated in a June report that New York City could reduce FOI-related costs by 66 percent – from $20 million per year down to $7 million – by adopting an open-data system and doing away with its “hodgepodge of paper-based methods that are expensive, slow and unreliable.”
So…What’s the Catch?
In their survey, chief information officers were asked to name the top three barriers to advancing open data in state government. Fifty-three percent cited “agencies’ willingness to publish data,” and 49 percent cited “the reliability of the data.”
Information is of little value to the public if it’s faulty or too complex to understand. It could become a way for agencies to claim they’re being transparent without actually providing anything useful.
Plus, some worry that public servants will self-censor if they know, for example, their emails are automatically being shared with the world….”

Selected Readings on Cities and Civic Technology

Curated on Posted on by Julia Root

By Julia Root and Stefaan Verhulst

The Living Library’s Selected Readings series seeks to build a knowledge base on innovative approaches for improving the effectiveness and legitimacy of governance. This curated and annotated collection of recommended works on the topic of civic innovation was originally published in 2014.

The last five years have seen a wave of new organizations, entrepreneurs and investment in cities and the field of civic innovation.  Two subfields, Civic Tech and Government Innovation, are particularly aligned with GovLab’s interest in the ways in which technology is and can be deployed to redesign public institutions and re-imagine governance.

The emerging field of civic technology, or “Civic Tech,” champions new digital platforms, open data and collaboration tools for transforming government service delivery and engagement with citizens. Government Innovation, while not a new field, has seen in the last five years a proliferation of new structures (e.g. Mayor’s Office of New Urban Mechanics), roles (e.g. Chief Technology/Innovation Officer) and public/private investment (e.g. Innovation Delivery Teams and Code for America Fellows) that are building a world-wide movement for transforming how government thinks about and designs services for its citizens.

There is no set definition for “civic innovation.” However, broadly speaking, it is about improving our cities through the implementation of tools, ideas and engagement methods that strengthen the relationship between government and citizens. The civic innovation field encompasses diverse actors from across the public, private and nonprofit spectrums. These can include government leaders, nonprofit and foundation professionals, urbanists, technologists, researchers, business leaders and community organizers, each of whom may use the term in a different way, but ultimately are seeking to disrupt how cities and public institutions solve problems and invest in solutions.

Selected Reading List (in alphabetical order)

Annotated Selected Readings (in alphabetical order)

Books

Goldsmith, Stephen, and Susan Crawford. The Responsive City: Engaging Communities Through Data-Smart Governance. 1 edition. San Francisco, CA: Jossey-Bass, 2014. http://bit.ly/1zvKOL0.

  • The Responsive City, a guide to civic engagement and governance in the digital age, is the culmination of research originating from the Data-Smart City Solutions initiative, an ongoing project at Harvard Kennedy School working to catalyze adoption of data projects on the city level.
  • The “data smart city” is one that is responsive to citizens, engages them in problem solving and finds new innovative solutions for dismantling entrenched bureaucracy.
  • The authors document case studies from New York City, Boston and Chicago to explore the following topics:
    • Building trust in the public sector and fostering a sustained, collective voice among communities;
    • Using data-smart governance to preempt and predict problems while improving quality of life;
    • Creating efficiencies and saving taxpayer money with digital tools; and
    • Spearheading these new approaches to government with innovative leadership.

Townsend, Anthony M. Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia. 1 edition. New York: W. W. Norton & Company, 2013. http://bit.ly/17Y4G0R.

  • In this book, Townsend illustrates how “cities worldwide are deploying technology to address both the timeless challenges of government and the mounting problems posed by human settlements of previously unimaginable size and complexity.”
  • He also considers “the motivations, aspirations, and shortcomings” of the many stakeholders involved in the development of smart cities, and poses a new civics to guide these efforts.
  • He argues that smart cities are not made smart by various, soon-to-be-obsolete technologies built into its infrastructure; instead, it is how citizens are using ever-changing and grassroots technologies to be “human-centered, inclusive and resilient” that will make cities ‘smart.’

Reports + Journal Articles

Black, Alissa, and Rachel Burstein. “The 2050 City – What Civic Innovation Looks Like Today and Tomorrow.” White Paper. New America Foundation – California Civic Innovation Project, June 2013. https://bit.ly/2GohMvw.

  • Through their interviews, the authors determine that civic innovation is not just a “compilation of projects” but that it can inspire institutional structural change.
  • Civic innovation projects that have a “technology focus can sound very different than process-related innovations”; however the outcomes are actually quite similar as they disrupt how citizens and government engage with one another.
  • Technology is viewed by some of the experts as an enabler of civic innovation – not necessarily the driver for innovation itself. What constitutes innovation is how new tools are implemented by government or by civic groups that changes the governing dynamic.

Patel, Mayur, Jon Sotsky, Sean Gourley, and Daniel Houghton. “Knight Foundation Report on Civic Technology.” Presentation. Knight Foundation, December 2013. http://slidesha.re/11UYgO0.

  • This reports aims to advance the field of civic technology, which compared to the tech industry as a whole is relatively young. It maps the field, creating a starting place for understanding activity and investment in the sector.
  • It defines two themes, Open Government and Civic Action, and identifies 11 clusters of civic tech innovation that fall into the two themes. For each cluster, the authors describe the type of activities and highlights specific organizations.
  • The report identified more than $430 million of private and philanthropic investment directed to 102 civic tech organizations from January 2011 to May 2013.

Open Plans. “Field Scan on Civic Technology.” Living Cities, November 2012. http://bit.ly/1HGjGih.

  • Commissioned by Living Cities and authored by Open Plans, the Field Scan investigates the emergent field of civic technology and generates the first analysis of the potential impact for the field as well as a critique for how tools and new methods need to be more inclusive of low-income communities in their use and implementation.
  • Respondents generally agreed that the tools developed and in use in cities so far are demonstrations of the potential power of civic tech, but that these tools don’t yet go far enough.
  • Civic tech tools have the potential to improve the lives of low-income people in a number of ways. However, these tools often fail to reach the population they are intended to benefit. To better understand this challenge, civic tech for low-income people must be considered in the broader context of their interactions with technology and with government.
  • Although hackathons are popular, their approach to problem solving is not always driven by community needs, and hackathons often do not produce useful material for governments or citizens in need.

Goldberg, Jeremy M. “Riding the Second Wave of Civic Innovation.” Governing, August 28, 2014. http://bit.ly/1vOKnhJ.

  • In this piece, Goldberg argues that innovation and entrepreneurship in local government increasingly require mobilizing talent from many sectors and skill sets.

Black, Alissa, and Burstein, Rachel. “A Guide for Making Innovation Offices Work.” IBM Center for the Business of Government, October 2014. http://bit.ly/1vOFZP4.

  • In this report, Burstein and Black examine the recent trend toward the creation of innovation offices across the nation at all levels of government to understand the structural models now being used to stimulate innovation—both internally within an agency, and externally for the agency’s partners and communities.
  • The authors conducted interviews with leadership of innovation offices of cities that include Philadelphia, Austin, Kansas City, Chicago, Davis, Memphis and Los Angeles.
  • The report cites examples of offices, generates a typology for the field, links to projects and highlights success factors.

Mulholland, Jessica, and Noelle Knell. “Chief Innovation Officers in State and Local Government (Interactive Map).” Government Technology, March 28, 2014. http://bit.ly/1ycArvX.

  • This article provides an overview of how different cities structure their Chief Innovation Officer positions and provides links to offices, projects and additional editorial content.
  • Some innovation officers find their duties merged with traditional CIO responsibilities, as is the case in Chicago, Philadelphia and New York City. Others, like those in Louisville and Nashville, have titles that reveal a link to their jurisdiction’s economic development endeavors.

Toolkits

Bloomberg Philanthropies. January 2014. “Transform Your City through Innovation: The Innovation Delivery Model for Making It Happen.” New York: Bloomberg Philanthropies. http://bloombg.org/120VrKB.

  • In 2011, Bloomberg Philanthropies funded a three-year innovation capacity program in five major United States cities— Atlanta, Chicago, Louisville, Memphis, and New Orleans – in which cities could hire top-level staff to develop and see through the implementation of solutions to top mayoral priorities such as customer service, murder, homelessness, and economic development, using a sequence of steps.
  • The Innovation Delivery Team Playbook describes the Innovation Delivery Model and describes each aspect of the model from how to hire and structure the team, to how to manage roundtables and run competitions.

Participatory sensing: enabling interactive local governance through citizen engagement

Curated on Posted on by Stefaan Verhulst

New White Paper by the Institute for a Broadband-Enabled Society (Australia): “Local government (such as the City of Melbourne) is accountable and responsible for establishment, execution and oversight of strategic objectives and resource management in the metropolis. Faced with a rising population, Council has in place a number of strategic plans to ensure it is able to deliver services that maintain (and ideally improve) the quality of life for its citizens (including residents, workers and visitors). This publication explores participatory sensing (PS) and issues associated with governance in the light of new information gathering capabilities that directly engage citizens in collecting data and providing contextual insight that has the potential to greatly enhance Council operations in managing these environments. Download: Participatory Sensing: Enabling interactive local governance through citizen engagement (pdf: 2.3mb)“

What do businesses really look for in open data?

Curated on Posted on by Stefaan Verhulst

Harvey Lewis in Computer Weekly:  “In 2015, the UK’s primary open data portal, www.data.gov.uk, will be six years old. The portal hosts approximately 20,000 official data sets from central government departments and their agencies, local authorities and other public sector bodies across the country. Just over half of these data sets are available as open data under the Open Government Licence (OGL). Data.gov.uk forms part of an international network of over three hundred open data efforts that have seen not just thousands but millions of data sets worldwide becoming freely available for personal or commercial use. [See http://datacatalogs.org and www.quandle.com].
…simply publishing open data does not guarantee that a business will use it…., if businesses are building new products or services, or relying on the data to inform their strategy, a number of characteristics other than just openness become critical in determining success:

  • Provenance – what is the source of the data and how it was collected? Is it authoritative?
  • Completeness and accuracy – are the examples and features of the data present and correct, and, if not, is the quality understood and documented?
  • Consistency – is the data published in a consistent, easy-to-access format and are any changes documented?
  • Timeliness – is the data available when it is needed for the time periods needed?
  • Richness – does the data contain a level of detail sufficient to answer our questions?
  • Guarantees of availability – will the data continue to be made available in the future?

If these characteristics cannot be guaranteed in open data or are unavailable except under a commercial licence then many businesses would prefer to pay to get them. While some public sector bodies – particularly the Trading Funds – have, over the years, established strong connections with business users of their data and understand their needs implicitly, the Open Data Institute is the first to cement these characteristics into a formal certification scheme for publishers of open data.
A campaign is needed to get publishers to adopt these certificates and to recognise that, economically at least, they are as important as Sir Tim Berners-Lee’s five-star scale for linked open data.  ….”

Restoring Confidence in Open, Shared and Personal Data

Curated on Posted on by Stefaan Verhulst

Report of the UK Digital Government Review: “It is obvious that government needs to be able to use data both to deliver services and to present information to public view. How else would government know which bank account to place a pension payment into, or a citizen know the results of an election or how to contact their elected representatives?

As more and more data is created, preserved and shared in ever-increasing volumes a number of urgent questions are begged: over opportunities and hazards; over the importance of using best-practice techniques, insights and technologies developed in the private sector, academia and elsewhere; over the promises and limitations of openness; and how all this might be articulated and made accessible to the public.

Government has already adopted “open data” (we will discuss this more in the next section) and there are now increasing calls for government to pay more attention to data analytics and so-called “big data” – although the first faltering steps to unlock benefits, here, have often ended in the discovery that using large-scale data is a far more nuanced business than was initially assumed

Debates around government and data have often been extremely high-profile – the NHS care.data [27] debate was raging while this review was in progress – but they are also shrouded in terms that can generate confusion and complexities that are not easily summarized.

In this chapter we will unpick some of these terms and some parts of the debate. This is a detailed and complex area and there is much more that could have been included [28]. This is not an area that can easily be summarized into a simple bullet-pointed list of policies.

Within this report we will use the following terms and definitions, proceeding to a detailed analysis of each in turn:

Type of Data

Definition [29]

Examples
1. Open Data Data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and sharealike Insolvency notices in the London Gazette
Government spending information
Public transport information
Official National Statistics
2. Shared Data Restricted data provided to restricted organisations or individuals for restricted purposes National Pupil Database
NHS care.data
Integrated health and social care
Individual census returns
3. Personal Data Data that relate to a living individual who can be identified from that data. For full legal definition see [30] Health records
Individual tax records
Insolvency notices in the London gazette
National Pupil Database
NB These definitions overlap. Personal data can exist in both open and shared data.

This social productivity will help build future economic productivity; in the meantime it will improve people’s lives and it will enhance our democracy. From our analysis it was clear that there was room for improvement…”

New Tool in Fighting Corruption: Open Data

Curated on Posted on by Stefaan Verhulst

Martin Tisne at Omidyar Network: “Yesterday in Brisbane, the G20 threw its weight behind open data by featuring it prominently in the G20 Anti-Corruption working action plan. Specifically, the action plan calls for effort in three related areas:

(1)   Prepare a G20 compendium of good practices and lessons learned on open data and its application in the fight against corruption
(2)   Prepare G20 Open Data Principles, including identifying areas or sectors where their application is particularly useful
(3)   Complete self‑assessments of G20 country open data frameworks and initiatives

Open data describes information that is not simply public, but that has been published in a manner that makes it easy to access and easy to compare and connect with other information.
This matters for anti corruption – if you are a journalist or a civil society activist investigating bribery and corruption those connections are everything. They tell you that an anonymous person (e.g. ‘Mr Smith’) who owns an obscure company registered in a tax haven is linked to a another company that has been illegally exporting timber from a neighboring country. That the said Mr. Smith is also the son-in-law of the mining minister of yet another country, who herself has been accused of embezzling mining revenues. As we have written elsewhere on this blog, investigative journalists, prosecution authorities, and civil society groups all need access to this linked data for their work.
The action plan also links open data to the wider G20 agenda, citing its impact on the ability of businesses to make better investment decisions. You can find the full detail here….”

Innovating Practice in a Culture of Expertise

Curated on Posted on by Stefaan Verhulst

Aleem Walji at SSI Review: “When I joined the World Bank five years ago to lead a new innovation practice, the organization asked me to help expand the space for experimentation and learning with an emphasis on emergent technologies. But that mandate was intimidating and counter-intuitive in an “expert-driven” culture. Experts want detailed plans, budgets, clear success indicators, and minimal risk. But innovation is about managing risk and navigating uncertainty intelligently. You fail fast and fail forward. It has been a step-by-step process, and the journey is far from over, but the World Bank today sees innovation as essential to achieving its mission.
It’s taught me a lot about seeding innovation in a culture of expertise, including phasing change across approaches to technology, teaming, problem solving, and ultimately leadership.
Innovating technologies: As a newcomer, my goal was not to try to change the World Bank’s culture. I was content to carve out a space where my team could try new things we couldn’t do elsewhere in the institution, learn fast, and create impact. Our initial focus was leveraging technologies with approaches that, if they took root, could be very powerful.
Over the first 18 to 24 months, we served as an incubator for ideas and had a number of successes that built on senior management’s support for increased access to information. The Open Data Initiative, for example, made our trove of information on countries, people, projects, and programs widely available and searchable. To our surprise, people came in droves to access it. We also launched the Mapping for Results initiative, which mapped project results and poverty data to show the relationship between where we lend and where the poor live, and the results of our work. These programs are now mainstream at the World Bank and have penetrated other development institutions….
Innovating teams: The lab idea—phase two—would require collaboration and experimentation in an unprecedented way. For example, we worked with other parts of the World Bank and a number of outside organizations to incubate the Open Development Technology Alliance, now part of the digital engagement unit of the World Bank. It worked to enhance accountability, and improve the delivery and quality of public services through technology-enabled citizen engagement such as using mobile phones, interactive mapping, and social media to draw citizens into collective problem mapping and problem solving….
Innovating problem solving: At the same time, we recognized that we face some really complex problems that the World Bank’s traditional approach of lending to governments and supervising development projects is not solving. For this, we needed another type of lab that innovated the very way we solve problems. We needed a deliberate process for experimenting, learning, iterating, and adapting. But that’s easier said than done. At our core, we are an expert-driven organization with know-how in disciplines ranging from agricultural economics and civil engineering to maternal health and early childhood development. Our problem-solving architecture is rooted in designing technical solutions to complicated problems. Yet the hardest problems in the world defy technical fixes. We work in contexts where political environments shift, leaders change, and conditions on the ground constantly evolve. Problems like climate change, financial inclusion, food security, and youth unemployment demand new ways of solving old problems.
The innovation we most needed was innovation in the leadership architecture of how we confront complex challenges. We share knowledge and expertise on the “what” of reform, but the “how” is what we need most. We need to marry know-how with do-how. We need multiyear, multi-stakeholder, and systems approaches to solving problems. We need to get better at framing and reframing problems, integrative thinking, and testing a range of solutions. We need to iterate and course-correct as we learn what works and doesn’t work in which context. That’s where we are right now with what we call “integrated leadership learning innovation”—phase four. It’s all about shaping an innovative process to address complex problems….”