Does Open Data Need Journalism?

Curated on Posted on by Stefaan Verhulst

Paper by Jonathan Stoneman at Reuters Institute for Journalism: “The Open Data movement really came into being when President Obama issued his first policy paper, on his first day in office in January 2009. The US government opened up thousands of datasets to scrutiny by the public, by journalists, by policy-makers. Coders and developers were also invited to make the data useful to people and businesses in all manner of ways. Other governments across the globe followed suit, opening up data to their populations.

Opening data in this way has not resulted in genuine openness, save in a few isolated cases. In the USA and a few European countries, developers have created apps and websites which draw on Open Data, but these are not reaching a mass audience.

At the same time, journalists are not seen by government as the end users of these data. Data releases, even in the best cases, are uneven, and slow, and do not meet the needs of journalists. Although thousands of journalists have been learning and adopting the new skills of datajournalism they have tended to work with data obtained through Freedom of Information (FOI) legislation.

Stories which have resulted from datajournalists’ efforts have rarely been front page news; in many cases data-driven stories have ended up as lesser stories on inside pages, or as infographics, which relatively few people look at.

In this context, therefore, Open Data remains outside the mainstream of journalism, and out of the consciousness of the electorate, begging the question, “what are Open Data for?”, or as one developer put it – “if Open Data is the answer, what was the question?” Openness is seen as a badge of honour – scores of national governments have signed pledges to make data open, often repeating the same kind of idealistic official language as the previous announcement of a conversion to openness. But these acts are “top down”, and soon run out of momentum, becoming simply openness for its own sake. Looking at specific examples, the United States is the nearest to a success story: there is a rich ecosystem – made up of government departments, interest groups and NGOs, the media, civil society – which allows data driven projects the space to grow and the airtime to make an impact. (It probably helped that the media in the US were facing an existential challenge urgent enough to force them to embrace new, inexpensive, ways of carrying out investigative reporting).

Elsewhere data are making less impact on journalism. In the UK the new openness is being exploited by a small minority. Where data are made published on the data.gov.uk website they are frequently out of date, incomplete, or of limited new value, so where data do drive stories, these tend to be data released under FOI legislation, and the resulting stories take the form of statistics and/or infographics.

In developing countries where Open Data Portals have been launched with a fanfare – such as Kenya, and more recently Burkina Faso – there has been little uptake by coders, journalists, or citizens, and the number of fresh datasets being published drops to a trickle, and are soon well out of date. Small, apparently randomly selected datasets are soon outdated and inertia sets in.

The British Conservative Party, pledging greater openness in its 2010 manifesto, foresaw armies of “Armchair Auditors” who would comb through the data and present the government with ideas for greater efficiency in the use of public funds. Almost needless to say, these armies have never materialised, and thousands of datasets go unscrutinised by anybody. 2 In countries like Britain large amounts of data are being published but going (probably) unread and unscrutinised by anybody. At the same time, the journalists who want to make use of data are getting what they need through FOI, or even by gathering data themselves. Open Data is thus being bypassed, and could become an irrelevance. Yet, the media could be vital agents in the quest for the release of meaningful, relevant, timely data.

Governments seem in no hurry to expand the “comfort zone” from which they release the data which shows their policies at their most effective, and keeping to themselves data which paints a gloomier picture. Journalists seem likely to remain in their comfort zone, where they make use of FOI and traditional sources of information. For their part, journalists should push for better data and use it more, working in collaboration with open data activists. They need to change the habits of a lifetime and discuss their sources: revealing the source and quality of data used in a story would in itself be as much a part of the advocacy as of the actual reporting.

If Open Data are to be part of a new system of democratic accountability, they need to be more than a gesture of openness. Nor should Open Data remain largely the preserve of companies using them for commercial purposes. Governments should improve the quality and relevance of published data, making them genuinely useful for journalists and citizens alike….(More)”

Push, Pull, and Spill: A Transdisciplinary Case Study in Municipal Open Government

Curated on Posted on by Stefaan Verhulst

Paper by Jan Whittington et al: “Cities hold considerable information, including details about the daily lives of residents and employees, maps of critical infrastructure, and records of the officials’ internal deliberations. Cities are beginning to realize that this data has economic and other value: If done wisely, the responsible release of city information can also release greater efficiency and innovation in the public and private sector. New services are cropping up that leverage open city data to great effect.

Meanwhile, activist groups and individual residents are placing increasing pressure on state and local government to be more transparent and accountable, even as others sound an alarm over the privacy issues that inevitably attend greater data promiscuity. This takes the form of political pressure to release more information, as well as increased requests for information under the many public records acts across the country.

The result of these forces is that cities are beginning to open their data as never before. It turns out there is surprisingly little research to date into the important and growing area of municipal open data. This article is among the first sustained, cross-disciplinary assessments of an open municipal government system. We are a team of researchers in law, computer science, information science, and urban studies. We have worked hand-in-hand with the City of Seattle, Washington for the better part of a year to understand its current procedures from each disciplinary perspective. Based on this empirical work, we generate a set of recommendations to help the city manage risk latent in opening its data….(More)”

Open Data: Six Stories About Impact in the UK

Curated on Posted on by Stefaan Verhulst

Laura Bacon at Omidyar Network: “In 2010, the year the United Kingdom launched its open data portal, a Transparency & Accountability Initiative report highlighted both the promise and potential of open data to improve services and create economic growth.

In the five years since, the UK’s progress in opening its data has been pioneering and swift, but not without challenges and questions about impact. It’s this qualified success that prompted us to commission this report in an effort to understand if the promise and potential of open data are being realized, and, specifically, to…

… explore and document open data’s social, cultural, political, and economic impact;

… shine a light on the range of sectors and ways in which open data can make a difference; and

… profile the open data value chain, including its supply, demand, use, and re-use.

The report’s author, Becky Hogge, finds that open data has had catalytic and significant impact and that time will likely reveal even further value. She also flags critical challenges and obstacles, including closed datasets, valuable data not currently being collected, and important privacy considerations….Download full report here.”

Open government: a new paradigm in social change?

Curated on Posted on by Stefaan Verhulst

Rosie Williams: In a recent speech to the Australian and New Zealand School of Government (ANSOG) annual conference, technology journalist and academic Suelette Drefyus explained the growing ‘information asymmetry’ that characterises the current-day relationship between government and citizenry.

According to Dreyfus:

‘Big Data makes government very powerful in its relationship with the citizen. This is even more so with the rise of intelligent systems, software that increasingly trawls, matches and analyses that Big Data. And it is moving toward making more decisions once made by human beings.’

The role of technology in the delivery of government services gives much food for thought in terms of both its implications for potential good and the potential dangers it may pose. The concept of open government is an important one for the future of policy and democracy in Australia. Open government has at its core a recognition that the world has changed, that the ways people engage and who they engage with has transformed in ways that governments around the world must respond to in both technological and policy terms.

As described in the ANSOG speech, the change within government in how it uses technology is well underway, however in many regards we are at the very beginning of understanding and implementing the potential of data and technology in providing solutions to many of our shared problems. Australia’s pending membership of the Open Government Partnership is integral to how Australia responds to these challenges. Membership of the multi-lateral partnership requires the Australian government to create a National Action Plan based on consultation and demonstrate our credentials in the areas of Fiscal Transparency, Access to Information, Income and Asset Disclosure, and Citizen Engagement.

What are the implications of the National Action Plan for policy consultation formulation, implementation and evaluation? In relative terms, Australia’s history with open government is fairly recent. Policies on open data have seen the roll out of data.gov.au – a repository of data published by government agencies and made available for re-use in efforts such as the author’s own financial transparency site OpenAus.

In this way citizen activity and government come together for the purposes of achieving open government. These efforts express a new paradigm in government and activism where the responsibility for solving the problems of democracy are shared between government and the people as opposed to the government ‘solving’ the problems of a passive, receptive citizenry.

As the famous whistle-blowers have shown, citizens are no longer passive but this new capability also requires a consciousness of the responsibilities and accountability that go along with the powers newly developed by citizen activists through technological change.

The opening of data and communication channels in the formulation of public policy provides a way forward to create both a better informed citizenry and also better informed policy evaluation. When new standards of transparency are applied to wicked problems what shortcomings does this highlight?

This question was tested with my recent request for a basic fact missing from relevant government research and reviews but key to social issues of homelessness and domestic violence….(More)”

Teaching Open Data for Social Movements: a Research Strategy

Curated on Posted on by Stefaan Verhulst

Alan Freihof Tygel and Maria Luiza Machado Campo at the Journal of Community Informatics: “Since the year 2009, the release of public government data in open formats has been configured as one of the main actions taken by national states in order to respond to demands for transparency and participation by the civil society. The United States and theUnited Kingdom were pioneers, and today over 46 countries have their own Open Government Data Portali , many of them fostered by the Open Government Partnership (OGP), an international agreement aimed at stimulating transparency.

The premise of these open data portals is that, by making data publicly available in re-usable formats, society would take care of building applications and services, and gain value from this data (Huijboom & Broek, 2011). According to the same authors, the discourse around open data policies also includes increasing democratic control and participation and strengthening law enforcement.

Several recent works argue that the impact of open data policies, especially the release of open data portals, is still difficult to assess (Davies & Bawa, 2012; Huijboom & Broek, 2011; Zuiderwijk, Janssen, Choenni, Meijer, & Alibaks, 2012). One important consideration is that “The gap between the promise and reality of OGD [Open Government Data] re-use cannot be addressed by technological solutions alone” (Davies, 2012). Therefore, sociotechnical approaches (Mumford, 1987) are mandatory.

The targeted users of open government data lie over a wide range that includes journalists, non-governmental organizations (NGO), civil society organizations (CSO), enterprises, researchers and ordinary citizens who want to audit governments’ actions. Among them, the focus of our research is on social (or grassroots) movements. These are groups of organized citizens at local, national or international level who drive some political action, normally placing themselves in opposition to the established power relations and claiming rights for oppressed groups.

A literature definition gives a social movement as “collective social actions with a socio-political and cultural approach, which enable distinct forms of organizing the population and expressing their demands” (Gohn, 2011).

Social movements have been using data in their actions repertory with several motivations (as can be seen in Table 1 and Listing 1). From our experience, an overview of several cases where social movements use open data reveals a better understanding of reality and a more solid basis for their claims as motivations. Additionally, in some cases data produced by the social movements was used to build a counter-hegemonic discourse based on data. An interesting example is the Citizen Public Depth Audit Movement which takes place in Brazil. This movement, which is part of an international network, claims that “significant amounts registered as public debt do not correspond to money collected through loans to the country” (Fattorelli, 2011), and thus origins of this debt should be proven. According to the movement, in 2014 45% of Brazil’s Federal spend was paid to debt services.

Recently, a number of works tried to develop comparison schemes between open data strategies (Atz, Heath, & Fawcet, 2015; Caplan et al., 2014; Ubaldi, 2013; Zuiderwijk & Janssen, 2014). Huijboom & Broek (2011) listed four categories of instruments applied by the countries to implement their open data policies:

  • voluntary approaches, such as general recommendations,
  • economic instruments,
  • legislation and control, and
  • education and training.

One of the conclusions is that the latter was used to a lesser extent than the others.

Social movements, in general, are composed of people with little experience of informatics, either because of a lack of opportunities or of interest. Although it is recognized that using data is important for a social movement’s objectives, the training aspect still hinders a wider use of it.

In order to address this issue, an open data course for social movements was designed. Besides building a strategy on open data education, the course also aims to be a research strategy to understand three aspects:

  • the motivations of social movements for using open data;
  • the impediments that block a wider and better use; and
  • possible actions to be taken to enhance the use of open data by social movements….(More)”

Open Data Impact: How Zillow Uses Open Data to Level the Playing Field for Consumers

Curated on Posted on by Stefaan Verhulst

Daniel Castro at US Dept of Commerce: “In the mid-2000s, several online data firms began to integrate real estate data with national maps to make the data more accessible for consumers. Of these firms, Zillow was the most effective at attracting users by rapidly growing its database, thanks in large part to open data. Zillow’s success is based, in part, on its ability to create tailored products that blend multiple data sources to answer customer’s questions about the housing market. Zillow’s platform lets customers easily compare neighborhoods and conduct thorough real estate searches through a single portal. This ensures a level playing field of information for home buyers, sellers and real estate professionals.

The system empowers consumers by providing them all the information needed to make well-informed decisions about buying or renting a home. For example, information from the Census Bureau’s American Community Survey helps answer people’s questions about what kind of housing they can afford in any U.S. market. Zillow also creates market analysis reports, which inform consumer about whether it is a good time to buy or sell, how an individual property’s value is likely to fluctuate over time, or whether it is better to rent or to own in certain markets. These reports can even show which neighborhoods are the top buyers’ or sellers’ markets in a given city. Zillow uses a wide range of government data, not just from the Census Bureau, to produce economic analyses and products it then freely provides to the public.

In addition to creating reports from synthesized data, Zillow has made a conscious effort to make raw data more usable. It has combined rental, mortgage, and other data into granular metrics on individual neighborhoods and zip codes. For example, the “Breakeven Horizon” is a metric that gives users a snapshot of how long they would need to own a home in a given area for the accrued cost of buying to be less than renting. Zillow creates this by comparing the up-front costs of buying a home versus the amount of interest that money could generate, and then analyzing how median rents and home values are likely to fluctuate, affecting both values. By creating metrics, rankings, and indices, Zillow makes raw or difficult-to-quantify data readily accessible to the public.

While real estate agents can be instrumental in the process of finding a new home or selling an old one, Zillow and other platforms add value by connecting consumers to a wealth of data, some of which may have been accessible before but was too cumbersome for the average user. Not only does this allow buyers and sellers to make more informed decisions about real estate, but it also helps to balance the share of knowledge. Buyers have more information than ever before on available properties, their valuations for specific neighborhoods, and how those valuations have changed in relation to larger markets. Sellers can use the same types of information to evaluate offers they receive, or decide whether to list their home in the first place. The success that Zillow and other companies like it have achieved in the real estate market is a testament to how effective they have been in harnessing data to address consumers’ needs and it is a marvelous example of the power of open data….(More)”

Good Governance by All Means

Curated on Posted on by Stefaan Verhulst

 at Huffington Post: “Citizens today have higher expectations and demand effective solutions to every day issues and challenges. From climate change to expedient postal services, governments are required to act with transparency and diligence. Public accountability demands us, public servants, to act with almost no margin of error and using the most open and transparent means available to achieve our goals. The name of the game is simple: government efforts should focus on building stronger, better and healthier relationships with civil society. Nobody should be left behind when tailoring public policy. For the Mexican Government, it is crystal clear, that such endeavor is no longer the State’s monopoly and thus, the pressing need for governments to use smarter and more efficient tool boxes, such as the one that the Open Government Partnership (OGP), provides. The buzzword is good governance by all means.

The High Level Segment of the 70th Session of the United Nations General Assembly was a milestone for the open government community. It allowed the 13 countries taking part of the OGP Steering Committee and several civil society organizations to endorse the Joint Declaration: Open Government for the Implementation of the 2030 Agenda for Sustainable Development. This declaration highlights the paramount importance of promoting the principles of open government (transparency, accountability, citizen participation and innovation) as key enablers of the Sustainable Development Goals. The Declaration particularly embraces Agenda 2030’s Goal 16 as a common target for all 66 OGP member countries. Our common goal is to continue building stronger institutions while weaving peaceful and inclusive societies. Our meeting in New York also allowed us to work with key players to develop the Open Data Charter that recognizes the value of having timely, comprehensive, accessible, and comparable data for the promotion of greater citizen engagement triggering development and innovation….(More)

Citizen-Generated Data and Governments: Towards a Collaborative Model

Curated on Posted on by Stefaan Verhulst

Civicus: “…we’re very happy today to launch “Citizen-Generated Data and Governments: Towards a Collaborative Model”.

This piece explores the idea that governments could host and publish citizen-generated data (CGD) themselves, and whether this could mean that data is applied more widely and in a more sustainable way. It was inspired by a recent meeting in Buenos Aires with Argentine civil society organizations and government representatives, hosted by the City of Buenos Aires Innovation and Open Government Lab (Laboratorio de innovación y Gobierno Abierto de la Ciudad de Buenos Aires).

Screen Shot 2015-10-26 at 20.58.06

The meeting was organized to explore how people within government think about citizen-generated data, and discuss what would be needed for them to consider it as a valid method of data generation. One of the most novel and exciting ideas that surfaced was the potential for government open data portals, such as that managed by the Buenos Aires Innovation Lab, to host and publish CGD.

We wrote this report to explore this issue further, looking at existing models of data collaboration and outlining our first thoughts on the benefits and obstacles this kind of model might face. We welcome feedback from those with deeper expertise into different aspects of citizen-generated data, and look forward to refining these thoughts in the future together with the broader community…(More)”

How open company data was used to uncover the powerful elite benefiting from Myanmar’s multi-billion dollar jade industry

Curated on Posted on by Stefaan Verhulst

OpenCorporates: “Today, we’re pleased to release a white paper on how OpenCorporates data was used to uncover the powerful elite benefiting from Myanmar’s multi-billion dollar jade industry, in a ground-breaking report from Global Witness. This investigation is an important case study on how open company data and identifiers are critical tool to uncover corruption and the links between companies and the real people benefitting from it.

This white paper shows how not only was it critical that OpenCorporates had this information (much of the information was removed from the official register during the investigation), but that the fact that it was machine-readable data, available via an API (data service), and programmatically combinable with other data was essential to discover the hidden connections between the key actors and the jade industry. Global Witness was able to analyse this data with the help of Open Knowledge.

In this white paper, we make recommendations about the collection and publishing of statutory company information as open data to facilitate the creation of a hostile environment for corruption by providing a rigorous framework for public scrutiny and due diligence.

You can find the white paper here or read it on Medium.”

Using data to improve the environment

Curated on Posted on by Stefaan Verhulst

Sir Philip Dilley at the UK Environment Agency: “We live in a data rich world. As an engineer I know the power of data in the design and implementation of new urban spaces, iconic buildings and the infrastructure on which we all depend.

Data also is a powerful force in helping us to protect the environment and it can be mined from a variety of sources.

Since the Victorian times naturalists have collected data on the natural world. At the Environment Agency we continue to use local enthusiasts to track rainfall, which we use to feed into and support local projections of flood risk. But the advent of computing power and the Government’s move to open data means we can now all use data in a new and exciting way. The result is a more informed approach to improving the environment and protecting people.

For the last 17 years the Environment Agency has used lasers in planes to map and scan the English landscape from above to help us carry out work such as flood modelling (data now available for everyone to use). The same information has been used to track changing coastal habitats and to help us use the power of nature to adapt to a changing climate.

We’ve used our LIDAR height data together with aerial photography to inform the location and design of major coastal realignment sites. The award-winning Medmerry project, which created 183 hectares of new coastal habitat and protects 348 properties from flooding, was based on this data-led approach.

Those who live near rivers or who use them for sport and recreation know the importance of getting up to date information on river flows. We already provide online services to the public so they can see current warnings and river levels information, but opening our data means everyone can get bespoke information through one postcode or location search.

We are not the only ones seeing the power of environmental data. Data entrepreneurs know how to get accurate and easily accessible information to the public. And that means that we can all make informed choices.FloodAlerts provides a graphical representation of flood warnings and gives localised updates every 15 minutes and Flood Risk Finder app provides flood risk profiles on any property in England, both using data made available for public use by the Environment Agency.

Our bathing waters data directs those who like to swim, surf or paddle with vital information on water quality. The Safer Seas Service app alerts water users when water quality is reduced at beaches and our bathing water data is also used by the Marine Conservation Society’s Good Beach Guide….(More)”