Open data, transparency and accountability


Topic guide by Liz Carolan: “…introduces evidence and lessons learned about open data, transparency and accountability in the international development context. It discusses the definitions, theories, challenges and debates presented by the relationship between these concepts, summarises the current state of open data implementation in international development, and highlights lessons and resources for designing and implementing open data programmes.

Open data involves the release of data so that anyone can access, use and share it. The Open DataCharter (2015) describes six principles that aim to make data easier to find, use and combine:

  • open by default
  • timely and comprehensive
  • accessible and usable
  • comparable and interoperable
  • for improved governance and citizen engagement
  • for inclusive development and innovation

One of the main objectives of making data open is to promote transparency.

Transparency is a characteristic of government, companies, organisations and individuals that are open in the clear disclosure of information, rules, plans, processes and actions. Trans­parency of information is a crucial part of this. Within a development context, transparency and accountability initiatives have emerged over the last decade as a way to address developmental failures and democratic deficits.

There is a strong intersection between open data and transparency as concepts, yet as fields of study and practice, they have remained somewhat separate. This guide draws extensively on analysis and evidence from both sets of literature, beginning by outlining the main concepts and the theories behind the relationships between them.

Data release and transparency are parts of the chain of events leading to accountability.  For open data and transparency initiatives to lead to accountability, the required conditions include:

  • getting the right data published, which requires an understanding of the politics of data publication
  • enabling actors to find, process and use information, and to act on any outputs, which requires an accountability ecosystem that includes equipped and empowered intermediaries
  • enabling institutional or social forms of enforceability or citizens’ ability to choose better services,which requires infrastructure that can impose sanctions, or sufficient choice or official support for citizens

Programmes intended to increase access to information can be impacted by and can affect inequality. They can also pose risks to privacy and may enable the misuse of data for the exploitation of individuals and markets.

Despite a range of international open data initiatives and pressures, developing countries are lagging behind in the implementation of reforms at government level, in the overall availability of data, and in the use of open data for transparency and accountability. What is more, there are signs that ‘open-washing’ –superficial efforts to publish data without full integration with transparency commitments – may be obscuring backsliding in other aspects of accountability.

The topic guide pulls together lessons and guidance from open data, transparency and accountability work,including an outline of technical and non-technical aspects of implementing a government open data initiative. It also lists further resources, tools and guidance….(More)”

Data for Policy: Data Science and Big Data in the Public Sector


Innar Liiv at OXPOL: “How can big data and data science help policy-making? This question has recently gained increasing attention. Both the European Commission and the White House have endorsed the use of data for evidence-based policy making.

Still, a gap remains between theory and practice. In this blog post, I make a number of recommendations for systematic development paths.

RESEARCH TRENDS SHAPING DATA FOR POLICY

‘Data for policy’ as an academic field is still in its infancy. A typology of the field’s foci and research areas are summarised in the figure below.

 

diagram1

 

Besides the ‘data for policy’ community, there are two important research trends shaping the field: 1) computational social science; and 2) the emergence of politicised social bots.

Computational social science (CSS) is an new interdisciplinary research trend in social science, which tries to transform advances in big data and data science into research methodologies for understanding, explaining and predicting underlying social phenomena.

Social science has a long tradition of using computational and agent-based modelling approaches (e.g.Schelling’s Model of Segregation), but the new challenge is to feed real-life, and sometimes even real-time information into those systems to get gain rapid insights into the validity of research hypotheses.

For example, one could use mobile phone call records to assess the acculturation processes of different communities. Such a project would involve translating different acculturation theories into computational models, researching the ethical and legal issues inherent in using mobile phone data and developing a vision for generating policy recommendations and new research hypothesis from the analysis.

Politicised social bots are also beginning to make their mark. In 2011, DARPA solicited research proposals dealing with social media in strategic communication. The term ‘political bot’ was not used, but the expected results left no doubt about the goals…

The next wave of e-government innovation will be about analytics and predictive models.  Taking advantage of their potential for social impact will require a solid foundation of e-government infrastructure.

The most important questions going forward are as follows:

  • What are the relevant new data sources?
  • How can we use them?
  • What should we do with the information? Who cares? Which political decisions need faster information from novel sources? Do we need faster information? Does it come with unanticipated risks?

These questions barely scratch the surface, because the complex interplay between general advancements of computational social science and hovering satellite topics like political bots will have an enormous impact on research and using data for policy. But, it’s an important start….(More)”

When Innovation Goes Wrong


Christian Seelos & Johanna Mair at Stanford Social Innovation Review: “Efforts by social enterprises to develop novel interventions receive a great deal of attention. Yet these organizations often stumble when it comes to turning innovation into impact. As a result, they fail to achieve their full potential. Here’s a guide to diagnosing and preventing several “pathologies” that underlie this failure….

The core purpose of an innovation process is the conversion of uncertainty into knowledge. Or to put it another way: Innovation is essentially a matter of learning. In fact, one critical insight that we have drawn from our research is that effective organizations approach innovation not with an expectation of success but with an expectation of learning. Innovators who expect success from innovation efforts will inevitably encounter disappointment, and the experience of failure will generate a blame culture in their organization that dramatically lowers their chance of achieving positive impact. But a focus on learning creates a sense of progress rather than a sense of failure. The high-impact organizations that we have studied owe much of their success to their wealth of accumulated knowledge—knowledge that often has emerged from failed innovation efforts.

 

Innovation uncertainty has multiple dimensions, and organizations need to be vigilant about addressing uncertainty in all of its forms. (See “Types of Innovation Uncertainty” below.) Let’s take a close look at three aspects of the innovation process that often involve a considerable degree of uncertainty.

Problem formulation | Organizations may incorrectly frame the problem that they aim to solve, and identifying that problem accurately may require several iterations and learning cycles…

Solution development | Even when an organization has an adequate understanding of a problem, it may not be able to access and deploy the resources needed to create an effective and robust solution….

Alignment with identity | Innovation may lead an organization in a direction that does not fit its culture or its sense of its purpose—its sense of “who we are.”…

In short, innovation plus scaling equals impact. Innovation is an investment of resources that creates a new potential; scaling creates impact by enacting that potential. Because innovation creates only the potential for impact, we advocate replacing the assumption that “innovation is good, and more is better” with a more critical view: Innovation, we argue, needs to prove itself on the basis of the impact that it actually creates. The goal is not innovation for its own sake but productive innovation.

Productive innovation depends on two factors: (1) an organization’s capacity for efficiently replacing innovation uncertainty with knowledge, and (2) its ability to scale up innovation outcomes by enhancing its organizational effectiveness. Innovation and scaling thus work together to form an overall social impact creation process. Over time, an investment in innovation—in the work of overcoming uncertainty—yields positive social impact, and the value of such impact will eventually exceed the cost of that investment. But that will be the case only if an organization is able to master the scaling part of this process….

 

 

Focusing on Pathologies

Through our study of social enterprises, we have devised a set of six pathologies—six ways that organizations limit their capacity for productive innovation. From the stage when people first develop (or fail to develop) the idea for an innovation to the stage when scaling efforts take off (or fail to take off), these pathologies adversely affect an organization’s ability to make its way through the social impact creation process. (See “Creating Social Impact: Six Innovation Pathologies to Avoid” below.) Organizations can greatly improve the impact of their innovation efforts by working to prevent or treat these pathologies.

Never getting started | In too many cases, organizations simply fail to invest seriously in the work of innovation. This pathology has many causes. People in organizations may have neither the time nor the incentive to develop or communicate new ideas. Or they may find that their ideas fall on deaf ears. Or they may have a tendency to discuss an idea endlessly—until the problem that gave rise to it has been replaced by another urgent problem or until an opportunity has vanished….

Pursuing too many bad ideas | Organizations in the social sector frequently fall into the habit of embracing a wide variety of ideas for innovation without regard to whether those ideas are sound. The recent obsession with “scientific” evaluation tools such as randomized controlled trials, or RCTs, exemplifies this tendency to favor costly ideas that may or may not deliver real benefits. As with other pathologies, many factors potentially contribute to this one. Funders may push their favorite solutions regardless of how well they understand the problems that those solutions target or how well a solution fits a particular organization. Or an organization may fail to invest in learning about the context of a problem before adopting a solution. Wasting scarce resources on the pursuit of bad ideas creates frustration and cynicism within an organization. It also increases innovation uncertainty and the likelihood of failure….

Stopping too early | In some instances, organizations are unable or unwilling to devote adequate resources to the development of worthy ideas. When resources are scarce and not formally dedicated to innovation processes, project managers will struggle to develop an idea and may have to abandon it prematurely. Too often, they end up taking the blame for failure, and others in their organization ignore the adverse circumstances that caused it. Decision makers then reallocate resources on an ad-hoc basis to other urgent problems or to projects that seem more important. As a result, even promising innovation efforts come to a grinding halt….

Stopping too late | Even more costly than stopping too early is stopping too late. In this pathology, an organization continues an innovation project even after the innovation proves to be ineffective or unworkable. This problem occurs, for example, when an unsuccessful innovation happens to be the pet project of a senior leader who has limited experience. Leaders who have recently joined an organization and who are keen to leave their mark rather than continue what their predecessor has built are particularly likely to engage in this pathology. Another cause of “stopping too late” is the assumption that a project budget needs to be spent. The consequences of this pathology are clear: Organizations expend scarce resources with little hope for success and without gaining any useful knowledge….

Scaling too little | To repeat an essential point that we made earlier: no scaling, no impact. This pathology—which involves a failure to move beyond the initial stages of developing, launching, and testing an intervention—is all too common in the social enterprise field. Thousands of inspired young people want to become social entrepreneurs. But few of them are willing or able to build an organization that can deliver solutions at scale. Too many organizations, therefore, remain small and lack the resources and capabilities required for translating innovation into impact….

Innovating again too soon | Too many organizations rush to launch new innovation projects instead of investing in efforts to scale interventions that they have already developed. The causes of this pathology are fairly well known: People often portray scaling as dull, routine work and innovation as its more attractive sibling. “Innovative” proposals thus attract funders more readily than proposals that focus on scaling. Reinforcing this bias is the preference among many funders for “lean projects” that reduce overhead costs to a minimum. These factors lead organizations to jump opportunistically from one innovation grant to another….(More)”

For Quick Housing Data, Hit Craigslist


Tanvi Misra at CityLab: “…housing researchers can use the Internet bulletin board for a more worthy purpose: as a source of fairly accurate, real-time data on the U.S. rental housing market.

A new paper in the Journal of Planning Education and Research analyzed 11 million Craigslist rental listings posted between May and July 2014 across the U.S. and found a treasure trove of information on regional and local housing trends. “Being able to track rental listings data from Craigslist is really useful for urban planners to take the pulse of [changing neighborhoods] much more quickly,” says Geoff Boeing, a researcher at University of California at Berkeley’s Urban Analytics Lab, who co-authored the paper with Paul Waddell, a Berkeley professor of planning and design.

Here are a couple of big takeaways from their deep dive down the CL rabbit hole:

Overall, Craigslist listings track with HUD data (except when they don’t)

The researchers compared median rents in different Craigslist domains (metropolitan areas, essentially) to the corresponding Housing and Urban Development median rents. In New Orleans and Oklahoma City, the posted and the official rents were very similar. But in other metros, they diverged significantly. In Las Vegas, for example, the Craigslist median rent was lower than the HUD median rent, but in New York, it was much, much higher.

“That’s important for local planners to be careful with because there are totally different cultures and ways that Craigslist is used in different cities,” Boeing explains. “The economies of the cities could very much affect how rentals are being posted. If they’re posting it higher [on Craigslist], they may negotiate down eventually. Or, if they’re posting it low, they could be expecting a bidding war with a bunch of tenants coming in.” …(More)”

How the Federal Government is thinking about Artificial Intelligence


Mohana Ravindranath at NextGov: “Since May, the White House has been exploring the use of artificial intelligence and machine learning for the public: that is, how the federal government should be investing in the technology to improve its own operations. The technologies, often modeled after the way humans take in, store and use new information, could help researchers find patterns in genetic data or help judges decide sentences for criminals based on their likelihood to end up there again, among other applications. …

Here’s a look at how some federal groups are thinking about the technology:

  • Police data: At a recent White House workshop, Office of Science and Technology Policy Senior Adviser Lynn Overmann said artificial intelligence could help police departments comb through hundreds of thousands of hours of body-worn camera footage, potentially identifying the police officers who are good at de-escalating situations. It also could help cities determine which individuals are likely to end up in jail or prison and officials could rethink programs. For example, if there’s a large overlap between substance abuse and jail time, public health organizations might decide to focus their efforts on helping people reduce their substance abuse to keep them out of jail.
  • Explainable artificial intelligence: The Pentagon’s research and development agency is looking for technology that can explain to analysts how it makes decisions. If people can’t understand how a system works, they’re not likely to use it, according to a broad agency announcement from the Defense Advanced Research Projects Agency. Intelligence analysts who might rely on a computer for recommendations on investigative leads must “understand why the algorithm has recommended certain activity,” as do employees overseeing autonomous drone missions.
  • Weather detection: The Coast Guard recently posted its intent to sole-source a contract for technology that could autonomously gather information about traffic, crosswind, and aircraft emergencies. That technology contains built-in artificial intelligence technology so it can “provide only operational relevant information.”
  • Cybersecurity: The Air Force wants to make cyber defense operations as autonomous as possible, and is looking at artificial intelligence that could potentially identify or block attempts to compromise a system, among others.

While there are endless applications in government, computers won’t completely replace federal employees anytime soon….(More)”

Rethinking Nudge: Libertarian paternalism and classical utilitarianism


Hiroaki Itai, Akira Inoue, and Satoshi Kodama in Special Issue on Nudging of The Tocqueville Review/La revue Tocqueville: “Recently, libertarian paternalism has been intensely debated. It recommends us to employ policies and practices that “nudge” ordinary people to make better choices without forcing them to do so. Nudging policies and practices have penetrated our society, in cases like purchasing life insurance or a residence. They are also used for preventing people from addictive acts that may be harmful to them in the long run, such as having too much sugary or fatty food. In nudging people to act rationally, various kinds of cognitive effects impacting the consumers’ decision-making process should be considered, given the growing influence of consumer advertising. Since libertarian paternalism makes use of such effects in light of the recent development of behavioral economics and cognitive psychology in a principled manner, libertarian paternalism and its justification of nudges attract our attention as an approach providing a normative guidance for our action. 

This paper has two aims: the first is to examine whether libertarian paternalism can give an appropriate theoretical foundation to the idea and practice of nudges. The second is to show that utilitarianism, or, more precisely, the classical version of utilitarianism, treats nudges in a more consistent and plausible manner. To achieve these two aims, first of all, we dwell on how Cass Sunstein—one of the founder of libertarian paternalism—misconceives Mill’s harm principle, and that this may prompt us to see that utilitarianism can reasonably legitimate nudging policies (section one). We then point to two biases that embarrass libertarian paternalism (the scientism bias and the dominant-culture bias), which we believe stem from the fact that libertarian paternalism assumes the informed preference satisfaction view of welfare (section two). We finally argue that classical utilitarianism not only can overcome the two biases, but can also reasonably endorse any system monitoring a choice architect to discharge his or her responsibility (section three)….(More)”

Smart Economy in Smart Cities


Book edited by Vinod Kumar, T. M.: “The present book highlights studies that show how smart cities promote urban economic development. The book surveys the state of the art of Smart City Economic Development through a literature survey. The book uses 13 in depth city research case studies in 10 countries such as the North America, Europe, Africa and Asia to explain how a smart economy changes the urban spatial system and vice versa. This book focuses on exploratory city studies in different countries, which investigate how urban spatial systems adapt to the specific needs of smart urban economy. The theory of smart city economic development is not yet entirely understood and applied in metropolitan regional plans. Smart urban economies are largely the result of the influence of ICT applications on all aspects of urban economy, which in turn changes the land-use system. It points out that the dynamics of smart city GDP creation takes ‘different paths,’ which need further empirical study, hypothesis testing and mathematical modelling. Although there are hypotheses on how smart cities generate wealth and social benefits for nations, there are no significant empirical studies available on how they generate urban economic development through urban spatial adaptation.  This book with 13 cities research studies is one attempt to fill in the gap in knowledge base….(More)”

Make Data Sharing Routine to Prepare for Public Health Emergencies


Jean-Paul Chretien, Caitlin M. Rivers, and Michael A. Johansson in PLOS Medicine: “In February 2016, Wellcome Trust organized a pledge among leading scientific organizations and health agencies encouraging researchers to release data relevant to the Zika outbreak as rapidly and widely as possible [1]. This initiative echoed a September 2015 World Health Organization (WHO) consultation that assessed data sharing during the recent West Africa Ebola outbreak and called on researchers to make data publicly available during public health emergencies [2]. These statements were necessary because the traditional way of communicating research results—publication in peer-reviewed journals, often months or years after data collection—is too slow during an emergency.

The acute health threat of outbreaks provides a strong argument for more complete, quick, and broad sharing of research data during emergencies. But the Ebola and Zika outbreaks suggest that data sharing cannot be limited to emergencies without compromising emergency preparedness. To prepare for future outbreaks, the scientific community should expand data sharing for all health research….

Open data deserves recognition and support as a key component of emergency preparedness. Initiatives to facilitate discovery of datasets and track their use [4042]; provide measures of academic contribution, including data sharing that enables secondary analysis [43]; establish common platforms for sharing and integrating research data [44]; and improve data-sharing capacity in resource-limited areas [45] are critical to improving preparedness and response.

Research sponsors, scholarly journals, and collaborative research networks can leverage these new opportunities with enhanced data-sharing requirements for both nonemergency and emergency settings. A proposal to amend the International Health Regulations with clear codes of practice for data sharing warrants serious consideration [46]. Any new requirements should allow scientists to conduct and communicate the results of secondary analyses, broadening the scope of inquiry and catalyzing discovery. Publication embargo periods, such as one under consideration for genetic sequences of pandemic-potential influenza viruses [47], may lower barriers to data sharing but may also slow the timely use of data for public health.

Integrating open science approaches into routine research should make data sharing more effective during emergencies, but this evolution is more than just practice for emergencies. The cause and context of the next outbreak are unknowable; research that seems routine now may be critical tomorrow. Establishing openness as the standard will help build the scientific foundation needed to contain the next outbreak.

Recent epidemics were surprises—Zika and chikungunya sweeping through the Americas; an Ebola pandemic with more than 10,000 deaths; the emergence of severe acute respiratory syndrome and Middle East respiratory syndrome, and an influenza pandemic (influenza A[H1N1]pdm09) originating in Mexico—and we can be sure there are more surprises to come. Opening all research provides the best chance to accelerate discovery and development that will help during the next surprise….(More)”

Why Zika, Malaria and Ebola should fear analytics


Frédéric Pivetta at Real Impact Analytics:Big data is a hot business topic. It turns out to be an equally hot topic for the non profit sector now that we know the vital role analytics can play in addressing public health issues and reaching sustainable development goals.

Big players like IBM just announced they will help fight Zika by analyzing social media, transportation and weather data, among other indicators. Telecom data takes it further by helping to predict the spread of disease, identifying isolated and fragile communities and prioritizing the actions of aid workers.

The power of telecom data

Human mobility contributes significantly to epidemic transmission into new regions. However, there are gaps in understanding human mobility due to the limited and often outdated data available from travel records. In some countries, these are collected by health officials in the hospitals or in occasional surveys.

Telecom data, constantly updated and covering a large portion of the population, is rich in terms of mobility insights. But there are other benefits:

  • it’s recorded automatically (in the Call Detail Records, or CDRs), so that we avoid data collection and response bias.
  • it contains localization and time information, which is great for understanding human mobility.
  • it contains info on connectivity between people, which helps understanding social networks.
  • it contains info on phone spending, which allows tracking of socio-economic indicators.

Aggregated and anonymized, mobile telecom data fills the public data gap without questioning privacy issues. Mixing it with other public data sources results in a very precise and reliable view on human mobility patterns, which is key for preventing epidemic spreads.

Using telecom data to map epidemic risk flows

So how does it work? As in any other big data application, the challenge is to build the right predictive model, allowing decision-makers to take the most appropriate actions. In the case of epidemic transmission, the methodology typically includes five steps :

  • Identify mobility patterns relevant for each particular disease. For example, short-term trips for fast-spreading diseases like Ebola. Or overnight trips for diseases like Malaria, as it spreads by mosquitoes that are active only at night. Such patterns can be deduced from the CDRs: we can actually find the home location of each user by looking at the most active night tower, and then tracking calls to identify short or long-term trips. Aggregating data per origin-destination pairs is useful as we look at intercity or interregional transmission flows. And it protects the privacy of individuals, as no one can be singled out from the aggregated data.
  • Get data on epidemic incidence, typically from local organisations like national healthcare systems or, in case of emergency, from NGOs or dedicated emergency teams. This data should be aggregated on the same level of granularity than CDRs.
  • Knowing how many travelers go from one place to another, for how long, and the disease incidence at origin and destination, build an epidemiological model that can account for the way and speed of transmission of the particular disease.
  • With an import/export scoring model, map epidemic risk flows and flag areas that are at risk of becoming the new hotspots because of human travel.
  • On that base, prioritize and monitor public health measures, focusing on restraining mobility to and from hotspots. Mapping risk also allows launching prevention campaigns at the right places and setting up the necessary infrastructure on time. Eventually, the tool reduces public health risks and helps stem the epidemic.

That kind of application works in a variety of epidemiological contexts, including Zika, Ebola, Malaria, Influenza or Tuberculosis. No doubt the global boom of mobile data will proof extraordinarily helpful in fighting these fierce enemies….(More)”

Open Data for Social Change and Sustainable Development


Special issue of the Journal of Community Informatics edited by Raed M. Sharif and Francois Van Schalkwyk: “As the second phase of the Emerging Impacts of Open Data in Developing Countries (ODDC) drew to a close, discussions started on a possible venue for publishing some of the papers that emerged from the research conducted by the project partners. In 2012 the Journal of Community Informatics published a special issue titled ‘Community Informatics and Open Government Data’. Given the journal’s previous interest in the field of open data, its established reputation and the fact that it is a peer-reviewed open access journal, the Journal of Community Informatics was approached and agreed to a second special issue with a focus on open data. A closed call for papers was sent out to the project research partners. Shortly afterwards, the first Open Data Research Symposium was held ahead of the International Open Data Conference 2015 in Ottawa, Canada. For the first time, a forum was provided to academics and researchers to present papers specifically on open data. Again there were discussions about an appropriate venue to publish selected papers from the Symposium. The decision was taken by the Symposium Programme Committee to invite the twenty plus presenters to submit full papers for consideration in the special issue.

The seven papers published in this special issue are those that were selected through a double-blind peer review process. Researchers are often given a rough ride by open data advocates – the research community is accused of taking too long, not being relevant enough and of speaking in tongues unintelligible to social movements and policy-makers. And yet nine years after the ground-breaking meeting in Sebastopol at which the eight principles of open government data were penned, seven after President Obama injected political legitimacy into a movement, and five after eleven nation states formed the global Open Government Partnership (OGP), which has grown six-fold in membership; an email crosses our path in which the authors of a high-level report commit to developing a comprehensive understanding of a continental open data ecosystem through an examination of open data supply. Needless to say, a single example is not necessarily representative of global trends in thinking about open data. Yet, the focus on government and on the supply of open data by open data advocates – with little consideration of open data use, the differentiation of users, intermediaries, power structures or the incentives that propel the evolution of ecosystems – is still all too common. Empirical research has already revealed the limitations of ‘supply it and they will use it’ open data practices, and has started to fill critical knowledge gaps to develop a more holistic understanding of the determinants of effective open data policy and practice. As open data policies and practices evolve, the need to capture the dynamics of this evolution and to trace unfolding outcomes becomes critical to advance a more efficient and progressive field of research and practice. The trajectory of the existing body of literature on open data and the role of public authorities, both local and national, in the provision of open data

As open data policies and practices evolve, the need to capture the dynamics of this evolution and to trace unfolding outcomes becomes critical to advance a more efficient and progressive field of research and practice. The trajectory of the existing body of literature on open data and the role of public authorities, both local and national, in the provision of open data is logical and needed in light of the central role of government in producing a wide range of types and volumes of data. At the same time, the complexity of open data ecosystem and the plethora of actors (local, regional and global suppliers, intermediaries and users) makes a compelling case for opening avenues for more diverse discussion and research beyond the supply of open data. The research presented in this special issue of the Journal of Community Informatics touches on many of these issues, sets the pace and contributes to the much-needed knowledge base required to promote the likelihood of open data living up to its promise. … (More)”