Three Paradoxes of Big Data

New Paper by Neil M. Richards and Jonathan H. King in the Stanford Law Review Online:Big data is all the rage. Its proponents tout the use of sophisticated analytics to mine large data sets for insight as the solution to many of our society’s problems. These big data evangelists insist that data-driven decisionmaking can now give us better predictions in areas ranging from college admissions to dating to hiring to medicine to national security and crime prevention. But much of the rhetoric of big data contains no meaningful analysis of its potential perils, only the promise. We don’t deny that big data holds substantial potential for the future, and that large dataset analysis has important uses today. But we would like to sound a cautionary note and pause to consider big data’s potential more critically. In particular, we want to highlight three paradoxes in the current rhetoric about big data to help move us toward a more complete understanding of the big data picture. First, while big data pervasively collects all manner of private information, the operations of big data itself are almost entirely shrouded in legal and commercial secrecy. We call this the Transparency Paradox. Second, though big data evangelists talk in terms of miraculous outcomes, this rhetoric ignores the fact that big data seeks to identify at the expense of individual and collective identity. We call this the Identity Paradox. And third, the rhetoric of big data is characterized by its power to transform society, but big data has power effects of its own, which privilege large government and corporate entities at the expense of ordinary individuals. We call this the Power Paradox. Recognizing the paradoxes of big data, which show its perils alongside its potential, will help us to better understand this revolution. It may also allow us to craft solutions to produce a revolution that will be as good as its evangelists predict.”

The US Constitution version 2.0

Luis Ibanez : “After ‘version 1.0’ of the US Constitution was released to the public on Sept 17, 1787 there was remaining discontent among several states regarding the powers assigned to the new Federal government and a lack of protections for fundamental individual freedoms and civil rights.

To fix this bug, the First Continental Congress voted on twelve Constitutional Amendments in September of 1789. Two of them failed to gain enough support and the remaining ten, collectively known as The Bill of Rights, were included in ‘version 2.0’ of the US Constitution, released in 1791.
This refactoring process was open source-minded on multiple levels.
First, the voice of the people (the community) was heard when expressing concern about defects (bugs) in the Constitution. In this case, the bugs related to the lack of sufficient protection for individual civil rights. There was no presumption of perfection or completeness in the US Constitution, and there was a will to improve it and make it better through an open political process.
Second, changes were proposed, discussed, and finally implemented. The discussion of these amendments is equivalent to code reviews that a typical open source software project will go through when adopting substantial changes. Note: the amendements were adopted without having to “fork” the project (the country), though later the country was deeply divided, resulting in the American Civil War in 1861…
As code and law, and community and society, come closer together, taking a fresh look at the history that led us here, sheds a bright light on how we can continue to work together, and how open source principles can continue to change the world for the better.”

Measuring the Internet Economy

New OECD Report: “The Internet began as a way of linking different computers over the phone network, but it now connects billions of users worldwide from wherever they happen to be via portable or fixed devices. The Internet began as an important tool for improving communication but has transformed into a universal technology supporting all virtually sectors across the economy, just like electricity or steam engine did in the past. Given the growing importance of the Internet as a policy tool, the question about the value of the Internet economy becomes particularly relevant.
There is a high level of interest, therefore, in being able to measure the size of the Internet economy as a way to understand the effects of various investment strategies, regulatory rulings and policy decisions. The existing OECD research presented in this report and in the Internet Economy Outlook illustrates the importance of establishing an international definition and the need to develop related policies. According to one of the approaches, at least 3.2% and up to 13.8% of business sector value added in the United States in 2011 could be attributed to Internet-related activities depending on the scope of the definition. It needs to be highlighted that the respective figures for 2010 were 3% and up to 13%. This indicates that the Internet economy has reported a steady growth rate since 2010.”

Participatory Budgeting Around the World

Jay Colburn, from the International Budget Partnership:  “Public participation in budget decision making can occur in many different forms. Participatory budgeting (PB) is an increasingly popular process in which the public is involved directly in making budgetary decisions, most often at the local level. The involvement of community members usually includes identifying and prioritizing the community’s needs and then voting on spending for specific projects.
PB was first developed in Porto Alegre, Brazil, in 1989 as an innovative reform to address the city’s severe inequality. Since then it has spread around the world. Though the specifics of how the PB process works varies depending on the context in which it is implemented, most PB processes have four basic similarities: 1) community members identify spending ideas; 2) delegates are selected to develop spending proposals based on those ideas; 3) residents vote on which proposals to fund; and 4) the government implements the chosen proposals.
During the 1990s PB spread throughout Brazil and across Latin America. Examples of participatory budgeting can now be found in every region of the world, including Central Asia, Europe, and the Middle East. As the use of PB has expanded, it has been adapted in many ways. One example is to incorporate new information and communication technologies as a way to broaden opportunities for participation (see Using Technology to Improve Transparency and Citizen Engagement in this newsletter for more on this topic.)…
There are also a number of different models of PB that have been developed, each with slightly different rules and processes. Using the different models and methods has expanded our knowledge on the potential impacts of PB. In addition to having demonstrable and measurable results on mobilizing public funds for services for the poor, participatory budgeting has also been linked to greater tax compliance, increased demands for transparency, and greater access to budget information and oversight.
However, not all instances of PB are equally successful; there are many variables to consider when weighing the impact of different cases. These can include the level and mechanisms of participation, information accessibility, knowledge of opportunities to participate, political context, and prevailing socioeconomic factors. There is a large and growing literature on the benefits and challenges of PB. The IBP Open Budgets Blog recently featured posts on participatory budgeting initiatives in Peru, Kyrgyzstan, and Kenya. While there are still many lessons to be learned about how PB can be used in different contexts, it is certainly a positive step toward increased citizen engagement in the budget process and influence over how public funds are spent.
For more information and resources on PB, visit the participatory budgeting Facebook group”

Smarter Than You Think: How Technology is Changing Our Minds for the Better

New book by Clive Thompson: “It’s undeniable—technology is changing the way we think. But is it for the better? Amid a chorus of doomsayers, Clive Thompson delivers a resounding “yes.” The Internet age has produced a radical new style of human intelligence, worthy of both celebration and analysis. We learn more and retain it longer, write and think with global audiences, and even gain an ESP-like awareness of the world around us. Modern technology is making us smarter, better connected, and often deeper—both as individuals and as a society.
In Smarter Than You Think Thompson shows that every technological innovation—from the written word to the printing press to the telegraph—has provoked the very same anxieties that plague us today. We panic that life will never be the same, that our attentions are eroding, that culture is being trivialized. But as in the past, we adapt—learning to use the new and retaining what’s good of the old.”

Cyberpsychology and New Media

A thematic reader, edited by Andrew Power, Grainne Kirwan:Cyberpsychology is the study of human interactions with the internet, mobile computing and telephony, games consoles, virtual reality, artificial intelligence, and other contemporary electronic technologies. The field has grown substantially over the past few years and this book surveys how researchers are tackling the impact of new technology on human behaviour and how people interact with this technology.

Examining topics as diverse as online dating, social networking, online communications, artificial intelligence, health-information seeking behaviour, education online, online therapies and cybercrime, Cyberpsychology and New Media book provides an in-depth overview of this burgeoning field, and allows those with little previous knowledge to gain an appreciation of the diversity of the research being undertaken in the area.”

OECD's Revised Guidelines on Privacy

OECD: “Over many decades the OECD has played an important role in promoting respect for privacy as a fundamental value and a condition for the free flow of personal data across borders. The cornerstone of OECD work on privacy is its newly revised Guidelines on the Protection of Privacy and Transborder Flows of Personal Data (2013).
Another key component of work in this area aims to improve cross-border co-operation among privacy law enforcement authorities.  This work produced an OECD Recommendation on Cross-border Co-operation in the Enforcement of Laws Protecting Privacy in 2007 and inspired the formation of the Global Privacy Enforcement Network, to which the OECD provides support.
Other projects have examined privacy notices and considered privacy in the context of horizontal issues such as radio frequency indentification (RFID), digital identity management, and looked at metrics to inform policy making in these areas. The important role of privacy is also addressed in the OECD Recommendation on Principles for Internet Policy Making (2011) and the Seoul Ministerial Declaration on the Future of the Internet Economy (2008).
Current work is examining privacy-related issues raised by large-scale data use and analytics. It is part of a broader project on the data-driven innovation and growth, which already produced a preliminary report identifying key issues.”

Here’s how the Recovery Act became a test case for open data

Andrea Peterson in the Washington Post: “Making sure that government money is spent efficiently and without fraud can be difficult. You need to collect the right data, get the information to the right people, and deal with the sheer volume of projects that need tracking. Open data make the job easier to draw comparisons across programs and agencies. And when data are released to the public, everyone can help be a government watchdog.
When President Obama was first elected in 2008, he promised transparency. Almost immediately after he was sworn into office, he had an opportunity to test that promise with the implementation of the Recovery Act. And it worked…. used geospatial technology to “allow Americans to drill down to their zip codes exactly where government money was being spent in their neighborhood.” It’s this micro-level of attention that increased accountability, according to Devaney.
“The degree of transparency forced them to get it right because they didn’t want to be embarrassed by their neighbors who they knew were going to these Web sites and could see what they were doing with the money.”
As to the second question of what data to collect: “I finally put my foot down and said no more than 100 pieces of data,” Devaney recalls, “So naturally, we came up to 99.” Of course, even with limiting themselves to that number of data points, transparency and fraud prevention was a daunting task, with the 300,000 some grantees to keep tabs on.
But having those data points in an open format was what allowed investigators to use “sophisticated cyber-technology and software to review and analyze Recovery-related data and information for any possible concerns or issues.” And they were remarkably successful on that end. A status report in October, 2010 showed “less than 0.2 percent of all reported awards currently have active fraud investigations.” Indeed, for Devaney’s  tenure leading the board he says the level of fraud hovered somewhere below half of one percent of all awards.”


thomas levine: “There are loads of open data portals There’s even portal about data portals. And each of these portals has loads of datasets.
OpenPrism is my most recent attempt at understanding what is going on in all of these portals. Read on if you want to see why I made it, or just go to the site and start playing with it.

People don’t know much about open data

Nobody seems to know what is in the data portals. Many people know about datasets that are relevant to their work, municipality, &c., but nobody seems to know about the availability of data on broader topics, and nobody seems to have a good way of finding out what is available.
If someone does know any of this, he probably works for an open data portal. Still, he probably doesn’t know much about what is going on in other portals.

Naive search method

One difficulty in discovering open data is the search paradigm.
Open data portals approach searching data as if data were normal prose; your search terms are some keywords, a category, &c., and your results are dataset titles and descriptions.
There are other approaches. For example, AppGen searches for datasets with the same variables as each other, and the results are automatically generated app prototypes.

Siloed open data portals

Another issue is that people tend to use data from only one portal; they use their local government’s portals or their organizations’ portals.
Let me give you a couple examples of why this should maybe be different. Perhaps I’m considering making an app to help people find parking, and I want to see what parking lot data are available before I put much work into the app. Or maybe I want to find all of the data about sewer overflows so that I can expand my initiative to reduce water pollution.
OpenPrism is one small attempt at making it easier to search. Rather than going to all of the different portals and making a separate search for each portal, you type your search in one search bar, and you get results from a bunch of different Socrata, CKAN and Junar portals.”

From public innovation to social innovation in the public sector

A review of the literature by Victor Bekkers, Lars Tummers & William Voorberg: “Innovation is a recurring issue in public administration. In doing so it can be considered as a ‘magic concept’ that is been used to frame the necessary transformation of the public sector in order to improve not only its effectiveness and efficiency but also its legitimacy. Innovation is a concept that inspires people and policy makers because it offers the promise of radical change. As such the desire to innovate the public sector has a long history which is sometimes linked to reform programs in order to meet budget cutbacks, to meet the introduction of new management and governance ideologies (like New Public Management or Open Government) or to meet the introduction of new information and communication technologies (like e- government)…
Our starting point for studying social innovations in the public sector is that social innovations take place in a specific environment in which different actors can be distinguished. These actors collaborate with each other in terms of sharing relevant resources in order to develop and implement new ideas, new ways of working or new ways of organizing. This implies that characteristics of the environment can be seen as a relevant drivers and barriers. These characteristics can either function as a trigger for innovation while at the same time they can also function as relevant constraints. Based on an analysis of the literature, we have found that the following aspects of the environment could function as important drivers and barriers of innovation:
  • The social and political complexity of the environment in which public organizations operate which leads to specific demands that function as an external ‘trigger’ for innovation
  • The characteristics and degree of the legal culture in a country or policy sector
  • The type of governance and state tradition in the country or policy sector
  • The allocation of resources, resource dependency and the quality of relationships within the networks among the involved stakeholders”