Selected Readings on Big Data

Curated on Posted on by Andrew Young

The Living Library’s Selected Readings series seeks to build a knowledge base on innovative approaches for improving the effectiveness and legitimacy of governance. This curated and annotated collection of recommended works on the topic of big data was originally published in 2014.

Big Data refers to the wide-scale collection, aggregation, storage, analysis and use of data. Government is increasingly in control of a massive amount of raw data that, when analyzed and put to use, can lead to new insights on everything from public opinion to environmental concerns. The burgeoning literature on Big Data argues that it generates value by: creating transparency; enabling experimentation to discover needs, expose variability, and improve performance; segmenting populations to customize actions; replacing/supporting human decision making with automated algorithms; and innovating new business models, products and services. The insights drawn from data analysis can also be visualized in a manner that passes along relevant information, even to those without the tech savvy to understand the data on its own terms (see The GovLab Selected Readings on Data Visualization).

Selected Reading List (in alphabetical order)

Annotated Selected Reading List (in alphabetical order)

Australian Government Information Management Office. The Australian Public Service Big Data Strategy: Improved Understanding through Enhanced Data-analytics Capability Strategy Report. August 2013. http://bit.ly/17hs2xY.

  • This Big Data Strategy produced for Australian Government senior executives with responsibility for delivering services and developing policy is aimed at ingraining in government officials that the key to increasing the value of big data held by government is the effective use of analytics. Essentially, “the value of big data lies in [our] ability to extract insights and make better decisions.”
  • This positions big data as a national asset that can be used to “streamline service delivery, create opportunities for innovation, identify new service and policy approaches as well as supporting the effective delivery of existing programs across a broad range of government operations.”

Bollier, David. The Promise and Peril of Big Data. The Aspen Institute, Communications and Society Program, 2010. http://bit.ly/1a3hBIA.

  • This report captures insights from the 2009 Roundtable exploring uses of Big Data within a number of important consumer behavior and policy implication contexts.
  • The report concludes that, “Big Data presents many exciting opportunities to improve modern society. There are incalculable opportunities to make scientific research more productive, and to accelerate discovery and innovation. People can use new tools to help improve their health and well-being, and medical care can be made more efficient and effective. Government, too, has a great stake in using large databases to improve the delivery of government services and to monitor for threats to national security.
  • However, “Big Data also presents many formidable challenges to government and citizens precisely because data technologies are becoming so pervasive, intrusive and difficult to understand. How shall society protect itself against those who would misuse or abuse large databases? What new regulatory systems, private-law innovations or social practices will be capable of controlling anti-social behaviors–and how should we even define what is socially and legally acceptable when the practices enabled by Big Data are so novel and often arcane?”

Boyd, Danah and Kate Crawford. “Six Provocations for Big Data.” A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society. September 2011. http://bit.ly/1jJstmz.

  • In this paper, Boyd and Crawford raise challenges to unchecked assumptions and biases regarding big data. The paper makes a number of assertions about the “computational culture” of big data and pushes back against those who consider big data to be a panacea.
  • The authors’ provocations for big data are:
    • Automating Research Changes the Definition of Knowledge
    • Claims to Objectivity and Accuracy are Misleading
    • Big Data is not always Better Data
    • Not all Data is Equivalent
    • Just Because it is accessible doesn’t make it ethical
    • Limited Access to Big Data creates New Digital Divide

The Economist Intelligence Unit. Big Data and the Democratisation of Decisions. October 2012. http://bit.ly/17MpH8L.

  • This report from the Economist Intelligence Unit focuses on the positive impact of big data adoption in the private sector, but its insights can also be applied to the use of big data in governance.
  • The report argues that innovation can be spurred by democratizing access to data, allowing a diversity of stakeholders to “tap data, draw lessons and make business decisions,” which in turn helps companies and institutions respond to new trends and intelligence at varying levels of decision-making power.

Manyika, James, Michael Chui, Brad Brown, Jacques Bughin, Richard Dobbs, Charles Roxburgh, and Angela Hung Byers. Big Data: The Next Frontier for Innovation, Competition, and Productivity.  McKinsey & Company. May 2011. http://bit.ly/18Q5CSl.

  • This report argues that big data “will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus, and that “leaders in every sector will have to grapple with the implications of big data.” 
  • The report offers five broad ways in which using big data can create value:
    • First, big data can unlock significant value by making information transparent and usable at much higher frequency.
    • Second, as organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days, and therefore expose variability and boost performance.
    • Third, big data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services.
    • Fourth, big sophisticated analytics can substantially improve decision-making.
    • Finally, big data can be used to improve the development of the next generation of products and services.

The Partnership for Public Service and the IBM Center for The Business of Government. “From Data to Decisions II: Building an Analytics Culture.” October 17, 2012. https://bit.ly/2EbBTMg.

  • This report discusses strategies for better leveraging data analysis to aid decision-making. The authors argue that, “Organizations that are successful at launching or expanding analytics program…systematically examine their processes and activities to ensure that everything they do clearly connects to what they set out to achieve, and they use that examination to pinpoint weaknesses or areas for improvement.”
  • While the report features many strategies for government decisions-makers, the central recommendation is that, “leaders incorporate analytics as a way of doing business, making data-driven decisions transparent and a fundamental approach to day-to-day management. When an analytics culture is built openly, and the lessons are applied routinely and shared widely, an agency can embed valuable management practices in its DNA, to the mutual benet of the agency and the public it serves.”

TechAmerica Foundation’s Federal Big Data Commission. “Demystifying Big Data: A Practical Guide to Transforming the Business of Government.” 2013. http://bit.ly/1aalUrs.

  • This report presents key big data imperatives that government agencies must address, the challenges and the opportunities posed by the growing volume of data and the value Big Data can provide. The discussion touches on the value of big data to businesses and organizational mission, presents case study examples of big data applications, technical underpinnings and public policy applications.
  • The authors argue that new digital information, “effectively captured, managed and analyzed, has the power to change every industry including cyber security, healthcare, transportation, education, and the sciences.” To ensure that this opportunity is realized, the report proposes a detailed big data strategy framework with the following steps: define, assess, plan, execute and review.

World Economic Forum. “Big Data, Big Impact: New Possibilities for International Development.” 2012. http://bit.ly/17hrTKW.

  • This report examines the potential for channeling the “flood of data created every day by the interactions of billions of people using computers, GPS devices, cell phones, and medical devices” into “actionable information that can be used to identify needs, provide services, and predict and prevent crises for the benefit of low-income populations”
  • The report argues that, “To realise the mutual benefits of creating an environment for sharing mobile-generated data, all ecosystem actors must commit to active and open participation. Governments can take the lead in setting policy and legal frameworks that protect individuals and require contractors to make their data public. Development organisations can continue supporting governments and demonstrating both the public good and the business value that data philanthropy can deliver. And the private sector can move faster to create mechanisms for the sharing data that can benefit the public.”

Prospects for Online Crowdsourcing of Social Science Research Tasks: A Case Study Using Amazon Mechanical Turk

Curated on Posted on by Stefaan Verhulst

New paper by Catherine E. Schmitt-Sands and Richard J. Smith: “While the internet has created new opportunities for research, managing the increased complexity of relationships and knowledge also creates challenges. Amazon.com has a Mechanical Turk service that allows people to crowdsource simple tasks for a nominal fee. The online workers may be anywhere in North America or India and range in ability. Social science researchers are only beginning to use this service. While researchers have used crowdsourcing to find research subjects or classify texts, we used Mechanical Turk to conduct a policy scan of local government websites. This article describes the process used to train and ensure quality of the policy scan. It also examines choices in the context of research ethics.”

The Effective Use of Crowdsourcing in E-Governance

Curated on Posted on by Stefaan Verhulst
Paper by Jayakumar Sowmya and Hussain Shafiq Pyarali: “The rise of Web 2.0 paradigm has empowered the Internet users to share information and generate content on social networking and media sharing platforms such as wikis and blogs. The trend of harnessing the wisdom of public using Web 2.0 distributed networks through open calls is termed as ‘Crowdsourcing’. In addition to businesses, this powerful idea of using collective intelligence or the ‘wisdom of crowd’ applies to different situations, such as in governments and non-profit organizations which have started utilizing crowdsourcing as an essential problem -solving tool. In addition, the widespread and easy access to technologies such as the Internet, mobile phones and other communication devices has resulted in an exponential growth in the use of crowdsourcing for government policy advocacy, e-democracy and e-governance during the past decade. However, utilizing collective intelligence and efforts of public to find solutions to real life problems using web 2.0 tools does come with its share of associated challenges and limitations. This paper aims at identifying and examining the value-adding strategies which contribute to the success of crowdsourcing in e-governance. The qualitative case study analysis and emphatic design methodology are employed to evaluate the effectiveness of the identified strategic and functional components, by analyzing the characteristics of some of the notable cases of crowdsourcing in e-governance and the findings are tabulated and discussed. The paper concludes with the limitations and the implications for future research”.

Organizational Innovation in Public Services: Forms and Governance

Curated on Posted on by Stefaan Verhulst

New edited book by Pekka Valkama, Stephen James Bailey, Ari-Veikko Anttiroiko: “Reforming public services has become an integral part of instituting austerity measures as governments around the world struggle to balance the books in the wake of the financial crisis. Vital public services and government departments have been given the seemingly impossible task of delivering better services to the public while receiving less funding. This excellent and highly original collection brings together contributors from across the globe to explore and analyse innovational methods aimed at helping overburdened and under-funded public services cope with the demands of austerity and continue to deliver high quality services to the public. In the process this book develops new theoretical models and analyses case studies to provide an important and timely insight into how to reform public services across the globe…
Table of Contents:
1. Contexts and Challenges of Organisational Innovation in Public Services
2. Supporting Organisational Innovation in the Public Sector: Creative Councils in England
3. Analysis Organisational Innovation in Public Services: Conceptual and Theoretical Issues
4. Agentifcation Processes and Agency Governance: Organisational Innovation at a Global Scale?
5. Corporatisation as Organisational Innovation
6. Mutulatisation and Public Services
7. Organisational Innovation in Public Procurement in Scotland: The Scottish Futures Trust (SFT)
8. Outsourcing Public Services: Process Innovation in Dutch Municipalities
9. Governance of Public Service Companies: Australian Cases and Examples
10. Governance of Social Enterprises as Producers of Public Services
11. Championing and Governing UK Public Service Mutuals
12. Improving Governance Arrangements for Academic Entrepreneurships
13. Governance and Accountability of Joint Ventures: A Swedish Case Study
14. Contractual Governance: A Social Learning Perspective
15. Lessons for the Governance of Organisational Innovations”

The New Eye of Government: Citizen Sentiment Analysis in Social Media

Curated on Posted on by Stefaan Verhulst
New paper by R. Arunachalam and S. Sarkar: “Governments across the world facing unique challenges today than ever before. In recent time, Arab Spring
phenomenon is an example of how Governments can be impacted if they ignore citizen sentiment. It is a growing trend that Governments are trying to move closer to the citizen-centric model, where the priorities and services would be driven according to citizen needs rather than Government capability. Such trends are
forcing the Governments in rethinking and reshaping their policies in citizen interactions. New disruptive technologies like cloud, mobile etc. are opening new opportunities to the Governments to enable innovations in such interactions.
The advent of Social Media is a recent addition to such disruptive socio-technical enablers. Governments are fast realizing that it can be a great vehicle to get closer to the citizens. It can provide deep insight in what citizens want. Thus, in the current gloomy climate of world economy today, Governments can reorganize and reprioritize the allocation limited funds, thereby creating maximum impact on citizens’ life. Building such insight is a non-trivial task because of the huge
volume of information that social media can generate. However, Sentiment Analysis or Opinion Mining can be a useful vehicle in this journey.
In this work, we presented a model and case study to analyze citizen sentiment from social media in helping the Governments to take decisions.”

Collaboration Between Government and Outreach Organizations: A Case Study of the Department of Veterans Affairs

Curated on Posted on by Stefaan Verhulst

“In this report, Drs. Lael Keiser and Susan Miller examine the critical role of non-governmental outreach organizations in assisting government agencies to determine benefit eligibility of citizens applying for services.  Many non-profits and other organizations help low-income applicants apply for Social Security, Medicaid, and the Supplemental Nutritional Assistance Program (SNAP, or food stamps).
Some outreach organizations help veterans navigate the complexity of the veterans disability benefits program.  These organizations include the American Legion, the Disabled American Veterans, and the Veterans of Foreign Wars, as well as state government-run veterans agencies.  Drs. Keiser and Miller interviewed dozens of managers from the Department of Veterans Affairs (VA) and outreach organizations about their interactions in helping veterans.  They found “there is indeed effective collaboration” and that these organizations serve a key role for veterans in processing their claims.  These organizations also help lighten the workload of VA benefit examiners by ensuring the paperwork is in order in advance, as well as serving as a communications conduit.
Drs. Keiser and Miller found variations in the effectiveness of the relationships between VA and outreach organization staffs and identified best practices for increasing effectiveness.  These lessons can be applied to other agencies that interactive frequently with outreach organizations that assist citizens in navigating the complexity of applying for various government benefit programs.
Listen to the interview on Federal News Radio.”

From Networked Publics to Issue Publics: Reconsidering the Public/Private Distinction in Web Science

Curated on Posted on by Stefaan Verhulst

New paper by Andreas Birkbak: “As an increasing part of everyday life becomes connected with the web in many areas of the globe, the question of how the web mediates political processes becomes still more urgent. Several scholars have started to address this question by thinking about the web in terms of a public space. In this paper, we aim to make a twofold contribution towards the development of the concept of publics in web science. First, we propose that although the notion of publics raises a variety of issues, two major concerns continue to be user privacy and democratic citizenship on the web. Well-known arguments hold that the complex connectivity of the web puts user privacy at risk and enables the enclosure of public debate in virtual echo chambers. Our first argument is that these concerns are united by a set of assumptions coming from liberal political philosophy that are rarely made explicit. As a second contribution, this paper points towards an alternative way to think about publics by proposing a pragmatist reorientation of the public/private distinction in web science, away from seeing two spheres that needs to be kept separate, towards seeing the public and the private as something that is continuously connected. The theoretical argument is illustrated by reference to a recently published case study of Facebook groups, and future research agendas for the study of web-mediated publics are proposed.”

Open Government in a Digital Age

Curated on Posted on by Stefaan Verhulst

An essay by Jonathan Reichental (Chief Information Officer of the City of Palo Alto) and Sheila Tucker (Assistant to the City Manager) that summarizes work done at the City that resulted in the Thomas H. Muehlenback Award for Excellent in Local Government: “Government is in a period of extraordinary change. Demographics are shifting. Fiscal constraints continue to challenge service delivery. Communities are becoming more disconnected with one another and their governments, and participation in civic affairs is rapidly declining. Adding to the complexities, technology is rapidly changing the way cities provide services, and conduct outreach and civic engagement. Citizens increasingly expect to engage with their government in much the same way they pay bills online or find directions using their smartphone where communication is interactive and instantaneous. The role of government of course is more complicated than simply improving transactions.
To help navigate these challenges, the City of Palo Alto has focused its effort on new ways of thinking and acting by leveraging our demographic base, wealth of intellectual talent and entrepreneurial spirit to engage our community in innovative problem solving. The City’s historic advantages in innovative leadership create a compelling context to push the possibilities of technology to solve civic challenges.
This case study examines how Palo Alto is positioning itself to maximize the use of technology to build a leading Digital City and make local government more inclusive, transparent and engage a broader base of its community in civic affairs….”

A promising phenomenon of open data: A case study of the Chicago open data project

Curated on Posted on by Stefaan Verhulst

Paper by Maxat Kassen in Government Information Quarterly: “This article presents a case study of the open data project in the Chicago area. The main purpose of the research is to explore empowering potential of an open data phenomenon at the local level as a platform useful for promotion of civic engagement projects and provide a framework for future research and hypothesis testing. Today the main challenge in realization of any e-government projects is a traditional top–down administrative mechanism of their realization itself practically without any input from members of the civil society. In this respect, the author of the article argues that the open data concept realized at the local level may provide a real platform for promotion of proactive civic engagement. By harnessing collective wisdom of the local communities, their knowledge and visions of the local challenges, governments could react and meet citizens’ needs in a more productive and cost-efficient manner. Open data-driven projects that focused on visualization of environmental issues, mapping of utility management, evaluating of political lobbying, social benefits, closing digital divide, etc. are only some examples of such perspectives. These projects are perhaps harbingers of a new political reality where interactions among citizens at the local level will play an more important role than communication between civil society and government due to the empowering potential of the open data concept.”

The Participatory Turn: Participatory Budgeting Comes to America

Curated on Posted on by Stefaan Verhulst

Thesis by Hollie Russon Gilman: “Participatory Budgeting (PB) has expanded to over 1,500 municipalities worldwide since its inception in Porto Alege, Brazil in 1989 by the leftist Partido dos Trabalhadores (Workers’ Party). While PB has been adopted throughout the world, it has yet to take hold in the United States. This dissertation examines the introduction of PB to the United States with the first project in Chicago in 2009, and proceeds with an in-depth case study of the largest implementation of PB in the United States: Participatory Budgeting in New York City. I assess the outputs of PB in the United States including deliberations, governance, and participation. I argue that PB produces better outcomes than the status quo budget process in New York City, while also transforming how those who participate understand themselves as citizens, constituents, Council members, civil society leaders and community stakeholders. However, there are serious challenges to participation, including high costs of engagement, process exhaustion, and perils of scalability. I devise a framework for assessment called “citizenly politics,” focusing on: 1) designing participation 2) deliberation 3) participation and 4) potential for institutionalization. I argue that while the material results PB produces are relatively modest, including more innovative projects, PB delivers more substantial non-material or existential results. Existential citizenly rewards include: greater civic knowledge, strengthened relationships with elected officials, and greater community inclusion. Overall, PB provides a viable and informative democratic innovation for strengthening civic engagement within the United States that can be streamlined and adopted to scale.”