Google’s flu fail shows the problem with big data

Curated on Posted on by Stefaan Verhulst

Adam Kucharski in The Conversation: “When people talk about ‘big data’, there is an oft-quoted example: a proposed public health tool called Google Flu Trends. It has become something of a pin-up for the big data movement, but it might not be as effective as many claim.
The idea behind big data is that large amount of information can help us do things which smaller volumes cannot. Google first outlined the Flu Trends approach in a 2008 paper in the journal Nature. Rather than relying on disease surveillance used by the US Centers for Disease Control and Prevention (CDC) – such as visits to doctors and lab tests – the authors suggested it would be possible to predict epidemics through Google searches. When suffering from flu, many Americans will search for information related to their condition….
Between 2003 and 2008, flu epidemics in the US had been strongly seasonal, appearing each winter. However, in 2009, the first cases (as reported by the CDC) started in Easter. Flu Trends had already made its predictions when the CDC data was published, but it turned out that the Google model didn’t match reality. It had substantially underestimated the size of the initial outbreak.
The problem was that Flu Trends could only measure what people search for; it didn’t analyse why they were searching for those words. By removing human input, and letting the raw data do the work, the model had to make its predictions using only search queries from the previous handful of years. Although those 45 terms matched the regular seasonal outbreaks from 2003–8, they didn’t reflect the pandemic that appeared in 2009.
Six months after the pandemic started, Google – who now had the benefit of hindsight – updated their model so that it matched the 2009 CDC data. Despite these changes, the updated version of Flu Trends ran into difficulties again last winter, when it overestimated the size of the influenza epidemic in New York State. The incidents in 2009 and 2012 raised the question of how good Flu Trends is at predicting future epidemics, as opposed to merely finding patterns in past data.
In a new analysis, published in the journal PLOS Computational Biology, US researchers report that there are “substantial errors in Google Flu Trends estimates of influenza timing and intensity”. This is based on comparison of Google Flu Trends predictions and the actual epidemic data at the national, regional and local level between 2003 and 2013
Even when search behaviour was correlated with influenza cases, the model sometimes misestimated important public health metrics such as peak outbreak size and cumulative cases. The predictions were particularly wide of the mark in 2009 and 2012:

Original and updated Google Flu Trends (GFT) model compared with CDC influenza-like illness (ILI) data. PLOS Computational Biology 9:10
Click to enlarge

Although they criticised certain aspects of the Flu Trends model, the researchers think that monitoring internet search queries might yet prove valuable, especially if it were linked with other surveillance and prediction methods.
Other researchers have also suggested that other sources of digital data – from Twitter feeds to mobile phone GPS – have the potential to be useful tools for studying epidemics. As well as helping to analysing outbreaks, such methods could allow researchers to analyse human movement and the spread of public health information (or misinformation).
Although much attention has been given to web-based tools, there is another type of big data that is already having a huge impact on disease research. Genome sequencing is enabling researchers to piece together how diseases transmit and where they might come from. Sequence data can even reveal the existence of a new disease variant: earlier this week, researchers announced a new type of dengue fever virus….”

The End of Hypocrisy

Curated on Posted on by Stefaan Verhulst

New paper by Henry Farrell and Martha Finnemore in Foreign Affairs: “The U.S. government seems outraged that people are leaking classified materials about its less attractive behavior. It certainly acts that way: three years ago, after Chelsea Manning, an army private then known as Bradley Manning, turned over hundreds of thousands of classified cables to the anti-secrecy group WikiLeaks, U.S. authorities imprisoned the soldier under conditions that the UN special rapporteur on torture deemed cruel and inhumane. The Senate’s top Republican, Mitch McConnell, appearing on Meet the Press shortly thereafter, called WikiLeaks’ founder, Julian Assange, “a high-tech terrorist.””
More recently, following the disclosures about U.S. spying programs by Edward Snowden, a former National Security Agency analyst, U.S. officials spent a great deal of diplomatic capital trying to convince other countries to deny Snowden refuge. And U.S. President Barack Obama canceled a long-anticipated summit with Russian President Vladimir Putin when he refused to comply.
Despite such efforts, however, the U.S. establishment has often struggled to explain exactly why these leakers pose such an enormous threat. Indeed, nothing in the Manning and Snowden leaks should have shocked those who were paying attention…
The deeper threat that leakers such as Manning and Snowden pose is more subtle than a direct assault on U.S. national security: they undermine Washington’s ability to act hypocritically and get away with it. Their danger lies not in the new information that they reveal but in the documented confirmation they provide of what the United States is actually doing and why…”

IRM releases United States report for public comment

Curated on Posted on by Stefaan Verhulst
“The Open Government Partnership’s Independent Reporting Mechanism (IRM) has launched its eighth progress reports for public comment; this one is on the United States and can be found below….
The United States’ action plan was highly varied and, in many respects, ambitious and innovative and significant progress was made on most of the commitments. While OGP implementation in the United States drew inspiration from an unprecedented consultation on open government during the implementation of the 2009 Open Government Directive, the dedicated public consultation for the OGP action plan was more limited and arguably more targeted.
Several of the commitments in the action plan focused on improving transparency; however, open government progress has been relatively slower in controversial areas such as national security, ethics reform, declassification of documents, and Freedom of Information Act reform.
The United States completed half of the commitments in its action plan, while the other half saw limited or substantial progress.
Due to the nature of the US government, wherein federal agencies are to some degree independent of the White House, much of the best participation took place within agencies. There were several notable examples of participation and collaboration at this level, including the commitments around the Extractive Industries Transparency Initiative, the National Dialogue on Federal Website Policy, and NASA’s Space Apps competition.
This report is a draft for public comment.  All interested parties are encouraged to comment on this blog or to send public comments to [email protected] until November 14. Comments will be collated and published, except where the requestor asks to be anonymous. Where substantive factual errors are identified, comments will be integrated into a final version of the report.”
 

United States IRM Report

7 Tactics for 21st-Century Cities

Curated on Posted on by Stefaan Verhulst

Abhi Nemani, co-director of Code for America: “Be it the burden placed on them by shrinking federal support, or the opportunity presented by modern technology, 21st-century cities are finding new ways to do things. For four years, Code for America has worked with dozens of cities, each finding creative ways to solve neighborhood problems, build local capacity and steward a national network. These aren’t one-offs. Cities are championing fundamental, institutional reforms to commit to an ongoing innovation agenda.
Here are a few of the ways how:

  1. …Create an office of new urban mechanics or appoint a chief innovation officer…
  2. …Appoint a chief data officer or create an office of performance management/enhancement…
  3. …Adopt the Gov.UK Design Principles, and require plain, human language on every interface….
  4. …Share open source technology with a sister city or change procurement rules to make it easier to redeploy civic tech….
  5. …Work with the local civic tech community and engage citizens for their feedback on city policy through events, tech and existing forums…
  6. …Create an open data policy and adopt open data specifications…
  7. …Attract tech talent into city leadership, and create training opportunities citywide to level up the tech literacy for city staff…”

From open data to open democracy

Curated on Posted on by Stefaan Verhulst

Article by : “Such debates further underscore the complexities of open data and where it might lead. While open data may be viewed by some inside and outside government as a technically-focused and largely incremental project based upon information formatting and accessibility (with the degree of openness subject to a myriad of security and confidentiality provisions), such an approach greatly limits its potential. Indeed, the growing ubiquity of mobile and smart devices, the advent of open source operating systems and social media platforms, and the growing commitment by governments themselves to expansive public engagement objectives, all suggest a widening scope.
Yet, what will incentivize the typical citizen to access open data and to partake in collective efforts to create public value? It is here where our digital culture may well fall short, emphasizing individualized service and convenience at the expense of civic responsibility and community-mindedness. For one American academic, this “citizenship deficit” erodes democratic legitimacy and renders our politics more polarized and less discursive. For other observers in Europe, notions of the digital divide are giving rise to new “data divides.”
The politics and practicalities of data privacy often bring further confusion. While privacy advocates call for greater protection and a culture of data activism among Internet users themselves, the networked ethos of online communities and commercialization fuels speed and sharing, often with little understanding of the ramifications of doing so. Differences between consumerism and citizenship are subtle yet profoundly important, while increasingly blurred and overlooked.
A key conundrum provincially and federally, within the Westminster confines of parliamentary democracy, is that open data is being hatched mainly from within the executive branch, whereas the legislative branch watches and withers. In devising genuine democratic openness, politicians and their parties must do more than post expenses online: they must become partners and advocates for renewal. A lesson of open source technology, however, is that systemic change demands an informed and engaged civil society, disgruntled with the status quo but also determined to act anew.
Most often, such actions are highly localized, even in a virtual world, giving rise to the purpose and meaning of smarter and more intelligent communities. And in Canada it bears noting that we see communities both large and small embracing open data and other forms of online experimentation such as participatory budgeting. It is often within small but connected communities where a virtuous cycle of online and in-person identities and actions can deepen and impact decision-making most directly.
How, then, do we reconcile traditional notions of top-down political federalism and national leadership with this bottom-up approach to community engagement and democratic renewal? Shifting from open data to open democracy is likely to be an uneven, diverse, and at times messy affair. Better this way than attempting to ordain top-down change in a centralized and standardized manner.”

What the Government Does with Americans’ Data

Curated on Posted on by Stefaan Verhulst

New paper from the Brennan Center for Justice: “After the attacks of September 11, 2001, the government’s authority to collect, keep, and share information about Americans with little or no basis to suspect wrongdoing dramatically expanded. While the risks and benefits of this approach are the subject of intense debate, one thing is certain: it results in the accumulation of large amounts of innocuous information about law-abiding citizens. But what happens to this data? In the search to find the needle, what happens to the rest of the haystack? For the first time in one report, the Brennan Center takes a comprehensive look at the multiple ways U.S. intelligence agencies collect, share, and store data on average Americans. The report, which surveys across five intelligence agencies, finds that non-terrorism related data can be kept for up to 75 years or more, clogging national security databases and creating opportunities for abuse, and recommends multiple reforms that seek to tighten control over the government’s handling of Americans’ information.”

Why Nudge?: The Politics of Libertarian Paternalism

Curated on Posted on by Stefaan Verhulst

New and forthcoming book by Cass Sunstein: “Based on a series of pathbreaking lectures given at Yale University in 2012, this powerful, thought-provoking work by national best-selling author Cass R. Sunstein combines legal theory with behavioral economics to make a fresh argument about the legitimate scope of government, bearing on obesity, smoking, distracted driving, health care, food safety, and other highly volatile, high-profile public issues. Behavioral economists have established that people often make decisions that run counter to their best interests—producing what Sunstein describes as “behavioral market failures.” Sometimes we disregard the long term; sometimes we are unrealistically optimistic; sometimes we do not see what is in front of us. With this evidence in mind, Sunstein argues for a new form of paternalism, one that protects people against serious errors but also recognizes the risk of government overreaching and usually preserves freedom of choice.
Against those who reject paternalism of any kind, Sunstein shows that “choice architecture”—government-imposed structures that affect our choices—is inevitable, and hence that a form of paternalism cannot be avoided. He urges that there are profoundly moral reasons to ensure that choice architecture is helpful rather than harmful—and that it makes people’s lives better and longer.”

Bright Spots of open government to be recognised at global summit

Curated on Posted on by Stefaan Verhulst

Press Release of the UK Cabinet Office: “The 7 shortlisted initiatives vying for the Bright Spots award show how governments in Open Government Partnership countries are working with citizens to sharpen governance, harness new technologies to increase public participation and improve government responsiveness.
At the Open Government Partnership summit in London on 31 October 2013 and 1 November 2013, participants will be able to vote for one of the shortlisted projects. The winning project – the Bright Spot – will be announced in the summit’s final plenary session….
The shortlisted entries for the Bright Spots prize – which will be awarded at the London summit – are:

  • Chile – ChileAtiende

The aim of ChileAtiende has been to simplify government to citizens by providing a one-stop shop for accessing public services. Today, ChileAtiende has more than 190 offices across the whole country, a national call centre and a digital platform, through which citizens can access multiple services and benefits without having to navigate multiple government offices.

  • Estonia – People’s Assembly

The People’s Assembly is a deliberative democracy tool, designed to encourage input from citizens on the government’s legislative agenda. This web-based platform allows ordinary citizens to propose policy solutions to problems including fighting corruption. Within 3 weeks, 1,800 registered users posted nearly 6,000 ideas and comments. Parliament has since set a timetable for the most popular proposals to be introduced in the formal proceedings.

  • Georgia – improvements to the Freedom of Information Act

Civil society organisations in Georgia have successfully used the government’s participation in OGP to advocate improvements to the country’s Freedom of Information legislation. Government agencies are now obliged to proactively publish information in a way that is accessible to anyone, and to establish an electronic request system for information.

  • Indonesia – complaints portal

LAPOR! (meaning “to report” in Indonesian) is a social media channel where Indonesian citizens can submit complaints and enquiries about development programmes and public services. Comments are transferred directly to relevant ministries or government agencies, which can respond via the website. LAPOR! now has more than 225,350 registered users and receives an average of 1,435 inputs per day.

  • Montenegro – Be Responsible app

“Be Responsible” is a mobile app that allows citizens to report local problems – from illegal waste dumps, misuse of official vehicles and irregular parking, to failure to comply with tax regulations and issues over access to healthcare and education.

  • Philippines – citizen audits

The Citizen Participatory Audit (CPA) project is exploring ways in which citizens can be directly engaged in the audit process for government projects and contribute to ensuring greater efficiency and effectiveness in the use of public resources. 4 pilot audits are in progress, covering public works, welfare, environment and education projects.

  • Romania – transparency in public sector recruitment

The PublicJob.ro website was set up to counter corruption and lack of transparency in civil service recruitment. PublicJob.ro takes recruitment data from public organisations and e-mails it to more than 20,000 subscribers in a weekly newsletter. As a result, it has become more difficult to manipulate the recruitment process.”

Beyond Transparency

Curated on Posted on by Stefaan Verhulst

New book on Open Data and the Future of Civic Innovation: The rise of open data in the public sector has sparked innovation, driven efficiency, and fueled economic development. And in the vein of high-profile federal initiatives like Data.gov and the White House’s Open Government Initiative, more and more local governments are making their foray into the field with Chief Data Officers, open data policies, and open data catalogs.
While still emerging, we are seeing evidence of the transformative potential of open data in shaping the future of our civic life. It’s at the local level that government most directly impacts the lives of residents—providing clean parks, fighting crime, or issuing permits to open a new business. This is where there is the biggest opportunity to use open data to reimagine the relationship between citizens and government.
Beyond Transparency is a cross-disciplinary survey of the open data landscape, in which practitioners share their own stories of what they’ve accomplished with open civic data. It seeks to move beyond the rhetoric of transparency for transparency’s sake and towards action and problem solving. Through these stories, we examine what is needed to build an ecosystem in which open data can become the raw materials to drive more effective decision-making and efficient service delivery, spur economic activity, and empower citizens to take an active role in improving their own communities….
This book is a resource for (and by) practitioners inside and outside government—from the municipal chief information officer to the community organizer to the civic-minded entrepreneur. Beyond Transparency is intended to capture and distill the community’s learnings around open data for the past four years. And we know that the community is going to continue learning. That’s why, in addition to the print version of the book which you can order on Amazon, we’ve also published the digital version of this book on this site under a Creative Commons license. The full text of this site is on GitHub — which means that anyone can submit a pull request with a suggested edit. Help us improve this resource for the community and write the next edition of Beyond Transparency by submitting your pull requests.
Code for America is a national nonprofit committed to building a government for the people, by the people, that works in the 21st century. Over the past four years, CfA has worked with dozens of cities to support civic innovation through open data. You can support this work by contributing to the book on GitHub, joining the CfA volunteer community (the Brigade), or connecting your city with CfA.

NEW Publication: “Reimagining Governance in Practice: Benchmarking British Columbia’s Citizen Engagement Efforts”

Curated on Posted on by Stefaan Verhulst

Over the last few years, the Government of British Columbia (BC), Canada has initiated a variety of practices and policies aimed at providing more legitimate and effective governance. Leveraging advances in technology, the BC Government has focused on changing how it engages with its citizens with the goal of optimizing the way it seeks input and develops and implements policy. The efforts are part of a broader trend among a wide variety of democratic governments to re-imagine public service and governance.
At the beginning of 2013, BC’s Ministry of Citizens’ Services and Open Government, now the Ministry of Technology, Innovation and Citizens’ Services, partnered with the GovLab to produce “Reimagining Governance in Practice: Benchmarking British Columbia’s Citizen Engagement Efforts.” The GovLab’s May 2013 report, made public today, makes clear that BC’s current practices to create a more open government, leverage citizen engagement to inform policy decisions, create new innovations, and provide improved public monitoring­—though in many cases relatively new—are consistently among the strongest examples at either the provincial or national level.
According to Stefaan Verhulst, Chief of Research at the GovLab: “Our benchmarking study found that British Columbia’s various initiatives and experiments to create a more open and participatory governance culture has made it a leader in how to re-imagine governance. Leadership, along with the elimination of imperatives that may limit further experimentation, will be critical moving forward. And perhaps even more important, as with all initiatives to re-imaging governance worldwide, much more evaluation of what works, and why, will be needed to keep strengthening the value proposition behind the new practices and polices and provide proof-of-concept.”
See also our TheGovLab Blog.