Neal Ungerleider at Co-Labs: “It’s not just for the private sector anymore: Government scientists are embracing crowdsourcing. At a White House-sponsored workshop in late November, representatives from more than 20 different federal agencies gathered to figure out how to integrate crowdsourcing and citizen scientists into various government efforts. The workshop is part of a bigger effort with a lofty goal: Building a set of best practices for the thousands of citizens who are helping federal agencies gather data, from the Environmental Protection Agency (EPA) to NASA….Perhaps the best known federal government crowdsourcing project is Nature’s Notebook, a collaboration between the U.S. Geological Survey and the National Park Service which asks ordinary citizens to take notes on plant and animal species during different times of year. These notes are then cleansed and collated into a massive database on animal and plant phenology that’s used for decision-making by national and local governments. The bulk of the observations, recorded through smartphone apps, are made by ordinary people who spend a lot of time outdoors….Dozens of government agencies are now asking the public for help. The Centers for Disease Control and Prevention runs a student-oriented, Mechanical Turk-style “micro-volunteering” service called CDCology, the VA crowdsources design of apps for homeless veterans, while the National Weather Service distributes a mobile app called mPING that asks ordinary citizens to help fine-tune public weather reports by giving information on local conditions. The Federal Communication Commission’s Measuring Broadband America app, meanwhile, allows citizens to volunteer information on their Internet broadband speeds, and the Environmental Protection Agency’s Air Sensor Toolbox asks users to track local air pollution….
As of now, however, when it comes to crowdsourcing data for government scientific research, there’s no unified set of standards or best practices. This can lead to wild variations in how various agencies collect data and use it. For officials hoping to implement citizen science projects within government, the roadblocks to crowdsourcing include factors that crowdsourcing is intended to avoid: limited budgets, heavy bureaucracy, and superiors who are skeptical about the value of relying on the crowd for data.
Benforado and Shanley also pointed out that government agencies are subject to additional regulations, such as the Paperwork Reduction Act, which can make implementation of crowdsourcing projects more challenging than they would be in academia or the private sector… (More)”
The Free 'Big Data' Sources Everyone Should Know
Bernard Marr at Linkedin Pulse: “…The moves by companies and governments to put large amounts of information into the public domain have made large volumes of data accessible to everyone….here’s my rundown of some of the best free big data sources available today.
Data.gov
The US Government pledged last year to make all government data available freely online. This site is the first stage and acts as a portal to all sorts of amazing information on everything from climate to crime. To check it out, click here.
US Census Bureau
A wealth of information on the lives of US citizens covering population data, geographic data and education. To check it out, click here. To check it out, click here.
European Union Open Data Portal
As the above, but based on data from European Union institutions. To check it out, click here.
Data.gov.uk
Data from the UK Government, including the British National Bibliography – metadata on all UK books and publications since 1950. To check it out, click here.
The CIA World Factbook
Information on history, population, economy, government, infrastructure and military of 267 countries. To check it out, click here.
Healthdata.gov
125 years of US healthcare data including claim-level Medicare data, epidemiology and population statistics. To check it out, click here.
NHS Health and Social Care Information Centre
Health data sets from the UK National Health Service. To check it out, click here.
Amazon Web Services public datasets
Huge resource of public data, including the 1000 Genome Project, an attempt to build the most comprehensive database of human genetic information and NASA’s database of satellite imagery of Earth. To check it out, click here.
Facebook Graph
Although much of the information on users’ Facebook profile is private, a lot isn’t – Facebook provide the Graph API as a way of querying the huge amount of information that its users are happy to share with the world (or can’t hide because they haven’t worked out how the privacy settings work). To check it out, click here.
Gapminder
Compilation of data from sources including the World Health Organization and World Bank covering economic, medical and social statistics from around the world. To check it out, click here.
Google Trends
Statistics on search volume (as a proportion of total search) for any given term, since 2004. To check it out, click here.
Google Finance
40 years’ worth of stock market data, updated in real time. To check it out, click here.
Google Books Ngrams
Search and analyze the full text of any of the millions of books digitised as part of the Google Books project. To check it out, click here.
National Climatic Data Center
Huge collection of environmental, meteorological and climate data sets from the US National Climatic Data Center. The world’s largest archive of weather data. To check it out, click here.
DBPedia
Wikipedia is comprised of millions of pieces of data, structured and unstructured on every subject under the sun. DBPedia is an ambitious project to catalogue and create a public, freely distributable database allowing anyone to analyze this data. To check it out, click here.
Topsy
Free, comprehensive social media data is hard to come by – after all their data is what generates profits for the big players (Facebook, Twitter etc) so they don’t want to give it away. However Topsy provides a searchable database of public tweets going back to 2006 as well as several tools to analyze the conversations. To check it out, click here.
Likebutton
Mines Facebook’s public data – globally and from your own network – to give an overview of what people “Like” at the moment. To check it out, click here.
New York Times
Searchable, indexed archive of news articles going back to 1851. To check it out, click here.
Freebase
A community-compiled database of structured data about people, places and things, with over 45 million entries. To check it out, click here.
Million Song Data Set
Metadata on over a million songs and pieces of music. Part of Amazon Web Services. To check it out, click here.”
See also Bernard Marr‘s blog at Big Data Guru
4 Tech Trends Changing How Cities Operate
at Governing: “Louis Brandeis famously characterized states as laboratories for democracy, but cities could be called labs for innovation or new practices….When Government Technology magazine (produced by Governing’s parent company, e.Republic, Inc.) published its annual Digital Cities Survey, the results provided an interesting look at how local governments are using technology to improve how they deliver services, increase production and streamline operations…the survey also showed four technology trends changing how local government operates and serves its citizens:
1. Open Data
…Big cities were the first to open up their data and gained national attention for their transparency. New York City, which passed an open data law in 2012, leads all cities with more than 1,300 data sets open to the public; Chicago started opening up data to the public in 2010 following an executive order and is second among cities with more than 600; and San Francisco, which was the first major city to open the doors to transparency in 2009, had the highest score from the U.S. Open Data Census for the quality of its open data.
But the survey shows that a growing number of mid-sized jurisdictions are now getting involved, too. Tacoma, Wash., has a portal with 40 data sets that show how the city is spending tax dollars on public works, economic development, transportation and public safety. Ann Arbor, Mich., has a financial transparency tool that reveals what the city is spending on a daily basis, in some cases….
2. ‘Stat’ Programs and Data Analytics
…First, the so-called “stat” programs are proliferating. Started by the New York Police Department in the 1980s, CompStat was a management technique that merged data with staff feedback to drive better performance by police officers and precinct captains. Its success led to many imitations over the years and, as the digital survey shows, stat programs continue to grow in importance. For example, Louisville has used its “LouieStat” program to cut the city’s bill for unscheduled employee overtime by $23 million as well as to spot weaknesses in performance.
Second, cities are increasing their use of data analytics to measure and improve performance. Denver, Jacksonville, Fla., and Phoenix have launched programs that sift through data sets to find patterns that can lead to better governance decisions. Los Angeles has combined transparency with analytics to create an online system that tracks performance for the city’s economy, service delivery, public safety and government operations that the public can view. Robert J. O’Neill Jr., executive director of the International City/County Management Association, said that both of these tech-driven performance trends “enable real-time decision-making.” He argued that public leaders who grasp the significance of these new tools can deliver government services that today’s constituents expect.
3. Online Citizen Engagement
…Avondale, Ariz., population 78,822, is engaging citizens with a mobile app and an online forum that solicits ideas that other residents can vote up or down.
In Westminster, Colo., population 110,945, a similar forum allows citizens to vote online about community ideas and gives rewards to users who engage with the online forum on a regular basis (free passes to a local driving range or fitness program). Cities are promoting more engagement activities to combat a decline in public trust in government. The days when a public meeting could provide citizen engagement aren’t enough in today’s technology-dominated world. That’s why social media tools, online surveys and even e-commerce rewards programs are popping up in cities around the country to create high-value interaction with its citizens.
4. Geographic Information Systems
… Cities now use them to analyze financial decisions to increase performance, support public safety, improve public transit, run social service activities and, increasingly, engage citizens about their city’s governance.
Augusta, Ga., won an award for its well-designed and easy-to-use transit maps. Sugar Land, Texas, uses GIS to support economic development and, as part of its citizen engagement efforts, to highlight its capital improvement projects. GIS is now used citywide by 92 percent of the survey respondents. That’s significant because GIS has long been considered a specialized (and expensive) technology primarily for city planning and environmental projects….”
Let’s kickstart science in America
David Lang at Ideas.Ted: “Science funding is broken. To fix it, we need to empower a new class of makers, citizen scientists and explorers
The troubling state of science funding in America goes by many names: sequestration, the profzi scheme, the postdocalypse. Because it can take extensive planning over years in academia to gain research funds from the National Science Foundation and the National Institutes of Health, “nobody takes risks anymore,” writes one researcher in his “Goodbye Academia” letter. “Nobody young jumps and tries totally new things, because it’s almost surely a noble way to suicide your career.” The result? We are on the verge of losing a generation of scientists at the exact moment we need to embolden them. Biologist Michael Eisen sums up the effects of the funding crunch: “It is an amazing time to do science, but an incredibly difficult time to be a scientist.”
It’s not all bad news for the thousands of science and conservation ideas that fall outside the traditional funding rubric. Fortunately, new citizen science models are emerging — along with a new class of philanthropic backers to fill the funding voids left by the NSF and the NIH. Our experience developing OpenROV (an open-source underwater robot) into one of the largest (by volume) underwater robot manufacturers in the world is illustrative of this shift.
Looking back at the sequence of events, it seems improbable that such a small amount of initial funding could have made such a large impact, but it makes perfect sense when you break down all the contributing factors. Two years ago, we weren’t even part of the oceanographic community. Our ideas and techniques were outside the playbook for experienced ocean engineers. And since we only had enough money to test the first thing, not the whole thing, we started by creating a prototype. Using TechShop equipment in San Francisco, we able to create several iterations of a low-cost underwater robot that was suitable for our purpose: exploring an underwater cave and looking for lost treasure. After sharing our designs online, we found a community of like-minded developers. Together we raised over $100,000 on Kickstarter to do a first run of manufacturing.
This experience made us think: How can we make more microsponsorship opportunities available in science, exploration and conservation? OpenExplorer was our response. Instead of providing seed funding, we’ve created a model that gives everyone a chance to sponsor new ideas, research and expeditions in science and engineering. One success: TED Fellow Asha de Vos‘s work on preventing the ship strike of blue whales in the Indian Ocean….”
States and democracy
New paper by Francis Fukuyama in the journal Democratization: “The state, rule of law, and democratic accountability are the three basic components of a modern political order. The state concentrates and uses power, while law and democracy constrain the exercise of power, indicating that there is an inherent tension between them. This article looks at ways in which the state and liberal democracy interact in three areas: citizen security, patronage and clientelism, and the formation of national identity. In all three areas, state and democracy act at cross purposes in some domains, and are mutually supportive in others. The reason for this complex relationship is that both state and democracy are themselves complex collections of institutions which interact on a multiplicity of levels. Understanding the relationship between state and democracy is important in policy terms because many recent initiatives to improve the quality of governance assume that state quality and democracy are mutually supportive, something that is not fully supported by the empirical evidence.”
Designing a Citizen Science and Crowdsourcing Toolkit for the Federal Government
In the 2013 Second Open Government National Action Plan, President Obama called on Federal agencies to harness the ingenuity of the public by accelerating and scaling the use of open innovation methods, such as citizen science and crowdsourcing, to help address a wide range of scientific and societal problems.
Citizen science is a form of open collaboration in which members of the public participate in the scientific process, including identifying research questions, collecting and analyzing data, interpreting results, and solving problems. Crowdsourcing is a process in which individuals or organizations submit an open call for voluntary contributions from a large group of unknown individuals (“the crowd”) or, in some cases, a bounded group of trusted individuals or experts.
Citizen science and crowdsourcing are powerful tools that can help Federal agencies:
- Advance and accelerate scientific research through group discovery and co-creation of knowledge. For instance, engaging the public in data collection can provide information at resolutions that would be difficult for Federal agencies to obtain due to time, geographic, or resource constraints.
- Increase science literacy and provide students with skills needed to excel in science, technology, engineering, and math (STEM). Volunteers in citizen science or crowdsourcing projects gain hands-on experience doing real science, and take that learning outside of the classroom setting.
- Improve delivery of government services with significantly lower resource investments.
- Connect citizens to the missions of Federal agencies by promoting a spirit of open government and volunteerism.
To enable effective and appropriate use of these new approaches, the Open Government National Action Plan specifically commits the Federal government to “convene an interagency group to develop an Open Innovation Toolkit for Federal agencies that will include best practices, training, policies, and guidance on authorities related to open innovation, including approaches such as incentive prizes, crowdsourcing, and citizen science.”
On November 21, 2014, the Office of Science and Technology Policy (OSTP) kicked off development of the Toolkit with a human-centered design workshop. Human-centered design is a multi-stage process that requires product designers to engage with different stakeholders in creating, iteratively testing, and refining their product designs. The workshop was planned and executed in partnership with the Office of Personnel Management’s human-centered design practice known as “The Lab” and the Federal Community of Practice on Crowdsourcing and Citizen Science (FCPCCS), a growing network of more than 100 employees from more than 20 Federal agencies….
The Toolkit will help further the culture of innovation, learning, sharing, and doing in the Federal citizen science and crowdsourcing community: indeed, the development of the Toolkit is a collaborative and community-building activity in and of itself.
The following successful Federal projects illustrate the variety of possible citizen science and crowdsourcing applications:
- The Citizen Archivist Dashboard (NARA) coordinates crowdsourced archival record tagging and document transcription. Recently, more than 170,000 volunteers indexed 132 million names of the 1940 Census in only five months, which NARA could not have done alone.
- Through Measuring Broadband America (FCC), 2 million volunteers collected and provided the FCC with data on their Internet speeds, data that FCC used to create a National Broadband Map revealing digital divides.
- In 2014, Nature’s Notebook (USGS, NSF) volunteers recorded more than 1 million observations on plants and animals that scientists use to analyze environmental change.
- Did You Feel It? (USGS) has enabled more than 3 million people worldwide to share their experiences during and immediately after earthquakes. This information facilitates rapid damage assessments and scientific research, particularly in areas without dense sensor networks.
- The mPING (NOAA) mobile app has collected more than 600,000 ground-based observations that help verify weather models.
- USAID anonymized and opened its loan guarantee data to volunteer mappers. Volunteers mapped 10,000 data points in only 16 hours, compared to the 60 hours officials expected.
- The Air Sensor Toolbox (EPA), together with training workshops, scientific partners, technology evaluations, and a scientific instrumentation loan program, empowers communities to monitor and report local air pollution.
In early 2015, OSTP, in partnership with the Challenges and Prizes Community of Practice, will convene Federal practitioners to develop the other half of the Open Innovation Toolkit for prizes and challenges. Stay tuned!”
Mapping information economy business with big data: findings from the UK
NESTA: “This paper uses innovative ‘big data’ resources to measure the size of the information economy in the UK.
Key Findings
- Counts of information economy firms are 42 per cent larger than SIC-based estimates
- Using ‘big data’ estimates, the research finds 225,800 information economy businesses in the UK
- Information economy businesses are highly clustered across the country, with very high counts in the Greater South East, notably London (especially central and east London), as well as big cities such as Manchester, Birmingham and Bristol
- Looking at local clusters, we find hotspots in Middlesbrough, Aberdeen, Brighton, Cambridge and Coventry, among others
Information and Communications Technologies – and the digital economy they support – are of enduring interest to researchers and policymakers. National and local government are particularly keen to understand the characteristics and growth potential of ‘their’ digital businesses.
Given the recent resurgence of interest in industrial policy across many developed countries, there is now substantial policy interest in developing stronger, more competitive digital economies. For example, the UK’s current industrial strategy combines horizontal interventions with support for seven key sectors, of which the ‘information economy’ is one.
The desire to grow high–tech clusters is often prominent in the policy mix – for instance, the UK’s Tech City UK initiative, Regional Innovation Clusters in the US and elements of ‘smart specialisation’ policies in the EU.
In this paper, NIESR and Growth Intelligence use novel ‘big data’ sources to improve our understanding of information economy businesses in the UK – that is, those involved in the production of ICTs. We use this experience to critically reflect on some of the opportunities and challenges presented by big data tools and analytics for economic research and policymaking.”
– See more at: http://www.nesta.org.uk/publications/mapping-information-economy-business-big-data-findings-uk-0#sthash.2ismEMr2.dpuf
Restoring Confidence in Open, Shared and Personal Data
Report of the UK Digital Government Review: “It is obvious that government needs to be able to use data both to deliver services and to present information to public view. How else would government know which bank account to place a pension payment into, or a citizen know the results of an election or how to contact their elected representatives?
As more and more data is created, preserved and shared in ever-increasing volumes a number of urgent questions are begged: over opportunities and hazards; over the importance of using best-practice techniques, insights and technologies developed in the private sector, academia and elsewhere; over the promises and limitations of openness; and how all this might be articulated and made accessible to the public.
Government has already adopted “open data” (we will discuss this more in the next section) and there are now increasing calls for government to pay more attention to data analytics and so-called “big data” – although the first faltering steps to unlock benefits, here, have often ended in the discovery that using large-scale data is a far more nuanced business than was initially assumed
Debates around government and data have often been extremely high-profile – the NHS care.data [27] debate was raging while this review was in progress – but they are also shrouded in terms that can generate confusion and complexities that are not easily summarized.
In this chapter we will unpick some of these terms and some parts of the debate. This is a detailed and complex area and there is much more that could have been included [28]. This is not an area that can easily be summarized into a simple bullet-pointed list of policies.
Within this report we will use the following terms and definitions, proceeding to a detailed analysis of each in turn:
Type of Data |
Definition [29] |
Examples |
| 1. Open Data | Data that can be freely used, reused and redistributed by anyone – subject only, at most, to the requirement to attribute and sharealike | Insolvency notices in the London Gazette Government spending information Public transport information Official National Statistics |
| 2. Shared Data | Restricted data provided to restricted organisations or individuals for restricted purposes | National Pupil Database NHS care.data Integrated health and social care Individual census returns |
| 3. Personal Data | Data that relate to a living individual who can be identified from that data. For full legal definition see [30] | Health records Individual tax records Insolvency notices in the London gazette National Pupil Database |
This social productivity will help build future economic productivity; in the meantime it will improve people’s lives and it will enhance our democracy. From our analysis it was clear that there was room for improvement…”
White House: Help Shape Public Participation
at the White House Blog: “Public participation — where citizens help shape and implement government programs — is a foundation of open, transparent, and engaging government services. From emergency management and regulatory development to science and education, better and more meaningful engagement with those who use public services can measurably improve government for everyone.
A team across the government is now working side-by-side with civil society organizations to deliver the first U.S. Public Participation Playbook, dedicated to providing best practices for how agencies can better design public participation programs, and suggested performance metrics for evaluating their effectiveness.
Developing a U.S. Public Participation Playbook has been an open government priority, and was included in both the first and second U.S. Open Government National Action Plans as part of the United States effort to increase public integrity in government programs. This resource reflects the commitment of the government and civic partners to measurably improve participation programs, and is designed using the same inclusive principles that it champions.
More than 30 Federal leaders from across diverse missions in public service have collaborated on draft best practices, or “plays,” lead by the General Services Administration’s inter-agency SocialGov Community. The playbook is not limited to digital participation, and is designed to address needs from the full spectrum of public participation programs.
The plays are structured to provide best practices, tangible examples, and suggested performance metrics for government activities that already exist or are under development. Some categories included in the plays include encouraging community development and outreach, empowering participants through public/private partnerships, using data to drive decisions, and designing for inclusiveness and accessibility.
In developing this new resource, the team has been reaching out to more than a dozen civil society organizations and stakeholders, asking them to contribute as the Playbook is created. The team would like your input as well! Over the next month, contribute your ideas to the playbook using Madison, an easy-to-use, open source platform that allows for accountable review of each contribution.
Through this process, the team will work together to ensure that the Playbook reflects the best ideas and examples for agencies to use in developing and implementing their programs with public participation in mind. This resource will be a living document, and stakeholders from inside or outside of government should continually offer new insights — whether new plays, the latest case studies, or the most current performance metrics — to the playbook.
We look forward to seeing the public participate in the creation and evolution of the Public Participation Playbook!”
Look to Government—Yes, Government—for New Social Innovations
Paper by Christian Bason and Philip Colligan: “If asked to identify the hotbed of social innovation right now, many people would likely point to the new philanthropy of Silicon Valley or the social entrepreneurship efforts supported by Ashoka, Echoing Green, and Skoll Foundation. Very few people, if any, would mention their state capital or Capitol Hill. While local and national governments may have promulgated some of the greatest advances in human history — from public education to putting a man on the moon — public bureaucracies are more commonly known to stifle innovation.
Yet, around the world, there are local, regional, and national government innovators who are challenging this paradigm. They are pioneering a new form of experimental government — bringing new knowledge and practices to the craft of governing and policy making; drawing on human-centered design, user engagement, open innovation, and cross-sector collaboration; and using data, evidence, and insights in new ways.
Earlier this year, Nesta, the UK’s innovation foundation (which Philip helps run), teamed up with Bloomberg Philanthropies to publish i-teams, the first global review of public innovation teams set up by national and city governments. The study profiled 20 of the most established i-teams from around the world, including:
- French Experimental Fund for Youth, which has supported more than 554 experimental projects (such as one that reduces school drop-out rates) that have benefited over 480,000 young people;
- Nesta’s Innovation Lab, which has run 70 open innovation challenges and programs supporting over 750 innovators working in fields as diverse as energy efficiency, healthcare, and digital education;
- New Orleans’ Innovation and Delivery team, which achieved a 19% reduction in the number of murders in the city in 2013 compared to the previous year.
How are i-teams achieving these results? The most effective ones are explicit about the goal they seek – be it creating a solution to a specific policy challenge, engaging citizenry in behaviors that help the commonweal, or transforming the way government behaves. Importantly, these teams are also able to deploy the right skills, capabilities, and methods for the job.
In addition, i-teams have a strong bias toward action. They apply academic research in behavioral economics and psychology to public policy and services, focusing on rapid experimentation and iteration. The approach stands in stark contrast to the normal routines of government.
Take for example, The UK’s Behavioural Insights Team (BIT), often called the Nudge Unit. It sets clear goals, engages the right expertise to prototype means to the end, and tests innovations rapidly in the field, to learn what’s not working and rapidly scales what is.
One of BIT’s most famous projects changed taxpayer behavior. BIT’s team of economists, behavioral psychologists, and seasoned government staffers came up with minor changes to tax letters, sent out by the UK Government, that subtlety introduced positive peer pressure. By simply altering the letters to say that most people in their local area had already paid their taxes, BIT was able to boost repayment rates by around 5%. This trial was part of a range of interventions, which have helped forward over £200 million in additional tax revenue to HM Revenue & Customs, the UK’s tax authority.
The Danish government’s internal i-team, MindLab (which Christian ran for 8 years) has likewise influenced citizen behavior….”