Open Data Barometer (second edition)


The second edition of the Open Data Barometer: “A global movement to make government “open by default” picked up steam in 2013 when the G8 leaders signed an Open Data Charter – promising to make public sector data openly available, without charge and in re-useable formats. In 2014 the G20 largest industrial economies followed up by pledging to advance open data as a tool against corruption, and the UN recognized the need for a “Data Revolution” to achieve global development goals.
However, this second edition of the Open Data Barometer shows that there is still a long way to go to put the power of data in the hands of citizens. Core data on how governments are spending our money and how public services are performing remains inaccessible or paywalled in most countries. Information critical to fight corruption and promote fair competition, such as company registers, public sector contracts, and land titles, is even harder to get. In most countries, proactive disclosure of government data is not mandated in law or policy as part of a wider right to information, and privacy protections are weak or uncertain.
Our research suggests some of the key steps needed to ensure the “Data Revolution” will lead to a genuine revolution in the transparency and performance of governments:

  • High-level political commitment to proactive disclosure of public sector data, particularly the data most critical to accountability
  • Sustained investment in supporting and training a broad cross-section of civil society and entrepreneurs to understand and use data effectively
  • Contextualizing open data tools and approaches to local needs, for example by making data visually accessible in countries with lower literacy levels.
  • Support for city-level open data initiatives as a complement to national-level programmes
  • Legal reform to ensure that guarantees of the right to information and the right to privacy underpin open data initiatives

Over the next six months, world leaders have several opportunities to agree these steps, starting with the United Nation’s high-level data revolution in Africa conference in March, Canada’s global International Open Data Conference in May and the G7 summit in Germany this June. It is crucial that these gatherings result in concrete actions to address the political and resource barriers that threaten to stall open data efforts….(More)”.

Federal Leaders Digital Insight Study


New report by the National Academy of Public Administration: “The Federal Leaders Digital Insight Study, conducted by the National Academy of Public Administration (the Academy) in collaboration with ICF, is the inaugural report designed to survey Federal Leaders’ perspectives about the pace with which the government is adopting, applying, and leveraging technological advancements in service to its constituencies.
The study found that Federal Leaders believe the government is reaping benefits from having adopted technology and that technology helps agencies achieve their missions. Further, Federal Leaders want the government to continue investing in technology as it evolves, yet they are concerned that the government cannot keep pace either in procuring rapidly changing digital technology or with the private sector’s use of it….Infographic—Federal Leaders Digital Insight Study: Key Findings

Computer-based personality judgments are more accurate than those made by humans


Paper by Wu Youyou, Michal Kosinski and David Stillwell at PNAS (Proceedings of the National Academy of Sciences): “Judging others’ personalities is an essential skill in successful social living, as personality is a key driver behind people’s interactions, behaviors, and emotions. Although accurate personality judgments stem from social-cognitive skills, developments in machine learning show that computer models can also make valid judgments. This study compares the accuracy of human and computer-based personality judgments, using a sample of 86,220 volunteers who completed a 100-item personality questionnaire. We show that (i) computer predictions based on a generic digital footprint (Facebook Likes) are more accurate (r = 0.56) than those made by the participants’ Facebook friends using a personality questionnaire (r = 0.49); (ii) computer models show higher interjudge agreement; and (iii) computer personality judgments have higher external validity when predicting life outcomes such as substance use, political attitudes, and physical health; for some outcomes, they even outperform the self-rated personality scores. Computers outpacing humans in personality judgment presents significant opportunities and challenges in the areas of psychological assessment, marketing, and privacy…(More)”.

The smartest cities rely on citizen cunning and unglamorous technology


at the Guardian: “We are lucky enough to live at a time in which a furious wave of innovation is breaking across the cities of the global south, spurred on both by the blistering pace of urbanisation, and by the rising popular demand for access to high-quality infrastructure that follows in its wake.
From Porto Alegre’s participatory budgeting and the literally destratifying cable cars of Caracas, to Nairobi’s “digital matatus” and the repurposed bus-ferries of Manila, the communities of the south are responsible for an ever-lengthening parade of social and technical innovations that rival anything the developed world has to offer for ingenuity and practical utility.
Nor is India an exception to this tendency. Transparent Chennai’s participatory maps and the work of the Mumbai-based practices CRIT and URBZ are better-known globally, but it is the tactics of daily survival devised by the unheralded multitude that really inspire urbanists. These techniques maximise the transactive capacity of the urban fabric, wrest the very last increment of value from the energy invested in the production of manufactured goods, and allow millions to eke a living, however precarious, from the most unpromising of circumstances. At a time of vertiginously spiralling economic and environmental stress globally, these are insights many of us in the developed north would be well advised to attend to – and by no means merely the poorest among us.
But, for whatever reason, this is not the face of urban innovation official India wants to share with the world – perhaps small-scale projects or the tactics of the poor simply aren’t dramatic enough to convey the magnitude and force of national ambition. We hear, instead, of schemes like Palava City, a nominally futuristic vision of digital technology minutely interwoven into the texture of everday urban life. Headlines were made around the planet this year when Narendra Modi’s government announced it had committed to building no fewer than 100 similarly “smart” cities….(More).”

Does Real-Time Feedback On Electricity Use Really Change Our Behavior?


Jessica Leber at Co.Exist: “Is information power? Or more to the point, does information about our energy usage help us consume less power?
Even though there are a growing number of smart devices and systems on the market designed to give feedback about energy and water usage, in hopes of nudging us to cut back, studies have shown mixed evidence as to whether they actually work in changing long-term behavior.
In September 2010, the developer of a new LEED Gold apartment building in Manhattan approached Columbia University’s Center for Research on Environmental Decisions with the idea of studying whether they could reduce energy use by giving occupants devices that gave them real-time feedback on their electricity demand. They chose the Modlet, a device made by ThinkEco, that monitors energy use at each outlet, appliance by appliance.
The results of the study, published recently as a working paper by the National Bureau of Economic Research, show that getting people to change their behavior is more complicated than it seems….(More).”

Public-sector digitization: The trillion-dollar challenge


Article by Cem Dilmegani, Bengi Korkmaz, and Martin Lundqvist from McKinsey: “Citizens and businesses now expect government information to be readily available online, easy to find and understand, and at low or no cost. Governments have many reasons to meet these expectations by investing in a comprehensive public-sector digital transformation. Our analysis suggests that capturing the full potential of govern­ment digitization could free up to $1 trillion annually in economic value worldwide, through improved cost and operational performance. Shared services, greater collaboration and inte­gra­tion, improved fraud management, and productivity enhancements enable system-wide efficiencies. At a time of increasing budgetary pressures, governments at national, regional, and local levels cannot afford to miss out on those savings.
Indeed, governments around the world are doing their best to meet citizen demand and capture benefits. More than 130 countries have online services. For example, Estonia’s 1.3 million residents can use electronic identification cards to vote, pay taxes, and access more than 160 services online, from unemployment benefits to property registration. Turkey’s Social Aid Infor­ma­tion System has consolidated multiple government data sources into one system to provide citizens with better access and faster decisions on its various aid programs. The United Kingdom’s gov.uk site serves as a one-stop information hub for all government departments. Such online services also provide greater access for rural populations, improve quality of life for those with physical infirmities, and offer options for those whose work and lifestyle demands don’t conform to typical daytime office hours.
However, despite all the progress made, most governments are far from capturing the full benefits of digitization. To do so, they need to take their digital transformations deeper, beyond the provision of online services through e-government portals, into the broader business of government itself. That means looking for opportunities to improve productivity, collabo­ration, scale, process efficiency, and innovation….
While digital transformation in the public sector is particularly challenging, a number of successful government initiatives show that by translating private-sector best practices into the public context it is possible to achieve broader and deeper public-sector digitization. Each of the six most important levers is best described by success stories….(More).”

Businesses dig for treasure in open data


Lindsay Clark in ComputerWeekly: “Open data, a movement which promises access to vast swaths of information held by public bodies, has started getting its hands dirty, or rather its feet.
Before a spade goes in the ground, construction and civil engineering projects face a great unknown: what is down there? In the UK, should someone discover anything of archaeological importance, a project can be halted – sometimes for months – while researchers study the site and remove artefacts….
During an open innovation day hosted by the Science and Technologies Facilities Council (STFC), open data services and technology firm Democrata proposed analytics could predict the likelihood of unearthing an archaeological find in any given location. This would help developers understand the likely risks to construction and would assist archaeologists in targeting digs more accurately. The idea was inspired by a presentation from the Archaeological Data Service in the UK at the event in June 2014.
The proposal won support from the STFC which, together with IBM, provided a nine-strong development team and access to the Hartree Centre’s supercomputer – a 131,000 core high-performance facility. For natural language processing of historic documents, the system uses two components of IBM’s Watson – the AI service which famously won the US TV quiz show Jeopardy. The system uses SPSS modelling software, the language R for algorithm development and Hadoop data repositories….
The proof of concept draws together data from the University of York’s archaeological data, the Department of the Environment, English Heritage, Scottish Natural Heritage, Ordnance Survey, Forestry Commission, Office for National Statistics, the Land Registry and others….The system analyses sets of indicators of archaeology, including historic population dispersal trends, specific geology, flora and fauna considerations, as well as proximity to a water source, a trail or road, standing stones and other archaeological sites. Earlier studies created a list of 45 indicators which was whittled down to seven for the proof of concept. The team used logistic regression to assess the relationship between input variables and come up with its prediction….”

Uncle Sam Wants You…To Crowdsource Science


at Co-Labs: “It’s not just for the private sector anymore: Government scientists are embracing crowdsourcing. At a White House-sponsored workshop in late November, representatives from more than 20 different federal agencies gathered to figure out how to integrate crowdsourcing and citizen scientists into various government efforts. The workshop is part of a bigger effort with a lofty goal: Building a set of best practices for the thousands of citizens who are helping federal agencies gather data, from the Environmental Protection Agency (EPA) to NASA….Perhaps the best known federal government crowdsourcing project is Nature’s Notebook, a collaboration between the U.S. Geological Survey and the National Park Service which asks ordinary citizens to take notes on plant and animal species during different times of year. These notes are then cleansed and collated into a massive database on animal and plant phenology that’s used for decision-making by national and local governments. The bulk of the observations, recorded through smartphone apps, are made by ordinary people who spend a lot of time outdoors….Dozens of government agencies are now asking the public for help. The Centers for Disease Control and Prevention runs a student-oriented, Mechanical Turk-style “micro-volunteering” service called CDCology, the VA crowdsources design of apps for homeless veterans, while the National Weather Service distributes a mobile app called mPING that asks ordinary citizens to help fine-tune public weather reports by giving information on local conditions. The Federal Communication Commission’s Measuring Broadband America app, meanwhile, allows citizens to volunteer information on their Internet broadband speeds, and the Environmental Protection Agency’s Air Sensor Toolbox asks users to track local air pollution….
As of now, however, when it comes to crowdsourcing data for government scientific research, there’s no unified set of standards or best practices. This can lead to wild variations in how various agencies collect data and use it. For officials hoping to implement citizen science projects within government, the roadblocks to crowdsourcing include factors that crowdsourcing is intended to avoid: limited budgets, heavy bureaucracy, and superiors who are skeptical about the value of relying on the crowd for data.
Benforado and Shanley also pointed out that government agencies are subject to additional regulations, such as the Paperwork Reduction Act, which can make implementation of crowdsourcing projects more challenging than they would be in academia or the private sector… (More)”

The Free 'Big Data' Sources Everyone Should Know


Bernard Marr at Linkedin Pulse: “…The moves by companies and governments to put large amounts of information into the public domain have made large volumes of data accessible to everyone….here’s my rundown of some of the best free big data sources available today.

Data.gov

The US Government pledged last year to make all government data available freely online. This site is the first stage and acts as a portal to all sorts of amazing information on everything from climate to crime. To check it out, click here.

US Census Bureau

A wealth of information on the lives of US citizens covering population data, geographic data and education. To check it out, click here. To check it out, click here.

European Union Open Data Portal

As the above, but based on data from European Union institutions. To check it out, click here.

Data.gov.uk

Data from the UK Government, including the British National Bibliography – metadata on all UK books and publications since 1950. To check it out, click here.

The CIA World Factbook

Information on history, population, economy, government, infrastructure and military of 267 countries. To check it out, click here.

Healthdata.gov

125 years of US healthcare data including claim-level Medicare data, epidemiology and population statistics. To check it out, click here.

NHS Health and Social Care Information Centre

Health data sets from the UK National Health Service. To check it out, click here.

Amazon Web Services public datasets

Huge resource of public data, including the 1000 Genome Project, an attempt to build the most comprehensive database of human genetic information and NASA’s database of satellite imagery of Earth. To check it out, click here.

Facebook Graph

Although much of the information on users’ Facebook profile is private, a lot isn’t – Facebook provide the Graph API as a way of querying the huge amount of information that its users are happy to share with the world (or can’t hide because they haven’t worked out how the privacy settings work). To check it out, click here.

Gapminder

Compilation of data from sources including the World Health Organization and World Bank covering economic, medical and social statistics from around the world. To check it out, click here.

Google Trends

Statistics on search volume (as a proportion of total search) for any given term, since 2004. To check it out, click here.

Google Finance

40 years’ worth of stock market data, updated in real time. To check it out, click here.

Google Books Ngrams

Search and analyze the full text of any of the millions of books digitised as part of the Google Books project. To check it out, click here.

National Climatic Data Center

Huge collection of environmental, meteorological and climate data sets from the US National Climatic Data Center. The world’s largest archive of weather data. To check it out, click here.

DBPedia

Wikipedia is comprised of millions of pieces of data, structured and unstructured on every subject under the sun. DBPedia is an ambitious project to catalogue and create a public, freely distributable database allowing anyone to analyze this data. To check it out, click here.

Topsy

Free, comprehensive social media data is hard to come by – after all their data is what generates profits for the big players (Facebook, Twitter etc) so they don’t want to give it away. However Topsy provides a searchable database of public tweets going back to 2006 as well as several tools to analyze the conversations. To check it out, click here.

Likebutton

Mines Facebook’s public data – globally and from your own network – to give an overview of what people “Like” at the moment. To check it out, click here.

New York Times

Searchable, indexed archive of news articles going back to 1851. To check it out, click here.

Freebase

A community-compiled database of structured data about people, places and things, with over 45 million entries. To check it out, click here.

Million Song Data Set

Metadata on over a million songs and pieces of music. Part of Amazon Web Services. To check it out, click here.”
See also Bernard Marr‘s blog at Big Data Guru

4 Tech Trends Changing How Cities Operate


at Governing: “Louis Brandeis famously characterized states as laboratories for democracy, but cities could be called labs for innovation or new practices….When Government Technology magazine (produced by Governing’s parent company, e.Republic, Inc.) published its annual Digital Cities Survey, the results provided an interesting look at how local governments are using technology to improve how they deliver services, increase production and streamline operations…the survey also showed four technology trends changing how local government operates and serves its citizens:

1. Open Data

…Big cities were the first to open up their data and gained national attention for their transparency. New York City, which passed an open data law in 2012, leads all cities with more than 1,300 data sets open to the public; Chicago started opening up data to the public in 2010 following an executive order and is second among cities with more than 600; and San Francisco, which was the first major city to open the doors to transparency in 2009, had the highest score from the U.S. Open Data Census for the quality of its open data.
But the survey shows that a growing number of mid-sized jurisdictions are now getting involved, too. Tacoma, Wash., has a portal with 40 data sets that show how the city is spending tax dollars on public works, economic development, transportation and public safety. Ann Arbor, Mich., has a financial transparency tool that reveals what the city is spending on a daily basis, in some cases….

2. ‘Stat’ Programs and Data Analytics

…First, the so-called “stat” programs are proliferating. Started by the New York Police Department in the 1980s, CompStat was a management technique that merged data with staff feedback to drive better performance by police officers and precinct captains. Its success led to many imitations over the years and, as the digital survey shows, stat programs continue to grow in importance. For example, Louisville has used its “LouieStat” program to cut the city’s bill for unscheduled employee overtime by $23 million as well as to spot weaknesses in performance.
Second, cities are increasing their use of data analytics to measure and improve performance. Denver, Jacksonville, Fla., and Phoenix have launched programs that sift through data sets to find patterns that can lead to better governance decisions. Los Angeles has combined transparency with analytics to create an online system that tracks performance for the city’s economy, service delivery, public safety and government operations that the public can view. Robert J. O’Neill Jr., executive director of the International City/County Management Association, said that both of these tech-driven performance trends “enable real-time decision-making.” He argued that public leaders who grasp the significance of these new tools can deliver government services that today’s constituents expect.

3. Online Citizen Engagement

…Avondale, Ariz., population 78,822, is engaging citizens with a mobile app and an online forum that solicits ideas that other residents can vote up or down.
In Westminster, Colo., population 110,945, a similar forum allows citizens to vote online about community ideas and gives rewards to users who engage with the online forum on a regular basis (free passes to a local driving range or fitness program). Cities are promoting more engagement activities to combat a decline in public trust in government. The days when a public meeting could provide citizen engagement aren’t enough in today’s technology-dominated  world. That’s why social media tools, online surveys and even e-commerce rewards programs are popping up in cities around the country to create high-value interaction with its citizens.

4. Geographic Information Systems

… Cities now use them to analyze financial decisions to increase performance, support public safety, improve public transit, run social service activities and, increasingly, engage citizens about their city’s governance.
Augusta, Ga., won an award for its well-designed and easy-to-use transit maps. Sugar Land, Texas, uses GIS to support economic development and, as part of its citizen engagement efforts, to highlight its capital improvement projects. GIS is now used citywide by 92 percent of the survey respondents. That’s significant because GIS has long been considered a specialized (and expensive) technology primarily for city planning and environmental projects….”