Jeanne Holm at Digital Gov: “Open government, open source, openness. These words are often used in talking about open data, but we sometimes forget that the root of all of this is an open community. Individuals working together to release government data and put it to use to help their neighbors and reach new personal goals.
This sense of community in the open data field shows up in many places. I see it when people volunteer at the National Day of Civic Hacking, crowdsource data integrity with MapGive, or mentor with Girls Who Code. And each day I see it on Open Data Stack Exchange, where people ask questions about open data issues, searches, or challenges, and strangers half a world away answer the question within an hour.
We launched the Open Data Stack Exchange in 2013 as a way of helping to build community and open up the knowledge in our emergent field. What started slowly, soon took off with 3,375 participants today having provided 1,592 answers to 721 questions. Anyone can ask a question. These have ranged from data requests (looking for specific hard-to-find data) to technical questions on parsing or visualizing data. More importantly, anyone can answer a question, too. You’ll notice from the numbers that most questions have more than one answer, with the asker being able to choose the best answer and everyone being able to vote the questions and answers up and down. The forum is loosely moderated (I’ve served as one of the moderators since inception), but predominantly self-governed. Google trusts this method and forum so much that within a few minutes of answering a question, it will pop to the top of the Google search results for that topic.
What are people asking on the Open Data Stack Exchange? One question is seeking applications being developed with open data, one is looking for a database of open databases and another seeks data about the Ebola outbreak. Answers, edits, comments, suggestions…all are part of the conversation and documentation of our collective open data knowledge. This type of community-vetted, open forum helps to evolve and preserve our collective wisdom into the future. I encourage people who ask questions of Data.gov to do so on Stack Exchange so that everyone can see the answer, and flag those for easy reference (OpenFDA does the same)…”
How Open Data Is Transforming City Life
Joel Gurin, The GovLab, at Techonomy: “Start a business. Manage your power use. Find cheap rents, or avoid crime-ridden neighborhoods. Cities and their citizens worldwide are discovering the power of “open data”—public data and information available from government and other sources that can help solve civic problems and create new business opportunities. By opening up data about transportation, education, health care, and more, municipal governments are helping app developers, civil society organizations, and others to find innovative ways to tackle urban problems. For any city that wants to promote entrepreneurship and economic development, open data can be a valuable new resource.
The urban open data movement has been growing for several years, with American cities including New York, San Francisco, Chicago, and Washington in the forefront. Now an increasing number of government officials, entrepreneurs, and civic hackers are recognizing the potential of open data. The results have included applications that can be used across many cities as well as those tailored to an individual city’s needs.
At first, the open data movement was driven by a commitment to transparency and accountability. City, state, and local governments have all released data about their finances and operations in the interest of good government and citizen participation. Now some tech companies are providing platforms to make this kind of city data more accessible, useful, and comparable. Companies like OpenGov and Govini make it possible for city managers and residents to examine finances, assess police department overtime, and monitor other factors that let them compare their city’s performance to neighboring municipalities.
Other new businesses are tapping city data to provide residents with useful, practical information. One of the best examples is NextBus, which uses metropolitan transportation data to tell commuters when to expect a bus along their route. Commuter apps like this have become common in cities in the U.S. and around the world. Another website, SpotCrime, collects, analyzes, and maps crime statistics to tell city dwellers which areas are safest or most dangerous and to offer crime alerts. And the Chicago-based Purple Binder helps people in need find city healthcare services. Many companies in the Open Data 500, the study of open data companies that I direct at the GovLab at NYU, use data from cities as well as other sources….
Some of the most ambitious uses of city data—with some of the greatest potential—focus on improving education. In Washington, the nonprofit Learn DC has made data about public schools available through a portal that state agencies, community organizations, and civic hackers can all use. They’re using it for collaborative research and action that, they say, has “empowered every DC parent to participate in shaping the future of the public education system.”…”
In democracy and disaster, emerging world embraces 'open data'
Jeremy Wagstaff’ at Reuters: “Open data’ – the trove of data-sets made publicly available by governments, organizations and businesses – isn’t normally linked to high-wire politics, but just may have saved last month’s Indonesian presidential elections from chaos.
Data is considered open when it’s released for anyone to use and in a format that’s easy for computers to read. The uses are largely commercial, such as the GPS data from U.S.-owned satellites, but data can range from budget numbers and climate and health statistics to bus and rail timetables.
It’s a revolution that’s swept the developed world in recent years as governments and agencies like the World Bank have freed up hundreds of thousands of data-sets for use by anyone who sees a use for them. Data.gov, a U.S. site, lists more than 100,000 data-sets, from food calories to magnetic fields in space.
Consultants McKinsey reckon open data could add up to $3 trillion worth of economic activity a year – from performance ratings that help parents find the best schools to governments saving money by releasing budget data and asking citizens to come up with cost-cutting ideas. All the apps, services and equipment that tap the GPS satellites, for example, generate $96 billion of economic activity each year in the United States alone, according to a 2011 study.
But so far open data has had a limited impact in the developing world, where officials are wary of giving away too much information, and where there’s the issue of just how useful it might be: for most people in emerging countries, property prices and bus schedules aren’t top priorities.
But last month’s election in Indonesia – a contentious face-off between a disgraced general and a furniture-exporter turned reformist – highlighted how powerful open data can be in tandem with a handful of tech-smart programmers, social media savvy and crowdsourcing.
“Open data may well have saved this election,” said Paul Rowland, a Jakarta-based consultant on democracy and governance…”
Assessing Social Value in Open Data Initiatives: A Framework
Paper by Gianluigi Viscusi, Marco Castelli and Carlo Batini in Future Internet Journal: “Open data initiatives are characterized, in several countries, by a great extension of the number of data sets made available for access by public administrations, constituencies, businesses and other actors, such as journalists, international institutions and academics, to mention a few. However, most of the open data sets rely on selection criteria, based on a technology-driven perspective, rather than a focus on the potential public and social value of data to be published. Several experiences and reports confirm this issue, such as those of the Open Data Census. However, there are also relevant best practices. The goal of this paper is to investigate the different dimensions of a framework suitable to support public administrations, as well as constituencies, in assessing and benchmarking the social value of open data initiatives. The framework is tested on three initiatives, referring to three different countries, Italy, the United Kingdom and Tunisia. The countries have been selected to provide a focus on European and Mediterranean countries, considering also the difference in legal frameworks (civic law vs. common law countries)”
Out in the Open: This Man Wants to Turn Data Into Free Food (And So Much More)
Klint Finley in Wired: “Let’s say your city releases a list of all trees planted on its public property. It would be a godsend—at least in theory. You could filter the data into a list of all the fruit and nut trees in the city, transfer it into an online database, and create a smartphone app that helps anyone find free food.
Such is promise of “open data”—the massive troves of public information our governments now post to the net. The hope is that, if governments share enough of this data with the world at large, hackers and entrepreneurs will find a way of putting it to good use. But although so much of this government data is now available, the revolution hasn’t exactly happened.
In far too many cases, the data just sits there on a computer server, unseen and unused. Sometimes, no one knows about the data, or no one knows what to do with it. Other times, the data is just too hard to work with. If you’re building that free food app, how do you update your database when the government releases a new version of the spreadsheet? And if you let people report corrections to the data, how do you contribute that data back to the city?
These are the sorts of problems that obsess 25-year-old software developer Max Ogden, and they’re the reason he built Dat, a new piece of open source software that seeks to restart the open data revolution. Basically, Dat is a way of synchronizing data between two or more sources, tracking any changes to that data, and handling transformations from one data format to another. The aim is a simple one: Ogden wants to make it easier for governments to share their data with a world of software developers.
That’s just the sort of thing that government agencies are looking for, says Waldo Jaquith, the director of US Open Data Institute, the non-profit that is now hosting Dat…
Git is a piece of software originally written by Linux creator Linus Torvalds. It keeps track of code changes and makes it easier to integrate code submissions from outside developers. Ogden realized what developers needed wasn’t a GitHub for data, but a Git for data. And that’s what Dat is.
Instead of CouchDB, Dat relies on a lightweight, open-source data storage system from Google called LevelDB. The rest of the software was written in JavaScript by Ogden and his growing number of collaborators, which enables them to keep things minimal and easily run the software on multiple operating systems, including Windows, Linux and Macintosh OS X….”
“
In far too many cases, the data just sits there on a computer server, unseen and unused. Sometimes, no one knows about the data, or no one knows what to do with it. Other times, the data is just too hard to work with. If you’re building that free food app, how do you update your database when the government releases a new version of the spreadsheet? And if you let people report corrections to the data, how do you contribute that data back to the city?
These are the sorts of problems that obsess 25-year-old software developer Max Ogden, and they’re the reason he built Dat, a new piece of open source software that seeks to restart the open data revolution. Basically, Dat is a way of synchronizing data between two or more sources, tracking any changes to that data, and handling transformations from one data format to another. The aim is a simple one: Ogden wants to make it easier for governments to share their data with a world of software developers.
That’s just the sort of thing that government agencies are looking for, says Waldo Jaquith, the director of US Open Data Institute, the non-profit that is now hosting Dat…
Git is a piece of software originally written by Linux creator Linus Torvalds. It keeps track of code changes and makes it easier to integrate code submissions from outside developers. Ogden realized what developers needed wasn’t a GitHub for data, but a Git for data. And that’s what Dat is.
Instead of CouchDB, Dat relies on a lightweight, open-source data storage system from Google called LevelDB. The rest of the software was written in JavaScript by Ogden and his growing number of collaborators, which enables them to keep things minimal and easily run the software on multiple operating systems, including Windows, Linux and Macintosh OS X….”
“
How technology is beating corruption
Jim Yong Kim at World Economic Forum: “Good governance is critical for all countries around the world today. When it doesn’t exist, many governments fail to deliver public services effectively, health and education services are often substandard and corruption persists in rich and poor countries alike, choking opportunity and growth. It will be difficult to reduce extreme poverty — let alone end it — without addressing the importance of good governance.
But this is not a hopeless situation. In fact, a new wave of progress on governance suggests we may be on the threshold of a transformational era. Countries are tapping into some of the most powerful forces in the world today to improve services and transparency. These forces include the spread of information technology and its convergence with grassroots movements for transparency, accountability and citizen empowerment. In some places, this convergence is easing the path to better-performing and more accountable governments.
The Philippines is a good example of a country embracing good governance. During a recent visit, I spoke with President Benigno Aquino about his plans to reduce poverty, create jobs, and ensure that economic growth is inclusive. He talked in great detail about how improving governance is a fundamentally important part of their strategy. The government has opened government data and contract information so citizens can see how their tax money is spent. The Foreign Aid Transparency Hub, launched after Typhoon Yolanda, offers a real-time look at pledges made and money delivered for typhoon recovery. Geo-tagging tools monitor assistance for people affected by the typhoon.
Opening budgets to scrutiny
This type of openness is spreading. Now many countries that once withheld information are opening their data and budgets to public scrutiny.
Late last year, my organization, the World Bank Group, established the Open Budgets Portal, a repository for budget data worldwide. So far, 13 countries have posted their entire public spending datasets online — including Togo, the first fragile state to do so.
In 2011, we helped Moldova become the first country in central Europe to launch an open data portal and put its expenditures online. Now the public and media can access more than 700 datasets, and are asking for more.
The original epicenter of the Arab Spring, Tunisia, recently passed a new constitution and is developing the first open budget data portal in the Middle East and North Africa. Tunisia has taken steps towards citizen engagement by developing a citizens’ budget and civil society-led platforms such as Marsoum41, to support freedom of information requests, including via mobile.
Using technology to improve services
Countries also are tapping into technology to improve public and private services. Estonia is famous for building an information technology infrastructure that has permitted widespread use of electronic services — everything from filing taxes online to filling doctors’ drug prescriptions.
In La Paz, Bolivia, a citizen feedback system known as OnTrack allows residents of one of the city’s marginalized neighbourhoods to send a text message on their mobile phones to provide feedback, make suggestions or report a problem related to public services.
In Pakistan, government departments in Punjab are using smart phones to collect real-time data on the activities of government field staff — including photos and geo-tags — to help reduce absenteeism and lax performance….”
Technology’s Crucial Role in the Fight Against Hunger
Crowdsourcing, predictive analytics and other new tools could go far toward finding innovative solutions for America’s food insecurity.
National Geographic recently sent three photographers to explore hunger in the United States. It was an effort to give a face to a very troubling statistic: Even today, one-sixth of Americans do not have enough food to eat. Fifty million people in this country are “food insecure” — having to make daily trade-offs among paying for food, housing or medical care — and 17 million of them skip at least one meal a day to get by. When choosing what to eat, many of these individuals must make choices between lesser quantities of higher-quality food and larger quantities of less-nutritious processed foods, the consumption of which often leads to expensive health problems down the road.
This is an extremely serious, but not easily visible, social problem. Nor does the challenge it poses become any easier when poorly designed public-assistance programs continue to count the sauce on a pizza as a vegetable. The deficiencies caused by hunger increase the likelihood that a child will drop out of school, lowering her lifetime earning potential. In 2010 alone, food insecurity cost America $167.5 billion, a figure that includes lost economic productivity, avoidable health-care expenses and social-services programs.
As much as we need specific policy innovations, if we are to eliminate hunger in America food insecurity is just one of many extraordinarily complex and interdependent “systemic” problems facing us that would benefit from the application of technology, not just to identify innovative solutions but to implement them as well. In addition to laudable policy initiatives by such states as Illinois and Nevada, which have made hunger a priority, or Arkansas, which suffers the greatest level of food insecurity but which is making great strides at providing breakfast to schoolchildren, we can — we must — bring technology to bear to create a sustained conversation between government and citizens to engage more Americans in the fight against hunger.
Identifying who is genuinely in need cannot be done as well by a centralized government bureaucracy — even one with regional offices — as it can through a distributed network of individuals and organizations able to pinpoint with on-the-ground accuracy where the demand is greatest. Just as Ushahidi uses crowdsourcing to help locate and identify disaster victims, it should be possible to leverage the crowd to spot victims of hunger. As it stands, attempts to eradicate so-called food deserts are often built around developing solutions for residents rather than with residents. Strategies to date tend to focus on the introduction of new grocery stores or farmers’ markets but with little input from or involvement of the citizens actually affected.
Applying predictive analytics to newly available sources of public as well as private data, such as that regularly gathered by supermarkets and other vendors, could also make it easier to offer coupons and discounts to those most in need. In addition, analyzing nonprofits’ tax returns, which are legally open and available to all, could help map where the organizations serving those in need leave gaps that need to be closed by other efforts. The Governance Lab recently brought together U.S. Department of Agriculture officials with companies that use USDA data in an effort to focus on strategies supporting a White House initiative to use climate-change and other open data to improve food production.
Such innovative uses of technology, which put citizens at the center of the service-delivery process and streamline the delivery of government support, could also speed the delivery of benefits, thus reducing both costs and, every bit as important, the indignity of applying for assistance.
Being open to new and creative ideas from outside government through brainstorming and crowdsourcing exercises using social media can go beyond simply improving the quality of the services delivered. Some of these ideas, such as those arising from exciting new social-science experiments involving the use of incentives for “nudging” people to change their behaviors, might even lead them to purchase more healthful food.
Further, new kinds of public-private collaborative partnerships could create the means for people to produce their own food. Both new kinds of financing arrangements and new apps for managing the shared use of common real estate could make more community gardens possible. Similarly, with the kind of attention, convening and funding that government can bring to an issue, new neighbor-helping-neighbor programs — where, for example, people take turns shopping and cooking for one another to alleviate time away from work — could be scaled up.
Then, too, advances in citizen engagement and oversight could make it more difficult for lawmakers to cave to the pressures of lobbying groups that push for subsidies for those crops, such as white potatoes and corn, that result in our current large-scale reliance on less-nutritious foods. At the same time, citizen scientists reporting data through an app would be able do a much better job than government inspectors in reporting what is and is not working in local communities.
As a society, we may not yet be able to banish hunger entirely. But if we commit to using new technologies and mechanisms of citizen engagement widely and wisely, we could vastly reduce its power to do harm.
State Open Data Policies and Portals
New report by Laura Drees and Daniel Castro at the Center for Data Innovation: “This report provides a snapshot of states’ efforts to create open data policies and portals and ranks states on their progress. The six top-scoring states are Hawaii, Illinois, Maryland, New York, Oklahoma, and Utah. Each of these states has established an open data policy that requires basic government data, such as expenditure information, as well as other agency data, to be published on their open data portals in a machine-readable format. These portals contain extensive catalogs of open data, are relatively simple to navigate, and provide data in machine-readable formats as required. The next highest-ranked state, Connecticut, offers a similarly serviceable, machine-readable open data portal that provides wide varieties of information, but its policy does not requiremachine readability. Of the next three top-ranking states, Texas’s and Rhode Island’s policies require neither machine readability nor government data beyond expenditures; New Hampshire’s policy requires machine readability and many types of data, but its open data portal is not yet fully functional. States creating new open data policies or portals, or refreshing old ones, have many opportunities to learn from the experiences of early adopters in order to fully realize the benefits of data-driven innovation.”
Download printer-friendly PDF.
Open Data: Going Beyond Solving Problems to Making the Impossible Possible
Lisa Abeyta at the Huffington Post: “As a global community, we are producing data at an astounding rate. The pace was recently described as a “new Google every four days” by the highly respected Andreesen Horowitz partner, Peter Levine, in a thought-provoking post addressing the challenge of making sense of this mountain of data.
“… we are now collecting more data each day, so much that 90 percent of the data in the world today has been created in the last two years alone. In fact, every day, we create 2.5 quintillion bytes of data — by some estimates that’s one new Google every four days, and the rate is only increasing. Our desire to use, interact, and learn from this data will become increasingly important and strategic to businesses and society as a whole.”
For the past year and a half, my cofounders and team have focused on what it will take to use, interact and learn from data being produced within the civic sector. It’s one thing to be able to build an app for civic; it’s quite another to build a platform that can manage multiple apps across multiple platforms while addressing the challenges plaguing the “wild west” nature of growth in the quickly emerging market of open data. I was recently invited to share our lessons learned and the promise of the future in mobile open data at the TEDxABQ Technology Salon. Here is a bit of what I shared:
Beyond Solving Problems to New Possibilities
When we first looked at the opportunities for creating apps built on open data, our priority was finding pain points for both cities and the people who lived there. We focused on solving real problems, and it led to some early success. We worked with the City of Albuquerque to deploy their ABQ RIDE app on iOS and Android platforms, and the app not only solved real problems for riders, it also saved real money for the city. The app has grown to over 20,000 regular users and continues to be one of the highest downloaded apps on our platform.
But recently, we’ve started asking questions that go beyond the basic, that change the experience or make things possible in ways that never were before. Here are two I’m incredibly proud to be a part of…”
Government opens up: 10k active government users on GitHub
GitHub: “In the summer of 2009, The New York Senate was the first government organization to post code to GitHub, and that fall, Washington DC quickly followed suit. By 2011, cities like Miami, Chicago, and New York; Australian, Canadian, and British government initiatives like Gov.uk; and US Federal agencies like the Federal Communications Commission, General Services Administration, NASA, and Consumer Financial Protection Bureau were all coding in the open as they began to reimagine government for the 21st century.
Fast forward to just last year: The White House Open Data Policy is published as a collaborative, living document, San Francisco laws are now forkable, and government agencies are accepting pull requests from every day developers.
This is all part of a larger trend towards government adopting open source practices and workflows — a trend that spans not only software, but data, and policy as well — and the movement shows no signs of slowing, with government usage on GitHub nearly tripling in the past year, to exceed 10,000 active government users today.
How government uses GitHub
When government works in the open, it acknowledges the idea that government is the world’s largest and longest-running open source project. Open data efforts, efforts like the City of Philadelphia’s open flu shot spec, release machine-readable data in open, immediately consumable formats, inviting feedback (and corrections) from the general public, and fundamentally exposing who made what change when, a necessary check on democracy.
Unlike the private sector, however, where open sourcing the “secret sauce” may hurt the bottom line, with government, we’re all on the same team. With the exception of say, football, Illinois and Wisconsin don’t compete with one another, nor are the types of challenges they face unique. Shared code prevents reinventing the wheel and helps taxpayer dollars go further, with efforts like the White House’s recently released Digital Services Playbook, an effort which invites every day citizens to play a role in making government better, one commit at a time.
However, not all government code is open source. We see that adopting these open source workflows for open collaboration within an agency (or with outside contractors) similarly breaks down bureaucratic walls, and gives like-minded teams the opportunity to work together on common challenges.
Government Today
It’s hard to believe that what started with a single repository just five years ago, has blossomed into a movement where today, more than 10,000 government employees use GitHub to collaborate on code, data, and policy each day….
You can learn more about GitHub in government at government.github.com, and if you’re a government employee, be sure to join our semi-private peer group to learn best practices for collaborating on software, data, and policy in the open.”