Report by Kathryn L.S. Pettit, Leah Hendey, Brianna Losoya, G. Thomas Kingsley at the Urban Institute: “The National Neighborhood Indicators Partnership (NNIP) is a network of local organizations that collect, organize, and use neighborhood data to tackle issues in their communities. As the movement for government transparency has spread at the local level, more NNIP partners are participating in the call for governments to release data and are using open data to provide information for decisionmaking and community engagement. Local NNIP partners and open data advocates have complementary strengths and should work together to more effectively advance open government data that benefits all residents.”
In Defense of Transit Apps
Mark Headd at Civic Innovations: “The civic technology community has a love-hate relationship with transit apps.
We love to, and often do, use the example of open transit data and the cottage industry of civic app development it has helped spawn as justification for governments releasing open data. Some of the earliest, most enduring and most successful civic applications have been built on transit data and there literally hundreds of different apps available.
The General Transit Feed Specification (GTFS), which has helped to encourage the release of transit data from dozens and dozens of transportation authorities across the country, is used as the model for the development of other open data standards. I once described work being done to develop a data standard for locations dispensing vaccinations as “GTFS for flu shots.”
But some in the civic technology community chafe at the overuse of transit apps as the example cited for the release of open data and engagement with outside civic hackers. Surely there are other examples we can point to that get at deeper, more fundamental problems with civic engagement and the operation of government. Is the best articulation of the benefits of open data and civic hacking a simple bus stop application?
Last week at Transparency Camp in DC, during a session I ran on open data, I was asked what data governments should focus on releasing as open data. I stated my belief that – at a minimum – governments should concentrate on The 3 B’s: Buses (transit data), Bullets (crime data) and Bucks (budget & expenditure data).
To be clear – transit data and the apps it helps generate are critical to the open data and civic technology movements. I think it is vital to exploring the role that transit apps have played in the development of the civic technology ecosystem and their impact on open data.
Story telling with transit data
Transit data supports more than just “next bus” apps. In fact, characterizing all transit apps this way does a disservice to the talented and creative people working to build things with transit data. Transit data supports a wide range of different visualizations that can tell an intimate, granular story about how a transit system works and how it’s operation impacts a city.
One inspiring example of this kind of app was developed recently by Mike Barry and Brian Card, and looked at the operation of MBTA in Boston. Their motive was simple:
We attempt to present this information to help people in Boston better understand the trains, how people use the trains, and how the people and trains interact with each other.
We’re able to tell nuanced stories about transit systems because the quality of data being released continues to expand and improve in quality. This happens because developers building apps in cities across the country have provided feedback to transit officials on what they want to see and the quality of what is provided.
Developers building the powerful visualizations we see today are standing on the shoulders of the people that built the “next bus” apps a few years ago. Without these humble apps, we don’t get to tell these powerful stories today.
Holding government accountable
Transit apps are about more than just getting to the train on time.
Support for transit system operations can run into the billions of dollars and affect the lives of millions of people in an urban area. With this much investment, it’s important that transit riders and taxpayers are able to hold officials accountable for the efficient operation of transit systems. To help us do this, we now have a new generation of transit apps that can examine things like the scheduled arrival and departure times of trains with their actual arrival and departure time.
Not only does this give citizens transparency into how well their transit system is being run, it offers a pathway for engagement – by knowing which routes are not performing close to scheduled times, transit riders and others can offer suggestions for changes and improvements.
A gateway to more open data
One of the most important things that transit apps can do is provide a pathway for more open data.
In Philadelphia, the city’s formal open data policy and the creation of an open data portal all followed after the efforts of a small group of developers working to obtain transit schedule data from the Southeastern Pennsylvania Transportation Authority (SEPTA). This group eventually built the region’s first transit app.
This small group pushed SEPTA to make their data open, and the Authority eventually embraced open data. This, in turn, raised the profile of open data with other city leaders and directly contributed to the adoption of an open data policy by the City of Philadelphia several years later. Without this simple transit app and the push for more open transit data, I don’t think this would have happened. Certainly not as soon as it did.
And it isn’t just big cities like Philadelphia. In Syracuse, NY – a small city with no tradition of civic hacking and no formal open data program – a group at a local hackathon decided that they wanted to build a platform for government open data.
The first data source they selected to focus on? Transit data. The first app they built? A transit app…”
The Art and Science of Data-driven Journalism
Alex Howard for the Tow Center for digital journalism: “Journalists have been using data in their stories for as long as the profession has existed. A revolution in computing in the 20th century created opportunities for data integration into investigations, as journalists began to bring technology into their work. In the 21st century, a revolution in connectivity is leading the media toward new horizons. The Internet, cloud computing, agile development, mobile devices, and open source software have transformed the practice of journalism, leading to the emergence of a new term: data journalism. Although journalists have been using data in their stories for as long as they have been engaged in reporting, data journalism is more than traditional journalism with more data. Decades after early pioneers successfully applied computer-assisted reporting and social science to investigative journalism, journalists are creating news apps and interactive features that help people understand data, explore it, and act upon the insights derived from it. New business models are emerging in which data is a raw material for profit, impact, and insight, co-created with an audience that was formerly reduced to passive consumption. Journalists around the world are grappling with the excitement and the challenge of telling compelling stories by harnessing the vast quantity of data that our increasingly networked lives, devices, businesses, and governments produce every day. While the potential of data journalism is immense, the pitfalls and challenges to its adoption throughout the media are similarly significant, from digital literacy to competition for scarce resources in newsrooms. Global threats to press freedom, digital security, and limited access to data create difficult working conditions for journalists in many countries. A combination of peer-to-peer learning, mentorship, online training, open data initiatives, and new programs at journalism schools rising to the challenge, however, offer reasons to be optimistic about more journalists learning to treat data as a source. (Download the report)”
How NYC Open Data and Reddit Saved New Yorkers Over $55,000 a Year
IQuantNY: “NYC generates an enormous amount of data each year, and for the most part, it stays behind closed doors. But thanks to the Open Data movement, signed into law by Bloomberg in 2012 and championed over the last several years by Borough President Gale Brewer, along with other council members, we now get to see a small slice of what the city knows. And that slice is growing.
There have been some detractors along the way; a senior attorney for the NYPD said in 2012 during a council hearing that releasing NYPD data in csv format was a problem because they were “concerned with the integrity of the data itself” and because “data could be manipulated by people who want ‘to make a point’ of some sort”. But our democracy is built on the idea of free speech; we let all the information out and then let reason lead the way.
In some ways, Open Data adds another check and balance into government: its citizens. I’ve watched the perfect example of this check work itself out over the past month. You may have caught my post that used parking ticket data to identify the fire hydrant in New York City that was generating the most income for the city in the form of fines: $33,000 a year. And on the next block, the second most profitable hydrant was generating $24,000 a year. That’s two consecutive blocks with hydrants generating over $55,000 a year. But there was a problem. In my post, I laid out why these two parking spots were extremely confusing and basically seemed like a trap; there was a wide “curb extension” between the street and the hydrant, making it appear like the hydrant was not by the street. Additionally, the DOT had painted parking spots right where you would be fined if you parked.
Once the data was out there, the hydrant took on a life of its own. First, it raised to the top of the nyc sub-reddit. That is basically one way that the internet voted that this is in-fact “interesting”. And that is how things go from small to big. From there, it travelled to the New York Observer, which was able to get a comment from the DOT. After that, it appeared in the New York Post, the post was republished in Gothamist and finally it even went global in the Daily Mail.
I guess the pressure was on the DOT at this point, as each media source reached out for comment, but what struck me was their response to the Observer:
“While DOT has not received any complaints about this location, we will review the roadway markings and make any appropriate alterations”
Why does someone have to complain in order for the DOT to see problems like this? In fact, the DOT just redesigned every parking sign in New York because some of the old ones were considered confusing. But if this hydrant was news to them, it implies that they did not utilize the very strongest source of measuring confusion on our streets: NYC parking tickets….”
A brief history of open data
Article by Luke Fretwell in FCW: “In December 2007, 30 open-data pioneers gathered in Sebastopol, Calif., and penned a set of eight open-government data principles that inaugurated a new era of democratic innovation and economic opportunity.
“The objective…was to find a simple way to express values that a bunch of us think are pretty common, and these are values about how the government could make its data available in a way that enables a wider range of people to help make the government function better,” Harvard Law School Professor Larry Lessig said. “That means more transparency in what the government is doing and more opportunity for people to leverage government data to produce insights or other great business models.”
The eight simple principles — that data should be complete, primary, timely, accessible, machine-processable, nondiscriminatory, nonproprietary and license-free — still serve as the foundation for what has become a burgeoning open-data movement.
In the seven years since those principles were released, governments around the world have adopted open-data initiatives and launched platforms that empower researchers, journalists and entrepreneurs to mine this new raw material and its potential to uncover new discoveries and opportunities. Open data has drawn civic hacker enthusiasts around the world, fueling hackathons, challenges, apps contests, barcamps and “datapaloozas” focused on issues as varied as health, energy, finance, transportation and municipal innovation.
In the United States, the federal government initiated the beginnings of a wide-scale open-data agenda on President Barack Obama’s first day in office in January 2009, when he issued his memorandum on transparency and open government, which declared that “openness will strengthen our democracy and promote efficiency and effectiveness in government.” The president gave federal agencies three months to provide input into an open-government directive that would eventually outline what each agency planned to do with respect to civic transparency, collaboration and participation, including specific objectives related to releasing data to the public.
In May of that year, Data.gov launched with just 47 datasets and a vision to “increase public access to high-value, machine-readable datasets generated by the executive branch of the federal government.”
When the White House issued the final draft of its federal Open Government Directive later that year, the U.S. open-government data movement got its first tangible marching orders, including a 45-day deadline to open previously unreleased data to the public.
Now five years after its launch, Data.gov boasts more than 100,000 datasets from 227 local, state and federal agencies and organizations….”
Open Government Will Reshape Latin America
Alejandro Guerrero at Medium: “When people think on the place for innovations, they typically think on innovation being spurred by large firms and small startups based in the US. And particularly in that narrow stretch of land and water called Silicon Valley.
However, the flux of innovation taking place in the intersection between technology and government is phenomenal and emerging everywhere. From the marble hallways of parliaments everywhere —including Latin America’s legislative houses— to office hubs of tech-savvy non-profits full of enthusiastic social changers —also including Latin American startups— a driving force is starting to challenge our conception of how government and citizens can and should interact. And few people are discussing or analyzing these developments.
Open Government in Latin America
The potential for Open Government to improve government’s decision-making and performance is huge. And it is particularly immense in middle income countries such as the ones in Latin America, where the combination of growing incomes, more sophisticated citizens’ demands, and broken public services is generating a large bottom-up pressure and requesting more creative solutions from governments to meet the enormous social needs, while cutting down corruption and improving governance.
It is unsurprising that citizens from all over Latin America are increasingly taking the streets and demanding better public services and more transparent institutions.
While these protests are necessarily short-lived and unarticulated —a product of growing frustration with government— they are a symptom with deeper causes that won’t go easily away, and these protests will most likely come back with increasing frequency and the unresolved frustration may eventually transmute in political platforms with more radical ideas to challenge the status quo.
Behind the scene, governments across the region still face enormous weaknesses in public management, ill-prepared and underpaid public officials carry on with their duties as the platonic idea of a demotivated workforce, and the opportunities for corruption, waste, and nepotism are plenty. The growing segment of more affluent citizens simply opt out from government and resort to private alternatives, thus exacerbating inequalities in the already most unequal region in the world. The crumbling middle classes and the poor can just resort to voicing their complaints. And they are increasingly doing so.
And here is where open government initiatives might play a transformative role, disrupting the way governments make decisions and work while empowering citizens in the process.
The preconditions for OpenGov are almost here
In Latin America, connectivity rates are growing fast (reaching 61% in 2013 for the Americas as a whole), close to 90% of the population owns a cellphone, and access to higher levels of education keeps growing (as an example, the latest PISA report indicates that Mexico went from 58% in 2003 to 70% high-schoolers in 2012). The social conditions for a stronger role of citizens in government are increasingly there.
Moreover, most Latin American countries passed transparency laws during the 2000s, creating the enabling environment for open government initiatives to flourish. It is thus unsurprising that the next generation of young government bureaucrats, on average more internet-savvy and better educated than its predecessors, is taking over and embracing innovations in government. And they are finding echo (and suppliers of ideas and apps!) among local startups and civil society groups, while also being courted by large tech corporations (think of Google or Microsoft) behind succulent government contracts associated with this form of “doing good”.
This is an emerging galaxy of social innovators, technologically-savvy bureaucrats, and engaged citizens providing a large crowd-sourcing community and an opportunity to test different approaches. And the underlying tectonic shifts are pushing governments towards that direction. For a sampler, check out the latest developments for Brazil, Argentina, Peru, Mexico, Colombia, Paraguay, Chile, Panama, Costa Rica, Guatemala, Honduras, Dominican Republic, Uruguay and (why not?) my own country, which I will include in the review often for the surprisingly limited progress of open government in this OECD member, which shares similar institutions and challenges with Latin America.
A Road Full of Promise…and Obstacles
Most of the progress in Latin America is quite recent, and the real impact is still often more limited once you abandon the halls of the Digital Government directorates and secretarías or look if you look beyond the typical government data portal. The resistance to change is as human as laughing, but it is particularly intense among the public sector side of human beings. Politics also typically plays a enormous role in resisting transparency open government, and in a context of weak institutions and pervasive corruption, the temptation to politically block or water down open data/open government projects is just too high. Selective release of data (if any) is too frequent, government agencies often act as silos by not sharing information with other government departments, and irrational fears by policy-makers combined with adoption barriers (well explained here) all contribute to deter the progress of the open government promise in Latin America…”
Special Issue on Innovation through Open Data
A Review of the State-of-the-Art and an Emerging Research Agenda in the Journal of Theoretical and Applied Electronic Commerce Research:
- Going Beyond Open Data: Challenges and Motivations for Smart Disclosure in Ethical Consumption (Djoko Sigit Sayogo, Jing Zhang, Theresa A. Pardo, Giri K. Tayi, Jana Hrdinova, David F. Andersen and Luis Felipe Luna-Reyes)
- Shaping Local Open Data Initiatives: Politics and Implications (Josefin Lassinantti, Birgitta Bergvall-Kåreborn and Anna Ståhlbröst)
- A State-of-the-Art Analysis of the Current Public Data Landscape from a Functional, Semantic and Technical Perspective (Michael Petychakis, Olga Vasileiou, Charilaos Georgis, Spiros Mouzakitis and John Psarras)
- Using a Method and Tool for Hybrid Ontology Engineering: an Evaluation in the Flemish Research Information Space (Christophe Debruyne and Pieter De Leenheer)
- A Metrics-Driven Approach for Quality Assessment of Linked Open Data (Behshid Behkamal, Mohsen Kahani, Ebrahim Bagheri and Zoran Jeremic)
- Open Government Data Implementation Evaluation (Peter Parycek, Johann Höchtl and Michael Ginner)
- Data-Driven Innovation through Open Government Data (Thorhildur Jetzek, Michel Avital and Niels Bjorn-Andersen)
Blueprint on "The Open Data Era in Health and Social Care"
The GovLab Press Release: NHS England and The Governance Lab at NYU (The GovLab) have today launched a blueprint – The Open Data Era in Health and Social Care – for accelerating the use of open data in health and care settings.
The availability of open data can empower citizens and help care providers, patients and researchers make better decisions, spur new innovations and identify efficiencies. The report was commissioned by NHS England and written by The GovLab, part of New York University and world leaders in the field of open data usage. It puts forward a proposal for how the health and care system can maximise the impact of sharing open data through establishing priorities and clear ways of measuring benefits.
Tim Kelsey, National Director for Patients and Information for NHS England, said:
“There’s an urgent need for the NHS to use better information and evidence to guide decision-making and investment. We know with scientific and medical research, the rate of discovery is accelerated by better access to data. This report will kick off a conversation about how we can use open data in the NHS to build a meaningful evidence base to support better investment in health and care services. Over the coming months, I’m keen to hear the views of colleagues on how we can take this forward and build an evidence base to improve outcomes for patients.”
Stefaan Verhulst, Co-founder and Chief Research and Development Officer of the GovLab:
“The blueprint lays out a detailed plan to start a conversation about how to gather the evidence needed to understand and assess the shape and size of the impact of open health data. It is important to pay a comparable level of attention to an analysis of open data’s potential benefits, as well as potential risks.”
Download the full report: thegovlab.org/nhs
Open Data Is Open for Business
Jeffrey Stinson at Stateline: ” Last month, web designer Sean Wittmeyer and colleague Wojciech Magda walked away with a $25,000 prize from the state of Colorado for designing an online tool to help businesses decide where to locate in the state.
The tool, called “Beagle Score,” is a widget that can be embedded in online commercial real estate listings. It can rate a location by taxes and incentives, zoning, even the location of possible competitors – all derived from about 30 data sets posted publicly by the state of Colorado and its municipalities.
The creation of Beagle Score is an example of how states, cities, counties and the federal government are encouraging entrepreneurs to take raw government data posted on “open data” websites and turn the information into products the public will buy.
“The (Colorado contest) opened up a reason to use the data,” said Wittmeyer, 25, of Fort Collins. “It shows how ‘open data’ can solve a lot of challenges. … And absolutely, we can make it commercially viable. We can expand it to other states, and fairly quickly.”
Open-data advocates, such as President Barack Obama’s former information chief Vivek Kundra, estimate a multibillion-dollar industry can be spawned by taking raw government data files on sectors such as weather, population, energy, housing, commerce or transportation and turn them into products for the public to consume or other industries to pay for.
They can be as simple as mobile phone apps identifying every stop sign you will encounter on a trip to a different town, or as intricate as taking weather and crops data and turning it into insurance policies farmers can buy.
States, Cities Sponsor ‘Hackathons’
At least 39 states and 46 cities and counties have created open-data sites since the federal government, Utah, California and the cities of San Francisco and Washington, D.C., began opening data in 2009, according to the federal site, Data.gov.
Jeanne Holm, the federal government’s Data.gov “evangelist,” said new sites are popping up and new data are being posted almost daily. The city of Los Angeles, for example, opened a portal last week.
In March, Democratic New York Gov. Andrew Cuomo said that in the year since it was launched, his state’s site has grown to some 400 data sets with 50 million records from 45 agencies. Available are everything from horse injuries and deaths at state race tracks to maps of regulated child care centers. The most popular data: top fishing spots in the state.
State and local governments are sponsoring “hackathons,” “data paloozas,” and challenges like Colorado’s, inviting businesspeople, software developers, entrepreneurs or anyone with a laptop and a penchant for manipulating data to take part. Lexington, Kentucky, had a civic hackathon last weekend. The U.S. Transportation Department and members of the Geospatial Transportation Mapping Association had a three-day data palooza that ended Wednesday in Arlington, Virginia.
The goals of the events vary. Some, like Arlington’s transportation event, solicit ideas for how government can present its data more effectively. Others seek ideas for mining it.
Aldona Valicenti, Lexington’s chief information officer, said many cities want advice on how to use the data to make government more responsive to citizens, and to communicate with them on issues ranging from garbage pickups and snow removal to upcoming civic events.
Colorado and Wyoming had a joint hackathon last month sponsored by Google to help solve government problems. Colorado sought apps that might be useful to state emergency personnel in tracking people and moving supplies during floods, blizzards or other natural disasters. Wyoming sought help in making its tax-and-spend data more understandable and usable by its citizens.
Unless there’s some prize money, hackers may not make a buck from events like these, and participate out of fun, curiosity or a sense of public service. But those who create an app that is useful beyond the boundaries of a particular city or state, or one that is commercially valuable to business, can make serious money – just as Beagle Score plans to do. Colorado will hold onto the intellectual property rights to Beagle Score for a year. But Wittmeyer and his partner will be able to profit from extending it to other states.
States Trail in Open Data
Open data is an outgrowth of the e-government movement of the 1990s, in which government computerized more of the data it collected and began making it available on floppy disks.
States often have trailed the federal government or many cities in adjusting to the computer age and in sharing information, said Emily Shaw, national policy manager for the Sunlight Foundation, which promotes transparency in government. The first big push to share came with public accountability, or “checkbook” sites, that show where government gets its revenue and how it spends it.
The goal was to make government more transparent and accountable by offering taxpayers information on how their money was spent.
The Texas Comptroller of Public Accounts site, established in 2007, offers detailed revenue, spending, tax and contracts data. Republican Comptroller Susan Combs’ office said having a one-stop electronic site also has saved taxpayers about $12.3 million in labor, printing, postage and other costs.
Not all states’ checkbook sites are as openly transparent and detailed as Texas, Shaw said. Nor are their open-data sites. “There’s so much variation between the states,” she said.
Many state legislatures are working to set policies for releasing data. Since the start of 2010, according to the National Conference of State Legislatures, nine states have enacted open-data laws, and more legislation is pending. But California, for instance, has been posting open data for five years without legislation setting policies.
Just as states have lagged in getting data out to the public, less of it has been turned into commercial use, said Joel Gurin, senior adviser at the Governance Lab at New York University and author of the book “Open Data Now.”
Gurin leads Open Data 500, which identifies firms that that have made products from open government data and turned them into regional or national enterprises. In April, it listed 500. It soon may expand. “We’re finding more and more companies every day,” he said. “…
Open Government Data: Helping Parents to find the Best School for their Kids
Radu Cucos at the Open Government Partnership blog: “…This challenge – finding the right school – is probably one of the most important decisions in many parents’ lives. Parents are looking for answers to questions such as which schools are located in safe neighborhoods, which ones have the highest teacher – students’ ratio, which schools have the best funding, which schools have the best premises or which ones have the highest grades average.
It is rarely an easy decision, but is made doubly difficult in the case of migrants. People residing in the same location for a long time know, more or less, which are the best education institutions in their city, town or village. For migrants, the situation is absolutely the opposite. They have to spend extra time and resources in identifying relevant information about schools.
Open Government Data is an effective solution which can ease the problem of a lack of accessible information about existing schools in a particular country or location. By adopting the Open Government Data policy in the educational field, governments release data about grades, funding, student and teacher numbers, data generated throughout time by schools, colleges, universities and other educational settings.
Developers then use this data for creating applications which portray information in easy accessible formats. Three of the best apps which I have come across are highlighted below:
- Discover Your School, developed under the Province of British Columbia of Canada Open Data Initiative, is a platform for parents who are interested in finding a school for their kids, learning about the school districts or comparing schools in the same area. The application provides comprehensive information, such as the number of students enrolled in schools each year, class sizes, teaching language, disaster readiness, results of skills assessment, and student and parent satisfaction. Information and data can be viewed in interactive formats, including maps. On top of that, Discover Your School engages parents in policy making and initiatives such as Erase Bullying or British Columbia Education Plan.
- The School Portal, developed under the Moldova Open Data Initiative, uses data made public by the Ministry of Education of Moldova to offer comprehensive information about 1529 educational institutions in the Republic of Moldova. Users of the portal can access information about schools yearly budgets, budget implementation, expenditures, school rating, students’ grades, schools’ infrastructure and communications. The School Portal has a tool which allows visitors to compare schools based on different criteria – infrastructure, students’ performance or annual budgets. The additional value of the portal is the fact that it serves as a platform for private sector entities which sell school supplies to advertise their products. The School Portal also allows parents to virtually interact with the Ministry of Education of Moldova or with a psychologist in case they need additional information or have concerns regarding the education of their children.
- RomaScuola, developed under the umbrella of the Italian Open Data Initiative, allows visitors to obtain valuable information about all schools in the Rome region. Distinguishing it from the two listed above is the ability to compare schools depending on such facets as frequency of teacher absence, internet connectivity, use of IT equipment for teaching, frequency of students’ transfer to other schools and quality of education in accordance with the percentage of issued diplomas.
Open data on schools has great value not only for parents but also for the educational system in general. Each country has its own school market, if education is considered as a product in this market. Perfect information about products is one of the main characteristics of competitive markets. From this perspective, giving parents the opportunity to have access to information about schools characteristics will contribute to the increase in the competitiveness of the schools market. Educational institutions will have incentives to improve their performance in order to attract more students…”