Open Data: Going Beyond Solving Problems to Making the Impossible Possible

Curated on Posted on by Stefaan Verhulst

at the Huffington Post: “As a global community, we are producing data at an astounding rate. The pace was recently described as a “new Google every four days” by the highly respected Andreesen Horowitz partner, Peter Levine, in a thought-provoking post addressing the challenge of making sense of this mountain of data.

“… we are now collecting more data each day, so much that 90 percent of the data in the world today has been created in the last two years alone. In fact, every day, we create 2.5 quintillion bytes of data — by some estimates that’s one new Google every four days, and the rate is only increasing. Our desire to use, interact, and learn from this data will become increasingly important and strategic to businesses and society as a whole.”

For the past year and a half, my cofounders and team have focused on what it will take to use, interact and learn from data being produced within the civic sector. It’s one thing to be able to build an app for civic; it’s quite another to build a platform that can manage multiple apps across multiple platforms while addressing the challenges plaguing the “wild west” nature of growth in the quickly emerging market of open data. I was recently invited to share our lessons learned and the promise of the future in mobile open data at the TEDxABQ Technology Salon. Here is a bit of what I shared:
Beyond Solving Problems to New Possibilities
When we first looked at the opportunities for creating apps built on open data, our priority was finding pain points for both cities and the people who lived there. We focused on solving real problems, and it led to some early success. We worked with the City of Albuquerque to deploy their ABQ RIDE app on iOS and Android platforms, and the app not only solved real problems for riders, it also saved real money for the city. The app has grown to over 20,000 regular users and continues to be one of the highest downloaded apps on our platform.
But recently, we’ve started asking questions that go beyond the basic, that change the experience or make things possible in ways that never were before. Here are two I’m incredibly proud to be a part of…”

An Air-Quality Monitor You Take with You

Curated on Posted on by Stefaan Verhulst

MIT Technology Review: “A startup is building a wearable air-quality monitor using a sensing technology that can cheaply detect the presence of chemicals around you in real time. By reporting the information its sensors gather to an app on your smartphone, the technology could help people with respiratory conditions and those who live in highly polluted areas keep tabs on exposure.
Berkeley, California-based Chemisense also plans to crowdsource data from users to show places around town where certain compounds are identified.
Initially, the company plans to sell a $150 wristband geared toward kids with asthma—of which there are nearly 7 million in the U.S., according to data from the Centers for Disease Control and Prevention— to help them identify places and pollutants that tend to provoke attacks,  and track their exposure to air pollution over time. The company hopes people with other respiratory conditions, and those who are just concerned about air pollution, will be interested, too.
In the U.S., air quality is monitored at thousands of stations across the country; maps and forecasts can be viewed online. But these monitors offer accurate readings only in their location.
Chemisense has not yet made its initial product, but it expects it will be a wristband using polymers treated with charged nanoparticles of carbon such that the polymers swell in the presence of certain chemical vapors, changing the resistance of a circuit.”

Cell Phone Guide For US Protesters, Updated 2014 Edition

Curated on Posted on by Stefaan Verhulst

EFF: “With major protests in the news again, we decided it’s time to update our cell phone guide for protestors. A lot has changed since we last published this report in 2011, for better and for worse. On the one hand, we’ve learned more about the massive volume of law enforcement requests for cell phone—ranging from location information to actual content—and widespread use of dedicated cell phone surveillance technologies. On the other hand, strong Supreme Court opinions have eliminated any ambiguity about the unconstitutionality of warrantless searches of phones incident to arrest, and a growing national consensus says location data, too, is private.”

Smart cities: moving beyond urban cybernetics to tackle wicked problems

Curated on Posted on by Stefaan Verhulst

Paper by Robert Goodspeed in the Cambridge Journal of Regions Economy and  Society: This article makes three related arguments. First, that although many definitions of the smart city have been proposed, corporate promoters say a smart city uses information technology to pursue efficient systems through real-time monitoring and control. Second, this definition is not new and equivalent to the idea of urban cybernetics debated in the 1970s. Third, drawing on a discussion of Rio de Janeiro’s Operations Center, I argue that viewing urban problems as wicked problems allows for more fundamental solutions than urban cybernetics, but requires local innovation and stakeholder participation. Therefore the last section describes institutions for municipal innovation and IT-enabled collaborative planning.”

Government opens up: 10k active government users on GitHub

Curated on Posted on by Stefaan Verhulst

GitHub: “In the summer of 2009, The New York Senate was the first government organization to post code to GitHub, and that fall, Washington DC quickly followed suit. By 2011, cities like Miami, Chicago, and New York; Australian, Canadian, and British government initiatives like Gov.uk; and US Federal agencies like the Federal Communications Commission, General Services Administration, NASA, and Consumer Financial Protection Bureau were all coding in the open as they began to reimagine government for the 21st century.
Fast forward to just last year: The White House Open Data Policy is published as a collaborative, living document, San Francisco laws are now forkable, and government agencies are accepting pull requests from every day developers.
This is all part of a larger trend towards government adopting open source practices and workflows — a trend that spans not only software, but data, and policy as well — and the movement shows no signs of slowing, with government usage on GitHub nearly tripling in the past year, to exceed 10,000 active government users today.

How government uses GitHub

When government works in the open, it acknowledges the idea that government is the world’s largest and longest-running open source project. Open data efforts, efforts like the City of Philadelphia’s open flu shot spec, release machine-readable data in open, immediately consumable formats, inviting feedback (and corrections) from the general public, and fundamentally exposing who made what change when, a necessary check on democracy.
Unlike the private sector, however, where open sourcing the “secret sauce” may hurt the bottom line, with government, we’re all on the same team. With the exception of say, football, Illinois and Wisconsin don’t compete with one another, nor are the types of challenges they face unique. Shared code prevents reinventing the wheel and helps taxpayer dollars go further, with efforts like the White House’s recently released Digital Services Playbook, an effort which invites every day citizens to play a role in making government better, one commit at a time.
However, not all government code is open source. We see that adopting these open source workflows for open collaboration within an agency (or with outside contractors) similarly breaks down bureaucratic walls, and gives like-minded teams the opportunity to work together on common challenges.

Government Today

It’s hard to believe that what started with a single repository just five years ago, has blossomed into a movement where today, more than 10,000 government employees use GitHub to collaborate on code, data, and policy each day….
You can learn more about GitHub in government at government.github.com, and if you’re a government employee, be sure to join our semi-private peer group to learn best practices for collaborating on software, data, and policy in the open.”

Opening Health Data: What Do Researchers Want? Early Experiences With New York's Open Health Data Platform.

Curated on Posted on by Stefaan Verhulst

Paper by Martin, Erika G. PhD, MPH; Helbig, Natalie PhD, MPA; and Birkhead, Guthrie S. MD, MPH in the Journal of Public Health Management & Practice: “Governments are rapidly developing open data platforms to improve transparency and make information more accessible. New York is a leader, with currently the only state platform devoted to health. Although these platforms could build public health departments’ capabilities to serve more researchers, agencies have little guidance on releasing meaningful and usable data.

Objective: Structured focus groups with researchers and practitioners collected stakeholder feedback on potential uses of open health data and New York’s open data strategy….

Results: There was low awareness of open data, with 67% of researchers reporting never using open data portals prior to the workshop. Participants were interested in data sets that were geocoded, longitudinal, or aggregated to small area granularity and capabilities to link multiple data sets. Multiple environmental conditions and barriers hinder their capacity to use health data for research. Although open data platforms cannot address all barriers, they provide multiple opportunities for public health research and practice, and participants were overall positive about the state’s efforts to release open data.

Conclusions: Open data are not ideal for some researchers because they do not contain individually identifiable data, indicating a need for tiered data release strategies. However, they do provide important new opportunities to facilitate research and foster collaborations among agencies, researchers, and practitioners.”

Can big data help build more wind and solar farms?

Curated on Posted on by Stefaan Verhulst

Rachael Post in The Guardian: “Convincing customers to switch to renewable energy is an uphill battle. But for a former political operative, finding business is as easy as mining a consumer behavior database…After his father died from cancer related to pollution from a coal-burning plant, Tom Matzzie, the former director of democratic activist group MoveOn.org, decided that he’d had enough with traditional dirty energy. But when he installed solar panels on his home, he discovered that the complicated permitting and construction process made switching to renewable energy difficult and unwieldy. The solution, he concluded, was to use his online campaigning and big data skills – honed from his years of working in politics – to find the most likely customers for renewables and convince them to switch. Ethical Electric was born.
Matzzie’s company isn’t the first to sell renewable energy, but it might be the smartest. For the most part, convincing people to switch away from dirty energy is an unprofitable and work-intensive process, requiring electrical company representatives to approach thousands of randomly chosen customers. Ethical Electric, on the other hand, uses a highly-targeted, strategic method to identify its potential customers.
From finding votes to finding customers
Matzzie, who is now CEO of Ethical Electric, explained that the secret lies in his company’s use of big data, a resource that he and his partners mastered on the political front lines. In the last few presidential elections, big data fundamentally changed the way candidates – and their teams – approached voters. “We couldn’t rely on voter registration lists to make assumptions about who would be willing to vote in the next election,” Matzzie said. “What happened in politics is a real revolution in data.”…”

City Service Development Kit (CitySDK)

Curated on Posted on by Stefaan Verhulst

What is CitySDK?: “Helping cities to open their data and giving developers the tools they need, the CitySDK aims for a step change in how to deliver services in urban environments. With governments around the world looking at open data as a kick start for their economies, CitySDK provides better and easier ways for the cities throughout the Europe to release their data in a format that is easy for the developers to re-use.
Taking the best practices around the world the project will foresee the development of a toolkit – CitySDK v1.0 – that can be used by any city looking to create a sustainable infrastructure of “city apps”.
Piloting the CitySDK
The Project focuses on three Pilot domains: Smart Participation, Smart Mobility and Smart Tourism. Within each of the three domains, a large-scale Lead Pilot is carried out in one city. The experiences of the Lead Pilot will be applied in the Replication Pilots in other Partner cities.
Funding
CitySDK is a 6.8 million Euro project, part funded by the European Commission. It is a Pilot Type B within the ICT Policy Support Programme of the Competitiveness and Framework Programme. It runs from January 2012-October 2014.”

Station display shows waiting commuters the best train carriage to get on

Curated on Posted on by Stefaan Verhulst

Springwise: “When a train arrives into a station, it’s often the case that travelers aren’t spread evenly along the platform and are huddled in the same spot. This is annoying for both commuters and operators because it means carriages get full while others are left empty and leads to longer boarding times. In the Netherlands, the NS Reisplanner Xtra app has already offered train users a way to find a seat using their smartphone. Now the country’s Edenspiekermann design agency has developed a platform-length LED display which provides real-time information on carriage crowdedness and other details.
Created for train operators ProRail and NS with the help of design researchers STBY, the service consists of a 180-meter long color LED strip that spans the length of the platform. The display aims to give commuters all the information they need to know where they should wait to get on the right carriage. Numbers show whether the carriage is first or standard class, and the exact position the doors will be is also marked. Symbols show the carriages that are best for bikes, buggies, wheelchairs and large luggage, as well as quiet carriages. The boards also work with infrared sensors located on each train that detect how full each carriage is. A green strip means there are seats available, a yellow strip indicates that the carriage is fairly crowded and a red strip means it’s full.
Website: www.edenspiekermann.com“

As Data Overflows Online, Researchers Grapple With Ethics

Curated on Posted on by Stefaan Verhulst

at The New York Times: “Scholars are exhilarated by the prospect of tapping into the vast troves of personal data collected by Facebook, Google, Amazon and a host of start-ups, which they say could transform social science research.

Once forced to conduct painstaking personal interviews with subjects, scientists can now sit at a screen and instantly play with the digital experiences of millions of Internet users. It is the frontier of social science — experiments on people who may never even know they are subjects of study, let alone explicitly consent.

“This is a new era,” said Jeffrey T. Hancock, a Cornell University professor of communication and information science. “I liken it a little bit to when chemistry got the microscope.”

But the new era has brought some controversy with it. Professor Hancock was a co-author of the Facebook study in which the social network quietly manipulated the news feeds of nearly 700,000 people to learn how the changes affected their emotions. When the research was published in June, the outrage was immediate…

Such testing raises fundamental questions. What types of experiments are so intrusive that they need prior consent or prompt disclosure after the fact? How do companies make sure that customers have a clear understanding of how their personal information might be used? Who even decides what the rules should be?

Existing federal rules governing research on human subjects, intended for medical research, generally require consent from those studied unless the potential for harm is minimal. But many social science scholars say the federal rules never contemplated large-scale research on Internet users and provide inadequate guidance for it.

Sinan Aral, a professor at the Massachusetts Institute of Technology’s Sloan School of Management who has conducted large-scale social experiments with several tech companies, said any new rules must be carefully formulated.

“We need to understand how to think about these rules without chilling the research that has the promise of moving us miles and miles ahead of where we are today in understanding human populations,” he said. Professor Aral is planning a panel discussion on ethics at a M.I.T. conference on digital experimentation in October. (The professor also does some data analysis for The New York Times Company.)

Mary L. Gray, a senior researcher at Microsoft Research and associate professor at Indiana University’s Media School, who has worked extensively on ethics in social science, said that too often, researchers conducting digital experiments work in isolation with little outside guidance.

She and others at Microsoft Research spent the last two years setting up an ethics advisory committee and training program for researchers in the company’s labs who are working with human subjects. She is now working with Professor Hancock to bring such thinking to the broader research world.

“If everyone knew the right thing to do, we would never have anyone hurt,” she said. “We really don’t have a place where we can have these conversations.”…