Smart Cities Turn Big Data Into Insight [Infographic]

Mark van Rijmenam in SmartDataCollective: “Cities around the globe are confronted with growing populations, aging infrastructure, reduced budgets, and the challenge of doing more with less. Applying big data technologies within cities can provide valuable insights that can keep a city habitable. The City of Songdo is a great example of a connected city, where all connected devices create a smart city that is optimized for the every-changing conditions in that same city. IBM recently released an infographic showing the vast opportunities of smart cities and the possible effects on the economy.”
Infographic Smarter Cities. Turning Big Data into Insight

The Art of Making City Code Beautiful

Nancy Scola in Next City: “Some rather pretty legal websites have popped up lately:, and, as of last Thursday, This is how municipal code would design itself if it actually wanted to be read.
The network of [city] sites is the output of The State Decoded, a project of the OpenGov Foundation (See correction below), which has its own fascinating provenance. That D.C.-based non-profit grew out of the fight in Congress over the SOPA and PIPA digital copyright bills a few winters ago. At the time, the office of Rep. Darrell Issa, more recently of Benghazi fame, built a platform called Madison that invited the public to help edit an alternative bill. Madison outlived the SOPA debate, and was spun out last summer as the flagship project of the OpenGov Foundation, helmed by former Issa staffer Seamus Kraft.
“What we discovered,” Kraft says, “is that co-authoring legislation is high up there on what [the public wants to] do with government information, but it’s not at the top.” What heads the list, he says, is simply knowing “what are the laws?” Pre-SanFranciscoCode, the city’s laws on everything from elections to electrical installations to transportation were trapped in an interface, run by publisher American Legal, that would not have looked out of place in “WarGames.” (Here’s the comparable “old” site for Chicago. It’s probably enough to say that Philadelphia’s comes with a “Frames/No Frames” option.) Madison needed a base of clean, structured municipal code upon which to function, and Kraft and company were finding that in cities across the country, that just didn’t exist.
Fixing the code, Kraft says, starts with him “unlawyering the text” that is either supplied to them by the city or scraped from online. This involves reading through the city code and looking for signposts that indicate when sections start, how provisions nest within them, and other structural cues that establish a pattern. That breakdown gets passed to the organization’s developers, who use it to automatically parse the full corpus. The process is time consuming. In San Francisco, 16 different patterns were required to capture each of the code’s sections. Often, the parser needs to be tweaked. “Sometimes it happens in a few minutes or a few hours,” Kraft says, “and sometimes it takes a few days.”

Over the long haul, Kraft has in mind adopting the customizability of YouVersion, the online digital Bible that allows users to choose fonts, colors and more. Kraft, a 2007 graduate of Georgetown who will cite the Catholic Church’s distributed structure as a model for networked government, proclaims YouVersion “the most kick-ass Bible you’ve ever seen. It’s stunning.” He’d like to do the same with municipal code, for the benefit of both the average American and those who have more regular engagement with local legal texts. “If you’re spending all day reading law,” he says, “you should at the very least have the most comfortable view possible.”


AskThem is a project of the Participatory Politics Foundation, a 501(c)3 non-profit organization with a mission to increase civic engagement. AskThem is supported by a charitable grant from the Knight Foundation’s Tech For Engagement initiative.
AskThem is a free & open-source website for questions-and-answers with public figures. It’s a not-for-profit tool for a stronger democracy, with open data for informed and engaged communities.
AskThem allows you to:

  • Find and ask questions to over 142,000 elected officials nationwide: federal, state and city levels of government.
  • Get signatures for your question or petition, have it delivered over email or Twitter, and push for a public response.
  • See questions from people near you, sign-on to questions you care about, and review answers from public figures.

It’s like a version of “We The People” for every elected official, from local city council members all the way up to U.S. senators. Enter your email above to be the first to ask a question when we launch and see previews of the site this Fall.
Elected officials: enter your email above and we’ll send you more information about signing up to answer questions on AskThem. It’s a free and non-partisan service to respond to your constituents in an open public forum and update them over email about your work. Or, be a leader in open-government and sign up now.
Issue-based organizations and media: sign up to help promote questions to government from people in your area. We’re working to launch with partnerships that build greater public accountability.
Previously known as the project, AskThem is open-source and uses open government data – our code is available on GitHub – contributions welcome. For more development updates & discussion, join our low-traffic Google Group.
We’re a small non-profit organization actively seeking charitable funding support – help us launch this powerful new tool for public dialogue! Email us for a copy of our non-profit funding prospectus. If you can make a tax-exempt gift to support our work, please donate to PPF via OpenCongress. More background on the project is available on our Knight NewsChallenge proposal from March 2013.
Questions, feedback, ideas? Email David Moore, Executive Director of PPF – david at, Twitter: @ppolitics; like our page on Facebook & follow @AskThemPPF on Twitter. Stay tuned!”

Towards an effective framework for building smart cities: Lessons from Seoul and San Francisco

New paper by JH Lee, MG Hancock, MC Hu in Technological Forecasting and Social Change: “This study aims to shed light on the process of building an effective smart city by integrating various practical perspectives with a consideration of smart city characteristics taken from the literature. We developed a framework for conducting case studies examining how smart cities were being implemented in San Francisco and Seoul Metropolitan City. The study’s empirical results suggest that effective, sustainable smart cities emerge as a result of dynamic processes in which public and private sector actors coordinate their activities and resources on an open innovation platform. The different yet complementary linkages formed by these actors must further be aligned with respect to their developmental stage and embedded cultural and social capabilities. Our findings point to eight ‘stylized facts’, based on both quantitative and qualitative empirical results that underlie the facilitation of an effective smart city. In elaborating these facts, the paper offers useful insights to managers seeking to improve the delivery of smart city developmental projects.”

Using Big Data to Ask Big Questions

Chase Davis in the SOURCE: “First, let’s dispense with the buzzwords. Big Data isn’t what you think it is: Every federal campaign contribution over the last 30-plus years amounts to several tens of millions of records. That’s not Big. Neither is a dataset of 50 million Medicare records. Or even 260 gigabytes of files related to offshore tax havens—at least not when Google counts its data in exabytes. No, the stuff we analyze in pursuit of journalism and app-building is downright tiny by comparison.
But you know what? That’s ok. Because while super-smart Silicon Valley PhDs are busy helping Facebook crunch through petabytes of user data, they’re also throwing off intellectual exhaust that we can benefit from in the journalism and civic data communities. Most notably: the ability to ask Big Questions.
Most of us who analyze public data for fun and profit are familiar with small questions. They’re focused, incisive, and often have the kind of black-and-white, definitive answers that end up in news stories: How much money did Barack Obama raise in 2012? Is the murder rate in my town going up or down?
Big Questions, on the other hand, are speculative, exploratory, and systemic. As the name implies, they are also answered at scale: Rather than distilling a small slice of a dataset into a concrete answer, Big Questions look at entire datasets and reveal small questions you wouldn’t have thought to ask.
Can we track individual campaign donor behavior over decades, and what does that tell us about their influence in politics? Which neighborhoods in my city are experiencing spikes in crime this week, and are police changing patrols accordingly?
Or, by way of example, how often do interest groups propose cookie-cutter bills in state legislatures?

Looking at Legislation

Even if you don’t follow politics, you probably won’t be shocked to learn that lawmakers don’t always write their own bills. In fact, interest groups sometimes write them word-for-word.
Sometimes those groups even try to push their bills in multiple states. The conservative American Legislative Exchange Council has gotten some press, but liberal groups, social and business interests, and even sororities and fraternities have done it too.
On its face, something about elected officials signing their names to cookie-cutter bills runs head-first against people’s ideal of deliberative Democracy—hence, it tends to make news. Those can be great stories, but they’re often limited in scope to a particular bill, politician, or interest group. They’re based on small questions.
Data science lets us expand our scope. Rather than focusing on one bill, or one interest group, or one state, why not ask: How many model bills were introduced in all 50 states, period, by anyone, during the last legislative session? No matter what they’re about. No matter who introduced them. No matter where they were introduced.
Now that’s a Big Question. And with some basic data science, it’s not particularly hard to answer—at least at a superficial level.

Analyze All the Things!

Just for kicks, I tried building a system to answer this question earlier this year. It was intended as an example, so I tried to choose methods that would make intuitive sense. But it also makes liberal use of techniques applied often to Big Data analysis: k-means clustering, matrices, graphs, and the like.
If you want to follow along, the code is here….
To make exploration a little easier, my code represents similar bills in graph space, shown at the top of this article. Each dot (known as a node) represents a bill. And a line connecting two bills (known as an edge) means they were sufficiently similar, according to my criteria (a cosine similarity of 0.75 or above). Thrown into a visualization software like Gephi, it’s easy to click around the clusters and see what pops out. So what do we find?
There are 375 clusters in total. Because of the limitations of our data, many of them represent vague, subject-specific bills that just happen to have similar titles even though the legislation itself is probably very different (think things like “Budget Bill” and “Campaign Finance Reform”). This is where having full bill text would come handy.
But mixed in with those bills are a handful of interesting nuggets. Several bills that appear to be modeled after legislation by the National Conference of Insurance Legislators appear in multiple states, among them: a bill related to limited lines travel insurance; another related to unclaimed insurance benefits; and one related to certificates of insurance.”

User-Generated Content Is Here to Stay

in the Huffington Post: “The way media are transmitted has changed dramatically over the last 10 years. User-generated content (UGC) has completely changed the landscape of social interaction, media outreach, consumer understanding, and everything in between. Today, UGC is media generated by the consumer instead of the traditional journalists and reporters. This is a movement defying and redefining traditional norms at the same time. Current events are largely publicized on Twitter and Facebook by the average person, and not by a photojournalist hired by a news organization. In the past, these large news corporations dominated the headlines — literally — and owned the monopoly on public media. Yet with the advent of smartphones and spread of social media, everything has changed. The entire industry has been replaced; smartphones have supplanted how information is collected, packaged, edited, and conveyed for mass distribution. UGC allows for raw and unfiltered movement of content at lightening speed. With the way that the world works today, it is the most reliable way to get information out. One thing that is for certain is that UGC is here to stay whether we like it or not, and it is driving much more of modern journalistic content than the average person realizes.
Think about recent natural disasters where images are captured by citizen journalists using their iPhones. During Hurricane Sandy, 800,000 photos uploaded onto Instagram with “#Sandy.” Time magazine even hired five iPhoneographers to photograph the wreckage for its Instagram page. During the May 2013 Oklahoma City tornadoes, the first photo released was actually captured by a smartphone. This real-time footage brings environmental chaos to your doorstep in a chillingly personal way, especially considering the photographer of the first tornado photos ultimately died because of the tornado. UGC has been monumental for criminal investigations and man-made catastrophes. Most notably, the Boston Marathon bombing was covered by UGC in the most unforgettable way. Dozens of images poured in identifying possible Boston bombers, to both the detriment and benefit of public officials and investigators. Though these images inflicted considerable damage to innocent bystanders sporting suspicious backpacks, ultimately it was also smartphone images that highlighted the presence of the Tsarnaev brothers. This phenomenon isn’t limited to America. Would the so-called Arab Spring have happened without social media and UGC? Syrians, Egyptians, and citizens from numerous nations facing protests can easily publicize controversial images and statements to be shared worldwide….
This trend is not temporary but will only expand. The first iPhone launched in 2007, and the world has never been the same. New smartphones are released each month with better cameras and faster processors than computers had even just a few years ago….”

Using Participatory Crowdsourcing in South Africa to Create a Safer Living Environment

New Paper by Bhaveer Bhana, Stephen Flowerday, and Aharon Satt in the International Journal of Distributed Sensor Networks: “The increase in urbanisation is making the management of city resources a difficult task. Data collected through observations (utilising humans as sensors) of the city surroundings can be used to improve decision making in terms of managing these resources. However, the data collected must be of a certain quality in order to ensure that effective and efficient decisions are made. This study is focused on the improvement of emergency and non-emergency services (city resources) through the use of participatory crowdsourcing (humans as sensors) as a data collection method (collect public safety data), utilising voice technology in the form of an interactive voice response (IVR) system.
The study illustrates how participatory crowdsourcing (specifically humans as sensors) can be used as a Smart City initiative focusing on public safety by illustrating what is required to contribute to the Smart City, and developing a roadmap in the form of a model to assist decision making when selecting an optimal crowdsourcing initiative. Public safety data quality criteria were developed to assess and identify the problems affecting data quality.
This study is guided by design science methodology and applies three driving theories: the Data Information Knowledge Action Result (DIKAR) model, the characteristics of a Smart City, and a credible Data Quality Framework. Four critical success factors were developed to ensure high quality public safety data is collected through participatory crowdsourcing utilising voice technologies.”

5 Ways Cities Are Using Big Data

Eric Larson in Mashable: “New York City released more than 200 high-value data sets to the public on Monday — a way, in part, to provide more content for open-sourced mapping projects like OpenStreetMap.
It’s one of the many releases since the Local Law 11 of 2012 passed in February, which calls for more transparency of the city government’s collected data.
But it’s not just New York: Cities across the world, large and small, are utilizing big data sets — like traffic statistics, energy consumption rates and GPS mapping — to launch projects to help their respective communities.
We rounded up a few of our favorites below….

1. Seattle’s Power Consumption

The city of Seattle recently partnered with Microsoft and Accenture on a pilot project to reduce the area’s energy usage. Using Microsoft’s Azure cloud, the project will collect and analyze hundreds of data sets collected from four downtown buildings’ management systems.
With predictive analytics, then, the system will work to find out what’s working and what’s not — i.e. where energy can be used less, or not at all. The goal is to reduce power usage by 25%.

2. SpotHero

Finding parking spots — especially in big cities — is undoubtably a headache.

SpotHero is an app, for both iOS and Android devices, that tracks down parking spots in a select number of cities. How it works: Users type in an address or neighborhood (say, Adams Morgan in Washington, D.C.) and are taken to a listing of available garages and lots nearby — complete with prices and time durations.
The app tracks availability in real-time, too, so a spot is updated in the system as soon as it’s snagged.
Seven cities are currently synced with the app: Washington, D.C., New York, Chicago, Baltimore, Boston, Milwaukee and Newark, N.J.

3. Adopt-a-Hydrant

Anyone who’s spent a winter in Boston will agree: it snows.

In January, the city’s Office of New Urban Mechanics released an app called Adopt-a-Hydrant. The program is mapped with every fire hydrant in the city proper — more than 13,000, according to a Harvard blog post — and lets residents pledge to shovel out one, or as many as they choose, in the almost inevitable event of a blizzard.
Once a pledge is made, volunteers receive a notification if their hydrant — or hydrants — become buried in snow.

4. Adopt-a-Sidewalk

Similar to Adopt-a-Hydrant, Chicago’s Adopt-a-Sidewalk app lets residents of the Windy City pledge to shovel sidewalks after snowfall. In a city just as notorious for snowstorms as Boston, it’s an effective way to ensure public spaces remain free of snow and ice — especially spaces belonging to the elderly or disabled.

If you’re unsure which part of town you’d like to “adopt,” just register on the website and browse the map — you’ll receive a pop-up notification for each street you swipe that’s still available.

5. Less Congestion for Lyon

Last year, researchers at IBM teamed up with the city of Lyon, France (about four hours south of Paris), to build a system that helps traffic operators reduce congestion on the road.

The system, called the “Decision Support System Optimizer (DSSO),” uses real-time traffic reports to detect and predict congestions. If an operator sees that a traffic jam is likely to occur, then, she/he can adjust traffic signals accordingly to keep the flow of cars moving smoothly.
It’s an especially helpful tool for emergencies — say, when an ambulance is en route to the hospital. Over time, the algorithms in the system will “learn” from its most successful recommendations, then apply that knowledge when making future predictions.”

Social media: its emerging importance and impact on citizen engagement

New article by Victoria Burton in International Affairs Forum that “examines the impact of social media which not only provides citizens alternative avenues to express themselves about government policies but presents new challenges and means for government to provide services to the public. An example is the CovJam online venture presented by Coventry City and IBM that used social media as part of a three-day brainstorming event about the city. Social media have facilitated government programs to carry out surveys and fine-tune services but perhaps the greatest aspect is that of greater public participation. Moving forward, it will be important to address social media across public sectors and establish strategies to leverage its advantages and benefits.”

Open Data’s Road to Better Transit

Stephen Goldsmith in GovTech: “Data is everywhere. It now costs less to capture, store and process data than ever before, thanks to better technology and economies of scale. And more than ever, the public expects government to use data to improve its services. Increasingly, government’s problem is not capturing the data, but having sufficient resources to clean and analyze the information in order to address issues, improve performance and make informed decisions.
In particular, public transit not only produces an immense volume of data, but it also stands to benefit from good analysis in the form of streamlined operations and a better rider experience. More than 200 transit agencies worldwide — from Buffalo to Budapest — are well on their way. They are publishing their schedules, fares and station locations to Google’s TransitDataFeed in a common format and for free. Such information is called open data, which is any data that’s publicly shared.
Open data allows anyone to download and use the information for his or her purposes, particularly software developers who can use it to create mobile and Web-based applications. Google, for example, incorporates the information into its Maps application to help riders plan trips and learn about service updates across bus, rail and bike systems. Other third parties have built successful apps on top of open transit data.
Innovations like these allow transit agencies to leverage external expertise and resources, and have also reduced customer service costs and increased ridership levels. In fact, some members of the American Public Transportation Association believe that open data initiatives have catalyzed more innovation throughout the industry than any other factor in the last three decades….
In Philadelphia, the City Planning Commission is using text message surveying to capture the opinions of transit riders across the demographic spectrum to determine the usefulness of a proposed rapid transit line into downtown. Philadelphia uses the transit information to inform its comprehensive city plan, but this digital citizen survey mechanism, created by a company called Textizen, is a platform that can be used by any government that wants to solicit feedback or begin a dialog with its citizens.
In 2012, Dubuque, Iowa, collaborated with IBM to run a Smarter Travel pilot study. The pilot used a mobile app and RFIDs to collect anonymous travel data from volunteer transit riders. The city has already used the data to open a new late-night bus line for third-shift workers and college students, and by next year will incorporate data into more route planning decisions.”