Can Big Data Measure Livability in Cities?

Curated on Posted on by Stefaan Verhulst

PlaceILive: “Big data helps us measure and predict consumer behavior, hurricanes and even pregnancies. It has revolutionized the way we access and use information. That being said, so far big data has not been able to tackle bigger issues like urbanization or improve the livability of cities.

A new startup, www.placeilive.com thinks big data should and can be used to measure livability. They aggregated open data from government institutions and social media to create a tool that can calculate just that. ….PlaceILive wants to help people and governments better understand their cities, so that they can make smarter decisions. Cities can be more sustainable, while its users save money and time when they are choosing a new home.

Not everyone is eager to read long lists of raw data. Therefore they created appealing user-friendly maps that visualize the statistics. Offering the user fast and accessible information on the neighborhoods that matter to them.

Another cornerstone of PlaceILive is their Life Quality Index: an algorithm that takes aspects like transportation, safety, and affordability into account. Making it possible for people to easily compare the livability of different houses. You can read more on the methodology and sources here.

life quality index press release

In its beta form, the site features five cities—New York City, Chicago, San Francisco, London and Berlin. When you click on the New York portal, for instance, you can search for the place you want to know more about by borough, zip code, or address. Using New York as an example, it looks like this….(More)

Why Google’s Waze Is Trading User Data With Local Governments

Curated on Posted on by Stefaan Verhulst

Parmy Olson at Forbes: “In Rio de Janeiro most eyes are on the final, nail-biting matches of the World Cup. Over in the command center of the city’s department of transport though, they’re on a different set of screens altogether.

Planners there are watching the aggregated data feeds of thousands of smartphones being walked or driven around a city, thanks to two popular travel apps, Waze and Moovit.

The goal is traffic management, and it involves swapping data for data. More cities are lining up to get access, and while the data the apps are sharing is all anonymous for now, identifying details could get more specific if cities like what they see, and people become more comfortable with being monitored through their smartphones in return for incentives.

Rio is the first city in the world to collect real-time data both from drivers who use the Waze navigation app and pedestrians who use the public-transportation app Moovit, giving it an unprecedented view on thousands of moving points across the sprawling city. Rio is also talking to the popular cycling app Strava to start monitoring how cyclists are moving around the city too.

All three apps are popular, consumer services which, in the last few months, have found a new way to make their crowdsourced data useful to someone other than advertisers. While consumers use Waze and Moovit to get around, both companies are flipping the use case and turning those millions of users into a network of sensors that municipalities can tap into for a better view on traffic and hazards. Local governments can also use these apps as a channel to send alerts.

On an average day in June, Rio’s transport planners could get an aggregated view of 110,000 drivers (half a million over the course of the month), and see nearly 60,000 incidents being reported each day – everything from built-up traffic, to hazards on the road, Waze says. Till now they’ve been relying on road cameras and other basic transport-department information.

What may be especially tantalizing for planners is the super-accurate read Waze gets on exactly where drivers are going, by pinging their phones’ GPS once every second. The app can tell how fast a driver is moving and even get a complete record of their driving history, according to Waze spokesperson Julie Mossler. (UPDATE: Since this story was first published Waze has asked to clarify that it separates users’ names and their 30-day driving info. The driving history is categorized under an alias.)

This passively-tracked GPS data “is not something we share,” she adds. Waze, which Google bought last year for $1.3 billion, can turn the data spigots on and off through its application programing interface (API).

Waze has been sharing user data with Rio since summer 2013 and it just signed up the State of Florida. It says more departments of transport are in the pipeline.

But none of these partnerships are making Waze any money. The app’s currency of choice is data. “It’s a two-way street,” says Mossler. “Literally.”

In return for its user updates, Waze gets real-time information from Rio on highways, from road sensors and even from cameras, while Florida will give the app data on construction projects or city events.

Florida’s department of transport could not be reached for comment, but one of its spokesmen recently told a local news station: “We’re going to share our information, our camera images, all of our information that comes from the sensors on the roadway, and Waze is going to share its data with us.”…

To get Moovit’s data, municipalities download a web interface that gives them an aggregated view of where pedestrians using Moovit are going. In return, the city feeds Moovit’s database with a stream of real-time GPS data for buses and trains, and can issue transport alerts to Moovit’s users. Erez notes the cities aren’t allowed to make “any sort of commercial approach to the users.”

Erez may be saving that for advertisers, an avenue he says he’s still exploring. For now getting data from cities is the bigger priority. It gives Moovit “a competitive advantage,” he says.

Cycling app Strava also recently started sharing its real-time user data as part of a paid-for service called Strava Metro.

Municipalities pay 80 cents a year for every Strava member being tracked. Metro only launched in May, but it already counts the state of Oregon; London, UK; Glasgow, Scotland; Queensland, Austalia and Evanston, Illinois as customers.
….
Privacy advocates will naturally want to keep a wary eye on what data is being fed to cities, and that it doesn’t leak or get somehow misused by City Hall. The data-sharing might not be ubiquitous enough for that to be a problem yet, and it should be noted that any kind of deal making with the public sector can get wrapped up in bureaucracy and take years to get off the ground.

For now Waze says it’s acting for the public good….(More)

Using Innovation and Technology to Improve City Services

Curated on Posted on by Stefaan Verhulst

New report from the IBM Center for the Business of Government: “In this report, Professor Greenberg examines a dozen cities across the United States that have award-winning reputations for using innovation and technology to improve the services they provide to their residents. She explores a variety of success factors associated with effective service delivery at the local level, including:

  • The policies, platforms, and applications that cities use for different purposes, such as public engagement, streamlining the issuance of permits, and emergency response
  • How cities can successfully partner with third parties, such as nonprofits, foundations, universities, and private businesses to improve service delivery using technology
  • The types of business cases that can be presented to mayors and city councils to support various changes proposed by innovators in city government

Professor Greenberg identifies a series of trends that drive cities to undertake innovations, such as the increased use of mobile devices by residents. Based on cities’ responses to these trends, she offers a set of findings and specific actions that city officials can act upon to create innovation agendas for their communities. Her report also presents case studies for each of the dozen cities in her review. These cases provide a real-world context, which will allow interested leaders in other cities to see how their own communities might approach similar innovation initiatives.

This report builds on two other IBM Center reports: A Guide for Making Innovation Offices Work, by Rachel Burstein and Alissa Black, and The Persistence of Innovation in Government: A Guide for Public Servants, by Sandford Borins, which examines the use of awards to stimulate innovation in government….(More)”

New Desktop Application Has Potential to Increase Asteroid Detection, Now Available to Public

Curated on Posted on by Stefaan Verhulst

NASA Press Release: “A software application based on an algorithm created by a NASA challenge has the potential to increase the number of new asteroid discoveries by amateur astronomers.

Analysis of images taken of our solar system’s main belt asteroids between Mars and Jupiter using the algorithm showed a 15 percent increase in positive identification of new asteroids.

During a panel Sunday at the South by Southwest Festival in Austin, Texas, NASA representatives discussed how citizen scientists have made a difference in asteroid hunting. They also announced the release of a desktop software application developed by NASA in partnership with Planetary Resources, Inc., of Redmond, Washington. The application is based on an Asteroid Data Hunter-derived algorithm that analyzes images for potential asteroids. It’s a tool that can be used by amateur astronomers and citizen scientists.

The Asteroid Data Hunter challenge was part of NASA’s Asteroid Grand Challenge. The data hunter contest series, which was conducted in partnership with Planetary Resources under a Space Act Agreement, was announced at the 2014 South by Southwest Festival and concluded in December. The series offered a total of $55,000 in awards for participants to develop significantly improved algorithms to identify asteroids in images captured by ground-based telescopes. The winning solutions of each piece of the contest combined to create an application using the best algorithm that increased the detection sensitivity, minimized the number of false positives, ignored imperfections in the data, and ran effectively on all computer systems.

“The Asteroid Grand Challenge is seeking non-traditional partnerships to bring the citizen science and space enthusiast community into NASA’s work,” said Jason Kessler, program executive for NASA’s Asteroid Grand Challenge. “The Asteroid Data Hunter challenge has been successful beyond our hopes, creating something that makes a tangible difference to asteroid hunting astronomers and highlights the possibility for more people to play a role in protecting our planet.”…

The new asteroid hunting application can be downloaded at:

http://topcoder.com/asteroids

For information about NASA’s Asteroid Grand Challenge, visit:

http://www.nasa.gov/asteroidinitiative

The perils of extreme democracy

Curated on Posted on by Stefaan Verhulst

The Economist: “California cannot pass timely budgets even in good years, which is one reason why its credit rating has, in one generation, fallen from one of the best to the absolute worst among the 50 states. How can a place which has so much going for it—from its diversity and natural beauty to its unsurpassed talent clusters in Silicon Valley and Hollywood—be so poorly governed? ….But as our special report this week argues, the main culprit has been direct democracy: recalls, in which Californians fire elected officials in mid-term; referendums, in which they can reject acts of their legislature; and especially initiatives, in which the voters write their own rules. Since 1978, when Proposition 13 lowered property-tax rates, hundreds of initiatives have been approved on subjects from education to the regulation of chicken coops.

This citizen legislature has caused chaos. Many initiatives have either limited taxes or mandated spending, making it even harder to balance the budget. Some are so ill-thought-out that they achieve the opposite of their intent: for all its small-government pretensions, Proposition 13 ended up centralising California’s finances, shifting them from local to state government. Rather than being the curb on elites that they were supposed to be, ballot initiatives have become a tool of special interests, with lobbyists and extremists bankrolling laws that are often bewildering in their complexity and obscure in their ramifications. And they have impoverished the state’s representative government. Who would want to sit in a legislature where 70-90% of the budget has already been allocated?

This has been a tragedy for California, but it matters far beyond the state’s borders. Around half of America’s states and an increasing number of countries have direct democracy in some form (article). Next month Britain will have its first referendum for years (on whether to change its voting system), and there is talk of voter recalls for aberrant MPs. The European Union has just introduced the first supranational initiative process. With technology making it ever easier to hold referendums and Western voters ever more angry with their politicians, direct democracy could be on the march.

And why not? There is, after all, a successful model: in Switzerland direct democracy goes back to the Middle Ages at the local level and to the 19th century at the federal. This mixture of direct and representative democracy seems to work well. Surely it is just a case of California (which explicitly borrowed the Swiss model) executing a good idea poorly?

Not entirely. Very few people, least of all this newspaper, want to ban direct democracy. Indeed, in some cases referendums are good things: they are a way of holding a legislature to account. In California reforms to curb gerrymandering and non-partisan primaries, both improvements, have recently been introduced by initiatives; and they were pushed by Arnold Schwarzenegger, a governor elected through the recall process. But there is a strong case for proceeding with caution, especially when it comes to allowing people to circumvent a legislature with citizen-made legislation.

The debate about the merits of representative and direct democracy goes back to ancient times. To simplify a little, the Athenians favoured pure democracy (“people rule”, though in fact oligarchs often had the last word); the Romans chose a republic, as a “public thing”, where representatives could make trade-offs for the common good and were accountable for the sum of their achievements. America’s Founding Fathers, especially James Madison and Alexander Hamilton, backed the Romans. Indeed, in their guise of “Publius” in the “Federalist Papers”, Madison and Hamilton warn against the dangerous “passions” of the mob and the threat of “minority factions” (ie, special interests) seizing the democratic process.

Proper democracy is far more than a perpetual ballot process. It must include deliberation, mature institutions and checks and balances such as those in the American constitution. Ironically, California imported direct democracy almost a century ago as a “safety valve” in case government should become corrupt. The process began to malfunction only relatively recently. With Proposition 13, it stopped being a valve and instead became almost the entire engine.

….More important, direct democracy must revert to being a safety valve, not the engine. Initiatives should be far harder to introduce. They should be shorter and simpler, so that voters can actually understand them. They should state what they cost, and where that money is to come from. And, if successful, initiatives must be subject to amendment by the legislature. Those would be good principles to apply to referendums, too….(More)”

Mission Control: A History of the Urban Dashboard

Curated on Posted on by Stefaan Verhulst

Shannon Mattern at Places: “We know what rocket science looks like in the movies: a windowless bunker filled with blinking consoles, swivel chairs, and shirt-sleeved men in headsets nonchalantly relaying updates from “Houston” to outer space. Lately, that vision of Mission Control has taken over City Hall. NASA meets Copacabana, proclaimed the New York Times, hailing Rio de Janeiro’s Operations Center as a “potentially lucrative experiment that could shape the future of cities around the world.” The Times photographed an IBM executive in front of a seemingly endless wall of screens integrating data from 30 city agencies, including transit video, rainfall patterns, crime statistics, car accidents, power failures, and more. 1

Futuristic control rooms have proliferated in dozens of global cities. Baltimore has its CitiStat Room, where department heads stand at a podium before a wall of screens and account for their units’ performance.  The Mayor’s office in London’s City Hall features a 4×3 array of iPads mounted in a wooden panel, which seems an almost parodic, Terry Gilliam-esque take on the Brazilian Ops Center. Meanwhile, British Prime Minister David Cameron commissioned an iPad app – the “No. 10 Dashboard” (a reference to his residence at 10 Downing Street) – which gives him access to financial, housing, employment, and public opinion data. As The Guardian reported, “the prime minister said that he could run government remotely from his smartphone.”

This is the age of Dashboard Governance, heralded by gurus like Stephen Few, founder of the “visual business intelligence” and “sensemaking” consultancy Perceptual Edge, who defines the dashboard as a “visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance.” A well-designed dashboard, he says — one that makes proper use of bullet graphs, sparklines, and other visualization techniques informed by the “brain science” of aesthetics and cognition — can afford its users not only a perceptual edge, but a performance edge, too. The ideal display offers a big-picture view of what is happening in real time, along with information on historical trends, so that users can divine the how and why and redirect future action. As David Nettleton emphasizes, the dashboard’s utility extends beyond monitoring “the current situation”; it also “allows a manager to … make provisions, and take appropriate actions.”….

The dashboard market now extends far beyond the corporate world. In 1994, New York City police commissioner William Bratton adapted former officer Jack Maple’s analog crime maps to create the CompStat model of aggregating and mapping crime statistics. Around the same time, the administrators of Charlotte, North Carolina, borrowed a business idea — Robert Kaplan’s and David Norton’s “total quality management” strategy known as the “Balanced Scorecard” — and began tracking performance in five “focus areas” defined by the City Council: housing and neighborhood development, community safety, transportation, economic development, and the environment. Atlanta followed Charlotte’s example in creating its own city dashboard.

In 1999, Baltimore mayor Martin O’Malley, confronting a crippling crime rate and high taxes, designed CitiStat, “an internal process of using metrics to create accountability within his government.” (This rhetoric of data-tested internal “accountability” is prevalent in early dashboard development efforts.) The project turned to face the public in 2003, when Baltimore launched a website of city operational statistics, which inspired DCStat (2005), Maryland’s StateStat (2007), and NYCStat (2008). Since then, myriad other states and metro areas — driven by a “new managerialist” approach to urban governance, committed to “benchmarking” their performance against other regions, and obligated to demonstrate compliance with sustainability agendas — have developed their own dashboards.

The Open Michigan Mi Dashboard is typical of these efforts. The state website presents data on education, health and wellness, infrastructure, “talent” (employment, innovation), public safety, energy and environment, financial health, and seniors. You (or “Mi”) can monitor the state’s performance through a side-by-side comparison of “prior” and “current” data, punctuated with a thumbs-up or thumbs-down icon indicating the state’s “progress” on each metric. Another click reveals a graph of annual trends and a citation for the data source, but little detail about how the data are actually derived. How the public is supposed to use this information is an open question….(More)”

Big Data for Social Good

Curated on Posted on by Stefaan Verhulst

Introduction to a Special Issue of the Journal “Big Data” by Catlett Charlie and Ghani Rayid: “…organizations focused on social good are realizing the potential as well but face several challenges as they seek to become more data-driven. The biggest challenge they face is a paucity of examples and case studies on how data can be used for social good. This special issue of Big Data is targeted at tackling that challenge and focuses on highlighting some exciting and impactful examples of work that uses data for social good. The special issue is just one example of the recent surge in such efforts by the data science community. …

This special issue solicited case studies and problem statements that would either highlight (1) the use of data to solve a social problem or (2) social challenges that need data-driven solutions. From roughly 20 submissions, we selected 5 articles that exemplify this type of work. These cover five broad application areas: international development, healthcare, democracy and government, human rights, and crime prevention.

“Understanding Democracy and Development Traps Using a Data-Driven Approach” (Ranganathan et al.) details a data-driven model between democracy, cultural values, and socioeconomic indicators to identify a model of two types of “traps” that hinder the development of democracy. They use historical data to detect causal factors and make predictions about the time expected for a given country to overcome these traps.

“Targeting Villages for Rural Development Using Satellite Image Analysis” (Varshney et al.) discusses two case studies that use data and machine learning techniques for international economic development—solar-powered microgrids in rural India and targeting financial aid to villages in sub-Saharan Africa. In the process, the authors stress the importance of understanding the characteristics and provenance of the data and the criticality of incorporating local “on the ground” expertise.

In “Human Rights Event Detection from Heterogeneous Social Media Graphs,” Chen and Neil describe efficient and scalable techniques to use social media in order to detect emerging patterns in human rights events. They test their approach on recent events in Mexico and show that they can accurately detect relevant human rights–related tweets prior to international news sources, and in some cases, prior to local news reports, which could potentially lead to more timely, targeted, and effective advocacy by relevant human rights groups.

“Finding Patterns with a Rotten Core: Data Mining for Crime Series with Core Sets” (Wang et al.) describes a case study with the Cambridge Police Department, using a subspace clustering method to analyze the department’s full housebreak database, which contains detailed information from thousands of crimes from over a decade. They find that the method allows human crime analysts to handle vast amounts of data and provides new insights into true patterns of crime committed in Cambridge…..(More)

Gamification harnesses the power of games to motivate

Curated on Posted on by Stefaan Verhulst

Kevin Werbach at the Conversation: “Walk through any public area and you’ll see people glued to their phones, playing mobile games like Game of War and Candy Crush Saga. They aren’t alone. 59% of Americans play video games, and contrary to stereotypes, 48% of gamers are women. The US$100 billion video game industry is among the least-appreciated business phenomena in the world today.

But this isn’t an article about video games. It’s about where innovative organizations are applying the techniques that make those games so powerfully engaging: everywhere else.

Gamification is the perhaps-unfortunate name for the growing practice of applying structural elements, design patterns, and psychological insights from game design to business, education, health, marketing, crowdsourcing and other fields. Over the past four years, gamification has gone through a cycle of (over-)hype and (overblown) disappointment common for technological trends. Yet if you look carefully, you’ll see it everywhere.

Tapping into pieces of games

Gamification involves two primary mechanisms. The first is to take design structures from games, such as levels, achievements, points, and leaderboards — in my book, For the Win, my co-author and I label them “game elements” — and incorporate them into activities. The second, more subtle but ultimately more effective, is to mine the rich vein of design techniques that game designers have developed over many years. Good games pull you in and carry you through a journey that remains engaging, using an evolving balance of challenges and a stream of well crafted, actionable feedback.

Many enterprises now use tools built on top of Salesforce.com’s customer relationship management platform to motivate employees through competitions, points and leaderboards. Online learning platforms such as Khan Academy commonly challenge students to “level up” by sprinkling game elements throughout the process. Even games are now gamified: Microsoft’s Xbox One and Sony’s PS4 consoles offer a meta-layer of achievements and trophies to promote greater game-play.

The differences between a gamified system that incorporates good design principles and one that doesn’t aren’t always obvious on the surface. They show up in the results.

Duolingo is an online language-learning app. It’s pervasively and thoughtfully gamified: points, levels, achievements, bonuses for “streaks,” visual progression indicators, even a virtual currency with various ways to spend it. The well integrated gamification is a major differentiator for Duolingo, which happens to be the most successful tool of its kind. With over 60 million registered users, it teaches languages to more people than the entire US public school system.

Most of the initial high-profile cases of gamification were for marketing: for example, USA Network ramped up its engagement numbers with web-based gamified challenges for fans of its shows, and Samsung gave points and badges for learning about its products.

Soon it became clear that other applications were equally promising. Today, organizations are using gamification to enhance employee performance, promote health and wellness activities, improve retention in online learning, help kids with cancer endure their treatment regimen, and teach people how to code, to name just a few examples. Gamification has potential anywhere that motivation is an important element of success.

Gamification works because our responses to games are deeply hard-wired into our psychology. Game design techniques can activate our innate desires to recognize patterns, solve puzzles, master challenges, collaborate with others, and be in the drivers’ seat when experiencing the world around us. They can also create a safe space for experimentation and learning. After all, why not try something new when you know that even if you fail, you’ll get another life?…(More)

Using open legislative data to map bill co-sponsorship networks in 15 countries

Curated on Posted on by Stefaan Verhulst

François Briatte at OpeningParliament.org: “A few years back, Kamil Gregor published a post under the title “Visualizing politics: Network analysis of bill sponsors”. His post, which focused on the lower chamber of the Czech Parliament, showed how basic social network analysis can support the exploration of parliamentary work, by revealing the ties that members of parliament create between each other through the co-sponsorship of private bills….In what follows, I would like to quickly report on a small research project that I have developed over the years, under the name “parlnet”.

Legislative data on bill co-sponsorship

This project looks at bill co-sponsorship networks in European countries. Many parliaments allow their members to co-sponsor each other’s private bills, which makes it possible to represent these parliaments as collaborative networks, where a tie exists between two MPs if they have co-sponsored legislation together.

This idea is not new: it was pioneered by James Fowler in the United States, and has been the subject of extensive research in American politics, both on the U.S. Congress and on state legislatures. Similar research also exists on the bill co-sponsorship networks of parliaments in Argentina, Chile andRomania.

Inspired by this research and by Baptiste Coulmont’s visualisation of the French lower chamber, I surveyed the parliamentary websites of the following countries:

  • all 28 current members of the European Union ;
  • 4 members of the EFTA: Iceland, Liechtenstein, Norway, and Switzerland

This search returned 19 parliamentary chambers from 15 countries for which it was (relatively) easy to extract legislative data, either through open data portals like data.riksdagen.se in Sweden ordata.stortinget.no in Norway, or from official parliamentary websites directly….After splitting the data into legislative periods separated by nationwide elections, I was able to draw a large collection of networks showing bill co-sponsorship in these 19 chambers….In this graph, each point (or node) is a Belgian MP, and each tie between two MPs indicates that they have co-sponsored at least one bill together. The colors and abbreviations used in the graph are party-related codes, which combine information on the parliamentary group and linguistic community of each MP.Because this kind of graph can be interesting to explore in more detail, I have also built interactive visualizations out of them, in order to show more detailed information on the MPs who participate in bill cosposorship…

The parlnet project was coded in R, and its code is public so that it might benefit from external contributions. The list of countries and chambers that it covers is not exhaustive: in some cases like Portugal, I simply failed to retrieve the data. More talented coders might therefore be able to add to the current database.

Bill cosponsorship networks illustrate how open legislative data provided by parliaments can be turned into interactive tools that easily convey some information about parliamentary work, including, but not limited to:

  • the role of parliamentary party leaders in managing the legislation produced by their groups
  • the impact of partisan discipline and ideology on legislative collaboration between MPs
  • the extent of cross-party cooperation in various parliamentary environments and chambers… (More)

31 cities agree to use EU-funded open innovation platform for better smart cities’ services

Curated on Posted on by Stefaan Verhulst

European Commission Press Release: “At CEBIT, 25 cities from 6 EU countries (Belgium, Denmark, Finland, Italy, Portugal and Spain) and 6 cities from Brazil will present Open & Agile Smart Cities Task Force (OASC), an initiative making it easier for city councils  and startups to improve smart city services (such as transport, energy efficiency, environmental or e-health services). This will be achieved thanks to FIWARE, an EU-funded, open source platform and cloud-based building blocks developed in the EU that can be used to develop a huge range of applications, from Smart Cities to eHealth, and from transport to disaster management. Many applications have already been built using FIWARE – from warnings of earthquakes to preventing food waste to Smartaxi apps. Find a full list of cities in the Background.

The OASC deal will allow cities to share their open data (collected from sensors measuring, for example, traffic flows) so that startups can develop apps and tools that benefit all citizens (for example, an app with traffic information for people on the move). Moreover, these systems will be shared between cities (so, an app with transport information developed in city A can be also adopted by city B, without the latter having to develop it from scratch); FIWARE will also give startups and app developers in these cities access to a global market for smart city services.

Cities from across the globe are trying to make the most of open innovation. This will allow them to include a variety of stakeholders in their activities (services are increasingly connected to other systems and innovative startups are a big part of this trend) and encourage a competitive yet attractive market for developers, thus reducing costs, increasing quality and avoiding vendor lock-in….(More)”