Big Data for Social Good

Curated on Posted on by Stefaan Verhulst

Introduction to a Special Issue of the Journal “Big Data” by Catlett Charlie and Ghani Rayid: “…organizations focused on social good are realizing the potential as well but face several challenges as they seek to become more data-driven. The biggest challenge they face is a paucity of examples and case studies on how data can be used for social good. This special issue of Big Data is targeted at tackling that challenge and focuses on highlighting some exciting and impactful examples of work that uses data for social good. The special issue is just one example of the recent surge in such efforts by the data science community. …

This special issue solicited case studies and problem statements that would either highlight (1) the use of data to solve a social problem or (2) social challenges that need data-driven solutions. From roughly 20 submissions, we selected 5 articles that exemplify this type of work. These cover five broad application areas: international development, healthcare, democracy and government, human rights, and crime prevention.

“Understanding Democracy and Development Traps Using a Data-Driven Approach” (Ranganathan et al.) details a data-driven model between democracy, cultural values, and socioeconomic indicators to identify a model of two types of “traps” that hinder the development of democracy. They use historical data to detect causal factors and make predictions about the time expected for a given country to overcome these traps.

“Targeting Villages for Rural Development Using Satellite Image Analysis” (Varshney et al.) discusses two case studies that use data and machine learning techniques for international economic development—solar-powered microgrids in rural India and targeting financial aid to villages in sub-Saharan Africa. In the process, the authors stress the importance of understanding the characteristics and provenance of the data and the criticality of incorporating local “on the ground” expertise.

In “Human Rights Event Detection from Heterogeneous Social Media Graphs,” Chen and Neil describe efficient and scalable techniques to use social media in order to detect emerging patterns in human rights events. They test their approach on recent events in Mexico and show that they can accurately detect relevant human rights–related tweets prior to international news sources, and in some cases, prior to local news reports, which could potentially lead to more timely, targeted, and effective advocacy by relevant human rights groups.

“Finding Patterns with a Rotten Core: Data Mining for Crime Series with Core Sets” (Wang et al.) describes a case study with the Cambridge Police Department, using a subspace clustering method to analyze the department’s full housebreak database, which contains detailed information from thousands of crimes from over a decade. They find that the method allows human crime analysts to handle vast amounts of data and provides new insights into true patterns of crime committed in Cambridge…..(More)

Using open legislative data to map bill co-sponsorship networks in 15 countries

Curated on Posted on by Stefaan Verhulst

François Briatte at OpeningParliament.org: “A few years back, Kamil Gregor published a post under the title “Visualizing politics: Network analysis of bill sponsors”. His post, which focused on the lower chamber of the Czech Parliament, showed how basic social network analysis can support the exploration of parliamentary work, by revealing the ties that members of parliament create between each other through the co-sponsorship of private bills….In what follows, I would like to quickly report on a small research project that I have developed over the years, under the name “parlnet”.

Legislative data on bill co-sponsorship

This project looks at bill co-sponsorship networks in European countries. Many parliaments allow their members to co-sponsor each other’s private bills, which makes it possible to represent these parliaments as collaborative networks, where a tie exists between two MPs if they have co-sponsored legislation together.

This idea is not new: it was pioneered by James Fowler in the United States, and has been the subject of extensive research in American politics, both on the U.S. Congress and on state legislatures. Similar research also exists on the bill co-sponsorship networks of parliaments in Argentina, Chile andRomania.

Inspired by this research and by Baptiste Coulmont’s visualisation of the French lower chamber, I surveyed the parliamentary websites of the following countries:

  • all 28 current members of the European Union ;
  • 4 members of the EFTA: Iceland, Liechtenstein, Norway, and Switzerland

This search returned 19 parliamentary chambers from 15 countries for which it was (relatively) easy to extract legislative data, either through open data portals like data.riksdagen.se in Sweden ordata.stortinget.no in Norway, or from official parliamentary websites directly….After splitting the data into legislative periods separated by nationwide elections, I was able to draw a large collection of networks showing bill co-sponsorship in these 19 chambers….In this graph, each point (or node) is a Belgian MP, and each tie between two MPs indicates that they have co-sponsored at least one bill together. The colors and abbreviations used in the graph are party-related codes, which combine information on the parliamentary group and linguistic community of each MP.Because this kind of graph can be interesting to explore in more detail, I have also built interactive visualizations out of them, in order to show more detailed information on the MPs who participate in bill cosposorship…

The parlnet project was coded in R, and its code is public so that it might benefit from external contributions. The list of countries and chambers that it covers is not exhaustive: in some cases like Portugal, I simply failed to retrieve the data. More talented coders might therefore be able to add to the current database.

Bill cosponsorship networks illustrate how open legislative data provided by parliaments can be turned into interactive tools that easily convey some information about parliamentary work, including, but not limited to:

  • the role of parliamentary party leaders in managing the legislation produced by their groups
  • the impact of partisan discipline and ideology on legislative collaboration between MPs
  • the extent of cross-party cooperation in various parliamentary environments and chambers… (More)

31 cities agree to use EU-funded open innovation platform for better smart cities’ services

Curated on Posted on by Stefaan Verhulst

European Commission Press Release: “At CEBIT, 25 cities from 6 EU countries (Belgium, Denmark, Finland, Italy, Portugal and Spain) and 6 cities from Brazil will present Open & Agile Smart Cities Task Force (OASC), an initiative making it easier for city councils  and startups to improve smart city services (such as transport, energy efficiency, environmental or e-health services). This will be achieved thanks to FIWARE, an EU-funded, open source platform and cloud-based building blocks developed in the EU that can be used to develop a huge range of applications, from Smart Cities to eHealth, and from transport to disaster management. Many applications have already been built using FIWARE – from warnings of earthquakes to preventing food waste to Smartaxi apps. Find a full list of cities in the Background.

The OASC deal will allow cities to share their open data (collected from sensors measuring, for example, traffic flows) so that startups can develop apps and tools that benefit all citizens (for example, an app with traffic information for people on the move). Moreover, these systems will be shared between cities (so, an app with transport information developed in city A can be also adopted by city B, without the latter having to develop it from scratch); FIWARE will also give startups and app developers in these cities access to a global market for smart city services.

Cities from across the globe are trying to make the most of open innovation. This will allow them to include a variety of stakeholders in their activities (services are increasingly connected to other systems and innovative startups are a big part of this trend) and encourage a competitive yet attractive market for developers, thus reducing costs, increasing quality and avoiding vendor lock-in….(More)”

Turning smartphones into personal, real-time pollution-location monitors

Curated on Posted on by Stefaan Verhulst

Kurzweil Newsletter: “Scientists reporting in the ACS journal Environmental Science & Technology have used smartphone and sensing technology to better pinpoint times and locations of the worst air pollution, which is associated with respiratory and cardiovascular problems.

Most such studies create a picture of exposure based on air pollution levels outside people’s homes. This approach ignores big differences in air quality in school and work environments. It also ignores spikes in pollution that happen over the course of the day such as during rush hour.

To fill in these gaps, Mark J. Nieuwenhuijsen and colleagues in Spain, The Netherlands, and the U.S. equipped 54 school children from from 29 different schools around Barcelona with smartphones that could track their location and physical activity. The children also received sensors that continuously measured the ambient levels of black carbon, a component of soot. Although most children spent less than 4 percent of their day traveling to and from school, this exposure contributed 13 percent of their total potential black carbon exposure.

The study was associated with BREATHE, an epidemiological study of the relation between air pollution and brain development.

The researchers conclude that mobile technologies could contribute valuable new insights into air pollution exposure….

More: Mark J. Nieuwenhuijsen, David Donaire-Gonzalez, Ioar Rivas, Montserrat de Castro, Marta Cirach, Gerard Hoek, Edmund Seto, Michael Jerrett, Jordi Sunyer. Variability in and Agreement between Modeled and Personal Continuously Measured Black Carbon Levels Using Novel Smartphone and Sensor Technologies. Environmental Science & Technology, 2015; 150209104136008 DOI: 10.1021/es505362x

Budgets for the People

Curated on Posted on by Stefaan Verhulst
Paolo Spada and Hollie Russon Gilman on Brazil’s Democratic Innovations in Foreign Affairs: “At a time of record low trust in public institutions, thousands of new channels for citizen involvement in government are opening across the world. They go further than electoral participation; they increase citizens’ ability to monitor, regulate, and, in some cases, directly affect political decision-making.Labeled by scholars as democratic innovations, these efforts strengthen existing democratic institutions and promote participation in politics that exceeds infrequent voting. Participatory budgeting (PB) is a standout within the realm of democratic innovations, allowing citizens direct control over portions of government spending. Thirteen Brazilian cities introduced PB programs in 1989. By 2001, there were more than 100 cities implementing PB in Brazil, and in 2015, there are thousands of cities adopting variants of the process worldwide.
Brazilian best practices were instrumental in showcasing the potential of PB to the world. A 2008 World Bank study showed that implementing the process for ten years reduced poverty and improved access to clean water. Recent research has also shown that municipal governments that adopt PB receive an increase in public spending for services such as sanitation and health care. Scholars have also found that PB leads to an eight percent increase in the number of civil society organizations within a given Brazilian municipality, highlighting how this democratic innovation promotes citizen engagement, efficacy, voice, transparency, accountability, and lower levels of corruption. On a global scale, the spread of PB could help build trust among a citizenry that craves more participatory, transparent, and accountable institutions….(More)”

Open data: how mobile phones saved bananas from bacterial wilt in Uganda

Curated on Posted on by Stefaan Verhulst

Anna Scott in The Guardian:”Bananas are a staple food in Uganda. Ugandans eat more of the fruit than any other country in the world. Each person eats on average 700g (about seven small bananas) a day, according to the International Food Policy Research Institute, and they provide up to 27% of the population’s calorie intake.
But since 2002 a disease known as banana bacterial wilt (BBW) has wiped out crops across the country. When plants are infected, they cannot absorb water so their leaves start to shrivel and they eventually die….
The Ugandan government drew upon open data – data that is licensed and made available for anyone to access and share – about the disease made available by Unicef’s community polling project Ureport to deal with the problem.
Ureport mobilises a network of nearly 300,000 volunteers across Uganda, who use their mobiles to report on issues that affect them, from polio immunisation to malaria treatment, child marriage, to crop failure. It gathers data from via SMS polls and publishes the results as open sourced, open datasets.
The results are sent back to community members via SMS along with treatment options and advice on how best to protect their crops. Within five days of the first SMS being sent out, 190,000 Ugandans had learned about the disease and knew how to save bananas on their farms.
Via the Ureport platform, the datasets can also be accessed in real-time by community members, NGOs and the Ugandan government, allowing them to target treatments to where they we needed most. They are also broadcast on radio shows and analysed in articles produced by Ureport, informing wider audiences of scope and nature of the disease and how best to avoid it….
A report published this week by the Open Data Institute (ODI) features stories from around the world which reflect how people are using open date in development. Examples range from accessing school results in Tanzania to building smart cities in Latin America….(More).”

Opening travel spending through civic intelligence, participation and co-creation

Curated on Posted on by Stefaan Verhulst
Joel Salas Suárez at the Open Government Partnership Blog: “When we were appointed by the Senate as Commissioners of the Access to Information Institute in Mexico (IFAI), we identified two high profile issues that had negatively affected the Institute’s image: the acquisition of its new building and the lack of transparency on international travel expenditure of the former Commissioners.

IFAI has to lead by example, so my fellow commissioners and I decided to tackle these two problems with transparency actions to send a clear message to the Mexican society and the international community in our first hundred days in office. First we created the website sede.ifai.mx to publish all the information about the new building procurement (a 45.6 million USD lease). Secondly, we decided to start our first civic innovation project, a joint venture with civil society organizations, to find the best way to publish information related to travel spending by IFAI’s public servants.
Travel expenditure of IFAI is comparatively smaller. During 2013 it allotted to 186,760 USD, 0.5% of the Institute’s budget (38.2 million USD). However, this expenditure has historically been of public interest and it should be. According to the 2013 Mexican Government Expenditure Review (the latest available) the Federal Level (Executive, Legislative and Judicial Powers, and Autonomous organs) spent close to 633 million USD in official travel (Chapter 3000, concept 3700). Therefore, we decided to tackle the problem and design a platform that would allow us to effectively publish information related to the public money spent on travel by public officials and the results obtained during these trips.
In order to do this, we worked with civil society experts in public participation, accountability and technology, Codeando México, SocialTIC and IMCO. Together we launched a public challenge to create an open source web application to publish information on official travel spending.
The challenge #RetoViajesTransparentes was a very successful experience. Close to a hundred participants registered 14 projects that competed to develop an app that IFAI would officially use and to win a 3,500 USD prize. The jury selected 3 finalists, who presented their projects on a public Google Hangout. The winner app is named Viajes Claros and is being used to publish travel expenditure information of IFAI at viajesclaros.ifai.mx.
This challenge has allowed us to shift focus from the inputs of official travel (i.e. the money spent) to the outputs or results attained in each trip. Viajes Claros opens relevant information to understand and evaluate the activities performed by the public servants during their trips. It also allowed us to co-create with society an open source tool that can be replicated in Mexico and other countries….(More)”.

Citizen Science in America’s DNA

Curated on Posted on by Stefaan Verhulst

Keynote by NOAA Chief Scientist, Dr. Richard Spinrad at the forum  entitled, Tracking a Changing Climate: “Citizen science is part of America’s DNA.  For centuries, citizens not trained in science have helped shaped our understanding of Earth.
Thomas Jefferson turned Lewis and Clark into citizen scientists when he asked them to explore the landscape, wildlife and weather during their journeys of the West.They investigated plants, animals and geography, and came back with maps, sketches and journals.  These new data were some of the first pieces of environmental intelligence defining our young nation.  President Jefferson instilled citizen science in my own agency’s DNA by creating the Survey of the Coast, a NOAA legacy agency focused on charting  and protecting the entire coast of our Nation.
The National Weather Service’s Cooperative Observer Program, begun in 1890, is an outstanding example of citizen science.  Last year, NOAA honored an observer who has provided weather observations every day for 80 years. Volunteer citizen scientists have transcribed more than 68,000 pages of Arctic ship logs, adding to the long-term climate record by populating a database with historic weather and sea ice observations. Also, citizen scientists are providing new estimates of cyclone intensity by interpreting satellite images.
There is tremendous value in the capability of citizen scientists to feed local data into their own communities’ forecasts. In September 2013, for example, formal observation systems and tracking instruments were washed out when extreme floods struck Colorado and New Mexico. By ensuring that real-time forecasts were still integrated into the National Weather Service Flood Warning System, the reports of about 200 citizen scientists contributed to what has been called the best mapped extreme rain event in Colorado history and possibly nationwide.
The Community Collaborative Rain, Hail and Snow (CoCoRaHS) Network played a pivotal role in this mapping. CoCoRaHS also shows how citizen science can help make data collection straightforward and inexpensive. To measure the impact and size of hail, for example, it uses a Styrofoam sheet covered with tin foil, creating a “hail pad” that has proven to be quite accurate.
The recognized value of citizen science is growing rapidly.  NOAA has an app to crowdsource real-time precipitation data. If you feel a raindrop, or spot a snowflake, report it through NOAA’s mPING app. Precipitation reports have already topped 600,000, and the National Weather Service uses them to fine-tune forecasts…(More).”

Can 311 Call Centers Improve Service Delivery? Lessons from New York and Chicago

Curated on Posted on by Stefaan Verhulst

Paper by Jane Wiseman: “This paper is the first of the IDB’s “Innovations in Public Service Delivery” series, which identifies and analyzes innovative experiences of promising practices in Latin America and the Caribbean and around the world to improve the quality and delivery of public services. It presents the 311 Programs in New York City and Chicago, leading 311 centers in the United States. “311” is the universal toll-free number that provides citizens with a single point of entry to a wide array of information and services in major cities. In the cities studied, these centers have evolved to support new models of service delivery management. This publication provides an overview of these programs, analyzing their design and implementation, results, and impacts, and identifying their success factors. The final section consolidates the lessons learned from these experiences, highlighting what policymakers and public officials should consider when developing similar solutions…Download in PDF“.

Why Is Democracy Performing So Poorly?

Curated on Posted on by Stefaan Verhulst

Essay by Francis Fukuyama in the Journal of Democracy: “The Journal of Democracy published its inaugural issue a bit past the midpoint of what Samuel P. Huntington labeled the “third wave” of democratization, right after the fall of the Berlin Wall and just before the breakup of the former Soviet Union. The transitions in Southern Europe and most of those in Latin America had already happened, and Eastern Europe was moving at dizzying speed away from communism, while the democratic transitions in sub-Saharan Africa and the former USSR were just getting underway. Overall, there has been remarkable worldwide progress in democratization over a period of almost 45 years, raising the number of electoral democracies from about 35 in 1970 to well over 110 in 2014.
But as Larry Diamond has pointed out, there has been a democratic recession since 2006, with a decline in aggregate Freedom House scores every year since then. The year 2014 has not been good for democracy, with two big authoritarian powers, Russia and China, on the move at either end of Eurasia. The “Arab Spring” of 2011, which raised expectations that the Arab exception to the third wave might end, has degenerated into renewed dictatorship in the case of Egypt, and into anarchy in Libya, Yemen, and also Syria, which along with Iraq has seen the emergence of a new radical Islamist movement, the Islamic State in Iraq and Syria (ISIS).
It is hard to know whether we are experiencing a momentary setback in a general movement toward greater democracy around the world, similar to a stock-market correction, or whether the events of this year signal a broader shift in world politics and the rise of serious alternatives to democracy. In either case, it is hard not to feel that the performance of democracies around the world has been deficient in recent years. This begins with the most developed and successful democracies, those of the United States and the European Union, which experienced massive economic crises in the late 2000s and seem to be mired in a period of slow growth and stagnating incomes. But a number of newer democracies, from Brazil to Turkey to India, have also been disappointing in their performance in many respects, and subject to their own protest movements.
Spontaneous democratic movements against authoritarian regimes continue to arise out of civil society, from Ukraine and Georgia to Tunisia and Egypt to Hong Kong. But few of these movements have been successful in leading to the establishment of stable, well-functioning democracies. It is worth asking why the performance of democracy around the world has been so disappointing.
In my view, a single important factor lies at the core of many democratic setbacks over the past generation. It has to do with a failure of institutionalization—the fact that state capacity in many new and existing democracies has not kept pace with popular demands for democratic accountability. It is much harder to move from a patrimonial or neopatrimonial state to a modern, impersonal one than it is to move from an authoritarian regime to one that holds regular, free, and fair elections. It is the failure to establish modern, well-governed states that has been the Achilles heel of recent democratic transitions… (More)”