Tackling quality concerns around (volunteered) big data


University of Twente: “… Improvements in online information communication and mobile location-aware technologies have led to a dramatic increase in the amount of volunteered geographic information (VGI) in recent years. The collection of volunteered data on geographic phenomena has a rich history worldwide. For example, the Christmas Bird Count has studied the impacts of climate change on spatial distribution and population trends of selected bird species in North America since 1900. Nowadays, several citizen observatories collect information about our environment. This information is complementary or, in some cases, essential to tackle a wide range of geographic problems.

Despite the wide applicability and acceptability of VGI in science, many studies argue that the quality of the observations remains a concern. Data collected by volunteers does not often follow scientific principles of sampling design, and levels of expertise vary among volunteers. This makes it hard for scientists to integrate VGI in their research.

Low quality, inconsistent, observations can bias analysis and modelling results because they are not representative for the variable studied, or because they decrease the ratio of signal to noise. Hence, the identification of inconsistent observations clearly benefits VGI-based applications and provide more robust datasets to the scientific community.

In their paper the researchers describe a novel automated workflow to identify inconsistencies in VGI. “Leveraging a digital control mechanism means we can give value to the millions of observations collected by volunteers” and “it allows a new kind of science where citizens can directly contribute to the analysis of global challenges like climate change” say Hamed Mehdipoor and Dr. Raul Zurita-Milla, who work at the Geo-Information Processing department of ITC….

While some inconsistent observations may reflect real, unusual events, the researchers demonstrated that these observations also bias the trends (advancement rates), in this case of the date of lilac flowering onset. This shows that identifying inconsistent observations is a pre-requisite for studying and interpreting the impact of climate change on the timing of life cycle events….(More)”

How Big Data is Helping to Tackle Climate Change


Bernard Marr at DataInformed: “Climate scientists have been gathering a great deal of data for a long time, but analytics technology’s catching up is comparatively recent. Now that cloud, distributed storage, and massive amounts of processing power are affordable for almost everyone, those data sets are being put to use. On top of that, the growing number of Internet of Things devices we are carrying around are adding to the amount of data we are collecting. And the rise of social media means more and more people are reporting environmental data and uploading photos and videos of their environment, which also can be analyzed for clues.

Perhaps one of the most ambitious projects that employ big data to study the environment is Microsoft’s Madingley, which is being developed with the intention of creating a simulation of all life on Earth. The project already provides a working simulation of the global carbon cycle, and it is hoped that, eventually, everything from deforestation to animal migration, pollution, and overfishing will be modeled in a real-time “virtual biosphere.” Just a few years ago, the idea of a simulation of the entire planet’s ecosphere would have seemed like ridiculous, pie-in-the-sky thinking. But today it’s something into which one of the world’s biggest companies is pouring serious money. Microsoft is doing this because it believes that analytical technology has finally caught up with the ability to collect and store data.

Another data giant that is developing tools to facilitate analysis of climate and ecological data is EMC. Working with scientists at Acadia National Park in Maine, the company has developed platforms to pull in crowd-sourced data from citizen science portals such as eBird and iNaturalist. This allows park administrators to monitor the impact of climate change on wildlife populations as well as to plan and implement conservation strategies.

Last year, the United Nations, under its Global Pulse data analytics initiative, launched the Big Data Climate Challenge, a competition aimed to promote innovate data-driven climate change projects. Among the first to receive recognition under the program is Global Forest Watch, which combines satellite imagery, crowd-sourced witness accounts, and public datasets to track deforestation around the world, which is believed to be a leading man-made cause of climate change. The project has been promoted as a way for ethical businesses to ensure that their supply chain is not complicit in deforestation.

Other initiatives are targeted at a more personal level, for example by analyzing transit routes that could be used for individual journeys, using Google Maps, and making recommendations based on carbon emissions for each route.

The idea of “smart cities” is central to the concept of the Internet of Things – the idea that everyday objects and tools are becoming increasingly connected, interactive, and intelligent, and capable of communicating with each other independently of humans. Many of the ideas put forward by smart-city pioneers are grounded in climate awareness, such as reducing carbon dioxide emissions and energy waste across urban areas. Smart metering allows utility companies to increase or restrict the flow of electricity, gas, or water to reduce waste and ensure adequate supply at peak periods. Public transport can be efficiently planned to avoid wasted journeys and provide a reliable service that will encourage citizens to leave their cars at home.

These examples raise an important point: It’s apparent that data – big or small – can tell us if, how, and why climate change is happening. But, of course, this is only really valuable to us if it also can tell us what we can do about it. Some projects, such as Weathersafe, which helps coffee growers adapt to changing weather patterns and soil conditions, are designed to help humans deal with climate change. Others are designed to tackle the problem at the root, by highlighting the factors that cause it in the first place and showing us how we can change our behavior to minimize damage….(More)”

Crowdsourced phone camera footage maps conflicts


Springwise: “The UN requires accurate proof when investigating possible war crimes, but with different sides of a conflict providing contradicting evidence, and the unsafe nature of the environment, gaining genuine insight can be problematic. A team based at Goldsmith’s University in the UK are using amateur footage to investigate.

Forensic Architecture makes use of the increasingly prevalent smartphone footage on social media networks. By crowdsourcing several viewpoints around a given location on an accurately 3D rendered map, the team are able to determine where explosive devices were used, and of what calibre. Key resources are smoke plumes from explosions, which provide a unique shape at any moment, allowing the team to map them and identify the smoke at the exact moment from various viewpoints, providing a dossier of evidence to build up evidence against a war crime.

While Forensic Architecture’s method has been developed to validate war crime atrocities, the potential uses in other areas where satellite data are not available are numerous — forest fire sources could be located based on smoke plumes, and potential crowd crush scenarios may be spotted before they occur….(More)”

An Introduction to System Mapping


Joelle Cook at FSG: “Much has been written recently about the importance of using a system lens, or focusing on system change, to make real progress against some of society’s toughest challenges. But what does that really mean? The following definition of system change resonated with us, fromNPC’s 2015 guide:

“Systems change is an intentional process designed to alter the status quo by shifting the function or structure of an identified system with purposeful interventions. It is a journey which can require a radical change in people’s attitudes as well as in the ways people work. Systems change aims to bring about lasting change by altering underlying structures and supporting mechanisms which make the system operate in a particular way. These can include policies, routines, relationships, resources, power structures and values.”

However, to change the system, you need to first understand the system, and mapping is a great way to do that. A “system,” as described by Julia Coffman in her 2007 framework for evaluating system change, is “a group of interacting, interrelated, and interdependent components that form a complex and unified whole.” A system’s overall purpose or goal is achieved through the actions and interactions of its components.

As you can imagine, there are a number of different ways you might approach mapping the system to represent system elements and connections. For example, you might create:

  • Actor maps, to show which individuals and/or organizations are key players in the space and how they are connected
  • Mind maps, that highlight various trends in the external environment that influence the issue at hand
  • Issue maps, which lay out the political, social, or economic issues affecting a given geography or constituency (often used by advocacy groups)
  • Causal-loop diagrams, that  focus on explicating the feedback loops (positive and negative) that lead to system behavior or functioning

For more information, see a blog from Innovation Network on systems mapping, Jeff Wasbes’blog on causal loop diagrams, and an example from the Hewlett Foundation’s Madison Initiative….(More)”

Questioning Smart Urbanism: Is Data-Driven Governance a Panacea?


 at the Chicago Policy Review: “In the era of data explosion, urban planners are increasingly relying on real-time, streaming data generated by “smart” devices to assist with city management. “Smart cities,” referring to cities that implement pervasive and ubiquitous computing in urban planning, are widely discussed in academia, business, and government. These cities are characterized not only by their use of technology but also by their innovation-driven economies and collaborative, data-driven city governance. Smart urbanism can seem like an effective strategy to create more efficient, sustainable, productive, and open cities. However, there are emerging concerns about the potential risks in the long-term development of smart cities, including political neutrality of big data, technocratic governance, technological lock-ins, data and network security, and privacy risks.

In a study entitled, “The Real-Time City? Big Data and Smart Urbanism,” Rob Kitchin provides a critical reflection on the potential negative effects of data-driven city governance on social development—a topic he claims deserves greater governmental, academic, and social attention.

In contrast to traditional datasets that rely on samples or are aggregated to a coarse scale, “big data” is huge in volume, high in velocity, and diverse in variety. Since the early 2000s, there has been explosive growth in data volume due to the rapid development and implementation of technology infrastructure, including networks, information management, and data storage. Big data can be generated from directed, automated, and volunteered sources. Automated data generation is of particular interest to urban planners. One example Kitchin cites is urban sensor networks, which allow city governments to monitor the movements and statuses of individuals, materials, and structures throughout the urban environment by analyzing real-time data.

With the huge amount of streaming data collected by smart infrastructure, many city governments use real-time analysis to manage different aspects of city operations. There has been a recent trend in centralizing data streams into a single hub, integrating all kinds of surveillance and analytics. These one-stop data centers make it easier for analysts to cross-reference data, spot patterns, identify problems, and allocate resources. The data are also often accessible by field workers via operations platforms. In London and some other cities, real-time data are visualized on “city dashboards” and communicated to citizens, providing convenient access to city information.

However, the real-time city is not a flawless solution to all the problems faced by city managers. The primary concern is the politics of big, urban data. Although raw data are often perceived as neutral and objective, no data are free of bias; the collection of data is a subjective process that can be shaped by various confounding factors. The presentation of data can also be manipulated to answer a specific question or enact a particular political vision….(More)”

Remaking Participation: Science, Environment and Emergent Publics


Book edited by Jason Chilvers and Matthew Kearnes: “Changing relations between science and democracy – and controversies over issues such as climate change, energy transitions, genetically modified organisms and smart technologies – have led to a rapid rise in new forms of public participation and citizen engagement. While most existing approaches adopt fixed meanings of ‘participation’ and are consumed by questions of method or critiquing the possible limits of democratic engagement, this book offers new insights that rethink public engagements with science, innovation and environmental issues as diverse, emergent and in the making. Bringing together leading scholars on science and democracy, working between science and technology studies, political theory, geography, sociology and anthropology, the volume develops relational and co-productionist approaches to studying and intervening in spaces of participation. New empirical insights into the making, construction, circulation and effects of participation across cultures are illustrated through examples ranging from climate change and energy to nanotechnology and mundane technologies, from institutionalised deliberative processes to citizen-led innovation and activism, and from the global north to global south. This new way of seeing participation in science and democracy opens up alternative paths for reconfiguring and remaking participation in more experimental, reflexive, anticipatory and responsible ways….(More)”

How big data and The Sims are helping us to build the cities of the future


The Next Web: “By 2050, the United Nations predicts that around 66 percent of the world’s population will be living in urban areas. It is expected that the greatest expansion will take place in developing regions such as Africa and Asia. Cities in these parts will be challenged to meet the needs of their residents, and provide sufficient housing, energy, waste disposal, healthcare, transportation, education and employment.

So, understanding how cities will grow – and how we can make them smarter and more sustainable along the way – is a high priority among researchers and governments the world over. We need to get to grips with the inner mechanisms of cities, if we’re to engineer them for the future. Fortunately, there are tools to help us do this. And even better, using them is a bit like playing SimCity….

Cities are complex systems. Increasingly, scientists studying cities have gone from thinking about “cities as machines”, to approaching “cities as organisms”. Viewing cities as complex, adaptive organisms – similar to natural systems like termite mounds or slime mould colonies – allows us to gain unique insights into their inner workings. …So, if cities are like organisms, it follows that we should examine them from the bottom-up, and seek to understand how unexpected large-scale phenomena emerge from individual-level interactions. Specifically, we can simulate how the behaviour of individual “agents” – whether they are people, households, or organisations – affect the urban environment, using a set of techniques known as “agent-based modelling”….These days, increases in computing power and the proliferation of big datagive agent-based modelling unprecedented power and scope. One of the most exciting developments is the potential to incorporate people’s thoughts and behaviours. In doing so, we can begin to model the impacts of people’s choices on present circumstances, and the future.

For example, we might want to know how changes to the road layout might affect crime rates in certain areas. By modelling the activities of individuals who might try to commit a crime, we can see how altering the urban environment influences how people move around the city, the types of houses that they become aware of, and consequently which places have the greatest risk of becoming the targets of burglary.

To fully realise the goal of simulating cities in this way, models need a huge amount of data. For example, to model the daily flow of people around a city, we need to know what kinds of things people spend their time doing, where they do them, who they do them with, and what drives their behaviour.

Without good-quality, high-resolution data, we have no way of knowing whether our models are producing realistic results. Big data could offer researchers a wealth of information to meet these twin needs. The kinds of data that are exciting urban modellers include:

  • Electronic travel cards that tell us how people move around a city.
  • Twitter messages that provide insight into what people are doing and thinking.
  • The density of mobile telephones that hint at the presence of crowds.
  • Loyalty and credit-card transactions to understand consumer behaviour.
  • Participatory mapping of hitherto unknown urban spaces, such as Open Street Map.

These data can often be refined to the level of a single person. As a result, models of urban phenomena no longer need to rely on assumptions about the population as a whole – they can be tailored to capture the diversity of a city full of individuals, who often think and behave differently from one another….(More)

Strengthening the Connective Links in Government


John M. Kamensky at the IBM Center for The Business of Government: “Over the past five years, the Obama administration has pursued a host of innovation-fostering initiatives that work to strengthen the connective links among and within federal agencies.

Many factors contribute to the rise of such efforts, including presidential support, statutory encouragement, and an ongoing evolution in the way government does its business. The challenge now is how to solidify the best of them so they remain in place beyond the upcoming 2017 presidential transition.

Increased Use of Collaborative Governance

Dr. Rosemary O’Leary, an astute observer of trends in government, describes how government has steadily increased its use of collaborative approaches in lieu of the traditional hierarchical, bureaucratic approach. According to O’Leary, there are several explanations for this shift:

  • First, “most public challenges are larger than one organization, requiring new approaches to addressing public issues” such as housing, pollution, transportation, and healthcare.
  • Second, collaboration helps to improve the effectiveness and performance of programs “by encouraging new ways of providing services.”
  • Third, technology advances in recent years have helped “organizations and their employees to share information in a way that is integrative and interoperable.”
  • Finally, “citizens are seeking additional avenues for engaging in governance, resulting in new and different forms of collaborative problem solving and decision making.”

Early in his administration, President Barack Obama publicly placed a premium on the use of collaboration. One of his first directives to federal agencies set the tone for how he envisioned his administration would govern, directing agencies to be “collaborative” and “use innovative tools, methods, and systems to cooperate among themselves, across levels of government, and with nonprofits, businesses and individuals.” To that end, the Obama administration undertook a series of supporting actions, including establishing crossagency priority goals around issues such as reducing veteran homelessness, data sharing, and streamlining the sharing of social media licenses between agencies. Tackling many of these issues successfully involved the transformative intersection of innovation and technology.

In 2010, when Congress passed a series of amendments to the Government Performance and Results Act (GPRA), it provided the statutory basis for a broader, more consistent use of collaboration as a way of implementing policies and programs. These changes put in place a series of administrative processes:

  • The designation of agency and cross-agency priority goals
  • The naming of goal leaders
  • The convening of a set of regular progress reviews

Taken together, these legislative changes embedded the value of collaboration into the administrative fabric of the governing bureaucracy. In addition, the evolution of technology tools and the advances in the use of social media has dramatically lowered the technical and bureaucratic barriers to working in a more collaborative environment….(More)”

How open company data was used to uncover the powerful elite benefiting from Myanmar’s multi-billion dollar jade industry


OpenCorporates: “Today, we’re pleased to release a white paper on how OpenCorporates data was used to uncover the powerful elite benefiting from Myanmar’s multi-billion dollar jade industry, in a ground-breaking report from Global Witness. This investigation is an important case study on how open company data and identifiers are critical tool to uncover corruption and the links between companies and the real people benefitting from it.

This white paper shows how not only was it critical that OpenCorporates had this information (much of the information was removed from the official register during the investigation), but that the fact that it was machine-readable data, available via an API (data service), and programmatically combinable with other data was essential to discover the hidden connections between the key actors and the jade industry. Global Witness was able to analyse this data with the help of Open Knowledge.

In this white paper, we make recommendations about the collection and publishing of statutory company information as open data to facilitate the creation of a hostile environment for corruption by providing a rigorous framework for public scrutiny and due diligence.

You can find the white paper here or read it on Medium.”

Using data to improve the environment


Sir Philip Dilley at the UK Environment Agency: “We live in a data rich world. As an engineer I know the power of data in the design and implementation of new urban spaces, iconic buildings and the infrastructure on which we all depend.

Data also is a powerful force in helping us to protect the environment and it can be mined from a variety of sources.

Since the Victorian times naturalists have collected data on the natural world. At the Environment Agency we continue to use local enthusiasts to track rainfall, which we use to feed into and support local projections of flood risk. But the advent of computing power and the Government’s move to open data means we can now all use data in a new and exciting way. The result is a more informed approach to improving the environment and protecting people.

For the last 17 years the Environment Agency has used lasers in planes to map and scan the English landscape from above to help us carry out work such as flood modelling (data now available for everyone to use). The same information has been used to track changing coastal habitats and to help us use the power of nature to adapt to a changing climate.

We’ve used our LIDAR height data together with aerial photography to inform the location and design of major coastal realignment sites. The award-winning Medmerry project, which created 183 hectares of new coastal habitat and protects 348 properties from flooding, was based on this data-led approach.

Those who live near rivers or who use them for sport and recreation know the importance of getting up to date information on river flows. We already provide online services to the public so they can see current warnings and river levels information, but opening our data means everyone can get bespoke information through one postcode or location search.

We are not the only ones seeing the power of environmental data. Data entrepreneurs know how to get accurate and easily accessible information to the public. And that means that we can all make informed choices.FloodAlerts provides a graphical representation of flood warnings and gives localised updates every 15 minutes and Flood Risk Finder app provides flood risk profiles on any property in England, both using data made available for public use by the Environment Agency.

Our bathing waters data directs those who like to swim, surf or paddle with vital information on water quality. The Safer Seas Service app alerts water users when water quality is reduced at beaches and our bathing water data is also used by the Marine Conservation Society’s Good Beach Guide….(More)”