How Much Development Data Is Enough?

Curated on Posted on by Stefaan Verhulst

Keith D. Shepherd at Project Syndicate: “Rapid advances in technology have dramatically lowered the cost of gathering data. Sensors in space, the sky, the lab, and the field, along with newfound opportunities for crowdsourcing and widespread adoption of the Internet and mobile telephones, are making large amounts of information available to those for whom it was previously out of reach. A small-scale farmer in rural Africa, for example, can now access weather forecasts and market prices at the tap of a screen.

This data revolution offers enormous potential for improving decision-making at every level – from the local farmer to world-spanning development organizations. But gathering data is not enough. The information must also be managed and evaluated – and doing this properly can be far more complicated and expensive than the effort to collect it. If the decisions to be improved are not first properly identified and analyzed, there is a high risk that much of the collection effort could be wasted or misdirected.

This conclusion is itself based on empirical analysis. The evidence is weak, for example, that monitoring initiatives in agriculture or environmental management have had a positive impact. Quantitative analysis of decisions across many domains, including environmental policy, business investments, and cyber security, has shown that people tend to overestimate the amount of data needed to make a good decision or misunderstand what type of data are needed.

Furthermore, grave errors can occur when large data sets are mined using machine algorithms without having first having properly examined the decision that needs to be made. There are many examples of cases in which data mining has led to the wrong conclusion – including in medical diagnoses or legal cases – because experts in the field were not consulted and critical information was left out of the analysis.

Decision science, which combines understanding of behavior with universal principles of coherent decision-making, limits these risks by pairing empirical data with expert knowledge. If the data revolution is to be harnessed in the service of sustainable development, the best practices of this field must be incorporated into the effort.

The first step is to identify and frame frequently recurring decisions. In the field of development, these include large-scale decisions such as spending priorities – and thus budget allocations – by governments and international organizations. But it also includes choices made on a much smaller scale: farmers pondering which crops to plant, how much fertilizer to apply, and when and where to sell their produce.

The second step is to build a quantitative model of the uncertainties in such decisions, including the various triggers, consequences, controls, and mitigants, as well as the different costs, benefits, and risks involved. Incorporating – rather than ignoring – difficult-to-measure, highly uncertain factors leads to the best decisions…..

The third step is to compute the value of obtaining additional information – something that is possible only if the uncertainties in all of the variables have been quantified. The value of information is the amount a rational decision-maker would be willing to pay for it. So we need to know where additional data will have value for improving a decision and how much we should spend to get it. In some cases, no further information may be needed to make a sound decision; in others, acquiring further data could be worth millions of dollars….(More)”

The Innovation the Grantmaking Process Needs

Curated on Posted on by Stefaan Verhulst

Beth Simone Noveck and Andrew Young (TheGovLab) at Governing: “Although traditional grants provide greater flexibility than a contract for the recipient to decide how, precisely, to use the funds to advance a particular goal, prize-backed challenges like those on Challenge.gov have the potential to reach more diverse experts. Challenges are just one example of innovations in the grantmaking process being tested in government, philanthropy and the private sector. These innovations in “open grantmaking” have the potential to yield more legitimate and more accountable processes than their closed-door antecedents. They also have the potential to produce more creative strategies for solving problems and, ultimately, more effective outcomes.

Certainly the time has come for innovation in grantmaking. Despite its importance, we have a decidedly 20th-century system in place for deciding how we make these billions of dollars of crucial public investments. To make the most of limited funding — and help build confidence in the ability of public investments to make a positive difference — it is essential for our government agencies to try more innovative approaches to designing, awarding and measuring their grantmaking activities.

In most instances, grantmaking follows a familiar lifecycle: An agency describes and publicizes the grant in a public call for proposals, qualifying individuals or entities send in applications, and the agencies select the winners through internal deliberations. Members of the public — including outside experts, past grantees and service recipients — often have few opportunities to provide meaningful input before, during or after the granting process. And after awarding grants, the agencies themselves usually have limited continuing interactions with those they fund.

The current closed-door system, to be sure, developed to safeguard the legitimacy and fairness of the process. From application to judging, most government grantmaking has been confidential and at arm’s length. For statutory, regulatory or even cultural reasons, the grantmaking process in many agencies is characterized by caution rather than by creativity.

But it doesn’t always have to be this way, and new, more open grantmaking innovations might prove to be more effective in many contexts. Here are 10 recommendations for innovating the grantmaking process drawn from examples of how government agencies, foundations and philanthropists are changing how they give out money:…(More)”

HereHere

Curated on Posted on by Stefaan Verhulst

“HereHere NYC generates weekly cartoons for NYC neighborhoods based on public data. We sum up how your neighborhood, or other NYC neighborhoods you care about, are doing via weekly email digest, neighborhood-specific Twitter & Instagram feeds, and with deeper data and context.

HereHere is a research project from FUSE Labs, Microsoft Research that explores:

  • Creating compelling stories with data to engage larger communities
  • Inventing new habits for connecting to the hyperlocal
  • Using cartoons as a tool to drive data engagement

HereHere does not use sentiment analysis, but uses a research platform with the intention of surfacing the most pertinent information with a human perspective. …

How It Works

Several times a day we grab the freshest NYC 311 data. The data comes in as a long list of categorized concerns issued by people in NYC (either via phone, email, or text message) and range from heating complaints to compliments to concerns about harboring bees and everything in between.

We separate the data by neighborhood for each of the 42 neighborhoods throughout the 5 boroughs of NYC, and count the total of each concern per neighborhood.

Next, we process the data through the Sentient Data Server. SDS equips each neighborhood with a personality (like a character in a movie or videogame) and we calculate the character’s response to the latest data based on pace, position and trend. For example, a neighborhood might be delighted if after several days of more than 30 heating complaints, heating complaints drops down to 0; or a neighborhood might be ashamed to see a sudden rise in homeless person assistance requests.

 

HereHere determines the most critical 311 issues for each neighborhood each week and uses that to procedurally generate a weekly cartoon for each neighborhood.

 HereHere summarizes the 311 concerns into categories for a quick sense of what’s happening in each neighborhood…(More)

Collective Intelligence in Law Reforms: When the Logic of the Crowds and the Logic of Policymaking Collide

Curated on Posted on by Stefaan Verhulst

Paper by Tanja Aitamurto: “…shows how the two virtues of collective intelligence – cognitive diversity and large crowds –turn into perils in crowdsourced policymaking. That is because of a conflict between the logic of the crowds and the logic of policymaking. The crowd’s logic differs from that of traditional policymaking in several aspects. To mention some of those: In traditional policymaking it is a small group of experts making proposals to the policy, whereas in crowdsourced policymaking, it is a large, anonymous crowd with a mixed level of expertise. The crowd proposes atomic ideas, whereas traditional policymaking is used to dealing with holistic and synthesized proposals. By drawing on data from a crowdsourced law-making process in Finland, the paper shows how the logics of the crowds and policymaking collide in practice. The conflict prevents policymaking fully benefiting from the crowd’s input, and it also hinders governments from adopting crowdsourcing more widely as a practice for deploying open policymaking practices….(More)”

How Facebook Makes Us Dumber

Curated on Posted on by Stefaan Verhulst

 in BloombergView: “Why does misinformation spread so quickly on the social media? Why doesn’t it get corrected? When the truth is so easy to find, why do people accept falsehoods?

A new study focusing on Facebook users provides strong evidence that the explanation is confirmation bias: people’s tendency to seek out information that confirms their beliefs, and to ignore contrary information.

Confirmation bias turns out to play a pivotal role in the creation of online echo chambers. This finding bears on a wide range of issues, including the current presidential campaign, the acceptance of conspiracy theories and competing positions in international disputes.

The new study, led by Michela Del Vicario of Italy’s Laboratory of Computational Social Science, explores the behavior of Facebook users from 2010 to 2014. One of the study’s goals was to test a question that continues to be sharply disputed: When people are online, do they encounter opposing views, or do they create the virtual equivalent of gated communities?

Del Vicario and her coauthors explored how Facebook users spread conspiracy theories (using 32 public web pages); science news (using 35 such pages); and “trolls,” which intentionally spread false information (using two web pages). Their data set is massive: It covers all Facebook posts during the five-year period. They explored which Facebook users linked to one or more of the 69 web pages, and whether they learned about those links from their Facebook friends.

In sum, the researchers find a lot of communities of like-minded people. Even if they are baseless, conspiracy theories spread rapidly within such communities.

More generally, Facebook users tended to choose and share stories containing messages they accept, and to neglect those they reject. If a story fits with what people already believe, they are far more likely to be interested in it and thus to spread it….(More)”

Initial Conditions Matter: Social Capital and Participatory Development

Curated on Posted on by Stefaan Verhulst

Paper by Lisa A. Cameron et al: “Billions of dollars have been spent on participatory development programs in the developing world. These programs give community members an active decision-making role. Given the emphasis on community involvement, one might expect that the effectiveness of this approach would depend on communities’ pre-existing social capital stocks. Using data from a large randomised field experiment of Community-Led Total Sanitation in Indonesia, we find that villages with high initial social capital built toilets and reduced open defecation, resulting in substantial health benefits. In villages with low initial stocks of social capital, the approach was counterproductive – fewer toilets were built than in control communities and social capital suffered….(More)”

Big Data: A Tool for Inclusion or Exclusion? Understanding the Issues

Curated on Posted on by Stefaan Verhulst

Press Release: “A new report from the Federal Trade Commission outlines a number of questions for businesses to consider to help ensure that their use of big data analytics, while producing many benefits for consumers, avoids outcomes that may be exclusionary or discriminatory.

“Big data’s role is growing in nearly every area of business, affecting millions of consumers in concrete ways,” said FTC Chairwoman Edith Ramirez. “The potential benefits to consumers are significant, but businesses must ensure that their big data use does not lead to harmful exclusion or discrimination.”

The report, Big Data: A Tool for Inclusion or Exclusion? Understanding the Issues, looks specifically at big data at the end of its lifecycle – how it is used after being collected and analyzed, and draws on information from the FTC’s 2014 workshop, “Big Data: A Tool for Inclusion or Exclusion?,” as well as the Commission’s seminar on Alternative Scoring Products. The Commission also considered extensive public comments and additional public research in compiling the report.

The report highlights a number of innovative uses of big data that are providing benefits to underserved populations, including increased educational attainment, access to credit through non-traditional methods, specialized health care for underserved communities, and better access to employment.

In addition, the report looks at possible risks that could result from biases or inaccuracies about certain groups, including more individuals mistakenly denied opportunities based on the actions of others, exposing sensitive information, creating or reinforcing existing disparities, assisting in the targeting of vulnerable consumers for fraud, creating higher prices for goods and services in lower-income communities and weakening the effectiveness of consumer choice.

The report outlines some of the various laws that apply to the use of big data, especially in regards to possible issues of discrimination or exclusion, including the Fair Credit Reporting Act, FTC Act and equal opportunity laws. It also provides a range of questions for businesses to consider when they examine whether their big data programs comply with these laws.

The report also proposes four key policy questions that are drawn from research into the ways big data can both present and prevent harms. The policy questions are designed to help companies determine how best to maximize the benefit of their use of big data while limiting possible harms, by examining both practical questions of accuracy and built-in bias as well as whether the company’s use of big data raises ethical or fairness concerns….(More)”

Managing Secrecy

Curated on Posted on by Stefaan Verhulst

Clare Birchall in the International Journal of Communication: “As many anthropologists and sociologists have long argued, understanding the meaning and place of secrets is central to an adequate representation of society. This article extends previous accounts of secrecy in social, governmental, and organizational settings to configure secrecy as one form of visibility management among others. Doing so helps to remove the secret from a post-Enlightenment value system that deems secrets bad and openness good. Once secrecy itself is seen as a neutral phenomenon, we can focus on the politicality or ethics of any particular distribution of the visible, sayable, and knowable. Alongside understanding the work secrecy performs in contemporary society, this article argues that we can also seek inspiration from the secret as a methodological tool and political tactic. Moving beyond the claim to privacy, a claim that has lost bite in this era of state and consumer dataveillance, a “right to opacity”—the right to not be transparent, legible, seen—might open up an experience of subjectivity and responsibility beyond the circumscribed demands of the current politicotechnological management of visibilities….(More)”

Open data set to reshape charity and activism in 2016

Curated on Posted on by Stefaan Verhulst

The Guardian: “In 2015 the EU launched the world’s first international data portal, the Chinese government pledged to make state data public, and the UK lost its open data crown to Taiwan. Troves of data were unlocked by governments around the world last year, but the usefulness of much of that data is still to be determined by the civic groups, businesses and governments who use it. So what’s in the pipeline? And how will the open data ecosystem grow in 2016? We asked the experts.

1. Data will be seen as infrastructure (Heather Savory, director general for data capability, Office for National Statistics)….

2. Journalists, charities and civil society bodies will embrace open data (Hetan Shah, executive director, the Royal Statistical Society)…3. Activists will take it upon themselves to create data

3. Activists will take it upon themselves to create data (Pavel Richter, chief executive, Open Knowledge International)….

 

4. Data illiteracy will come at a heavy price (Sir Nigel Shadbolt, principal, Jesus College, Oxford, professorial research fellow in computer science, University of Oxford and chairman and co-founder of the Open Data Institute…)

5. We’ll create better tools to build a web of data (Dr Elena Simperl, associate professor, electronics and computer science, University of Southampton) …(More)”

Privacy, security and data protection in smart cities: a critical EU law perspective

Curated on Posted on by Stefaan Verhulst

CREATe Working Paper by Lilian Edwards: “Smart cities” are a buzzword of the moment. Although legal interest is growing, most academic responses at least in the EU, are still from the technological, urban studies, environmental and sociological rather than legal, sectors2 and have primarily laid emphasis on the social, urban, policing and environmental benefits of smart cities, rather than their challenges, in often a rather uncritical fashion3 . However a growing backlash from the privacy and surveillance sectors warns of the potential threat to personal privacy posed by smart cities . A key issue is the lack of opportunity in an ambient or smart city environment for the giving of meaningful consent to processing of personal data; other crucial issues include the degree to which smart cities collect private data from inevitable public interactions, the “privatisation” of ownership of both infrastructure and data, the repurposing of “big data” drawn from IoT in smart cities and the storage of that data in the Cloud.

This paper, drawing on author engagement with smart city development in Glasgow as well as the results of an international conference in the area curated by the author, argues that smart cities combine the three greatest current threats to personal privacy, with which regulation has so far failed to deal effectively; the Internet of Things(IoT) or “ubiquitous computing”; “Big Data” ; and the Cloud. While these three phenomena have been examined extensively in much privacy literature (particularly the last two), both in the US and EU, the combination is under-explored. Furthermore, US legal literature and solutions (if any) are not simply transferable to the EU because of the US’s lack of an omnibus data protection (DP) law. I will discuss how and if EU DP law controls possible threats to personal privacy from smart cities and suggest further research on two possible solutions: one, a mandatory holistic privacy impact assessment (PIA) exercise for smart cities: two, code solutions for flagging the need for, and consequences of, giving consent to collection of data in ambient environments….(More)