Big data’s ‘streetlight effect’: where and how we look affects what we see


 at the Conversation: “Big data offers us a window on the world. But large and easily available datasets may not show us the world we live in. For instance, epidemiological models of the recent Ebola epidemic in West Africa using big data consistently overestimated the risk of the disease’s spread and underestimated the local initiatives that played a critical role in controlling the outbreak.

Researchers are rightly excited about the possibilities offered by the availability of enormous amounts of computerized data. But there’s reason to stand back for a minute to consider what exactly this treasure trove of information really offers. Ethnographers like me use a cross-cultural approach when we collect our data because family, marriage and household mean different things in different contexts. This approach informs how I think about big data.

We’ve all heard the joke about the drunk who is asked why he is searching for his lost wallet under the streetlight, rather than where he thinks he dropped it. “Because the light is better here,” he said.

This “streetlight effect” is the tendency of researchers to study what is easy to study. I use this story in my course on Research Design and Ethnographic Methods to explain why so much research on disparities in educational outcomes is done in classrooms and not in students’ homes. Children are much easier to study at school than in their homes, even though many studies show that knowing what happens outside the classroom is important. Nevertheless, schools will continue to be the focus of most research because they generate big data and homes don’t.

The streetlight effect is one factor that prevents big data studies from being useful in the real world – especially studies analyzing easily available user-generated data from the Internet. Researchers assume that this data offers a window into reality. It doesn’t necessarily.

Looking at WEIRDOs

Based on the number of tweets following Hurricane Sandy, for example, it might seem as if the storm hit Manhattan the hardest, not the New Jersey shore. Another example: the since-retired Google Flu Trends, which in 2013 tracked online searches relating to flu symptoms to predict doctor visits, but gave estimates twice as high as reports from the Centers for Disease Control and Prevention. Without checking facts on the ground, researchers may fool themselves into thinking that their big data models accurately represent the world they aim to study.

The problem is similar to the “WEIRD” issue in many research studies. Harvard professor Joseph Henrich and colleagues have shown that findings based on research conducted with undergraduates at American universities – whom they describe as “some of the most psychologically unusual people on Earth” – apply only to that population and cannot be used to make any claims about other human populations, including other Americans. Unlike the typical research subject in psychology studies, they argue, most people in the world are not from Western, Educated, Industrialized, Rich and Democratic societies, i.e., WEIRD.

Twitter users are also atypical compared with the rest of humanity, giving rise to what our postdoctoral researcher Sarah Laborde has dubbed the “WEIRDO” problem of data analytics: most people are not Western, Educated, Industrialized, Rich, Democratic and Online.

Context is critical

Understanding the differences between the vast majority of humanity and that small subset of people whose activities are captured in big data sets is critical to correct analysis of the data. Considering the context and meaning of data – not just the data itself – is a key feature of ethnographic research, argues Michael Agar, who has written extensively about how ethnographers come to understand the world….(https://theconversation.com/big-datas-streetlight-effect-where-and-how-we-look-affects-what-we-see-58122More)”

Towards a critique of cybernetic urbanism: The smart city and the society of control


Maroš Krivý at Planning Theory: “The smart city has become a hegemonic notion of urban governance, transforming and supplanting planning. The first part of this article reviews current critiques of this notion. Scholars present three main arguments against the smart city: that it is incompatible with an informal character of the city, that it subjects the city to corporate power and that it reproduces social and urban inequalities. It is argued that these critiques either misunderstand how power functions in the smart city or fail to address it as a specific modality of entrepreneurial urban governance. The second part advances an alternative critique, contending that the smart city should be understood as an urban embodiment of the society of control (Deleuze). The smart city is embedded in the intellectual framework of second order cybernetics and articulates urban subjectivity in terms of data flows. Planning as a political practice is superseded by an environmental-behavioural control, in which subjectivity is articulated supra-individually (permeating the city with sensing nodes) and infra-individually (making citizens into sensing nodes)….(More)”

An Unintended Side Effect of Transparency


Stephen Engelberg in ProPublica: “In 2013, ProPublica released Prescriber Checkup, a database that detailed the prescribing habits of hundreds of thousands of doctors across the country.

ProPublica reporters used the data — which reflected prescriptions covered by Medicare’s massive drug program, known as part D — to uncover several important findings. The data showed doctors often prescribed narcotic painkillers and antipsychotic drugs in quantities that could be dangerous for their patients, many of whom were elderly. The reporters also found evidence that some doctors wrote far, far more prescriptions than their peers for expensive brand-name drugs for which there were cheaper generic alternatives. And we found instances of probable fraud that had gone undetected by the government.

The data proved equally useful for others: Doctors themselves turned to Prescriber Checkup to assess how they compared to their peers. Medical plan administrators and hospitals checked it to see whether their doctors were following best practices in treating patients. Law enforcement officials searched the database for leads on fraud and illicit trafficking in pain medications. Patients turned to the data to vet their doctors’ drug choices and compare them with others in their specialties.

Recently, though, we picked up clear signs that some readers are using the data for another purpose: To search for doctors likely to prescribe them some widely abused drugs, many of them opioids.

Like nearly everyone on the web, we use Google Analytics to collect data on our site. So far this year, it appears that perhaps as many as 25 percent of Prescriber Checkup’s page views involve narcotic painkillers, anti-anxiety medications, and amphetamines….(More)”

Society’s biggest problems need more than a nudge


 at the Conversation: “So-called “nudge units” are popping up in governments all around the world.

The best-known examples include the U.K.’s Behavioural Insights Team, created in 2010, and the White House-based Social and Behavioral Sciences Team, introduced by the Obama administration in 2014. Their mission is to leverage findings from behavioral science so that people’s decisions can be nudged in the direction of their best intentions without curtailing their ability to make choices that don’t align with their priorities.

Overall, these – and other – governments have made important strides when it comes to using behavioral science to nudge their constituents into better choices.

Yet, the same governments have done little to improve their own decision-making processes. Consider big missteps like the Flint water crisis. How could officials in Michigan decide to place an essential service – safe water – and almost 100,000 people at risk in order to save US$100 per day for three months? No defensible decision-making process should have allowed this call to be made.

When it comes to many of the big decisions faced by governments – and the private sector – behavioral science has more to offer than simple nudges.

Behavioral scientists who study decision-making processes could also help policy-makers understand why things went wrong in Flint, and how to get their arms around a wide array of society’s biggest problems – from energy transitions to how to best approach the refugee crisis in Syria.

When nudges are enough

The idea of nudging people in the direction of decisions that are in their own best interest has been around for a while. But it was popularized in 2008 with the publication of the bestseller “Nudge“ by Richard Thaler of the University of Chicago and Cass Sunstein of Harvard.

A common nudge goes something like this: if we want to eat better but are having a hard time doing it, choice architects can reengineer the environment in which we make our food choices so that healthier options are intuitively easier to select, without making it unrealistically difficult to eat junk food if that’s what we’d rather do. So, for example, we can shelve healthy foods at eye level in supermarkets, with less-healthy options relegated to the shelves nearer to the floor….

Sometimes a nudge isn’t enough

Nudges work for a wide array of choices, from ones we face every day to those that we face infrequently. Likewise, nudges are particularly well-suited to decisions that are complex with lots of different alternatives to choose from. And, they are advocated in situations where the outcomes of our decisions are delayed far enough into the future that they feel uncertain or abstract. This describes many of the big decisions policy-makers face, so it makes sense to think the solution must be more nudge units.

But herein lies the rub. For every context where a nudge seems like a realistic option, there’s at least another context where the application of passive decision support would be either be impossible – or, worse, a mistake.

Take, for example, the question of energy transitions. These transitions are often characterized by the move from infrastructure based on fossil fuels to renewables to address all manner of risks, including those from climate change. These are decisions that society makes infrequently. They are complex. And, the outcomes – which are based on our ability to meet conflicting economic, social and environmental objectives – will be delayed.

But, absent regulation that would place severe restrictions on the kinds of options we could choose from – and which, incidentally, would violate the freedom-of-choice tenet of choice architecture – there’s no way to put renewable infrastructure options at proverbial eye level for state or federal decision-makers, or their stakeholders.

Simply put, a nudge for a decision like this would be impossible. In these cases, decisions have to be made the old-fashioned way: with a heavy lift instead of a nudge.

Complex policy decisions like this require what we call active decision support….(More)”

Where are Human Subjects in Big Data Research? The Emerging Ethics Divide


Paper by Jacob Metcalf and Kate Crawford: “There are growing discontinuities between the research practices of data science and established tools of research ethics regulation. Some of the core commitments of existing research ethics regulations, such as the distinction between research and practice, cannot be cleanly exported from biomedical research to data science research. These discontinuities have led some data science practitioners and researchers to move toward rejecting ethics regulations outright. These shifts occur at the same time as a proposal for major revisions to the Common Rule — the primary regulation governing human-subjects research in the U.S. — is under consideration for the first time in decades. We contextualize these revisions in long-running complaints about regulation of social science research, and argue data science should be understood as continuous with social sciences in this regard. The proposed regulations are more flexible and scalable to the methods of non-biomedical research, but they problematically exclude many data science methods from human-subjects regulation, particularly uses of public datasets. The ethical frameworks for big data research are highly contested and in flux, and the potential harms of data science research are unpredictable. We examine several contentious cases of research harms in data science, including the 2014 Facebook emotional contagion study and the 2016 use of geographical data techniques to identify the pseudonymous artist Banksy. To address disputes about human-subjects research ethics in data science,critical data studies should offer a historically nuanced theory of “data subjectivity” responsive to the epistemic methods, harms and benefits of data science and commerce….(More)”

Jakarta’s plans for predictive government


 at GovInsider: “Jakarta is predicting floods and traffic using complaints data, and plans to do so for dengue as well.

Its Smart City Unit has partnered with startup Qlue to build a dashboard, analysing data from online complaints, sensors and traffic apps. “Our algorithms can predict several things related to our reports such as flood, traffic, and others”, Qlue co-founder and CEO Rama Raditya told GovInsider.

Take floods, for instance. Using trends in complaints from citizens, water level history from sensors and weather data, it can predict the intensity of floods in specific locations next year. “They can predict what will happen when they compare the weather with the flood conditions from last year”, he said.

The city will start to predict dengue hotspots from next year, Rama said. The dashboard was not originally looking at dengue, but after receiving “thousands of complaints on dengue locations”, the government is now looking into this data. “Next year our algorithm will allow the government to know before it happens so they can prepare the amount of medication and so on within each district,” he said.

The dashboard is paired with an app. The app started with collecting citizens’ complaints and has been expanding with new features. It now has a virtual reality section to explore tourist sites in the city. Next week it is launching an augmented reality feature giving directions to nearby ATMs, restaurants,mosques and parks, Rama said.

Qlue has become a strategic part of the Jakarta administration, with the Governor himself using it to decide who to fire and promote. Following its rise in the capital city, it is now being used by 12 other cities across Indonesia: Bandung, Makassar, Bali, Manado, Surabaya, Bogor, Depok, Palembang, Bekasi,Yogyakarta, Riau and Semarang….(More)

Citizen Generated Data In Practice


DataShift: “No-one can communicate the importance of citizen-generated data better than those who are actually working with it. At DataShift, we want to highlight the civil society organisations who have told us about the tangible results they have achieved through innovative approaches to harnessing data from citizens.

Each essay profiles the objectives, challenges and targets of an organisation using data generated by citizens to achieve their goals. We hope that the essays in this collection can help more people feel more confident about asking questions of the data that affects their lives, and taking a hands-on approach to creating it. (More)

ESSAYS

VOZDATA

People and collaborative technology are helping to redefine Argentina’s fourthestate

SCIENCE FOR CHANGE KOSOVO (SFCK)

Collaborative citizen science to tackleKosovo’s air pollution problem and simultaneously engage with a politically disenfranchised generation of young people

On Iceland’s Crowdsourced Constitution


Larry Lessig: “In the history of constitutions across the world, America has had a unique place: Ours was the first constitution ratified by the people in convention. But Iceland has now done something much more significant: For the first time in the history of the world, and using a technology only possible in the21st century, the people of a nation have crafted their own constitution through an open and inclusive crowd-sourcing process. Yet astonishingly,that constitution remains unenforced.

As everyone in [Iceland] knows, after the financial disasters of 2008, the citizens of Iceland began a process to claim back their own sovereignty.Building on the values identified by 1,000 randomly selected citizens,Icelanders launched a process to crowdsource a new constitution. That initiative was then ratified when the Parliament established a procedure for selecting delegates to a drafting commission. More than 500 citizens ran to serve on that 25 person commission. Over four months, the commissioners met to draft a constitution, with their work made available for public comment throughout the process. More than 3600 comments were offered by the public, leading to scores of modifications. The final draft, adopted unanimously, was then sent to the parliament and to the people. More than2/3ds of voters endorsed the document in a non-binding referendum as the basis of a new constitution.

Never in the history of constitutionalism has anything like this ever been done. If democracy is rule by the people — if the sovereignty of a democratic nation is ultimately the people — then this process and the constitution it produced is as authentic and binding as any in the world. Yet the parliament of Iceland has refused to allow this constitution to go into effect. And the question that anyone in the movements for democracy across the world must ask is just this: By what right?

No doubt, the procedure for crafting and ultimately ratifying the constitution included as the final step Parliament’s sanction — just as the procedure for selecting a government in Britain is subject ultimately to theQueen’s sanction. But the Queen understands the limited power that right conveys — if Britain is to call itself a democracy. And the same is true ofIceland. When the people have acted as they have here — by crafting a constitution in the most inclusive and reflective way that has ever, in the history of constitutionalism, happened, and then endorsed that work by a popular vote, by what moral authority does the Parliament now say no? No doubt, there are parts of the constitution that some don’t like. But democracy is not a promise of perfection. And no constitution in the history of the world has ever been loved by everyone it affected — just ask the million African slaves whose freedom was made unconstitutional through1808 by America’s popularly ratified constitution.

The question for Iceland is, who is sovereign? Is it the people or is it not?And if it is the people, will the people demand that their will be respected?…(More)”

Insights On Collective Problem-Solving: Complexity, Categorization And Lessons From Academia


Part 3 of an interview series by Henry Farrell for the MacArthur Research Network on Opening Governance: “…Complexity theorists have devoted enormous energy and attention to thinking about how complex problems, in which different factors interact in ways that are hard to predict, can best be solved. One key challenge is categorizing problems, so as to understand which approaches are best suited to addressing them.

Scott Page is the Leonid Hurwicz Collegiate Professor of Complex Systems at the University of Michigan, Ann Arbor, and one of the world’s foremost experts on diversity and problem-solving. I asked him a series of questions about how we might use insights from academic research to think better about how problem solving works.

Henry: One of the key issues of collective problem-solving is what you call the ‘problem of problems’ – the question of identifying which problems we need to solve. This is often politically controversial – e.g., it may be hard to get agreement that global warming, or inequality, or long prison sentences are a problem. How do we best go about identifying problems, given that people may disagree?

Scott: In a recent big think paper on the potential of diversity for collective problem solving in Scientific American, Katherine Phillips writes that group members must feel validated, that they must share a commitment to the group, and they must have a common goal if they are going to contribute. This implies that you won’t succeed in getting people to collaborate by setting an agenda from on high and then seeking to attract diverse people to further that agenda.

One way of starting to tackle the problem of problems is to steal a rule of thumb from Getting to Yes, by getting to think people about their broad interests rather than the position that they’re starting from. People often agree on their fundamental desires but disagree on how they can be achieved. For example, nearly everyone wants less crime, but they may disagree over whether they think the solution to crime involves tackling poverty or imposing longer prison sentences. If you can get them to focus on their common interest in solving crime rather than their disagreements, you’re more likely to get them to collaborate usefully.

Segregation amplifies the problem of problems. We live in towns and neighborhoods segregated by race, income, ideology, and human capital. Democrats live near Democrats and Republicans near Republicans. Consensus requires integration. We must work across ideologies. Relatedly, opportunity requires more than access. Many people grow up not knowing any engineers, dentists, doctors, lawyers, and statisticians. This isolation narrows the set of careers they consider and it reduces the diversity of many professions. We cannot imagine lives we do not know.

Henry: Once you get past the problem of problems, you still need to identify which kind of problem you are dealing with. You identify three standard types of problems: solution problems, selection problems and optimization problems. What – very briefly – are the key differences between these kinds of problems?

Scott: I’m constantly pondering the potential set of categories in which collective intelligence can emerge. I’m teaching a course on collective intelligence this semester and the undergraduates and I developed an acronym SCARCE PIGS to describe the different types of domains. Here’s the brief summary:

  • Predict: when individuals combine information, models, or measurements to estimate a future event, guess an answer, or classify an event. Examples might involve betting markets, or combined efforts to guess a quantity, such as Francis Galton’s example of people at a fair trying to guess the weight of a steer.
  • Identify: when individuals have local, partial, or possibly erroneous knowledge and collectively can find an object. Here, an example is DARPA’s Red Balloon project.
  • Solve: when individuals apply and possibly combine higher order cognitive processes and analytic tools for the purpose of finding or improving a solution to a task. Innocentive and similar organizations provide examples of this.
  • Generate: when individuals apply diverse representations, heuristics, and knowledge to produce something new. An everyday example is creating a new building.
  • Coordinate: when individuals adopt similar actions, behaviors, beliefs, or mental frameworks by learning through local interactions. Ordinary social conventions such as people greeting each other are good examples.
  • Cooperate: when individuals take actions, not necessarily in their self interest, that collectively produce a desirable outcome. Here, think of managing common pool resources (e.g. fishing boats not overfishing an area that they collectively control).
  • Arrange: when individuals manipulate items in a physical or virtual environment for their own purposes resulting in an organization of that environment. As an example, imagine a student co-op which keeps twenty types of hot sauce in its pantry. If each student puts whichever hot sauce she uses in the front of the pantry, then on average, the hot sauces will be arranged according to popularity, with the most favored hot sauces in the front and the least favored lost in the back.
  • Respond: when individuals react to external or internal stimuli creating collective responses that maintains system level functioning. For example, when yellow jackets attack a predator to maintain the colony, they are displaying this kind of problem solving.
  • Emerge: when individual parts create a whole that has categorically distinct and new functionalities. The most obvious example of this is the human brain….(More)”

Workplace innovation in the public sector


Eurofound: “Innovative organisational practices in the workplace, which aim to make best use of human capital, are traditionally associated with the private sector. The nature of the public sector activities makes it more difficult to identify these types of internal innovation in publicly funded organisations.

It is widely thought that public sector organisations are neither dynamic nor creative and are typified by a high degree of inertia. Yet the necessity of innovation ought not to be dismissed. The public sector represents a quarter of total EU employment, and it is of critical importance as a provider and regulator of services. Improving how it performs has a knock-on effect not only for private sector growth but also for citizens’ satisfaction. Ultimately, this improves governance itself.

So how can innovative organisation practices help in dealing with the challenges faced by the public sector? Eurofound, as part of a project on workplace innovation in European companies, carried out case studies of both private and public sector organisations. The findings show a number of interesting practices and processes used.

Employee participation

The case studies from the public sector, some of which are described below, demonstrate the central role of employee participation in the implementation of workplace innovation and its impacts on organisation and employees. They indicate that innovative practices have resulted in enhanced organisational performance and quality of working life.

It is widely thought that changes in the public sector are initiated as a response to government policies. This is often true, but workplace innovation may also be introduced as a result of well-designed initiatives driven by external pressures (such as the need for a more competitive public service) or internal pressures (such as a need to update the skills map to better serve the public).

Case study findings

The state-owned Lithuanian energy company Lietuvos Energijos Gamyba (140 KB PDF) encourages employee participation by providing a structured framework for all employees to propose improvements. This has required a change in managerial approach and has spread a sense of ownership horizontally and vertically in the company. The Polish public transport company Jarosław City Transport (191 KB PDF), when faced with serious financial stability challenges, as well as implementing operational changes, set up ways for employees’ voices to be heard, which enabled a contributory dialogue and strengthened partnerships. Consultation, development of mutual trust, and common involvement ensured an effective combination of top-down and bottom-up initiatives.

The Lithuanian Post, AB Lietuvos Pastas (136 KB PDF) experienced a major organisation transformation in 2010 to improve efficiency and quality of service. Through a programme of ‘Loyalty day’ monthly visits, both top and middle management of the central administration visit any part of the company and work with colleagues in other units. Under budgetary pressure to ‘earn their money’, the Danish Vej and Park Bornholm (142 KB PDF) construction services in roads, parks and forests had to find innovative solutions to deal with a merger and privatisation. Their intervention had the characteristics of workplace partnership with a new set of organisational values set from the bottom up. Self-managing teams are essential for the operation of the company.

The world of education has provided new structures that provide better outcomes for students. The South West University of Bulgaria (214 KB PDF) also operates small self-managing teams responsible for employee scheduling. Weekly round-tables encourage participation in collectively finding solutions, creating a more effective environment in which to respond to the competitive demands of education provision.

In Poland, an initiative by the Pomeranian Library (185 KB PDF) improved employee–management dialogue and communication through increased participation. The initiative is a response to the new frameworks for open access to knowledge for users, with the library mirroring the user experience through its own work practices.

Through new dialogue, government advisory bodies have also developed employee-led improvement. Breaking away from a traditional hierarchy is considered important in achieving a more flexible work organisation. Under considerable pressure, the top-heavy management of the British Geological Survey (89 KB PDF) now operates a flexible matrix that promotes innovative and entrepreneurial ways of working. And in Germany, Niersverband (138 KB PDF), a publicly owned water-management company innovated through training, learning, reflection partnerships and workplace partnerships. New occupational profiles were developed to meet external demands. Based on dialogue concerning workplace experiences and competences, employees acquired new qualifications that allowed the company to be more competitive.

In the Funen Village Museum in Odense, Denmark, (143 KB PDF) innovation came about at the request of staff looking for more flexibility in how they work. Formerly most of their work was maintenance tasks, but now they can now engage more with visitors. Control of schedules has moved to the team rather than being the responsibility of a single manager. As a result, museum employees are now hosts as well as craftspeople. They no longer feel ‘forgotten’ and are happier in their work….(More)”

The report Workplace innovation in European companies provides a full analysis of the case studies.

The 51 case studies and the  list of companies (PDF 119 KB) the case studies are based on are available for download.