The deception that lurks in our data-driven world


Alexis C. Madrigal at Fusion: “…There’s this amazing book called Seeing Like a State, which shows how governments and other big institutions try to reduce the vast complexity of the world into a series of statistics that their leaders use to try to comprehend what’s happening.

The author, James C. Scott, opens the book with an extended anecdote about the Normalbaum. In the second half of the 18th century, Prussian rulers wanted to know how many “natural resources” they had in the tangled woods of the country. So, they started counting. And they came up with these huge tables that would let them calculate how many board-feet of wood they could pull from a given plot of forest. All the rest of the forest, everything it did for the people and the animals and general ecology of the place was discarded from the analysis.

The world proved too unruly. Their data wasn’t perfect.

But the world proved too unruly. Their data wasn’t perfect. So they started creating new forests, the Normalbaum, planting all the trees at the same time, and monoculturing them so that there were no trees in the forest that couldn’t be monetized for wood. “The fact is that forest science and geometry, backed by state power, had the capacity to transform the real, diverse, and chaotic old-growth forest into a new, more uniform forest that closely resembled the administrative grid of its techniques,” Scott wrote.

normal forrest plan

The spreadsheet became the world! They even planted the trees in rows, like a grid.

German foresters got very scientific with their fertilizer applications and management practices. And the scheme really worked—at least for a hundred years. Pretty much everyone across the world adopted their methods.

Then the forests started dying.

“In the German case, the negative biological and ultimately commercial consequences of the stripped-down forest became painfully obvious only after the second rotation of conifers had been planted,” Scott wrote.

The complex ecosystem that underpinned the growth of these trees through generations—all the microbial and inter-species relationships—were torn apart by the rigor of the Normalbaum. The nutrient cycles were broken. Resilience was lost. The hidden underpinnings of the world were revealed only when they were gone. The Germans, like they do, came up with a new word for what happened: Waldsterben, or forest death.

The hidden underpinnings of the world were revealed only when they were gone.

Sometimes, when I look out at our world—at the highest level—in which thin data have come to stand in for huge complex systems of human and biological relationships, I wonder if we’re currently deep in the Normalbaum phase of things, awaiting the moment when Waldsterbensets in.

Take the ad-supported digital media ecosystem. The idea is brilliant: capture data on people all over the web and then use what you know to show them relevant ads, ads they want to see. Not only that, but because it’s all tracked, unlike broadcast or print media, an advertiser can measure what they’re getting more precisely. And certainly the digital advertising market has grown, taking share from most other forms of media. The spreadsheet makes a ton of sense—which is one reason for the growth predictions that underpin the massive valuations of new media companies.

But scratch the surface, like Businessweek recently did, and the problems are obvious. A large percentage of the traffic to many stories and videos consists of software pretending to be human.

“The art is making the fake traffic look real, often by sprucing up websites with just enough content to make them appear authentic,” Businessweek says. “Programmatic ad-buying systems don’t necessarily differentiate between real users and bots, or between websites with fresh, original work, and Potemkin sites camouflaged with stock photos and cut-and-paste articles.”

Of course, that’s not what high-end media players are doing. But the cheap programmatic ads, fueled by fake traffic, drive down the pricesacross the digital media industry, making it harder to support good journalism. Meanwhile, users of many sites are rebelling against the business model by installing ad blockers.

The advertisers and ad-tech firms just wanted to capture user data to show them relevant ads. They just wanted to measure their ads more effectively. But placed into the real-world, the system that grew up around these desires has reshaped the media landscape in unpredictable ways.

We’ve deceived ourselves into thinking data is a camera, but it’s really an engine. Capturing data about something changes the way that something works. Even the mere collection of stats is not a neutral act, but a way of reshaping the thing itself….(More)”

Governments’ Self-Disruption Challenge


Mohamed A. El-Erian at Project Syndicate: “One of the most difficult challenges facing Western governments today is to enable and channel the transformative – and, for individuals and companies, self-empowering – forces of technological innovation. They will not succeed unless they become more open to creative destruction, allowing not only tools and procedures, but also mindsets, to be revamped and upgraded. The longer it takes them to meet this challenge, the bigger the lost opportunities for current and future generations.
Self-empowering technological innovation is all around us, affecting a growing number of people, sectors, and activities worldwide. Through an ever-increasing number of platforms, it is now easier than ever for households and corporations to access and engage in an expanding range of activities – from urban transportation to accommodation, entertainment, and media. Even the regulation-reinforced, fortress-like walls that have traditionally surrounded finance and medicine are being eroded.

…In fact, Western political and economic structures are, in some ways, specifically designed to resist deep and rapid change, if only to prevent temporary and reversible fluctuations from having an undue influence on underlying systems. This works well when politics and economies are operating in cyclical mode, as they usually have been in the West. But when major structural and secular challenges arise, as is the case today, the advanced countries’ institutional architecture acts as a major obstacle to effective action….Against this background, a rapid and comprehensive transformation is clearly not feasible. (In fact, it may not even be desirable, given the possibility of collateral damage and unintended consequences.) The best option for Western governments is thus to pursue gradual change, propelled by a variety of adaptive instruments, which would reach a critical mass over time.
Such tools include well-designed public-private partnerships, especially when it comes to modernizing infrastructure; disruptive outside advisers – selected not for what they think, but for how they think – in the government decision-making process; mechanisms to strengthen inter-agency coordination so that it enhances, rather than retards, policy responsiveness; and broader cross-border private-sector linkages to enhance multilateral coordination.
How economies function is changing, as relative power shifts from established, centralized forces toward those that respond to the unprecedented empowerment of individuals. If governments are to overcome the challenges they face and maximize the benefits of this shift for their societies, they need to be a lot more open to self-disruption. Otherwise, the transformative forces will leave them and their citizens behind….(More)”

Big Data and Mass Shootings


Holman W. Jenkins in the Wall Street Journal: “As always, the dots are connected after the fact, when the connecting is easy. …The day may be coming, sooner than we think, when such incidents can be stopped before they get started. A software program alerts police to a social-media posting by an individual of interest in their jurisdiction. An algorithm reminds them why the individual had become a person of interest—a history of mental illness, an episode involving a neighbor. Months earlier, discreet inquires by police had revealed an unhealthy obsession with weapons—key word, unhealthy. There’s no reason why gun owners, range operators and firearms dealers shouldn’t be a source of information for local police seeking information about who might merit special attention.

Sound scary? Big data exists to find the signal among the noise. Your data is the noise. It’s what computerized systems seek to disregard in their quest for information that actually would be useful to act on. Big data is interested in needles, not hay.

Still don’t trust the government? You’re barking up an outdated tree. Consider the absurdly ancillary debate last year on whether the government should be allowed to hold telephone “metadata” when the government already holds vastly more sensitive data on all of us in the form of tax, medical, legal and census records.

All this seems doubly silly given the spacious information about each of us contained in private databases, freely bought and sold by marketers. Bizarre is the idea that Facebook should be able to use our voluntary Facebook postings to decide what we might like to buy, but police shouldn’t use the same information to prevent crime.

Hitachi, the big Japanese company, began testing its crime-prediction software in several unnamed American cities this month. The project, called Hitachi Visualization Predictive Crime Analytics, culls crime records, map and transit data, weather reports, social media and other sources for patterns that might otherwise go unnoticed by police.

Colorado-based Intrado, working with LexisNexis and Motorola Solutions, already sells police a service that instantly scans legal, business and social-media records for information about persons and circumstances that officers may encounter when responding to a 911 call at a specific address. Hundreds of public safety agencies find the system invaluable though that didn’t stop the city of Bellingham, Wash., from rejecting it last year on the odd grounds that such software must be guilty of racial profiling.

Big data is changing how police allocate resources and go about fighting crime. …It once was freely asserted that police weren’t supposed to prevent crime, only solve it. But recent research shows investment in policing actually does reduce crime rates—and produces a large positive return measured in dollars and cents. A day will come when failing to connect the dots in advance of a mass-shooting won’t be a matter for upturned hands. It will be a matter for serious recrimination…(More)

Weak States, Poor Countries


Angus Deaton in Project Syndicate: “Europeans tend to feel more positively about their governments than do Americans, for whom the failures and unpopularity of their federal, state, and local politicians are a commonplace. Yet Americans’ various governments collect taxes and, in return, provide services without which they could not easily live their lives.

Americans, like many citizens of rich countries, take for granted the legal and regulatory system, the public schools, health care and social security for the elderly, roads, defense and diplomacy, and heavy investments by the state in research, particularly in medicine. Certainly, not all of these services are as good as they might be, nor held in equal regard by everyone; but people mostly pay their taxes, and if the way that money is spent offends some, a lively public debate ensues, and regular elections allow people to change priorities.

All of this is so obvious that it hardly needs saying – at least for those who live in rich countries with effective governments. But most of the world’s population does not.

In much of Africa and Asia, states lack the capacity to raise taxes or deliver services. The contract between government and governed – imperfect in rich countries – is often altogether absent in poor countries. The New York cop was little more than impolite (and busy providing a service); in much of the world, police prey on the people they are supposed to protect, shaking them down for money or persecuting them on behalf of powerful patrons.

Even in a middle-income country like India, public schools and public clinics face mass (unpunished) absenteeism. Private doctors give people what (they think) they want – injections, intravenous drips, and antibiotics – but the state does not regulate them, and many practitioners are entirely unqualified.

Throughout the developing world, children die because they are born in the wrong place – not of exotic, incurable diseases, but of the commonplace childhood illnesses that we have known how to treat for almost a century. Without a state that is capable of delivering routine maternal and child health care, these children will continue to die.

Likewise, without government capacity, regulation and enforcement do not work properly, so businesses find it difficult to operate. Without properly functioning civil courts, there is no guarantee that innovative entrepreneurs can claim the rewards of their ideas.

The absence of state capacity – that is, of the services and protections that people in rich countries take for granted – is one of the major causes of poverty and deprivation around the world. Without effective states working with active and involved citizens, there is little chance for the growth that is needed to abolish global poverty.

Unfortunately, the world’s rich countries currently are making things worse. Foreign aid – transfers from rich countries to poor countries – has much to its credit, particularly in terms of health care, with many people alive today who would otherwise be dead. But foreign aid also undermines the development of local state capacity….

One thing that we can do is to agitate for our own governments to stop doing those things that make it harder for poor countries to stop being poor. Reducing aid is one, but so is limiting the arms trade, improving rich-country trade and subsidy policies, providing technical advice that is not tied to aid, and developing better drugs for diseases that do not affect rich people. We cannot help the poor by making their already-weak governments even weaker….(More)”

Viscous Open Data: The Roles of Intermediaries in an Open Data Ecosystem


François van Schalkwyk, Michelle Willmers & Maurice McNaughton in Journal: “Information Technology for Development”: “Open data have the potential to improve the governance of universities as public institutions. In addition, open data are likely to increase the quality, efficacy and efficiency of the research and analysis of higher education systems by providing a shared empirical base for critical interrogation and reinterpretation. Drawing on research conducted by the Emerging Impacts of Open Data in Developing Countries project, and using an ecosystems approach, this research paper considers the supply, demand and use of open data as well as the roles of intermediaries in the governance of South African public higher education. It shows that government’s higher education database is a closed and isolated data source in the data ecosystem; and that the open data that are made available by government is inaccessible and rarely used. In contrast, government data made available by data intermediaries in the ecosystem are being used by key stakeholders. Intermediaries are found to play several important roles in the ecosystem: (i) they increase the accessibility and utility of data; (ii) they may assume the role of a “keystone species” in a data ecosystem; and (iii) they have the potential to democratize the impacts and use of open data. The article concludes that despite poor data provision by government, the public university governance open data ecosystem has evolved because intermediaries in the ecosystem have reduced the viscosity of government data. Further increasing the fluidity of government open data will improve access and ensure the sustainability of open data supply in the ecosystem….(More)”

Applying Design Approaches to Policy Making: Discovering Policy Lab


Lucy Kimbell: “During 2014-2015 I spent three days a week embedded as an academic researcher in Policy Lab, a team in the Cabinet Office working closely with government departments and with the UK Civil Service’s policy profession. This was enabled by a fellowship from the Arts and Humanities Research Council while I was principal research fellow at the University of Brighton, building on other work including Mapping Social Design Research and Practice. Policy Lab is one example of an innovation team inside government, a growing network and set of practices.

The report summarises findings from the research informed by literature in design studies and organisation studies. It uses a format inspired by graphic novels in order to open up the work of interpretation about the role of design approaches in policy making and government….(More)” Click to download report (6MB, PDF)

Can non-Western democracy help to foster political transformation?


Richard Youngs at Open Democracy: “…many non-Western countries are showing signs of a newly-vibrant civic politics, organized in ways that are not centered on NGOs but on more loosely structured social movements in participatory forms of democracy where active citizenship is crucial—not just structured or formal, representative democratic institutions. Bolivia is a good example.

Many Western governments were skeptical about President Evo Morales’ political project, fearing that he would prove to be just as authoritarian as Hugo Chavez in Venezuela. But some Western donors (including Germany and the European Union) have already increased their support to indigenous social movements in Bolivia because they’ve become a vital channel of influence and accountability between government and society.

Secondly, it’s clear that the political dimensions of democracy will be undermined if economic conditions and inequalities are getting worse, so democracy promotion efforts need to be delinked from pressures to adopt neo-liberal economic policies. Western interests need to do more to prove that they are not supporting democracy primarily as a means to further their economic interest in ‘free markets.’ That’s why the European Union is supporting a growing number of projects designed to build up social insurance schemes during the early phases of democratic transitions. European diplomats, at least, say that they see themselves as supporters of social and economic democracy.

Donors are becoming more willing to support the role of labor unions in pro-democracy coalition-building; and to protect labor standards as a crucial part of political transitions in countries as diverse as Tunisia, Georgia, China, Egypt and Ecuador. But they should do more to assess how the embedded structures of economic power can undermine the quality of democratic processes. Support for civil society organizations that are keen on exploring heterodox economic models should also be stepped up.

Thirdly, non-Western structures and traditions can help to reduce violent conflict successfully. Tribal chiefs, traditional decision-making circles and customary dispute resolution mechanisms are commonplace in Africa and Asia, and have much to teach their counterparts in the West. In Afghanistan, for example, international organizations realized that the standard institutions of Western liberal democracy were gaining little traction, and were probably deepening rather than healing pre-existing divisions, so they’ve started to support local-level deliberative forums instead.

Something similar is happening in the Balkans, where the United States and the European Union are giving priority to locally tailored, consensual power-sharing arrangements. The United Nations is working with customary justice systems in Somalia. And in South Sudan and Kenya, donors have worked with tribal chiefs and supported traditional authorities to promote a better understanding of human rights and gender justice issues. These forms of power-sharing and ‘consensual communitarianism’ can be quite effective in protecting minorities while also encouraging dialogue and deliberation.

As these brief examples show, different countries can both offer and receive ideas about democratic transformation regardless of geography, though this is never straightforward. It involves finding a balance between defending genuinely-universal norms on the one hand, and encouraging democratic experimentation on the other. This is a thin line to walk, and it requires, for example, recognition that the basic precepts of liberal democracy are not synonymous with what can be seen as an amoral individualism, particularly in highly religious communities.

Pro-democracy reformers and civic groups in non-Western countries often take international organizations to task for pushing too hard on questions of ‘Western liberal rights’ rather than supporting variations to the standard, individualist template, even where tribal structures and traditional conflict-resolution mechanisms work reasonably well. This has led to resistance against international support in places as diverse as Libya, Mali and Pakistan…..

Academic critical theorists argue that Western democracy promoters fail to take alternative models of democracy on board because they would endanger their own geostrategic and economic interests….(More)”

We Need Both Networks and Communities


Henry Mintzberg at HBR: “If you want to understand the difference between a network and a community, ask your Facebook friends to help paint your house.

Social media certainly connects us to whoever is on the other end of the line, and so extends our social networks in amazing ways. But this can come at the expense of deeper personal relationships. When it feels like we’re up-to-date on our friends’ lives through Facebook or Instagram, we may become less likely to call them, much less meet up. Networks connect; communities care.

….A century or two ago, the word community “seemed to connote a specific group of people, from a particular patch of earth, who knew and judged and kept an eye on one another, who shared habits and history and memories, and could at times be persuaded to act as a whole on behalf of a part.” In contrast,the word has now become fashionable to describe what are really networks, as in the “business community”—”people with common interests [but] not common values, history, or memory.”

Does this matter for managing in the digital age, even for dealing with our global problems? It sure does. In a 2012 New York Times column, Thomas Friedman reported asking an Egyptian friend about the protest movements in that country: “Facebook really helped people to communicate, but not to collaborate,” he replied. Friedman added that “at their worst, [social media sites] can become addictive substitutes for real action.” That is why, while the larger social movements, as in Cairo’s Tahrir Square or on Wall Street, may raise consciousness about the need for renewal in society, it is the smaller social initiatives, usually developed by small groups in communities, that do much of the renewing….

We tend to make a great fuss about leadership these days, but communityship is more important. The great leaders create, enhance, and support a sense of community in their organizations, and that requires hands-on management. Hence managers have get beyond their individual leadership, to recognize the collective nature of effective enterprise.

Especially for operating around the globe, electronic communication has become essential. But the heart of enterprise remains rooted in personal collaborative relationships, albeit networked by the new information technologies. Thus, in localities and organizations, across societies and around the globe, beware of “networked individualism“ where people communicate readily while struggling to collaborate.

The new digital technologies, wonderful as they are in enhancing communication, can have a negative effect on collaboration unless they are carefully managed. An electronic device puts us in touch with a keyboard, that’s all….(More)”

A new model to explore non-profit social media use for advocacy and civic engagement


David Chapman, Katrina Miller-Stevens, John C Morris, and Brendan O’Hallarn in First Monday: “In an age when electronic communication is ubiquitous, non-profit organizations are actively using social media platforms as a way to deliver information to end users. In spite of the broad use of these platforms, little scholarship has focused on the internal processes these organizations employ to implement these tools. A limited number of studies offer models to help explain an organization’s use of social media from initiation to outcomes, yet few studies address a non-profit organization’s mission as the driver to employ social media strategies and tactics. Furthermore, the effectiveness of social media use is difficult for non-profit organizations to measure. Studies that attempt to address this question have done so by viewing social media platform analytics (e.g., Facebook analytics) or analyzing written content by users of social media (Nah and Saxton, 2013; Auger, 2013; Uzunoğlu and Misci Kip, 2014; or Guo and Saxton, 2014). The value added of this study is to present a model for practice (Weil, 1997) that explores social media use and its challenges from a non-profit organization’s mission through its desired outcome, in this case an outcome of advocacy and civic engagement.

We focus on one non-profit organization, Blue Star Families, that actively engages in advocacy and civic engagement. Blue Star Families was formed in 2009 to “raise the awareness of the challenges of military family life with our civilian communities and leaders” (Blue Star Families, 2010). Blue Star Families is a virtual organization with no physical office location. Thus, the organization relies on its Web presence and social media tools to advocate for military families and engage service members and their families, communities, and citizens in civic engagement activities (Blue Star Families, 2010).

The study aims to provide organizational-level insights of the successes and challenges of working in the social media environment. Specifically, the study asks: What are the processes non-profit organizations follow to link organizational mission to outcomes when using social media platforms? What are the successes and challenges of using social media platforms for advocacy and civic engagement purposes? In our effort to answer these questions, we present a new model to explore non-profit organizations’ use of social media platforms by building on previous models and frameworks developed to explore the use of social media in the public, private, and non-profit sectors.

This research is important for three reasons. First, most previous studies of social media tend to employ models that focus on the satisfaction of the social media tools for organizational members, rather than the utility of social media as a tool to meet organizational goals. Our research offers a means to explore the utility of social media from an organization perspective. Second, the exemplar case for our research, Blue Star Families, Inc., is a non-profit organization whose mission is to create and nurture a virtual community spread over a large geographical — if not global — area. Because Blue Star Families was founded as an online organization that could not exist without social media, it provides a case for which social media is a critical component of the organization’s activity. Finally, we offer some “lessons learned” from our case to identify issues for other organizations seeking to create a significant social media presence.

This paper is organized as follows: first, the growth of social media is briefly addressed to provide background context. Second, previous models and frameworks exploring social media are discussed. This is followed by a presentation of a new model exploring the use of social media from an organizational perspective, starting with the driver of a non-profit organization’s mission, to its desired outcomes of advocacy and civic engagement. Third, the case study methodology is explained. Next, we present an analysis and discussion applying the new model to Blue Star Families’ use of social media platforms. We conclude by discussing the challenges of social media revealed in the case study analysis, and we offer recommendations to address these challenges….(More)”

How the USGS uses Twitter data to track earthquakes


Twitter Blog: “After the disastrous Sichuan earthquake in 2008, people turned to Twitter to share firsthand information about the earthquake. What amazed many was the impression that Twitter was faster at reporting the earthquake than the U.S. Geological Survey (USGS), the official government organization in charge of tracking such events.

This Twitter activity wasn’t a big surprise to the USGS. The USGS National Earthquake Information Center (NEIC) processes about 2,000 realtime earthquake sensors, with the majority based in the United States. That leaves a lot of empty space in the world with no sensors. On the other hand, there are hundreds of millions of people using Twitter who can report earthquakes. At first, the USGS staff was a bit skeptical that Twitter could be used as a detection system for earthquakes – but when they looked into it, they were surprised at the effectiveness of Twitter data for detection.

USGS staffers Paul Earle, a seismologist, and Michelle Guy, a software developer, teamed up to look at how Twitter data could be used for earthquake detection and verification. By using Twitter’s Public API, they decided to use the same time series event detection method they use when detecting earthquakes. This gave them a baseline for earthquake-related chatter, but they decided to dig in even further. They found that people Tweeting about actual earthquakes kept their Tweets really short, even just to ask, “earthquake?” Concluding that people who are experiencing earthquakes aren’t very chatty, they started filtering out Tweets with more than seven words. They also recognized that people sharing links or the size of the earthquake were significantly less likely to be offering firsthand reports, so they filtered out any Tweets sharing a link or a number. Ultimately, this filtered stream proved to be very significant at determining when earthquakes occurred globally.

USGS Modeling Twitter Data to Detect Earthquakes

While I was at the USGS office in Golden, Colo. interviewing Michelle and Paul, three earthquakes happened in a relatively short time. Using Twitter data, their system was able to pick up on an aftershock in Chile within one minute and 20 seconds – and it only took 14 Tweets from the filtered stream to trigger an email alert. The other two earthquakes, off Easter Island and Indonesia, weren’t picked up because they were not widely felt…..

The USGS monitors for earthquakes in many languages, and the words used can be a clue as to the magnitude and location of the earthquake. Chile has two words for earthquakes: terremotoand temblor; terremoto is used to indicate a bigger quake. This one in Chile started with people asking if it was a terremoto, but others realizing that it was a temblor.

As the USGS team notes, Twitter data augments their own detection work on felt earthquakes. If they’re getting reports of an earthquake in a populated area but no Tweets from there, that’s a good indicator to them that it’s a false alarm. It’s also very cost effective for the USGS, because they use Twitter’s Public API and open-source software such as Kibana and ElasticSearch to help determine when earthquakes occur….(More)”