Hien To, Seon Ho Kim, and Cyrus Shahabi: “Efficient and thorough data collection and its timely analysis are critical for disaster response and recovery in order to save peoples lives during disasters. However, access to comprehensive data in disaster areas and their quick analysis to transform the data to actionable knowledge are challenging. With the popularity and pervasiveness of mobile devices, crowdsourcing data collection and analysis has emerged as an effective and scalable solution. This paper addresses the problem of crowdsourcing mobile videos for disasters by identifying two unique challenges of 1) prioritizing visualdata collection and transmission under bandwidth scarcity caused by damaged communication networks and 2) analyzing the acquired data in a timely manner. We introduce a new crowdsourcing framework for acquiring and analyzing the mobile videos utilizing fine granularity spatial metadata of videos for a rapidly changing disaster situation. We also develop an analytical model to quantify the visual awareness of a video based on its metadata and propose the visual awareness maximization problem for acquiring the most relevant data under bandwidth constraints. The collected videos are evenly distributed to off-site analysts to collectively minimize crowdsourcing efforts for analysis. Our simulation results demonstrate the effectiveness and feasibility of the proposed framework….(More)”
Data Science of the People, for the People, by the People: A Viewpoint on an Emerging Dichotomy
Paper by Kush R. Varshney: “This paper presents a viewpoint on an emerging dichotomy in data science: applications in which predictions of datadriven algorithms are used to support people in making consequential decisions that can have a profound effect on other people’s lives and applications in which data-driven algorithms act autonomously in settings of low consequence and large scale. An example of the first type of application is prison sentencing and of the second type is selecting news stories to appear on a person’s web portal home page. It is argued that the two types of applications require data, algorithms and models with vastly different properties along several dimensions, including privacy, equitability, robustness, interpretability, causality, and openness. Furthermore, it is argued that the second type of application cannot always be used as a surrogate to develop methods for the first type of application. To contribute to the development of methods for the first type of application, one must really be working on the first type of application….(More)”
Beyond the Quantified Self: Thematic exploration of a dataistic paradigm
Minna Ruckenstein and Mika Pantzar in New Media and Society: “This article investigates the metaphor of the Quantified Self (QS) as it is presented in the magazine Wired (2008–2012). Four interrelated themes—transparency, optimization, feedback loop, and biohacking—are identified as formative in defining a new numerical self and promoting a dataist paradigm. Wired captures certain interests and desires with the QS metaphor, while ignoring and downplaying others, suggesting that the QS positions self-tracking devices and applications as interfaces that energize technological engagements, thereby pushing us to rethink life in a data-driven manner. The thematic analysis of the QS is treated as a schematic aid for raising critical questions about self-quantification, for instance, detecting the merging of epistemological claims, technological devices, and market-making efforts. From this perspective, another definition of the QS emerges: a knowledge system that remains flexible in its aims and can be used as a resource for epistemological inquiry and in the formation of alternative paradigms….(More)”
Where the right to know comes from
Michael Schudson in Columbia Journalism Review: “…what began as an effort to keep the executive under check by the Congress became a law that helped journalists, historians, and ordinary citizens monitor federal agencies. Nearly 50 years later, it may all sound easy and obvious. It was neither. And this burst of political engagement is rarely, if ever, mentioned by journalists themselves as an exception to normal “acts of journalism.”
But how did it happen at all? In 1948, the American Society of Newspaper Editors set up its first-ever committee on government restrictions on the freedom to gather and publish news. It was called the “Committee on World Freedom of Information”—a name that implied that limiting journalists’ access or straightforward censorship was a problem in other countries. The committee protested Argentina’s restrictions on what US correspondents could report, censorship in Guatemala, and—closer to home—US military censorship in occupied Japan.
When the ASNE committee turned to the problem of secrecy in the US government in the early 1950s, it chose to actively criticize such secrecy, but not to “become a legislative committee.” Even in 1953, when ASNE leaders realized that significant progress on government secrecy might require federal legislation, they concluded that “watching all such legislation” would be an important task for the committee, but did not suggest taking a public position.
Representative Moss changed this. Moss was a small businessman who had served several terms in the California legislature before his election to Congress in 1952. During his first term, he requested some data from the Civil Service Commission about dismissals of government employees on suspicion of disloyalty. The commission flatly turned him down. “My experience in Washington quickly proved that you had a hell of a time getting any information,” Moss recalled. Two years later, a newly re-elected Moss became chair of a House subcommittee on government information….(More)”
Crowdsourced research: Many hands make tight work
Raphael Silberzahn & Eric L. Uhlmann in Nature: “…For many research problems, crowdsourcing analyses will not be the optimal solution. It demands a huge amount of resources for just one research question. Some questions will not benefit from a crowd of analysts: researchers’ approaches will be much more similar for simple data sets and research designs than for large and complex ones. Importantly, crowdsourcing does not eliminate all bias. Decisions must still be made about what hypotheses to test, from where to get suitable data, and importantly, which variables can or cannot be collected. (For instance, we did not consider whether a particular player’s skin tone was lighter or darker than that of most of the other players on his team.) Finally, researchers may continue to disagree about findings, which makes it challenging to present a manuscript with a clear conclusion. It can also be puzzling: the investment of more resources can lead to less-clear outcomes.
“Under the current system, strong storylines win out over messy results.”
Still, the effort can be well worth it. Crowdsourcing research can reveal how conclusions are contingent on analytical choices. Furthermore, the crowdsourcing framework also provides researchers with a safe space in which they can vet analytical approaches, explore doubts and get a second, third or fourth opinion. Discussions about analytical approaches happen before committing to a particular strategy. In our project, the teams were essentially peer reviewing each other’s work before even settling on their own analyses. And we found that researchers did change their minds through the course of analysis.
Crowdsourcing also reduces the incentive for flashy results. A single-team project may be published only if it finds significant effects; participants in crowdsourced projects can contribute even with null findings. A range of scientific possibilities are revealed, the results are more credible and analytical choices that seem to sway conclusions can point research in fruitful directions. What is more, analysts learn from each other, and the creativity required to construct analytical methodologies can be better appreciated by the research community and the public.
Of course, researchers who painstakingly collect a data set may not want to share it with others. But greater certainty comes from having an independent check. A coordinated effort boosts incentives for multiple analyses and perspectives in a way that simply making data available post-publication does not.
The transparency resulting from a crowdsourced approach should be particularly beneficial when important policy issues are at stake. The uncertainty of scientific conclusions about, for example, the effects of the minimum wage on unemployment, and the consequences of economic austerity policies should be investigated by crowds of researchers rather than left to single teams of analysts.
Under the current system, strong storylines win out over messy results. Worse, once a finding has been published in a journal, it becomes difficult to challenge. Ideas become entrenched too quickly, and uprooting them is more disruptive than it ought to be. The crowdsourcing approach gives space to dissenting opinions.
Scientists around the world are hungry for more-reliable ways to discover knowledge and eager to forge new kinds of collaborations to do so. Our first project had a budget of zero, and we attracted scores of fellow scientists with two tweets and a Facebook post.
Researchers who are interested in starting or participating in collaborative crowdsourcing projects can access resources available online. We have publicly shared all our materials and survey templates, and the Center for Open Science has just launched ManyLab, a web space where researchers can join crowdsourced projects….(More).
See also Nature special collection:reproducibility
Meaningful meetings: how can meetings be made better?
Geoff Mulgan at NESTA: “Many of us spend much of our time in meetings and at conferences. But too often these feel like a waste of time, or fail to make the most of the knowledge and experience of the people present.
Meetings have changed – with much more use of online tools, and a growing range of different meeting formats. But our sense is that meetings could be much better run and achieve better results.
This paper tries to help. It summarises some of what’s known about how meetings work well or badly; makes recommendations about how to make meetings better; and showcases some interesting recent innovations. It forms part of a larger research programme at Nesta on collective intelligence which is investigating how groups and organisations can make the most of their brains, and of the technologies they use.
We hope the paper will be helpful to anyone designing or running meetings of any kind, and that readers will contribute good examples, ideas and evidence which can be added into future versions….(More)”
The deception that lurks in our data-driven world
Alexis C. Madrigal at Fusion: “…There’s this amazing book called Seeing Like a State, which shows how governments and other big institutions try to reduce the vast complexity of the world into a series of statistics that their leaders use to try to comprehend what’s happening.
The author, James C. Scott, opens the book with an extended anecdote about the Normalbaum. In the second half of the 18th century, Prussian rulers wanted to know how many “natural resources” they had in the tangled woods of the country. So, they started counting. And they came up with these huge tables that would let them calculate how many board-feet of wood they could pull from a given plot of forest. All the rest of the forest, everything it did for the people and the animals and general ecology of the place was discarded from the analysis.
But the world proved too unruly. Their data wasn’t perfect. So they started creating new forests, the Normalbaum, planting all the trees at the same time, and monoculturing them so that there were no trees in the forest that couldn’t be monetized for wood. “The fact is that forest science and geometry, backed by state power, had the capacity to transform the real, diverse, and chaotic old-growth forest into a new, more uniform forest that closely resembled the administrative grid of its techniques,” Scott wrote.
The spreadsheet became the world! They even planted the trees in rows, like a grid.
German foresters got very scientific with their fertilizer applications and management practices. And the scheme really worked—at least for a hundred years. Pretty much everyone across the world adopted their methods.
Then the forests started dying.
“In the German case, the negative biological and ultimately commercial consequences of the stripped-down forest became painfully obvious only after the second rotation of conifers had been planted,” Scott wrote.
The complex ecosystem that underpinned the growth of these trees through generations—all the microbial and inter-species relationships—were torn apart by the rigor of the Normalbaum. The nutrient cycles were broken. Resilience was lost. The hidden underpinnings of the world were revealed only when they were gone. The Germans, like they do, came up with a new word for what happened: Waldsterben, or forest death.
Sometimes, when I look out at our world—at the highest level—in which thin data have come to stand in for huge complex systems of human and biological relationships, I wonder if we’re currently deep in the Normalbaum phase of things, awaiting the moment when Waldsterbensets in.
Take the ad-supported digital media ecosystem. The idea is brilliant: capture data on people all over the web and then use what you know to show them relevant ads, ads they want to see. Not only that, but because it’s all tracked, unlike broadcast or print media, an advertiser can measure what they’re getting more precisely. And certainly the digital advertising market has grown, taking share from most other forms of media. The spreadsheet makes a ton of sense—which is one reason for the growth predictions that underpin the massive valuations of new media companies.
But scratch the surface, like Businessweek recently did, and the problems are obvious. A large percentage of the traffic to many stories and videos consists of software pretending to be human.
“The art is making the fake traffic look real, often by sprucing up websites with just enough content to make them appear authentic,” Businessweek says. “Programmatic ad-buying systems don’t necessarily differentiate between real users and bots, or between websites with fresh, original work, and Potemkin sites camouflaged with stock photos and cut-and-paste articles.”
Of course, that’s not what high-end media players are doing. But the cheap programmatic ads, fueled by fake traffic, drive down the pricesacross the digital media industry, making it harder to support good journalism. Meanwhile, users of many sites are rebelling against the business model by installing ad blockers.
The advertisers and ad-tech firms just wanted to capture user data to show them relevant ads. They just wanted to measure their ads more effectively. But placed into the real-world, the system that grew up around these desires has reshaped the media landscape in unpredictable ways.
We’ve deceived ourselves into thinking data is a camera, but it’s really an engine. Capturing data about something changes the way that something works. Even the mere collection of stats is not a neutral act, but a way of reshaping the thing itself….(More)”
Governments’ Self-Disruption Challenge
Mohamed A. El-Erian at Project Syndicate: “One of the most difficult challenges facing Western governments today is to enable and channel the transformative – and, for individuals and companies, self-empowering – forces of technological innovation. They will not succeed unless they become more open to creative destruction, allowing not only tools and procedures, but also mindsets, to be revamped and upgraded. The longer it takes them to meet this challenge, the bigger the lost opportunities for current and future generations.
Self-empowering technological innovation is all around us, affecting a growing number of people, sectors, and activities worldwide. Through an ever-increasing number of platforms, it is now easier than ever for households and corporations to access and engage in an expanding range of activities – from urban transportation to accommodation, entertainment, and media. Even the regulation-reinforced, fortress-like walls that have traditionally surrounded finance and medicine are being eroded.
…In fact, Western political and economic structures are, in some ways, specifically designed to resist deep and rapid change, if only to prevent temporary and reversible fluctuations from having an undue influence on underlying systems. This works well when politics and economies are operating in cyclical mode, as they usually have been in the West. But when major structural and secular challenges arise, as is the case today, the advanced countries’ institutional architecture acts as a major obstacle to effective action….Against this background, a rapid and comprehensive transformation is clearly not feasible. (In fact, it may not even be desirable, given the possibility of collateral damage and unintended consequences.) The best option for Western governments is thus to pursue gradual change, propelled by a variety of adaptive instruments, which would reach a critical mass over time.
Such tools include well-designed public-private partnerships, especially when it comes to modernizing infrastructure; disruptive outside advisers – selected not for what they think, but for how they think – in the government decision-making process; mechanisms to strengthen inter-agency coordination so that it enhances, rather than retards, policy responsiveness; and broader cross-border private-sector linkages to enhance multilateral coordination.
How economies function is changing, as relative power shifts from established, centralized forces toward those that respond to the unprecedented empowerment of individuals. If governments are to overcome the challenges they face and maximize the benefits of this shift for their societies, they need to be a lot more open to self-disruption. Otherwise, the transformative forces will leave them and their citizens behind….(More)”
Big Data and Mass Shootings
Holman W. Jenkins in the Wall Street Journal: “As always, the dots are connected after the fact, when the connecting is easy. …The day may be coming, sooner than we think, when such incidents can be stopped before they get started. A software program alerts police to a social-media posting by an individual of interest in their jurisdiction. An algorithm reminds them why the individual had become a person of interest—a history of mental illness, an episode involving a neighbor. Months earlier, discreet inquires by police had revealed an unhealthy obsession with weapons—key word, unhealthy. There’s no reason why gun owners, range operators and firearms dealers shouldn’t be a source of information for local police seeking information about who might merit special attention.
Sound scary? Big data exists to find the signal among the noise. Your data is the noise. It’s what computerized systems seek to disregard in their quest for information that actually would be useful to act on. Big data is interested in needles, not hay.
Still don’t trust the government? You’re barking up an outdated tree. Consider the absurdly ancillary debate last year on whether the government should be allowed to hold telephone “metadata” when the government already holds vastly more sensitive data on all of us in the form of tax, medical, legal and census records.
All this seems doubly silly given the spacious information about each of us contained in private databases, freely bought and sold by marketers. Bizarre is the idea that Facebook should be able to use our voluntary Facebook postings to decide what we might like to buy, but police shouldn’t use the same information to prevent crime.
Hitachi, the big Japanese company, began testing its crime-prediction software in several unnamed American cities this month. The project, called Hitachi Visualization Predictive Crime Analytics, culls crime records, map and transit data, weather reports, social media and other sources for patterns that might otherwise go unnoticed by police.
Colorado-based Intrado, working with LexisNexis and Motorola Solutions, already sells police a service that instantly scans legal, business and social-media records for information about persons and circumstances that officers may encounter when responding to a 911 call at a specific address. Hundreds of public safety agencies find the system invaluable though that didn’t stop the city of Bellingham, Wash., from rejecting it last year on the odd grounds that such software must be guilty of racial profiling.
Big data is changing how police allocate resources and go about fighting crime. …It once was freely asserted that police weren’t supposed to prevent crime, only solve it. But recent research shows investment in policing actually does reduce crime rates—and produces a large positive return measured in dollars and cents. A day will come when failing to connect the dots in advance of a mass-shooting won’t be a matter for upturned hands. It will be a matter for serious recrimination…(More)“
Weak States, Poor Countries
Angus Deaton in Project Syndicate: “Europeans tend to feel more positively about their governments than do Americans, for whom the failures and unpopularity of their federal, state, and local politicians are a commonplace. Yet Americans’ various governments collect taxes and, in return, provide services without which they could not easily live their lives.
Americans, like many citizens of rich countries, take for granted the legal and regulatory system, the public schools, health care and social security for the elderly, roads, defense and diplomacy, and heavy investments by the state in research, particularly in medicine. Certainly, not all of these services are as good as they might be, nor held in equal regard by everyone; but people mostly pay their taxes, and if the way that money is spent offends some, a lively public debate ensues, and regular elections allow people to change priorities.
All of this is so obvious that it hardly needs saying – at least for those who live in rich countries with effective governments. But most of the world’s population does not.
In much of Africa and Asia, states lack the capacity to raise taxes or deliver services. The contract between government and governed – imperfect in rich countries – is often altogether absent in poor countries. The New York cop was little more than impolite (and busy providing a service); in much of the world, police prey on the people they are supposed to protect, shaking them down for money or persecuting them on behalf of powerful patrons.
Even in a middle-income country like India, public schools and public clinics face mass (unpunished) absenteeism. Private doctors give people what (they think) they want – injections, intravenous drips, and antibiotics – but the state does not regulate them, and many practitioners are entirely unqualified.
Throughout the developing world, children die because they are born in the wrong place – not of exotic, incurable diseases, but of the commonplace childhood illnesses that we have known how to treat for almost a century. Without a state that is capable of delivering routine maternal and child health care, these children will continue to die.
Likewise, without government capacity, regulation and enforcement do not work properly, so businesses find it difficult to operate. Without properly functioning civil courts, there is no guarantee that innovative entrepreneurs can claim the rewards of their ideas.
The absence of state capacity – that is, of the services and protections that people in rich countries take for granted – is one of the major causes of poverty and deprivation around the world. Without effective states working with active and involved citizens, there is little chance for the growth that is needed to abolish global poverty.
Unfortunately, the world’s rich countries currently are making things worse. Foreign aid – transfers from rich countries to poor countries – has much to its credit, particularly in terms of health care, with many people alive today who would otherwise be dead. But foreign aid also undermines the development of local state capacity….
One thing that we can do is to agitate for our own governments to stop doing those things that make it harder for poor countries to stop being poor. Reducing aid is one, but so is limiting the arms trade, improving rich-country trade and subsidy policies, providing technical advice that is not tied to aid, and developing better drugs for diseases that do not affect rich people. We cannot help the poor by making their already-weak governments even weaker….(More)”