Kenneth Neil Cukier and Viktor Mayer-Schoenberger in Foreign Affairs: “Everyone knows that the Internet has changed how businesses operate, governments function, and people live. But a new, less visible technological trend is just as transformative: “big data.” Big data starts with the fact that there is a lot more information floating around these days than ever before, and it is being put to extraordinary new uses. Big data is distinct from the Internet, although the Web makes it much easier to collect and share data. Big data is about more than just communication: the idea is that we can learn from a large body of information things that we could not comprehend when we used only smaller amounts.”
Gideon Rose, editor of Foreign Affairs, sits down with Kenneth Cukier, data editor of The Economist (video):
Crowd diagnosis could spot rare diseases doctors miss
New Scientist: “Diagnosing rare illnesses could get easier, thanks to new web-based tools that pool information from a wide variety of sources…CrowdMed, launched on 16 April at the TedMed conference in Washington DC, uses crowds to solve tough medical cases.
Anyone can join CrowdMed and analyse cases, regardless of their background or training. Participants are given points that they can then use to bet on the correct diagnosis from lists of suggestions. This creates a prediction market, with diagnoses falling and rising in value based on their popularity, like stocks in a stock market. Algorithms then calculate the probability that each diagnosis will be correct. In 20 initial test cases, around 700 participants identified each of the mystery diseases as one of their top three suggestions….
Frustrated patients and doctors can also turn to FindZebra, a recently launched search engine for rare diseases. It lets users search an index of rare disease databases looked after by a team of researchers. In initial trials, FindZebra returned more helpful results than Google on searches within this same dataset.”
White House: Unleashing the Power of Big Data
Tom Kalil, Deputy Director for Technology and Innovation at OSTP : “As we enter the second year of the Big Data Initiative, the Obama Administration is encouraging multiple stakeholders, including federal agencies, private industry, academia, state and local government, non-profits, and foundations to develop and participate in Big Data initiatives across the country. Of particular interest are partnerships designed to advance core Big Data technologies; harness the power of Big Data to advance national goals such as economic growth, education, health, and clean energy; use competitions and challenges; and foster regional innovation.
The National Science Foundation has issued a request for information encouraging stakeholders to identify Big Data projects they would be willing to support to achieve these goals. And, later this year, OSTP, NSF, and other partner agencies in the Networking and Information Technology R&D (NITRD) program plan to convene an event that highlights high-impact collaborations and identifies areas for expanded collaboration between the public and private sectors.”
Work-force Science and Big Data
Steve Lohr from the New York Times: “Work-force science, in short, is what happens when Big Data meets H.R….Today, every e-mail, instant message, phone call, line of written code and mouse-click leaves a digital signal. These patterns can now be inexpensively collected and mined for insights into how people work and communicate, potentially opening doors to more efficiency and innovation within companies.
Digital technology also makes it possible to conduct and aggregate personality-based assessments, often using online quizzes or games, in far greater detail and numbers than ever before. In the past, studies of worker behavior were typically based on observing a few hundred people at most. Today, studies can include thousands or hundreds of thousands of workers, an exponential leap ahead.
“The heart of science is measurement,” says Erik Brynjolfsson, director of the Center for Digital Business at the Sloan School of Management at M.I.T. “We’re seeing a revolution in measurement, and it will revolutionize organizational economics and personnel economics.”
The data-gathering technology, to be sure, raises questions about the limits of worker surveillance. “The larger problem here is that all these workplace metrics are being collected when you as a worker are essentially behind a one-way mirror,” says Marc Rotenberg, executive director of the Electronic Privacy Information Center, an advocacy group. “You don’t know what data is being collected and how it is used.”
David Brooks on Big Data
David Brooks in NYT: “Over the past few centuries, there have been many efforts to come up with methods to help predict human behavior — what Leon Wieseltier of The New Republic calls mathematizing the subjective. The current one is the effort to understand the world by using big data.
Other efforts to predict behavior were based on models of human nature. The people using big data don’t presume to peer deeply into people’s souls. They don’t try to explain why people are doing things. They just want to observe what they are doing. The theory of big data is to have no theory, at least about human nature. You just gather huge amounts of information, observe the patterns and estimate probabilities about how people will act in the future….
One of my take-aways is that big data is really good at telling you what to pay attention to. It can tell you what sort of student is likely to fall behind. But then to actually intervene to help that student, you have to get back in the world of causality, back into the world of responsibility, back in the world of advising someone to do x because it will cause y.”
Big Data, Big Brains
“This report on Big Data is the first MeriTalk Beacon, a new series of reports designed to shed light and provide direction on far reaching issues in government and technology. Since Beacons are designed to tackle broad concepts, each Beacon report relies on insight from a small number of big thinkers in the topic area. Less data. More insight. Real knowledge…Mankind created 150 exabytes (billion gigabytes) of data in 2005, and 1,800 exabytes in 20112; growth that only continues to accelerate. Every minute, users: Upload 48 hours of video to YouTube; Send 204 million emails; Spend $207,000 via the web; Create 571 new websites. Within the Federal government; U.S. drone aircraft sent back 24 years worth of video footage in just 2009. Every 24 hours, NASA’s Curiosity rover can send nearly three gigabytes of data, collecting in mere days the equivalent of all human knowledge through the death of Augustus Caesar – from Mars.”
Big Data Challenge to Transform Health Care Delivery
BPC Press Release: “Today, the Bipartisan Policy Center (BPC), Heritage Provider Network (HPN), and The Advisory Board Company launched the Care Transformation Prize Series, a national contest to address the most daunting data problems U.S. health care organizations face as they implement new delivery system and payment reforms.
The goal of this big data challenge is to help health care organizations more effectively use data to drive improvements in health care cost and quality. The series was announced at a BPC-hosted event today that featured a forward-thinking discussion on the strategies that providers, health plans, and states are using to harness data to help Americans lower their health care costs and improve care.”
Big Data can help keep the peace
NextGov story: “Some of the same social media analyses that have helped Google and the Centers for Disease Control and Prevention spot warning signs of a flu outbreak could be used to detect the rumblings of violent conflict before it begins, scholars said in a paper released this week.
Kenyan officials used essentially this system to track hate speech on Facebook, blogs and Twitter in advance of that nation’s 2013 presidential election, which brought Uhuru Kenyatta to power.
Similar efforts to track Syrian social media have been able to identify ceasefire violations within 15 minutes of when they occur, according to the paper on New Technology and the Prevention of Violence and Conflict prepared by the United States Agency for International Development, the United Nations Development Programme and the International Peace Institute and presented at the United States Institute of Peace Friday.”
Data Science: The Numbers of Our Lives
NYT on the integration of data science within universities: “In the last few years, dozens of programs under a variety of names have sprung up in response to the excitement about Big Data, not to mention the six-figure salaries for some recent graduates.”
June 1: National Day of Civic Hacking
From Peter Welsch at the White House: “On the first weekend in June, civic activists, technology experts, and entrepreneurs around the country will gather together for the National Day of Civic Hacking. By combining their expertise with new technologies and publicly released data, participants hope to build tools that help others in their own neighborhoods and across the United States”.
Apply for the National Day of Civic Hacking at the White House. The deadline for applications is 5:00pm on Friday, April 19.