science

How to Win a Science Contest

Curated on March 31, 2016May 29, 2019 by Stefaan Verhulst

Nathan Collins at Pacific Standard: “…there are contests like the DARPA Robotics Challenge, which gives prizes for solving particularly difficult problems, like how to prevent an autonomous vehicle from crashing.

But who wins such contests, and how? One might think it’s the science insiders, since they have the knowledge and background to solve difficult scientific problems. It’s hard to imagine, for example, a political scientist solving a major problem in theoretical physics. At the same time, insiders can become inflexible, having been so ensconced in a particular way of thinking that they can’t see outside of the box, let alone think outside it.

Unfortunately, most of what we know about insiders, outsiders, and scientific success is anecdotal. (Hedy Lamarr, the late actress and co-inventor of a key wireless technology, is a prominent anecdote, but still just an anecdote.) To remedy that, Oguz Ali Acar and Jan van den Ende decided to conduct a proper study. For data, they looked to InnoCentive, an online platform that “crowdsource[s] innovative solutions from the world’s smartest people, who compete to provide ideas and solutions to important business, social, policy, scientific, and technical challenges,” according to its website.

Acar and van den Ende surveyed 230 InnoCentive contest participants, who reported how much expertise they had related to the last problem they’d solved, along with how much experience they had solving similar problems in the past, regardless of whether it was related to their professional expertise. The researchers also asked how many different scientific fields problem solvers had looked to for ideas, and how much effort they’d put into their solutions. For each of the solvers, the researchers then looked at all the contests that person won and computed their odds of winning—a measure of creativity, they argue, since contests are judged in part on the solutions’ creativity.

That data revealed an intuitive, though not entirely simple pattern. Insiders (think Richard Feynman in physics) were more likely to win a contest when they cast a wide net for ideas, while outsiders (like Lamarr) performed best when they focused on one scientific or technological domain. In other words, outsiders—who may bring a useful new perspective to bear—should bone up on the problem they’re trying to solve, while insiders, who’ve already done their homework, benefit from thinking outside the box.

Still, there’s something both groups can’t do without: hard work. “[I]f insiders … spend significant amounts of time seeking out knowledge from a wide variety of other fields, they are more likely to be creative in that domain,” Acar and van den Ende write, and if outsiders work hard, they “can turn their lack of knowledge in a domain into an advantage.”….(More)”

How to stop being so easily manipulated by misleading statistics

Curated on March 30, 2016August 15, 2018 by Stefaan Verhulst

Q&A by Akshat Rathi in Quartz: “There are three kinds of lies: Lies, damned lies, and statistics.” Few people know the struggle of correcting such lies better than David Spiegelhalter. Since 2007, he has been the Winton professor for the public understanding of risk (though he prefers “statistics” to “risk”) at the University of Cambridge.In a sunlit hotel room in Washington DC, Quartz caught up with Spiegelhalter recently to talk about his unique job. The conversation sprawled from the wisdom of eating bacon (would you swallow any other known carcinogen?), to the serious crime of manipulating charts, to the right way to talk about rare but scary diseases.

In a sunlit hotel room in Washington DC, Quartz caught up with Spiegelhalter recently to talk about his unique job. The conversation sprawled from the wisdom of eating bacon (would you swallow any other known carcinogen?), to the serious crime of manipulating charts, to the right way to talk about rare but scary diseases.

When he isn’t fixing people’s misunderstandings of numbers, he works to communicate numbers better so that misunderstandings can be avoided from the beginning. The interview is edited and condensed for clarity….

What’s a recent example of misrepresentation of statistics that drove you bonkers?
I got very grumpy at an official graph of British teenage pregnancy rates that apparently showed they had declined to nearly zero. Until I realized that the bottom part of the axis had been cut off, which made it impossible to visualize the (very impressive) 50% reduction since 2000.You once said graphical representation of data does not always communicate what we think it communicates. What do you mean by that?
Graphs can be as manipulative as words. Using tricks such as cutting axes, rescaling things, changing data from positive to negative, etc. Sometimes putting zero on the y-axis is wrong. So to be sure that you are communicating the right things, you need to evaluate the message that people are taking away. There are no absolute rules. It all depends on what you want to communicate….

Poorly communicated risk can have a severe effect. For instance, the news story about the risk that pregnant women are exposing their unborn child to when they drink alcohol caused stress to one of our news editors who had consumed wine moderately through her pregnancy.

I think it’s irresponsible to say there is a risk when they actually don’t know if there is one. There is scientific uncertainty about that.

“‘Absence of evidence is not evidence of absence.’ I hate that phrase…It’s always used in a manipulative way.” In such situations of unknown risk, there is a phrase that is often used: “Absence of evidence is not evidence of absence.” I hate that phrase. I get so angry when people use that phrase. It’s always used in a manipulative way. I say to them that it’s not evidence of absence, but if you’ve looked hard enough you’ll see that most of the time the evidence shows a very small effect, if at all.

So on the risks of drinking alcohol while being pregnant, the UK’s health authority said that as a precautionary step it’s better not to drink. That’s fair enough. This honesty is important. To say that we don’t definitely know if drinking is harmful, but to be safe we say you shouldn’t. That’s treating people as adults and allowing them to use their own judgement.

Science is a bigger and bigger part of our lives. What is the limitation in science journalism right now and how can we improve it?...(More)

The Function of—and Need for—Institutional Review Boards

Curated on March 29, 2016August 3, 2018 by Stefaan Verhulst

Review by Iain Brassington of The Censor’s Hand: The Misregulation of Human-Subject Research (Carl E. Schneider, The MIT Press): “Scientific research can be a laborious and frustrating process even before it gets started—especially when it involves living human subjects. Universities and other research institutions maintain Institutional Review Boards that scrutinize research proposals and their methodologies, consent and privacy procedures, and so on. Similarly intensive reviews are required when the intention is to use human tissue—if, say, tissue from diagnostic cancer biopsies could potentially be used to gauge the prevalence of some other illness across the population. These procedures can generate absurdities. A doctor who wanted to know which television characters children recognized, for example, was advised to seek ethics committee approval, and told that he needed to do a pilot study as a precursor.

Today’s IRB system is the response to a historic problem: academic researchers’ tendency to behave abominably when left unmonitored. Nazi medical and pseudomedical experiments provide an obvious and well-known reference, but such horrors are not found only in totalitarian regimes. The Tuskegee syphilis study, for example, deliberately left black men untreated over the course of decades so researchers could study the natural course of the disease. On a much smaller but equally disturbing scale is the case of Dan Markingson, a 26-year-old University of Michigan graduate. Suffering from psychotic illness, Markingson was coercively enrolled in a study of antipsychotics to which he could not consent, and concerns about his deteriorating condition were ignored. In 2004, he was found dead, having almost decapitated himself with a box cutter.

Many thoughtful ethicists are aware of the imperfections of IRBs. They have worried publicly for some time that the IRB system, or parts of it, may claim an authority with which even many bioethicists are uncomfortable, and hinder science for no particularly good reason. Does the system need re-tuning, a total re-build, or something even more drastic?

When it comes to IRBs, Carl E. Schneider, a professor of law and internal medicine at the University of Michigan, belongs to the abolitionist camp. In The Censor’s Hand: The Misregulation of Human-Subject Research, he presents the case against the IRB system plainly. It is a case that rests on seven related charges.

IRBs, Schneider posits, cannot be shown to do good, with regulators able to produce “no direct evidence that IRBs prevented harm”; that an IRB at least went through the motions of reviewing the trial in which Markingson died might be cited as evidence of this. On top of that, he claims, IRBs sometimes cause harm, at least insofar as they slow down medical innovation. They are built to err on the side of caution, since “research on humans” can cover a vast range of activities and disciplines, and they struggle to take this range into proper account. Correspondingly, they “lack a legible and convincing ethics”; the autonomy of IRBs means that they come to different decisions on identical cases. (In one case, an IRB thought that providing supplemental vitamin A in a study was so dangerous that it should not be allowed; another thought that withholding it in the same study was so dangerous that it should not be allowed.) IRBs have unrealistically high expectations of their members, who are often fairly ad hoc groupings with no obvious relevant expertise. They overemphasize informed consent, with the unintended consequence that cramming every possible eventuality into a consent form makes it utterly incomprehensible. Finally, Schneider argues, IRBs corrode free expression by restricting what researchers can do and how they can do it….(More)”

Innovation Prizes in Practice and Theory

Curated on March 24, 2016October 23, 2018 by Stefaan Verhulst

Paper by Michael J. Burstein and Fiona Murray: “Innovation prizes in reality are significantly different from innovation prizes in theory. The former are familiar from popular accounts of historical prizes like the Longitude Prize: the government offers a set amount for a solution to a known problem, like £20,000 for a method of calculating longitude at sea. The latter are modeled as compensation to inventors in return for donating their inventions to the public domain. Neither the economic literature nor the policy literature that led to the 2010 America COMPETES Reauthorization Act — which made prizes a prominent tool of government innovation policy — provides a satisfying justification for the use of prizes, nor does either literature address their operation. In this article, we address both of these problems. We use a case study of one canonical, high profile innovation prize — the Progressive Insurance Automotive X Prize — to explain how prizes function as institutional means to achieve exogenously defined innovation policy goals in the face of significant uncertainty and information asymmetries. Focusing on the structure and function of actual innovation prizes as an empirical matter enables us to make three theoretical contributions to the current understanding of prizes. First, we offer a stronger normative justification for prizes grounded in their status as a key institutional arrangement for solving a specified innovation problem. Second, we develop a model of innovation prize governance and then situate that model in the administrative state, as a species of “new governance” or “experimental” regulation. Third, we derive from those analyses a novel framework for choosing among prizes, patents, and grants, one in which the ultimate choice depends on a trade off between the efficacy and scalability of the institutional solution….(More)”

Big data, meet behavioral science

Curated on March 14, 2016August 3, 2018 by Stefaan Verhulst

Ben Castleman at Brookings: “America’s community colleges offer the promise of a more affordable pathway to a bachelor’s degree. Students can pay substantially less for the first two years of college, transfer to a four-year college or university, and still earn their diploma in the same amount of time. At least in theory. Most community college students—80 percent of them—enter with the intention to transfer, but only 20 percent actually do so within five years of entering college. This divide represents a classic case of what behavioralists call an intention-action gap.

Why would so many students who enter community colleges intending to transfer fail to actually do so? Put yourself in the shoes of a 20-something community college student. You’ve worked hard for the past couple years, earning credits and paying a lot less in tuition than you would have if you had enrolled immediately in a four-year college or university. But now you want to transfer, so that you can complete your bachelor’s degree. How do you figure out where to go? Ideally you’d probably like to find a college that would take most of your credits, where you’re likely to graduate from, and where the degree is going to count for something in the labor market. A college advisor could probably help you figure this out,but at many community colleges there are at least 1,000 other students assigned to your advisor, so you might have a hard time getting a quality meeting. Some states have articulation agreements between two- and four-year institutions that guarantee admission for students who complete certain course sequences and perform at a high enough level. But these agreements are often dense and inaccessible.

The combination of big data and behavioral insights has the potential to help students navigate these complex decisions and successfully follow through on their intentions. Big data analytic techniques allow us to identify concrete transfer pathways where students are positioned to succeed; behavioral insights ensure we communicate these options in a way that maximizes students’ engagement and responsiveness…..A growing body of innovative research has demonstrated that, by applying behavioral science insights to the way we communicate with students and families about the opportunities and resources available to them, we can help people navigate these complex decisions and experience better outcomes as a result. A combination of simplified information, reminders, and access to assistance have improved achievement and attainment up and down the education pipeline, nudging parents to practice early-literacy activities with their kids or check in with their high schoolers about missed assignments, andencouraging students to renew their financial aid for college….

These types of big data techniques are already being used in some education sectors. For instance, a growing number of colleges use predictive analytics to identify struggling students who need additional assistance, so faculty and administrators can intervene before the student drops out. But frequently there is insufficient attention, once the results of these predictive analyses are in hand, about how to communicate the information in a way that is likely to lead to behavior change among students or educators. And much of the predictive analytics work has been on the side of plugging leaks in the pipeline (e.g. preventing drop-outs from higher education), rather than on the side of proactively sending students and families personalized information about educational and career pathways where they are likely to flourish…(More)”

Accelerating Discovery with New Tools and Methods for Next Generation Social Science

Curated on March 11, 2016August 3, 2018 by Stefaan Verhulst

DARPA: “The explosive growth of global digital connectivity has opened new possibilities for designing and conducting social science research. Once limited by practical constraints to experiments involving just a few dozen participants—often university students or other easily available groups—or to correlational studies of large datasets without any opportunity for determining causation, scientists can now engage thousands of diverse volunteers online and explore an expanded range of important topics and questions. If new tools and methods for harnessing virtual or alternate reality and massively distributed platforms could be developed and objectively validated, many of today’s most important and vexing challenges in social science—such as identifying the primary drivers of social cooperation, instability and resilience—might be made more tractable, with benefits for domains as broad as national security, public health, and economics.

To begin to assess the research opportunities provided by today’s web-connected world and advanced technologies, DARPA today launched its Next Generation Social Science (NGS2) program. The program aims to build and evaluate new methods and tools to advance rigorous, reproducible social science studies at scales necessary to develop and validate causal models of human social behaviors. The program will draw upon and build across a wide array of disciplines—including social sciences like sociology, economics, political science, anthropology, and psychology, as well as information and computer sciences, physics, biology and math.

As an initial focus, NGS2 will challenge researchers to develop and use these new tools and methods to identify causal mechanisms of “collective identity” formation—how a group of individuals becomes a unified whole, and how under certain circumstances that community breaks down into a chaotic mix of disconnected individuals.

“Social science has done a remarkable job of helping us understand ourselves as the highly social creatures we are, but the field has long acknowledged and rued some frustrating research limitations, including technical and logistical limits to experimentally studying large, representative populations and the challenges of replicating key studies to better understand the limits of our knowledge,” said Adam Russell, DARPA program manager. “As a result, it’s been difficult for social scientists to determine what variables matter most in explaining their observations of human social systems and to move from documenting correlation to identifying causation.”

On top of those methodological and analytic limitations, Russell said, the field is inherently challenged because of its subject matter: human beings, with all their complex variability and seeming unpredictability. “Physicists have joked about how much more difficult their field would be if atoms or electrons had personalities, but that’s exactly the situation faced by social scientists,” he said.

By developing and applying new methods and models to larger, more diverse, and more representative groups of individuals—such as through web-based global gaming and alternate reality platforms—NGS2 seeks to validate new tools that may empower social science in the same way that sophisticated telescopes and microscopes have helped advance astronomy and biology….(More)”

Revolutionizing Innovation: Users, Communities, and Open Innovation

Curated on March 10, 2016August 3, 2018 by Stefaan Verhulst

Book edited by Dietmar Harhoff and Karim R. Lakhani: “The last two decades have witnessed an extraordinary growth of new models of managing and organizing the innovation process that emphasizes users over producers. Large parts of the knowledge economy now routinely rely on users, communities, and open innovation approaches to solve important technological and organizational problems. This view of innovation, pioneered by the economist Eric von Hippel, counters the dominant paradigm, which cast the profit-seeking incentives of firms as the main driver of technical change. In a series of influential writings, von Hippel and colleagues found empirical evidence that flatly contradicted the producer-centered model of innovation. Since then, the study of user-driven innovation has continued and expanded, with further empirical exploration of a distributed model of innovation that includes communities and platforms in a variety of contexts and with the development of theory to explain the economic underpinnings of this still emerging paradigm. This volume provides a comprehensive and multidisciplinary view of the field of user and open innovation, reflecting advances in the field over the last several decades.

The contributors—including many colleagues of Eric von Hippel—offer both theoretical and empirical perspectives from such diverse fields as economics, the history of science and technology, law, management, and policy. The empirical contexts for their studies range from household goods to financial services. After discussing the fundamentals of user innovation, the contributors cover communities and innovation; legal aspects of user and community innovation; new roles for user innovators; user interactions with firms; and user innovation in practice, describing experiments, toolkits, and crowdsourcing, and crowdfunding…(More)”

How Google Optimized Healthy Office Snacks

Curated on March 8, 2016August 3, 2018 by Stefaan Verhulst

Zoe Chance, Ravi Dhar, Michelle Hatzis and Michiel Bakker at Harvard Business Review: “Employers need simple, low-cost ways of helping employees make healthy choices. The effects of poor health and obesity cost U.S. companies $225 billion every year, according to the Centers for Disease Control, and this number is quickly rising. Although some employer-sponsored wellness programs have yielded high returns — Johnson & Johnson reported a 170% return on wellness spending in the 2000s — the employee wellness industry as a whole has struggled to prove its value.

Wellness initiatives often fail because they rely on outdated methods of engagement, placing too much emphasis on providing information. Extensive evidence from behavioral economics has shown that information rarely succeeds in changing behavior or building new habits for fitness and food choices. Telling people why and how to improve their health fails to elicit behavior changes because behavior often diverges from intentions. This is particularly true for food choices because our self-control is taxed by any type of depletion, including hunger. And the necessity of making food decisions many times a day means we can’t devote much processing power to each choice, so our eating behaviors tend to be habit- and instinct-driven. With a clearer understanding of the influences on choice — context and impulsivity, for instance — companies can design environments that reinforce employees’ healthy choices, limit potential lapses, and save on health care costs.

Jointly, the Google Food Team and the Yale Center for Customer Insights have been studying how behavioral economics can improve employee health choices. We’ve run multiple field experiments to understand how small “tweaks” can nudge behavior toward desirable outcomes and yield outsized benefits. To guide these interventions, we distilled scattered findings from behavioral science into a simple framework, the four Ps of behavior change:

Process
Persuasion
Possibilities
Person

The framework helped us structure a portfolio of strategies for making healthy choices easier and more enticing and making unhealthy choices harder and less tempting. Below, we present a brief example of each point of intervention….(More)”

Design for policy and public services

Curated on March 7, 2016October 23, 2018 by Stefaan Verhulst

The Centre for Public Impact: “Traditional approaches to policymaking have left policymakers and citizens looking for alternative solutions. Despite the best of intentions, the standard model of dispassionate expert analysis and subsequent implementation by a professional bureaucracy has, generally, led to siloed solutions and outcomes for citizens that fall short of what might be possible.

The discipline of design may well provide an answer to this problem by offering a collection of methods which allow civil servants to generate insights based on citizens’ needs, aspirations and behaviours. In doing so, it changes the view of citizens from seeing them as anonymous entities to complex humans with complex needs to match. The potential of this new approach is already becoming clear – just ask the medical teams and patients at Norway’s Oslo University Hospital. Women with a heightened risk of developing breast cancer had previously been forced to wait up to three months before receiving an appointment for examination and diagnosis. A redesign reduced this wait to just three days.

In-depth user research identified the principal issues and pinpointed the lack of information about the referral process as a critical problem. The designers also interviewed 40 hospital employees of all levels to find out about their daily schedules and processes. Governments have always drawn inspiration from fields such as sociology and economics. Design methods are not (yet) part of the policymaking canon, but such examples help explain why this may be about to change….(More)”