Inside the world’s ‘what works’ teams

Curated on Posted on by Stefaan Verhulst

Jen Gold at What Works Blog: “There’s a small but growing band of government teams around the world dedicated to making experiments happen. The Cabinet Office’s What Works Team, set up in 2013, was the first of its kind. But you’ll now find them in Canada, the US, Finland, Australia, Colombia, and the UAE.

All of these teams work across government to champion the testing and evaluation of new approaches to public service delivery. This blog takes a look at the many ways in which we’re striving to make experimentation the norm in our governments.

Unsurprisingly we’re all operating in very different contexts. Some teams were set up in response to central requirements for greater experimentation. Take Canada, for instance. In 2016 the Treasury Board directed departments and agencies to devote a fixed proportion of programme funds to “experimenting with new approaches” (building on Prime Minister Trudeau’s earlier instruction to Ministers). An Innovation and Experimentation Team was then set up in the Treasury Board to provide some central support.

Finland’s Experimentation Office, based in the Prime Minister’s Office, is in a similar position. The team supports the delivery of Prime Minister Juha Sipilä’s 2016 national action plan that calls for “a culture of experimentation” in public services and a series of flagship policy experiments.

Others, like the US Office of Evaluation Sciences (OES) and the Behavioural Economics Team of the Australian Government (BETA), grew out of political interest in using behavioural science experiments in public policy. But these teams now run experiments in a much broader set of areas.

What unites us is a focus on helping public servants generate and use new evidence in policy decisions and service delivery….(More)”.

Listening to the people who think we are wrong

Curated on Posted on by Stefaan Verhulst

Larry Kramer at the Hewlett Foundation: “Among the most corrosive developments of recent years—one that predates the election of Donald Trump—has been a breakdown in our ability to debate and reason with others with whom we disagree. The term du jour, “tribalism,” replaced the earlier “polarization” precisely to capture the added ingredient of animosity that has made even conversation across partisan divides difficult. Mistrust and hostility have been grafted onto disagreement about ideas.

Political scientists differ about how widespread the phenomenon is—some seeing it shared broadly across American society, while others believe it confined to activist elites. I lean toward the latter view, though the disease seems to be spreading awfully fast. The difference hardly matters, because activists drive and shape public debates. And, either way, the resulting take-no-prisoners politics threatens the future of democratic government, which presupposes disagreement and depends on willingness to work through and across differences from a sense of shared community....

Learning to listen with empathy matters for a number of reasons. An advocate needs to see an opponent’s argument in its strongest light, not only to counter the position effectively, but also to fully understand his or her own position—its weaknesses as well as its strengths—and so be properly prepared to defend it. Nor is this the only reason, because adversarial advocacy is only part of what lawyers do. Most legal work involves bargaining among conflicting interests and finding ways to settle disputes. Good lawyers know how to negotiate and cooperate; they know (in the phrase made famous by Roger Fisher and William Ury) how to “get to yes”—something made vastly easier if one fully and fairly comprehends both sides of an issue. There is a reason lawyers have historically constituted such a disproportionate share of our legislators and executives, and it’s not because they know how to argue. It is because they know how to find common ground.

Not that compromising is always the right thing to do. Without doubt, there are matters of principle too important to relinquish, and instances in which an adversary is too inflexible or too extreme to accommodate. In today’s public discourse, moreover, outright fabrication has become, if not quite acceptable, increasingly common. But one cannot know if or when these are the case unless and until one has examined the other side’s position honestly and confronted the weaknesses in one’s own position fearlessly…

Three techniques in particular pervade the practice of paying heed to an opposing argument without condescending to meet it:

  • First, there is the “straw man” method—a tried and true practice that involves taking the weakest or most extreme or least plausible argument in favor of a position and acting as if it were the only argument for that position; a variation of this method takes the most extreme and unattractive advocates for a position and treats them as typical.
  • Second is the practice of attributing bad motives to one’s opponents. Those employing this approach assume that people who take a contrary position know in their hearts that they are wrong and make the arguments they do for some inappropriate reason, such as racism or self-interest, that makes it easy to ignore what they have to say.
  • Third, a relatively new entrant, is what might be called the identity excuse: “We don’t need to listen to them because they are [blank].” Then fill in the blank with whatever identity you think warrants dismissal: a white male, a Black Lives Matter supporter, a Trump voter, a Democrat, the oil industry, a union, someone who received money for their work, and so on….(More)”.

Nudge, Boost or Design? Limitations of behavioral policy under social interaction.

Curated on Posted on by Stefaan Verhulst

Paper by Samuli Reijula, Jaakko Kuorikoski et al: “Nudge and boost are two competing approaches to applying the psychology of reasoning and decision making to improve policy. Whereas nudges rely on manipulation of choice architecture to steer people towards better choices, the objective of boosts is to develop good decision-making competences. Proponents of both approaches claim capacity to enhance social welfare through better individual decisions.

We suggest that such efforts should involve a more careful analysis of how individual and social welfare are related in the policy context. First, individual rationality is not always sufficient or necessary for improving collective outcomes. Second, collective outcomes of complex social interactions among individuals are largely ignored by the focus of both nudge and boost on individual decisions. We suggest that the design of mechanisms and social norms can sometimes lead to better collective outcomes than nudge and boost, and present conditions under which the three approaches (nudge, boost, and design) can be expected to enhance social welfare….(More)”.

New mathematical model can help save endangered species

Curated on Posted on by Stefaan Verhulst

Blogpost by Majken Brahe and Ellegaard Christensen: “What does the blue whale have in common with the Bengal tiger and the green turtle? They share the risk of extinction and are classified as endangered species. There are multiple reasons for species to die out, and climate changes is among the main reasons.

The risk of extinction varies from species to species depending on how individuals in its populations reproduce and how long each animal survives. Understanding the dynamics of survival and reproduction can support management actions to improve a specie’s chances of surviving.

Mathematical and statistical models have become powerful tools to help explain these dynamics. However, the quality of the information we use to construct such models is crucial to improve our chances of accurately predicting the fate of populations in nature.

Colchero’s research focuses on mathematically recreating the population dynamics by better understanding the species’s demography. He works on constructing and exploring stochastic population models that predict how a certain population (for example an endangered species) will change over time.

These models include mathematical factors to describe how the species’ environment, survival rates and reproduction determine to the population’s size and growth. For practical reasons some assumptions are necessary.

Two commonly accepted assumptions are that survival and reproduction are constant with age, and that high survival in the species goes hand in hand with reproduction across all age groups within a species. Colchero challenged these assumptions by accounting for age-specific survival and reproduction, and for trade-offs between survival and reproduction. This is, that sometimes conditions that favor survival will be unfavorable for reproduction, and vice versa.

For his work Colchero used statistics, mathematical derivations, and computer simulations with data from wild populations of 24 species of vertebrates. The outcome was a significantly improved model that had more accurate predictions for a species’ population growth.

Despite the technical nature of Fernando’s work, this type of model can have very practical implications as they provide qualified explanations for the underlying reasons for the extinction. This can be used to take management actions and may help prevent extinction of endangered species….(More)”

Participation 2.0? Crowdsourcing Participatory Development @ DFID

Curated on Posted on by Stefaan Verhulst

Paper by Anke Schwittay, Paul Braund: “Through an empirical analysis of Amplify, a crowdsourcing platform funded by the UK’s Department for International Development (DFID), we examine the potential of ICTs to afford more participatory development. Especially interactive Web2.0 technologies are often assumed to enable the participation of marginalized groups in their development, through allowing them to modify content and generate their own communication. 

We use the concepts of platform politics and voice to show that while Amplify managers and designers invested time and resources to include the voices of Amplify beneficiaries on the platform and elicit their feedback on projects supported via the platform, no meaningful participation took place. Our analysis of the gaps between participatory rhetoric, policy and practice concludes with suggestions for how ICTs could be harnessed to contribute to meaningful participatory development that matters materially and politically.,,,(More)”

Data Was Supposed to Fix the U.S. Education System. Here’s Why It Hasn’t.

Curated on Posted on by Stefaan Verhulst

Simon Rodberg at Harvard Business School: “For too long, the American education system failed too many kids, including far too many poor kids and kids of color, without enough public notice or accountability. To combat this, leaders of all political persuasions championed the use of testing to measure progress and drive better results. Measurement has become so common that in school districts from coast to coast you can now find calendars marked “Data Days,” when teachers are expected to spend time not on teaching, but on analyzing data like end-of-year and mid-year exams, interim assessments, science and social studies and teacher-created and computer-adaptive tests, surveys, attendance and behavior notes. It’s been this way for more than 30 years, and it’s time to try a different approach.

The big numbers are necessary, but the more they proliferate, the less value they add. Data-based answers lead to further data-based questions, testing, and analysis; and the psychology of leaders and policymakers means that the hunt for data gets in the way of actual learning. The drive for data responded to a real problem in education, but bad thinking about testing and data use has made the data cure worse than the disease….

The leadership decision at stake is how much data to collect. I’ve heard variations on “In God we trust; all others bring data” at any number of conferences and beginning-of-school-year speeches. But the mantra “we believe in data” is actually only shorthand for “we believe our actions should be informed by the best available data.” In education, that mostly means testing. In other fields, the kind of process is different, but the issue is the same. The key question is not, “will the data be useful?” (of course it can be) or, “will the data be interesting?” (Yes, again.) The proper question for leaders to ask is: will the data help us make better-enough decisions to be worth the cost of getting and using it? So far, the answer is “no.”

Nationwide data suggests that the growth of data-driven schooling hasn’t worked even by its own lights. Harvard professor Daniel Koretz says “The best estimate is that test-based accountability may have produced modest gains in elementary-school mathematics but no appreciable gains in either reading or high-school mathematics — even though reading and mathematics have been its primary focus.”

We wanted data to help us get past the problem of too many students learning too little, but it turns out that data is an insufficient, even misleading answer. It’s possible that all we’ve learned from our hyper-focus on data is that better instruction won’t come from more detailed information, but from changing what people do. That’s what data-driven reform is meant for, of course: convincing teachers of the need to change and focusing where they need to change….(More)”.

All of Us Research Program Expands Data Collection Efforts with Fitbit

Curated on Posted on by Stefaan Verhulst

NIH Press Release: “The All of Us Research Program has launched the Fitbit Bring-Your-Own-Device (BYOD) project. Now, in addition to providing health information through surveys, electronic health records, and biosamples, participants can choose to share data from their Fitbit accounts to help researchers make discoveries. The project is a key step for the program in integrating digital health technologies for data collection.

Digital health technologies, like mobile apps and wearable devices, can gather data outside of a hospital or clinic. This data includes information about physical activity, sleep, weight, heart rate, nutrition, and water intake, which can give researchers a more complete picture of participants’ health. The All of Us Research Program is now gathering this data in addition to surveys, electronic health record information, physical measurements, and blood and urine samples, working to make the All of Us resource one of the largest and most diverse data sets of its kind for health research.

“Collecting real-world, real-time data through digital technologies will become a fundamental part of the program,” said Eric Dishman, director of the All of Us Research Program. “This information, in combination with many other data types, will give us an unprecedented ability to better understand the impact of lifestyle and environment on health outcomes and, ultimately, develop better strategies for keeping people healthy in a very precise, individualized way.”…

All of Us is developing additional plans to incorporate digital health technologies. A second project with Fitbit is expected to launch later in the year. It will include providing devices to a limited number of All of Us participants who will be randomly invited to take part, to enable them to share wearable data with the program. And All of Us will add connections to other devices and apps in the future to further expand data collection efforts and engage participants in new ways….(More)”.

The Future of Civic Engagement

Curated on Posted on by Stefaan Verhulst

Report by Hollie Russon Gilman: “The 2018 mid-term voter turnout was the highest in 50 years. While vital, voting can’t sustain civic engagement in the long term. So, how do we channel near-term activism into long-term civic engagement?  In her essay, Gilman paints a picture of how new institutional structures, enabled by new technologies, could lead to a new “civic layer” in society that results in “a more responsive, participatory, collaborative, and adaptive future for civic engagement in governance decision making.”

Creating a New “Civic Layer.” The longer-term future presents an opportunity to set up institutionalized structures for engagement across local, state, and federal levels of government—creating a “civic layer.” Its precise form will evolve, but the basic concept is to establish a centralized interface within a com- munity to engage residents in governance decision making that interweaves digital and in-person engagement. People will earn “civic points” for engagement across a variety of activities—including every time they sign a petition, report a pot hole, or volunteer in their local community.

While creating a civic layer will require new institutional approaches, emerging technologies such as the Internet of Things (IoT), artificial intelligence (AI), and distributed ledger (e.g., blockchain) will also play a critical enabling role. These technologies will allow new institutional models to expand the concept of citizen coproduction of services in building a more responsive, connected, and engaged citizenry.

The following examples show different collaborative governance and technology components that will comprise the civic layer.  Each could be expanded and become interwoven into the fabric of civic life.

Use Collaborative Policymaking Models to Build a Civic Layer.  While we currently think of elections as a primary mode of citizen engagement with government, in the medium- to long-range future we could see collaborative policy models that become the de facto way people engage to supplement elections. Several of these engagement models are on the local level. However, with the formation of a civic layer these forms of engagement could become integrated into a federated structure enabling more scale, scope, and impact. Following are two promising models.

  • Participatory Budgeting can be broadly defined as the participation of citizens in the decision-making process of how to allocate their community’s budget among different priorities and in the monitoring of public spending. The process first came to the United States in 2009 through the work of the nonprofit Participatory Budgeting Project. Unlike traditional budget consultations held by some governments—which often amount to “selective listening” exercises—with participatory budgeting, citizens have an actual say in how a portion of a government’s investment budget is spent, with more money often allocated to poorer communities. Experts estimate that up to 2,500 local governments around the world have implemented participatory budgeting,
  • Citizens’Jury is another promising collaborative policymaking engagement model, pioneered in the 1980s and currently advocated by the nonprofit Jefferson Center in Minnesota. Three counties in rural Minnesota use this method as a foundation for Rural Climate Dialogues—regular gatherings where local residents hear from rural experts, work directly with their neighbors to design actionable community and policy recommendations, and share their feedback with public officials at a statewide meeting of rural Minnesota citizens, state agency representatives, and nonprofit organizations….(More)”.

The Internet of Bodies: A Convenient—and, Yes, Creepy—New Platform for Data Discovery

Curated on Posted on by Stefaan Verhulst

David Horrigan at ALM: “In the Era of the Internet of Things, we’ve become (at least somewhat) comfortable with our refrigerators knowing more about us than we know about ourselves and our Apple watches transmitting our every movement. The Internet of Things has even made it into the courtroom in cases such as the hot tub saga of Amazon Echo’s Alexa in State v. Bates and an unfortunate wife’s Fitbit in State v. Dabate.

But the Internet of Bodies?…

The Internet of Bodies refers to the legal and policy implications of using the human body as a technology platform,” said Northeastern University law professor Andrea Matwyshyn, who works also as co-director of Northeastern’s Center for Law, Innovation, and Creativity (CLIC).

“In brief, the Internet of Things (IoT) is moving onto and inside the human body, becoming the Internet of Bodies (IoB),” Matwyshyn added….


The Internet of Bodies is not merely a theoretical discussion of what might happen in the future. It’s happening already.

Former U.S. Vice President Dick Cheney revealed in 2013 that his physicians ordered the wireless capabilities of his heart implant disabled out of concern for potential assassin hackers, and in 2017, the U.S. Food and Drug Administration recalled almost half a million pacemakers over security issues requiring a firmware update.

It’s not just former vice presidents and heart patients becoming part of the Internet of Bodies. Northeastern’s Matwyshyn notes that so-called “smart pills” with sensors can report back health data from your stomach to smartphones, and a self-tuning brain implant is being tested to treat Alzheimer’s and Parkinson’s.

So, what’s not to like?

Better with Bacon?

“We are attaching everything to the Internet whether we need to or not,” Matwyshyn said, calling it the “Better with Bacon” problem, noting that—as bacon has become a popular condiment in restaurants—chefs are putting it on everything from drinks to cupcakes.

“It’s great if you love bacon, but not if you’re a vegetarian or if you just don’t like bacon. It’s not a bonus,” Matwyshyn added.

Matwyshyn’s bacon analogy raises interesting questions: Do we really need to connect everything to the Internet? Do the data privacy and data protection risks outweigh the benefits?

The Northeastern Law professor divides these IoB devices into three generations: 1) “body external” devices, such as Fitbits and Apple watches, 2) “body internal” devices, including Internet-connected pacemakers, cochlear implants, and digital pills, and 3) “body embedded” devices, hardwired technology where the human brain and external devices meld, where a human body has a real time connection to a remote machine with live updates.

Chip Party for Chipped Employees

A Wisconsin company, Three Square Market, made headlines in 2017—including an appearance on The Today Show—when the company microchipped its employees, not unlike what veterinarians do with the family pet. Not surprisingly, the company touted the benefits of implanting microchips under the skin of employees, including being able to wave one’s hand at a door instead of having to carry a badge or use a password….(More)”.

Whatever happened to evidence-based policy making?

Curated on Posted on by Stefaan Verhulst

Speech by Professor Gary Banks: “One of the challenges in talking about EBPM (evidence-based policy making), which I had not fully appreciated last time, was that it means different things to different people, especially academics. As a result, disagreements, misunderstandings and controversies (or faux controversies) have abounded. And these may have contributed to the demise of the expression, if not the concept.

For example, some have interpreted the term EBPM so literally as to insist that the word “based” be replaced by “influenced”, arguing that policy decisions are rarely based on evidence alone. That of course is true, but few using the term (myself included) would have thought otherwise. And I am sure no-one in an audience such as this, especially in our nation’s capital, believes policy decisions could derive solely from evidence — or even rational analysis!

If you’ll pardon a quotation from my earlier address: “Values, interests, personalities, timing, circumstance and happenstance – in short, democracy – determine what actually happens” (EBPM: What is it? How do we get it?). Indeed it is precisely because of such multiple influences, that “evidence” has a potentially significant role to play.

So, adopting the position from Alice in Wonderland, I am inclined to stick with the term EBPM, which I choose to mean an approach to policy-making that makes systematic provision for evidence and analysis. Far from the deterministic straw man depicted in certain academic articles, it is an approach that seeks to achieve policy decisions that are better informed in a substantive sense, accepting that they will nevertheless ultimately be – and in a democracy need to be — political in nature.

A second and more significant area of debate concerns the meaning and value of “evidence” itself. There are a number of strands involved.

Evidentiary elitism?

One relates to methodology, and can be likened to the differences between the thresholds for a finding of guilt under civil and criminal law (“balance of probabilities” versus “beyond reasonable doubt”).

Some analysts have argued that, to be useful for policy, evidence must involve rigorous unbiased research techniques, the “gold standard” for which are “randomized control trials”. The “randomistas”, to use the term which headlines Andrew Leigh’s new book (Leigh, 2018), claim that only such a methodology is able to truly tell us “what works”

However adopting this exacting standard from the medical research world would leave policy makers with an excellent tool of limited application. Its forte is testing a specific policy or program relative to business as usual, akin to drug tests involving a placebo for a control group. And there are some inspiring examples of insights gained. But for many areas of public policy the technique is not practicable. Even where it is, it requires that a case has to some extent already been made. And while it can identify the extent to which a particular program “works”, it is less useful for understanding why, or whether something else might work even better.

That is not to say that any evidence will do. Setting the quality bar too low is the bigger problem in practice and the notion of a hierarchy of methodologies is helpful. However, no such analytical tools are self-sufficient for policy-making purposes and in my view are best thought of as components of a “cost benefit framework” – one that enables comparisons of different options, employing those estimation techniques that are most fit for purpose. Though challenging to populate fully with monetized data, CBA provides a coherent conceptual basis for assessing the net social impacts of different policy choices – which is what EBPM must aspire to as its contribution to (political) policy decisions….(More)”.