Jesper Christiansen et al at Nesta: “…we share some initial reflections from this work with the hope of prompting a useful discussion about how to articulate the value of experimentation as well as what to consider when strategically planning and doing experiments in government contexts.
Reflection 1: Experimentation as a way of accelerating learning and exploring “the room of the non-obvious”
Governments need to increase their pace and agility in learning about which ideas have the highest potential value-creation and make people’s lives the rationale of governing.
Experimental approaches accelerate learning by systematically testing assumptions and identifying knowledge gaps. What is there to be known about the problem and the function, fit and probability of a suggested solution? Experimentation helps fill these gaps without allocating too much time or resource, and helps governments accelerate the exploration of new potential solution spaces.
This approach is often a key contribution of government policy labs and public sector innovation teams. Units like Lab para la Ciudad in Mexico City, Alberta Co-Lab in Canada, Behavioural insights and Design Unit in Singapore, MindLab in Denmark and Policy Lab in the UK are specifically set up to promote, develop and/or embed experimental approaches and accelerate user-centred learning in different levels of government.
In addition, creating a culture of experimentation extends the policy options available by creating a political environment to test non-linear approaches to wicked problems. In our training, we often distinguish between “the room of the obvious” and the “room of the non-obvious”. By designing portfolios of experiments that include – by deliberate design – the testing of at least some non-linear, non-obvious solutions, government officials can move beyond the automatic mode of many policy interventions and explore the “room of the non-obvious” in a safe-to-fail context (think barbers to prevent suicides or dental insurance to prevent deforestation).
Reflection 2: Experimentation as a way of turning uncertainty into risk
In everyday language, uncertainty and risk are two notions that are often used interchangeably; yet they are very different concepts. Take, for example, the implementation of a solution. Risk is articulated in terms of the probability that the solution will generate a certain outcome. It is measurable (e.g. based on existing data there is X per cent chance of success, or X per cent chance of failure) and qualitative risk factors can be developed and described.
Uncertainty, on the other hand, is a situation where there is a lack of probabilities. There is no prior data on how the solution might perform; future outcomes are not known, and can therefore not be measured. The chance of success can be 0 per cent, 100 per cent, or anything in between (see table below).
There is often talk of the need for government to become more of a ‘risk taker’, or to become better at ‘managing risk’. But as Marco Steinberg, founder of strategic design practice Snowcone & Haystack, recently reminded us, risk-management – where probabilities are known – is actually something that governments do quite well. Issues arise when governments’ legacies can’t shape current solutions: when governments have to deal with the uncertainty of complex challenges by adapting or creating entirely new service systems to fit the needs of our time.
For example, when transforming a health system to fit the needs of our time, little can be known about the probabilities in terms of what might work when establishing a new practice. Or when transforming a social care system to accommodate the lives of vulnerable families, entirely new concepts for solutions need to be explored. “If you don’t have a map showing the way, you have to write one yourself,” as Sam Rye puts it in his inspirational example on the use of experimental cards at The Labs Wananga….
Reflection 3: Experimentation as a way to reframe failure and KPIs
Reflection 4: Experimentation on a continuum between exploration and validation
Reflection 5: Experimentation as cultural change…(More)”.