Geoff Mulgan: “How do you measure a programme of government reform? What counts as evidence that it’s working or not? I’ve been asked this question many times, so this very brief note suggests some simple answers – mainly prompted by seeing a few writings on this question which I thought confused some basic points.”
Any type of reform programme will combine elements at very different levels. These may include:
- A new device – for example, adjusting the wording in an official letter or a call centre script to see what impact this has on such things as tax compliance.
- A new kind of action – for example a new way of teaching maths in schools, treating patients with diabetes, handling prison leavers.
- A new kind of policy – for example opening up planning processes to more local input; making welfare payments more conditional.
- A new strategy – for example a scheme to cut carbon in cities, combining retrofitting of housing with promoting bicycle use; or a strategy for public health.
- A new approach to strategy – for example making more use of foresight, scenarios or big data.
- A new approach to governance – for example bringing hitherto excluded groups into political debate and decision-making.
This rough list hopefully shows just how different these levels are in their nature. Generally as we go down the list the following things rise:
- The number of variables and the complexity of the processes involved
- The timescales over which any judgements can be made
- The difficultness involved in making judgements about causation
- The importance of qualitative relative to quantitative assessment”