Blog by Ed Humpherson at Data & Policy: “At the Office for Statistics Regulation, thinking about these questions is our day job. We set the standards for Government statistics and data through our Code of Practice for Statistics. And we review how Government departments are living up to these standards when they publish data and statistics. We routinely look at Government statistics are used in public debate.
Based on this, I would propose four factors that ensure that new data sources and tools serve the public good. They do so when:
- When data quality is properly tested and understood:
As my colleague Penny Babb wrote recently in a blog: “‘Don’t trust the data. If you’ve found something interesting, something has probably gone wrong!”. People who work routinely with data develop a sort of innate scepticism, which Penny’s blog captures neatly. Understanding the limitations of both the data, and the inferences you make about the data, are the starting point for any appropriate role for data and policy. Accepting results and insights from new data at face value is a mistake. Much better to test the quality, explore the risks of mistakes, and only then to share findings and conclusions.
2. When the risks of misleadingness are considered:
At OSR, we have an approach to misleadingness that focuses on whether a misuse of data might lead a listener to a wrong conclusion. In fact, by “wrong” we don’t mean in some absolute sense of objective truth; more that if they received the data presented in a different and more faithful way, they would change their mind. Here’s a really simple example: someone might hear that, of two neighbouring countries, one has a much lower fatality rate, when comparing deaths to positive tests for Covid-19. …
3. When the data fill gaps
Data gaps come in several forms. One gap, highlighted by the interest in real-time economic indicators, is timing. Economic statistics don’t really tell us what’s going on right now. Figures like GDP, trade and inflation tells us about some point in the (admittedly quite) recent past. This is the attraction of the real-time economic indicators, which the Bank of England have drawn on in their decisions during the pandemic. They give policymakers a much more real-time feel by filling in this timing gap.
Other gaps are not about time but about coverage….
4. When the data are available
Perhaps the most important thing for data and policy is to democratise the notion of who the data are for. Data (and policy itself) are not just for decision-making elites. They are a tool to help people make sense of their world, what is going on in their community, helping frame and guide the choices they make.
For this reason, I often instinctively recoil at narratives of data that focus on the usefulness of data to decision-makers. Of course, we are all decision-makers of one kind or another, and data can help us all. But I always suspect that the “data for decision-makers” narrative harbours an assumption that decisions are made by senior, central, expert people, who make decisions on behalf of society; people who are, in the words of the musical Hamilton, in the room where it happens. It’s this implication that I find uncomfortable.
That’s why, during the pandemic, our work at the Office for Statistics Regulation has repeatedly argued that data should be made available. We have published a statement that any management information referred to by a decision maker should be published clearly and openly. We call this equality of access.
We fight for equality of access. We have secured the publication of lots of data — on positive Covid-19 cases in England’s Local Authorities, on Covid-19 in prisons, on antibody testing in Scotland…. and several others.
Data and policy are a powerful mix. They offer huge benefits to society in terms of defining, understanding and solving problems, and thereby in improving lives. We should be pleased that the coming together of data and policy is being sped-up by the pandemic.
But to secure these benefits, we need to focus on four things: quality, misleadingness, gaps, and public availability….(More)”