The problem with the data revolution in four Venn diagrams


Morten Jerven in The Guardian: “In August, UN secretary-general Ban Ki Moon named his independent expert advisory group, 24 experts tasked with providing recommendations on how best to use data to deliver the sustainable development goals….On 6 November, those recommendations were published in a report entitled A world that counts, a cleverly crafted motivational manifesto, but by no means a practical roadmap on how to apply a “data revolution” to the future development agenda.
I have previously written about this in more detail, but essentially, the report’s key weakness is that it conflates several terms, and assumes automatic relationships between things such as “counting” and “knowing”.
Using four Venn diagrams, I’ve tried to illustrate some of the main misconceptions.

Not everything that counts can be counted

venn 1
Image 1

The report strongly suggests that everything that matters can be counted. We know that this is not true. If the guiding principle for the sustainable development goals is to make decisions as if everything can be counted, the end result will be very misleading.

Data is not the same as statistics

venn 2
Image 2

The “data revolution” hype is just one of many places where the difference between statistics and data is misunderstood. Data is not the same as numbers. Data literally mean ‘what is given’, so when we speak of data we are talking about observations – quantitative or qualitative, or even figurative – that can be used to get information.
To keep talking about data when we mean statistics may sound better, but it only leads to confusion. The report (pdf) calls on the UN to establish “a process whereby key stakeholders create a Global Consensus on Data”. What is that supposed to mean? That statement is meaningless if you exchange the word “data” with “observations”, “knowledge” or ‘evidence’. It can, however, make sense if you talk about “statistics”.
International organisations do have a natural role when it comes to developing global standards for official statistics. Reaching a global consensus on how observations and evidence constitute knowledge is futile.

More data does not mean better decisions…

There are other methods to knowing than through counting…(More)”