Winny de Jong: “At the intersection of data and journalism, lots can go wrong. Merely taking precautions might not be enough….
Half True Is False
But this approach is not totally foolproof.
“In data journalism, we cannot settle for ‘half-true.’ Anything short of true is wrong – and we cannot afford to be wrong.” Unlike fact-checking websites such as Politifact, which invented ‘scales’ for truthfulness, from false to true and everything in between, data journalism should always be true.
No Pants on Fire: Politifact’s Truth-O-Meter.
True but Wrong
But even when your story is true, Gebeloff said you still could still be wrong. “You can do the math correctly, but get the context wrong, fail to acknowledge uncertainties or not describe your findings correctly.”
Fancy Math
When working on a story, journalists should consider whether they use “fancy math” – think statistics – or “standard math.” “Using fancy math you can explore complex relationships, but at the same time your story will be harder to explain.”…
Targets as a Source
…To make sure you’re not going to be wrong, you should share your findings. “Don’t just share findings with experts, share them with hostile experts too,” Gebeloff advises. “Use your targets as a source. If there’s a blowback, you want to know before publication – and include the blowback in the publication.”
How Not To Be Wrong Checklist
Here’s why you want to use this checklist, which is based on Gebeloff’s presentation: a half truth is false, and data journalism should always be true. But just being true is not enough. Your story can be mathematically true but wrong in context or explanation. You should want your stories to be true and not wrong.
- Check your data carefully:
- Pay attention to dates.
- Check for spelling and duplicates.
- Identify outliers.
- Statistical significance alone is not news.
- Prevent base year abuse: if something is a trend, it should be true in general not just if you cherrypick a base year.
- Make sure your data represents reality.
- As you work, keep a data diary that records what you’ve done and how you’ve done it. You should be able to reproduce your calculations.
- Make sure you explain the methods you used – your audience should be able to understand how you find a story.
- Play offense and defense simultaneously. Go for the maximum possible story, but at all times think of why you might be wrong, or what your target would say in response.
- Use your targets as a source to find blowbacks before publication.
- As part of the proofing process, create a footnotes file. Identify each fact and give it a number. Then, for each fact, list which document it came from, how you know it and the proof. Fix what needs to be fixed.
Additional links of interest: the slides of Robert Gebeloff’s how not to be wrong presentation, and the methodology notes and data from the “Race Behind Bars” series….(More)”