Blog and book by Erica Thompson: “Mathematical models are here to stay. Whether they are determining supply chain vulnerabilities, demonstrating regulatory compliance, or informing policies for a zero-carbon future, quantitative models are at the heart of modern societies. And as computers become more powerful and more readily accessible, artificial intelligence and machine learning models are also being applied in many new areas.
Given that, we urgently need to understand how best to use and work with models to make good and responsible decisions. Statistician George Box was quite right to point out that “all models are wrong”. They are necessarily simplifications of the messy reality we want to get to grips with. But many quantitative methods for working with models basically assume that the model is right, or at least that it can accurately estimate the range of plausible outcomes.
If the model is not quite perfect, we can expect some of its outputs to be wrong (not just inaccurate). In that case, the information that is offered as decision support could be misleading. We have two options here. We could remain in what I call model land and just expect to have to say “what a shame, we made the wrong decision” occasionally. In some circumstances that might be a reasonable answer, but if we are making decisions about critical infrastructure or selling a product that might be unsafe to millions of people, then we have both a legal and ethical responsibility to do better, to get out of model land and understand how relevant our model results are for the real world.
So what’s the second option? You won’t be surprised to know that it isn’t easy. In my new book, I consider some of the implications of working with imperfect models and the kinds of strategies that we need to adopt to make best use of the information they contain. One theme that I explore is the need to understand the role of expert judgement in constructing, calibrating, evaluating, and using models, and the way that that expert judgement might be shaped by our social context.
Experts make models – and that’s a very good thing, because who would want to rely on a model created by a non-expert? But their expertise is often limited, and it comes from a particular background and set of experiences. Indeed, you can often find equally qualified experts who will disagree about the right assumptions to make when constructing a model and who give different advice about how to achieve the stated aims. Then the decision-maker – probably a non-expert – will be in the difficult position of trying to adjudicate between different models from different experts, weighing up their relative credibility…(More)”.