Article by Sarah O’Connor: “In the early 20th century, a gifted engineer called Frederick Winslow Taylor embarked on an ambitious task: to extract knowledge from inside the heads of workers on America’s factory floors.
In the eyes of Taylor, who would go on to become one of the world’s first management consultants, factory workers possessed a “mass of rule-of-thumb or traditional knowledge” which had been “handed down from man to man by word of mouth” or “almost unconsciously learnt through personal observation”. Taylor thought it was about time this knowledge was “codified or systematically analysed or described”. To that end, he sent managers with stopwatches and notebooks on to shop floors to observe, time and record every stage of every job.
More than a century later, employers of white-collar professionals are beginning to confront a similar challenge. It is becoming increasingly clear that the knowledge required to make AI models genuinely powerful in a swath of workplaces is currently locked inside employees’ heads.
This isn’t true in every workplace. AI models have transformed the software profession, for example, because the task of writing code is testable and rules-based and there were vast reams of training data publicly available, thanks to online forums like Stack Overflow.
But for many other jobs, that sort of data just does not exist on the web. Indeed, some subtle but important skills are very hard to codify at all, which is why they are often learnt through experience and osmosis. This sort of tacit knowledge was famously summed up by the scientist and philosopher Michael Polanyi with the phrase: “we can know more than we can tell”.
As a result, general-purpose AI models are simply not very good at many specific tasks which require both domain and institutional knowledge. Investment firm Bridgewater Associates recently experimented, for example, with using LLMs to do something their human professionals do all the time: parsing reams of news stories and financial documents for information that might be relevant to their investment decisions.
While this could be a useful timesaver, Bridgewater found that variants of Gemini, Claude and GPT only tended to get it right about 50 per cent of the time..(More)”.
