Article by James Watson: “You know your ranking model is influential when national governments change policies with the explicit goal of boosting their position on your index. That was the power of the Ease of Doing Business Index (also known as Doing Business) until 2021.
However, the index’s success became its downfall. Some governments set up dedicated teams with an explicit goal of improving the country’s performance on the index. If those teams’ activity was solely focussed on positive policy reform, that would be great; unfortunately, in at least some cases, they were simply trying to game the results.
World Bank’s Business Ready Index
Index ranking optimisation (aka gaming the results)
To give an example of how that could happen, we need to take a brief detour into the world of qualitative indicators. Bear with me. In many indexes grappling with complex topics, there is a perennial problem of data availability. Imagine you want to measure the number of days it takes to set up a new business (this was one of the indicators in Doing Business). You will find that most of the time the data either doesn’t exist or is rarely updated by governments. Instead, put very simplistically, you’d need to ask a few experts or businesses for their views, and use those to create a numerical score for your index.
This is a valid approach, and it’s used in a lot of studies. Take Transparency International’s long-running Corruption Perceptions Index (CPI). Transparency International goes to great lengths to use robust and comparable data across countries, but measuring actual corruption is not viable — for obvious reasons. So the CPI does something different, and the clue is in the name: it measures people’s perceptions of corruption. It asks local businesses and experts whether they think there’s much bribery, nepotism and other forms of corruption in their country. This foundational input is then bolstered with other data points. The data doesn’t aim to measure corruption; instead, it’s about assessing which countries are more, or less, corrupt.
Transparency International’s Corruption Perceptions Index (CPI)
This technique can work well, but it got a bit shaky as Doing Business’s fame grew. Some governments that were anxious to move up the rankings started urging the World Bank to tweak the methodology used to assess their ratings, or to use the views of specific experts. The analysts responsible for assessing a country’s scores and data points were put under significant pressure, often facing strong criticism from governments that didn’t agree with their assessments. In the end, an internal review showed that a number of countries’ scores had been improperly manipulated…The criticism must have stung, because the team behind the World Bank’s new Business Ready report has spent three years trying to address those issues. The new methodology handbook lands with a thump at 704 pages…(More)”.