Using rankings to incentivise performance improvements
By Fenton Whelan & Will Anderson
Rankings can drive big improvements in performance. They can also generate disagreements and gaming. Understanding the nuances of ranking systems is key to using them effectively.

Used effectively, rankings harness competitive forces to incentivise higher performance

The World Bank’s ‘Doing Business’ ranks nearly all countries annually on ten ease of doing business topics. In part, the ranking seeks to harness competitive forces by “encouraging economies to compete towards efficient regulation”. For example China’s recent 32 place increase on the ranking was supposedly heavily influenced by India’s 30 place increase the year prior. Both nations achieved these increases by reforming parts of their business environment.

The 10 Ease of Doing Business topics used in 2019

1. Starting a business
2. Dealing with construction permits
3. Getting electricity
4. Registering property
5. Getting credit

6. Protecting minority investors
7. Paying taxes
8. Trading across borders
9. Enforcing contracts
10. Resolving insolvency


However rankings can also be prone to unintended consequences

The Doing Business ranking has some limitations. For example, it only uses data collected from the country's largest city (sub-national indicators are beginning to be incorporated for some countries). Additionally the ranking captures data mostly related to a specific size of limited liability company, excluding all other types of common business structures. These types of methodological limitations can lead to disagreements about whether the ranking accurately depicts reality. For example disagreements over the ranking’s methodological robustness arose as a result of Chile’s ranking wildly fluctuating scores between 2006-2017.


Limitations in ranking design can also lead to gaming. For example, the India edition of the Huffington Post recently suggested that India’s rapid increase on the Doing Business ranking was the result of “cracking the code” rather than substantive economic reforms.

Careful design may reduce disagreements and gaming

There are many ways to develop rankings, each with their own nuanced advantages and limitations (outlined below). Understanding these nuances can help ensure that rankings are used effectively.

Ranking system



First to last

Participants are ranked first to last (e.g. 40 districts listed 1 to 40)

  • Simple to understand

  • Intuitive to most people

  • Engaging and motivating

  • Small improvements still show up in ranking

  • Someone is always last, regardless of overall performance

  • Hard to correct for starting points

  • Misleading when the range is narrow

Quartiles or divisions

Participants are divided into a number of groups based on performance (e.g. top third, middle third, bottom third)

  • Simple to understand

  • Avoids ‘margin of error’ differences in data affecting ranking

  • Avoids signalling out individuals

  • There is always a bottom group

  • Performance within groups can be wide

  • Top quartile may still not be good

  • Improvements may not register (e.g. within top quartile)

Performance thresholds

Participants are scored based on whether they meet certain thresholds (e.g. above 90%)

  • Reasonably simple to use

  • Allows everyone to succeed

  • Difficult to use in systems where there is a wide variation in starting points

  • May not reward improvement for individuals who’s starting point is low

Based on directions

Participants are scored based on whether they are improving (e.g. improved by 10% this month)

  • Reasonably simple to use

  • Allows everyone to succeed

  • Rewards most recent improvements

  • May be confusing and misleading (very bad areas score highly if they are improving)

  • Becomes difficult to use once performance approaches its maximum

Based on targets

Participants are scored based on whether they meet targets, which may be different for each participant

  • Complicated and difficult to explain

  • Target calculation may be controversial

  • May entrench existing patterns of behaviour

  • Fairer and more realistic than performance thresholds where staring performance varies widely

Based on trajectories

Participants are scored based on whether they meet a trajectory, which may be different for each participant

  • Complicated and difficult to explain

  • Calculation may be controversial

  • May entrench existing patterns of behaviour

  • Allows high level of fine tuning to individual circumstances and variables

About the authors

Fenton Whelan founded Acasus. He has fifteen years of experience in public health and education development. Will Anderson supports business development and research at Acasus.

Lessons from two decades of school reform in New York City
By Fenton Whelan
06 June 2022
New York City’s schools have improved significantly and can teach us a lot about reform.
Learn more
Rapidly mapping health facilities in fragile and conflict-affected contexts: lessons from Somalia
By Jonny Barty & Abdiwahab Abdullahi Elmi
26 May 2022
Somalia is one of the most difficult places to provide health services, following decades of conflict and political instability, and..
Learn more
Lessons learnt: Rebuilding health service delivery after conflict
By Iman Abdulkarim Mubarek & Miki Wondimeneh
05 July 2022
The crisis in Ethiopia's northern region has caused significant disruption across the health service. Regions such as Afar, Amhara and..
Learn more