Nudging Science Towards Fairer Evaluations: Evidence From Peer Review

Paper by Inna Smirnova, Daniel M. Romero, and Misha Teplitskiy: “Peer review is widely used to select scientific projects for funding and publication, but there is growing evidence that it is biased towards prestigious individuals and institutions. Although anonymizing submissions can reduce prestige bias, many organizations do not implement anonymization, in part because enforcing it can be prohibitively costly. Here, we examine whether nudging but not forcing authors to anonymize their submissions reduces prestige bias. We partnered with IOP Publishing, one of the largest academic publishers, which adopted a policy strongly encouraging authors to anonymize their submissions and staggered the policy rollout across its physics journal portfolio. We examine 156,015 submissions to 57 peer-reviewed journals received between January 2018 and February 2022 and measure author prestige with citations accrued at submission time. Higher prestige first authors were less likely to anonymize. Nevertheless, for low-prestige authors, the policy increased positive peer reviews by 2.4% and acceptance by 5.6%. For middle- and high-prestige authors, the policy decreased positive reviews (1.8% and 1%) and final acceptance (4.6% and 2.2%). The policy did not have unintended consequences on reviewer recruitment or the characteristics of submitting authors. Overall, nudges are a simple, low-cost, and effective method to reduce prestige bias and should be considered by organizations for which enforced-anonymization is impractical…(More)”.