Rational sycophants and catastrophic risks

Paper by Robert Axelrod  and Scott E. Page: “Sycophants praise and support leaders’ proposals to gain personal and professional advantage. A rational sycophant is an advisor who supports actions they expect to be harmful even when rewards and punishments for good and bad advice are equal in magnitude. Rational sycophancy arises when the outcome distribution of a proposed action has a negative expected value but a positive median (outcome asymmetry). The risk is greatest when a small yet meaningful fraction of outcomes are catastrophic, which occurs in long-tailed distributions. Given that single realizations from long-tailed distributions reveal little about the underlying distribution, even after outcomes are observed, a leader may be unable to distinguish rational sycophancy from wise counsel. As a result, rational sycophants may gain influence and increase the likelihood of catastrophic policy outcomes…(More)”