Paper by Jane R. Bambauer and Tal Zarsky: “Most of the discourse on algorithmic decision-making, whether it comes in the form of praise or warning, assumes that algorithms apply to a static world. But automated decision-making is a dynamic process. Algorithms attempt to estimate some difficult-to-measure quality about a subject using proxies, and the subjects in turn change their behavior in order to game the system and get a better treatment for themselves (or, in some cases, to protest the system.) These behavioral changes can then prompt the algorithm to make corrections. The moves and counter-moves create a dance that has great import to the fairness and efficiency of a decision-making process. And this dance can be structured through law. Yet existing law lacks a clear policy vision or even a coherent language to foster productive debate.
This Article provides the foundation. We describe gaming and counter-gaming strategies using credit scoring, criminal investigation, and corporate reputation management as key examples. We then show how the law implicitly promotes or discourages these behaviors, with mixed effects on accuracy, distributional fairness, efficiency, and autonomy….(More)”.