MetroLab Network: “Predictive analytical tools are already being put to work within human service agencies to help make vital decisions about when and how to intervene in the lives of families and communities. The sector may not be entirely comfortable with this trend, but it should not be surprised. Predictive models are in wide use within the justice and education sectors and, more to the point, they work: risk assessment is fundamental to what social services do, and these tools can help agencies respond more quickly to prevent harm, to create more personalized interventions, and allocate scarce public resources to where they can do the most good.
There is also a strong case that predictive risk models (PRM) can reduce bias in decision-making. Designing a predictive model forces more explicit conversations about how agencies think about different risk factors and how they propose to guard against disadvantaging certain demographic or socioeconomic groups. And the standard that agencies are trying to improve upon is not perfect equity—it is the status quo, which is neither transparent nor uniformly fair. Risk scores do not eliminate the possibility of personal or institutional prejudice but they can make it more apparent by providing a common reference point.
That the use of predictive analytics in social services can reduce bias is not to say that it will. Careless or unskilled development of these predictive tools could worsen disparities among clients receiving social services. Child and civil rights advocates rightly worry about the potential for “net widening”—drawing more people in for unnecessary scrutiny by the government. They worry that rather than improving services for vulnerable clients, these models will replicate the biases in existing public data sources and expose them to greater trauma. Bad models scale just as quickly as good ones, and even the best of them can be misused.
The stakes here are real: for children and families that interact with these social systems and for the reputation of the agencies that turn to these tools. What, then, should a public leader know about risk modeling, and what lessons does it offer about how to think about data science, data stewardship, and the public interest?…(More)”.