Weapon of Math Destruction: “risk-based” sentencing models
There was a recent New York Times op-ed by Sonja Starr entitled Sentencing, by the Numbers (hat tip Jordan Ellenberg and Linda Brown) which described the widespread use – in 20 states so far and growing – of predictive models in sentencing.
The idea is to use a risk score to help inform sentencing of offenders. The risk is, I guess, supposed to tell us how likely the person is to commit another act in the future, although that’s not specified. From the article:
The basic problem is that the risk scores are not based on the defendant’s crime. They are primarily or wholly based on prior characteristics: criminal history (a legitimate criterion), but also factors unrelated to conduct. Specifics vary across states, but common factors include unemployment, marital status, age, education, finances, neighborhood, and family background, including family members’ criminal history.
I knew about the existence of such models, at least in the context of prisoners with mental disorders in England, but I didn’t know how widespread it had become here. This is a great example of a weapon of math destruction and I will be using this in my book.
A few comments:
- I’ll start with the good news. It is unconstitutional to use information such as family member’s criminal history against someone. Eric Holder is fighting against the use of such models.
- It is also presumably unconstitutional to jail someone longer for being poor, which is what this effectively does. The article has good examples of this.
- The modelers defend this crap as “scientific,” which is the worst abuse of science and mathematics imaginable.
- The people using this claim they only use it for as a way to mitigate sentencing, but letting a bunch of rich white people off easier because they are not considered “high risk” is tantamount to sentencing poor minorities more.
- It is a great example of confused causality. We could easily imagine a certain group that gets arrested more often for a given crime (poor black men, marijuana possession) just because the police have that practice for whatever reason (Stop & Frisk). Then model would then consider any such man at a higher risk of repeat offending, but that’s not because any particular person is actually more likely to do it, but because the police are more likely to arrest that person for it.
- It also creates a negative feedback loop on the most vulnerable population: the model will impose longer sentencing on the population it considers most risky, which will in turn make them even riskier in the future, if “length of time in prison previously” is used as an attribute in the model, which is surely is.
- Not to be cynical, but considering my post yesterday, I’m not sure how much momentum will be created to stop the use of such models, considering how discriminatory it is.
- Here’s an extreme example of preferential sentencing which already happens: rich dude Robert H Richards IV raped his 3-year-old daughter and didn’t go to jail because the judge ruled he “wouldn’t fare well in prison.”
- How great would it be if we used data and models to make sure rich people went to jail just as often and for just as long as poor people for the same crime, instead of the other way around?
mathbabe 
If we are to believe Beau Biden, son of Joe Biden, it had to do with the difficulty of winning the case.
http://www.delawareonline.com/story/news/crime/2014/04/03/beau-biden-defends-handling-du-pont-heir-sex-case/7255629/
http://www.killingmycareer.com/the-sociopathic-business-model/beau-biden-son-of-joe-supports-assault-on-children/
LikeLike
He “pleaded guilty to fourth-degree rape, a felony that can carry a 15-year prison sentence.”
http://www.businessinsider.com/du-pont-heir-robert-h-richards-iv-got-probation-2014-3
LikeLike
I am not saying the DuPont heir is innocent. If it was up to me I would let him hang by his private parts forever.. Just reported what Biden claimed. There are plenty of well known cases of injustice: OJ Simpson murdered his wife and got away. Poetic justice is that he’s in jail for another crime. Kennedy clan members get special treatment, starting with Ted “Chappaquiddick” Kennedy, where he either murdered Mary Jo Kopechne, or just left her there to die. But its not only the rich and famous who get special treatment. It’s enough that Al Sharpton gets involved in your case (Tawana Brawley). Our justice system is imperfect. Using data doesn’t make it any better. Are there better justice systems? I have not come across one.
LikeLike
I would argue, however, it’s fundamentally different when injustice is meted out by a mathematical equation. Especially in our society, where such “objective” outputs are considered unassailable…despite worldwide economic collapse as evidence to the contrary.
Without going into details, I build models for a company known for a very high level of technical competency (not in data analytics). I am shocked at the inability of people with advanced technical degrees to understand the most basic aspects of using and visualizing data, let alone the caveats and limitations of data-driven models. If the equations or the outputs look sophisticated enough, or if you tell them the fancy name of the technique you’ve applied in building the model, they trust it without questioning whether the approach was reasonable or the output sensible. I shudder to think what people with even less grounding in math would do with these kinds of outputs.
LikeLike
Is justice better when it’s random? Is it any worse if a bad mathematical model is picked? I don’t know. I hear your argument and I understand it, But justice is so approximate to begin with.
LikeLike
I don’t think injustice is more tolerable when it’s random. But I do think that when the justice system behaves randomly, it makes people more willing to challenge it. By glossing over that randomness (or codifying the inherent biases in how we currently make sentencing decisions), we risk making people complacent in the face of institutionalized injustice. That’s what makes this practice chilling, in my view.
LikeLike
but it’s not at all random, it systematically discriminates against the poor and against minorities.
On Tue, Aug 12, 2014 at 11:47 PM, mathbabe wrote:
>
LikeLike
I do not nor did I believe Beau Biden’s argument that it was too difficult of a case to win (as DuPont heir Richards admitted guilt); and, as Beau Biden now readies for a race for governor in 2016 (follow the money) I hope that people remember that he supported a judge who gave a child rapist no jail time. As for a predictable pattern once gender/race or religion of an individual are ignored by the courts then we look for inconsistent & contradictory language to actions as the trigger for The Sociopathic Business Model™. It’s about making better choices across the board. Thanks for linking me to the page. Really enjoyed your post Cathy.
http://www.killingmycareer.com/the-sociopathic-business-model/
LikeLike
I was peripherally involved in an analysis of the NYC arrest to arraignment process a decade or so ago, and at that time they used a relatively simple model developed by the NYCJA for ROR (and bail IIRC) recommendations. IIRC the model was deliberately simple and open (see the Pretrial FTA section of http://www.nycja.org/library.php). As they are somewhat in your backyard this may be of some interest.
LikeLike
Cathy I agree predictive analytics abuse is out there and I wrote about it in healthcare with doctors being knocked off the network with United Healthcare. They are the kings of analytics with health insurers and are a company with a past. I think they still hold the highest derivatives fine back in 2008 for backdating stock and then the AMA sued them and won a class action suit against them. The class action suit was for mathematically paying doctors and patients short on out of network claims for 15 years before auditors in Cuomo’s office caught it. Now the the #2 person at Medicare is the former CEO of this United subsidiary, Ingenix aka Optum now who developed the models to short pay MDs. This is only the start of more to come with this company.
You may have seen some of the news to where doctors, with no reason given, the company not willing to even discuss the doctors why they have been cut loose occurring in various states. The doctors are those who take care of Medicare Advantage seniors. So now you can old people to the scheme. Of course we all know that insurers all run mortality risk assessments on all of us as well as buying all our credit card data and some are buying Axciom data now too, so are they going to skip doctors in this process with risk assessments to save and make money, of course not.
Their math and execution of the models got so bad in Maryland that they bid and won a contract but the other side had already fired enough doctors to where they had none to see the patients in the contract they won. This was not discovered until all patient records had been transferred to United from the other insurer who was running the plan for the retirees and United had to bow out as patients could not find a doctor in network! In this area they had all been cut loose with a model. Again we can’t see it or exactly prove anything here, but when a company will not even talk to the doctors to tell them why they were eliminated, we have an issue of unfairness here.
One MD wrote a letter and said whatever it would take he would do it to reduce his payments if that were the issue in Tennessee and nothing. He was cut loose by United and he specialized in geriatrics and had over 100 patients that now, now when open enrollment occurs again, that would have to find a new doctor. United of course was taking action to pretty much “assign” these folks a new doctor. Well the math again was a mess and instead of being sent to a new primary MD, they were being assigned to OBGYNs and neurosurgeons as their new family practice doctor!
So again, what other course do we have but to think this is exactly what is happening there? They are doing risk assessments on what they think will be the quality of care plus the doctor’s mortality and who knows what else they are tossing in there. I hope this boils up there soon as well. United wants everyone to see a nurse practitioner eventually and only a doctor when you absolutely need one to keep cost down. United is the most hated health insurance company around right now and for good reason. So here’s one more area of abuse with predictive models and this is growing as well, scary.
http://ducknetweb.blogspot.com/2014/08/the-secret-scoring-of-americas.html
LikeLike
Please. Every one of the variables used in these assessment tools is already used haphazardly by judges and prosecutors in the sentencing process with virtually no oversight or recourse through later data analysis specifically compiled to demonstrate the biases you’re outlining. One reason the “reform” groups have pushed these models is because the research has shown that you could expect lower, not higher sentences, for most of those convicted. Pre-sentence investigations by probation officials prior to judicial decisions typically have more and if-fyer variables, plus subjective, open-ended info and will return to the standard in lieu of these more visible and refined tools. And even those were better than judges going with Stephen Colbert’s “gut,” which in their infinite wisdom produces just decisions almost all somehow divisible by 6.
We didn’t get the biased and unfair sentencing results we have today, including the disproportionate sentences for “high risk” individuals, from these instruments, and blasting them without understanding how the system actually operates in fact continues the empowerment of the court officials who have turned us into the most incarcerated nation on the planet. What you need to worry about is that, once these systems do come into play in major ways, those Powers That Be and their political constituents like private prisons and media-genic “victims’ representatives,” along with the new “data-driven” media, will insist on using their own specially informed tools. The protested models, which were not created just yesterday and do have years of experience and adjustment built in, will not likely withstand the efforts of those with the most power in the process. THAT’s when what you’re fearful of will be worthy of your concerns.
If you want to really help, insist that these tools not be used to sentence upfront in individual cases but to identify the sentences most associated with positive, non-recidivistic results for groups of offenders in the same risk categories. I guarantee you that in almost every case in every state and the federal level those sentences will be lower than the averages for those risk categories with those types of offenses. Then insist that public officials stop promoting and executing sentences that are associated with more recidivism and victimization after release. Demand to know why these self-interested “public safety first” blowhards are claiming their tougher sentences protect people when they in fact can be shown to be leading to more crime and more victims (of a wide variety) than would occur if the shorter sentences or other dispositions were used. Then be prepared to point to how the existing system, with its lack of data that could be used effectively as proposed by the “reform” groups, seems not to be “crime control” but “system and turf empowerment and protection.” Finally, say it in very simple terms that even the US Attorney General could understand.
LikeLike
I wonder if it’s not the models themselves, but rather their use that is the issue. In discussing this with my wife this morning, she made the (I think correct) suggestion that if the models were applied to other areas, they might be more useful. The specific situation that we discussed was potentially using a predictive model to help guide strategies for guidance after parole, for example. This would (potentially) be a more positive application of a model and would not suffer (as much) the types of spurious causality assumptions that Cathy mentioned in her original post. Instead, the models could be used as a form of useful dimensionality reduction, i.e. people with certain contextual background factors tend to benefit most from remediation X, or intervention Y when (say) being paroled.
I’m not sure that such an approach would work, but it at least seems that, prima facie, it would be a more fruitful application of predictive modeling than the current application to sentencing.
LikeLike
Paul Meehl wrote a book, Clinical vs Statistical Prediction (1986), showing that generally statistical prediction does better than clinical. I think, however, that if decision-makers were given feedback about the results of their earlier decisions, i.e., a hybrid system, they might do a lot better than purely statistical assessments. When dealing with individuals and behavior, not all relevant variables can be specified — or measured. And clinicians may be better in gauging these.
LikeLike