Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime
A few months ago I publicly pined for a more scientific audit of the Chicago Police Department’s “Heat List” system. The excerpt from that blogpost:
…the Chicago Police Department uses data mining techniques of social media to determine who is in gangs. Then they arrest scores of people on their lists, and finally they tout the accuracy of their list in part because of the percentage of people who were arrested who were also on their list. I’d like to see a slightly more scientific audit of this system.
Thankfully, my request has officially been fulfilled!
Yesterday I discovered via Marcos Carreiro on Twitter, that a paper has been written entitled Predictions put into practice: a quasi-experimental evaluation of Chicago’s predictive policing pilot, written by
The paper’s main result upheld my suspicions:
Individuals on the SSL are not more or less likely to become a victim of a homicide or shooting than the comparison group, and this is further supported by city-level analysis. The treated group is more likely to be arrested for a shooting.
Inside the paper, they make the following important observations. First, crime rates have been going down over time, and the “Heat List” system has not effected that trend. An excerpt:
…the statistically significant reduction in monthly homicides predated the introduction of the SSL, and that the SSL did not cause further reduction in the average number of monthly homicides above and beyond the pre-existing trend.
Here’s an accompanying graphic:
This is a really big and important point, one that smart people like Gillian Tett get thrown off by when discussing predictive policing tools. We cannot automatically attribute success to any policing policy in the context of meta-effects.
Next, being on the list doesn’t protect you:
However, once other demographics, criminal history variables, and social network risk have been controlled for using propensity score weighting and doubly-robust regression modeling, being on the SSL did not significantly reduce the likelihood of being a murder or shooting victim, or being arrested for murder.
But it does make it more likely for you to get surveilled by police:
Seventy-seven percent of the SSL subjects had at least one contact card over the year following the intervention, with a mean of 8.6 contact cards, and 60 % were arrested at some point, with a mean of 1.53 arrests. In fact, almost 90 % had some sort of interaction with the Chicago PD (mean = 10.72 interactions) during the year-long observation window. This increased surveillance does appear to be caused by being placed on the SSL. Individuals on SSL were 50 % more likely to have at least one contact card and 39 % more likely to have any interaction (including arrests, contact cards, victimizations, court appearances, etc.) with the Chicago PD than their matched comparisons in the year following the intervention. There was no statistically significant difference in their probability of being arrested or incapacitated8 (see Table 4). One possibility for this result, however, is that, given the emphasis by commanders to make contact with this group, these differences are due to increased reporting of contact cards for SSL subjects.
And, most importantly, being on the list means you are likely to be arrested for shooting, but it doesn’t cause that to be true:
In other words, the additional contact with police did not result in an increased likelihood for arrests for shooting, that is, the list was not a catalyst for arresting people for shootings. Rather, individuals on the list were people more likely to be arrested for a shooting regardless of the increased contact.
That also comes with an accompanying graphic:
From now on, I’ll refer to Chicago’s “Heat List” as a way for the police to predict their own future harassment and arrest practices.
Reblogged this on Matthews' Blog.
LikeLike
The monthly Chicago homicide graphics shows a very strong yearly seasonality. Can someone explains why more people are killed in certain months rather than others ? Are temperature and weather important factors when killing people ?
LikeLike
People stay inside during the winter.
LikeLike
It suggest many questions. Are most homicides happening in the street? Can we know how many are perpetrated indoors? Are death caused by police shootings included in the graphics?
LikeLike
Mathbabe is quite correct. Chicago is COLD in the winter time. I went there to visit a friend a few years back. I arrived in cargo pants and a T-shirt with people staring at me like I was crazy (it was still mid 70’s in my home town of Wilmington, NC). When I walked outside the double-door of the airport, the cold ripped into my lungs and flash froze their air that was already inside. At least that’s what it felt like.
I’ll never visit that freezing hell outside June – August again.
JamesNT
LikeLike
So you are only visiting when the probability of being shot is at the highest ?
LikeLike
The probability of being shot during a short summer visit is very small, whereas the probability of being freezing cold in winter is 1.
LikeLike
U of Chicago Law School has a T-Shirt with “Hell does freeze over” on the back.
LikeLike
Crime rates are higher in the summer everywhere.
http://www.governing.com/topics/public-justice-safety/gov-summer-crime-rates-increases-police.html
LikeLike
So the solution to crime is to abolish summer!
LikeLike
You are missing the most important consequence. If the lists are worthless in capturing the murderers, and given that police have limited resources, this means that capturing the actual murderers is delayed and costs more.
LikeLike
Once again we see the dystopian future predicted by Philip K. Dick in his story “Minority Report”. Yes it was made into a movie.
LikeLike