Home > Uncategorized > Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime

Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime

August 18, 2016

A few months ago I publicly pined for a more scientific audit of the Chicago Police Department’s “Heat List” system. The excerpt from that blogpost:

…the Chicago Police Department uses data mining techniques of social media to determine who is in gangs. Then they arrest scores of people on their lists, and finally they tout the accuracy of their list in part because of the percentage of people who were arrested who were also on their list. I’d like to see a slightly more scientific audit of this system.

Thankfully, my request has officially been fulfilled!

Yesterday I discovered via Marcos Carreiro on Twitter, that a paper has been written entitled Predictions put into practice: a quasi-experimental evaluation of Chicago’s predictive policing pilot, written by Priscillia Hunt, and John S. Hollywood and published in the Journal of Experimental Criminology.

The paper’s main result upheld my suspicions:

Individuals on the SSL are not more or less likely to become a victim of a homicide or shooting than the comparison group, and this is further supported by city-level analysis. The treated group is more likely to be arrested for a shooting.

Inside the paper, they make the following important observations. First, crime rates have been going down over time, and the “Heat List” system has not effected that trend. An excerpt:

…the statistically significant reduction in monthly homicides predated the introduction of the SSL, and that the SSL did not cause further reduction in the average number of monthly homicides above and beyond the pre-existing trend.

Here’s an accompanying graphic:

Screen Shot 2016-08-18 at 6.14.39 AM.png

This is a really big and important point, one that smart people like Gillian Tett get thrown off by when discussing predictive policing tools. We cannot automatically attribute success to any policing policy in the context of meta-effects.

Next, being on the list doesn’t protect you:

However, once other demographics, criminal history variables, and social network risk have been controlled for using propensity score weighting and doubly-robust regression modeling, being on the SSL did not significantly reduce the likelihood of being a murder or shooting victim, or being arrested for murder.

But it does make it more likely for you to get surveilled by police:

Seventy-seven percent of the SSL subjects had at least one contact card over the year following the intervention, with a mean of 8.6 contact cards, and 60 % were arrested at some point, with a mean of 1.53 arrests. In fact, almost 90 % had some sort of interaction with the Chicago PD (mean = 10.72 interactions) during the year-long observation window. This increased surveillance does appear to be caused by being placed on the SSL. Individuals on SSL were 50 % more likely to have at least one contact card and 39 % more likely to have any interaction (including arrests, contact cards, victimizations, court appearances, etc.) with the Chicago PD than their matched comparisons in the year following the intervention. There was no statistically significant difference in their probability of being arrested or incapacitated8 (see Table 4). One possibility for this result, however, is that, given the emphasis by commanders to make contact with this group, these differences are due to increased reporting of contact cards for SSL subjects.

And, most importantly, being on the list means you are likely to be arrested for shooting, but it doesn’t cause that to be true:

In other words, the additional contact with police did not result in an increased likelihood for arrests for shooting, that is, the list was not a catalyst for arresting people for shootings. Rather, individuals on the list were people more likely to be arrested for a shooting regardless of the increased contact.

That also comes with an accompanying graphic:

Screen Shot 2016-08-18 at 6.29.13 AM.png

From now on, I’ll refer to Chicago’s “Heat List” as a way for the police to predict their own future harassment and arrest practices.

Categories: Uncategorized
  1. August 18, 2016 at 8:05 am

    Reblogged this on Matthews' Blog.


  2. August 18, 2016 at 9:05 am

    The monthly Chicago homicide graphics shows a very strong yearly seasonality. Can someone explains why more people are killed in certain months rather than others ? Are temperature and weather important factors when killing people ?


    • August 18, 2016 at 9:06 am

      People stay inside during the winter.


      • August 18, 2016 at 9:46 am

        It suggest many questions. Are most homicides happening in the street? Can we know how many are perpetrated indoors? Are death caused by police shootings included in the graphics?


    • August 18, 2016 at 9:22 am

      Mathbabe is quite correct. Chicago is COLD in the winter time. I went there to visit a friend a few years back. I arrived in cargo pants and a T-shirt with people staring at me like I was crazy (it was still mid 70’s in my home town of Wilmington, NC). When I walked outside the double-door of the airport, the cold ripped into my lungs and flash froze their air that was already inside. At least that’s what it felt like.

      I’ll never visit that freezing hell outside June – August again.



      • August 18, 2016 at 9:41 am

        So you are only visiting when the probability of being shot is at the highest ?


        • August 18, 2016 at 10:46 am

          The probability of being shot during a short summer visit is very small, whereas the probability of being freezing cold in winter is 1.


      • September 4, 2016 at 7:50 am

        U of Chicago Law School has a T-Shirt with “Hell does freeze over” on the back.


  3. August 18, 2016 at 10:45 am
    • August 18, 2016 at 12:26 pm

      So the solution to crime is to abolish summer!


  4. Peter Pap
    August 18, 2016 at 12:18 pm

    You are missing the most important consequence. If the lists are worthless in capturing the murderers, and given that police have limited resources, this means that capturing the actual murderers is delayed and costs more.


  5. September 4, 2016 at 7:47 am

    Once again we see the dystopian future predicted by Philip K. Dick in his story “Minority Report”. Yes it was made into a movie.


  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: