Gillian Tett gets it very wrong on racial profiling
Last Friday Gillian Tett ran a profoundly disturbing article in the Financial Times entitled Mapping Crime – Or Stirring Hate? (hat tip Marcos Carreira), which makes me sad to say this given how much respect I normally have for her regarding her coverage of the financial crisis.
In the article, Tett describes the predictive policing model used by the Chicago police force, which told the police where to go to find criminals based on where people had been arrested in the past.
Her article reads like an advertisement for racist profiling. First she deftly and indirectly claims the model is super successful at lowering the murder rate without actually coming out and saying so (since she actually has only correlative evidence):
And when Weis launched the programme in early 2010, together with a clever policeman-cum-computer expert called Brett Goldstein, it delivered impressive results. In the first year the murder rate fell 5 per cent and then continued to tumble. Indeed by the summer of 2011 it looked as if Chicago’s annual death toll would soon drop below 400, the lowest since 1965. “The homicide rates for that summer were just crazy low compared to what we had been,” Weis observes.
But then, following his departure from the force, the programme was wound down in late 2011. And, tragically, the murder rate immediately rose again.
Here’s the thing, it’s really hard to actually know why murder rates go up and down. In New York City we’ve been using Stop & Frisk as the violent crime rates have been steadily lowering in this city (and many others), and for a long time Bloomberg took credit for that through the Stop & Frisk practice. But when Stop & Frisk rates went down, murder rates didn’t shoot up. Just saying. And that’s ignoring how reliable the police data is, which is another issue. Let’s take a look at her evidence for a longer time frame:
She’s talking about that small uptick at the end, which to the naked eye could well be statistical noise.
The reason I’m pointing out her bad statistics is that she needs them to set up the following, truly disturbing paragraphs (emphasis mine):
But while racism is rightly deemed unacceptable, computer programs pose more subtle questions. If a spreadsheet forecast has a racial imbalance, is this likely to reinforce existing human biases, or racial profiling? Or is a weather map of crime simply a neutral tool? To put it another way, does the benefit of using predictive policing outweigh any worries about political risk?
Personally, I think it does. After all, as the former CPD computer experts point out, the algorithms in themselves are neutral. “This program had absolutely nothing to do with race… but multi-variable equations,” argues Goldstein. Meanwhile, the potential benefits of predictive policing are profound.
No, Gillian Tett, there is no such thing as a neutral tool. No algorithm focused on human behavior is neutral. Anything which is trained on historical human behavior embeds and codifies historical and cultural practices. Specifically, this means that the fact that black Americans are nearly four times as likely as whites to be arrested on charges of marijuana possession even though the two groups use the drug at similar rates would be seen by such a model (or rather, by the people who deploy the model) as a fact of nature that is neutral and true. But it is in fact a direct consequence of systemic racism.
Put it another way: if we allowed a model to be used for college admissions in 1870, we’d still have 0.7% of women going to college. Thank goodness we didn’t have big data back then!
This is very scary to me, when even Gillian Tett, who famously predicted the financial crisis in 2006, can be fooled. We clearly have a lot of work to do.
mathbabe 
This reminds me of an anecdote a professor of economics told me about his department. He said the women on the faculty were suing the department head because they discovered they were getting paid less, although their credentials were just as good as the male faculty members. The department head apparently agreed with this — his defense was that he negotiated salaries in exactly the same way with women and men, but that the women didn’t negotiate as well as the men. Now what do you do about that?
LikeLike
Put a woman in charge of salary negotiations. 🙂
LikeLike
In a HS where I was superintendent one of the principals used a variation of this algorithm to good effect. He mapped the places where student fights broke out and deployed hall monitors to those areas. As a result he reduced suspensions for fighting and smoking to almost zero. This kind of data-mapping IS neutral: it provides information on WHERE a crime is committed not WHO or WHY the crime is committed. It is akin to identifying intersections where traffic accidents occur so that those can either be monitored or upgraded. What am I missing?
LikeLike
You’re probably not missing anything, in that case. Unfortunately it’s different with policing: the people living in targeted neighborhoods are typically harassed for very low-level, petty crimes like pot possession and selling cigarettes, not murder. Then they go to jail for such stupid stuff and a negative feedback loop is formed. It’s a collateral damage problem.
Thanks for pointing it out!
LikeLike
Thanks to the homicide graph (although not all homicides are murders), I don’t see that Tett has any evidence at all for the effectiveness of predictive policing on reducing the murder rate. Isn’t the data better reported like this?
The program delivered ordinary results. In the first year the murder rate fell 5 per cent, continuing the trend since 1994, but then started to level out. In the summer of 2011 the murder rate approached 400, the lowest since 1965. Weis said, ““The homicide rates for that summer were just crazy low compared to what we [sic] had been.” He was apparently referring to the high rates over 900 in the early 1990s. In 2004 the homicide rate dropped precipitously from around 600 to fewer than 500, and remained below 500 for every year but one since then. Tragically the rate jumped to around 500 again in 2012. The program ended in late 2011, after Weis had left the force.
LikeLike
About the [sic]. I thought that there should be a “they” instead of “we”. I now think that it is more likely that “been” is a typo for “seen”.
And, OC, the 5% drop in 2010 could be seen as noise, part of natural variation since 2004 instead of part of the trend since the early 1990s.
LikeLike
First of all, unlike a rose, a homicide is not a homicide is not a homicide. Intimate partner homicides (usually indoors) may have more to do with unemployment than police strategy.Second, it’s always difficult to attribute changes in crime of any type to policy changes without a randomized control trial, which the evidence-based policing (and the Journal of Experimental Criminology) has been promoting. For example, the much-touted “broken windows” strategy reduced crime in NYC during the 90s; what, pray tell, reduced crime in Dallas and Houston during that same period? The free rider effect-at-a-distance?
It’s often been noted that police chiefs are quick to tout their policies when crime is reduced, but are mum when it increases. The best strategy for a roving police chief, like Bratton, is to go where crime is increasing “out of control,” because all he has to do is wait for regression to the mean — and be considered a hero.
LikeLike
Homicide or murder is usually the “successful” result of shooting. The statistic to watch is shootings, which does not depend on the skill level or “luck” of the shooter. Media reports claim that NYC shootings are up 13% since the end of stop and frisk. Do you have stats that show otherwise?
LikeLike
Can you give me a reference for that statistic?
LikeLike
http://www.nydailynews.com/new-york/nyc-crime/homicides-shootings-rise-citywide-article-1.1823173
LikeLike
Statistical behavior is what is being modeled. Yes, the model might be in a black community and yes that means that it is modeling those blacks, not necessarily any others! The important fact, is that it is modeling the actors in that space. It indicates a cultural bias of a community which allows this behavior to exist and continue around them. It seems that these people might be ignorant of the facts around bullying, and other aggressive behaviors which when left unchecked at young ages, become embedded sub-conscience reactions in situations where things are stimulating or driving such reactions. This is just one example. Stretching the conclusion to mean all blacks might have this behavior is a miss-judgement that I don’t see happening in the individuals involved.
Community uninvolvement is what we need to address in all parts of our society today. You can feel bad about this particular application to a black community, but it would work equally well in any other place where this behavior was prevalent, and a single race was the dominant race in the population numbers. If it was a Caucasian population, we’d think Yeah, go get the criminals. But still, the problem is societal declination.
Try and point out the good things we could learn about this. Demand that the people of this community and others become players in the solution instead of ignoring the problem.
In the Middle East, people have been tortured, ruled by brutality and controlled for so long that they have no idea how to stand up to abuse and refuse to conform. They don’t want to die! Death is what happens to nonconformists in that part of the world. These people need our help to discover how to participate and demand freedom based ideals and behaviors in their societies.
It’s going to take generations of redevelopment to fix these kinds of things. This is not a small issue! We are going to have to have activities in all parts of the world to correct these things that we’ve ignored and allowed to manifest into complete chaos.
In the U.S., we have a long history of standing up for freedom (ours and others), because we recognize our own responsibility for being a part of the solution by driving ourselves and others to better behaviors. We hold each other responsible for our actions, and demand that we each do our part to keep our society functioning.
LikeLike
About this – “black Americans are nearly four times as likely as whites to be arrested on charges of marijuana possession even though the two groups use the drug at similar rates” … In my opinion, this is not indicative of racism but of corruption. The entire “law” enforcement system benefits from the war on drugs. If they would go after all segments of society, I have no doubt that these laws would have been repealed long ago. If they go after no one, no one benefits. Smaller police force without tanks and automatic weapons, and none of that lovely seized property? Which police chief would want that? Fewer prosecutions might mean fewer lawyers and judges, and where would all the future tough on crime politicians come from then, huh? What if there were fewer parole officers, fewer prison guards and prisons; what would happen to the profits of the private prison system? And it’s not just the jobs, livelihoods and money involved there. Many participants in the systems have almost god-like powers. Like one former police chief recently wrote in NY Times op ed: “When officer tells you to do something, you better do it, if you don’t want to get seriously hurt. Save your protests for later, when it is safe.” How much more god-like can a man get?
LikeLike
And more: “Anything which is trained on historical human behavior embeds and codifies historical and cultural practices.” So train it on homicide, robbery and rape, FBI data is readily available. Id’ like if you would post your own analysis some day.
LikeLike
it’s the lead: http://www.motherjones.com/environment/2013/01/lead-crime-link-gasoline
LikeLike
I agree with the message, but I will nitpick on the language.
You say that the algorithm is not neutral. I think it *is* neutral. And that’s a problem.
To explain why, I’ll start with a short story from Romania. Most Romanians dislike gypsies, for two main reasons: (1) gypsies commit many crimes, and (2) a disproportionate number of stories about Romania are in fact about crimes committed by gypsies. Both of these are without doubt true. Because most Romanians are aware of these facts, they will not hire a gypsy. So, if you happen to be a gypsy in Romania, it’s pretty much impossible to get a job. Being human, you need food. So, you start committing crimes. If your child says “I’m hungry”, it’s very difficult to not do something about it, even if it’s against the law.
That’s a positive feedback loop. (You say ‘negative’ in a comment, which is technically wrong. Yes, of course it’s bad, but that doesn’t make it ‘negative’ in the technical sense.)
Once you recognize this positive feedback loop, you realize that you must *do* something to break it. The very definition of positive feedback loop is that it perpetuates itself. Being neutral won’t solve the problem.
LikeLike
Thanks for the thoughtful response. I agree that “negative” was the wrong word choice.
And great example with the gypsies. It’s exactly the same issue with predictive policing, and it is racism. The difference is that with the predictive policing model, it’s called “scientific” and “neutral,” whereas when shopkeepers don’t hire a gypsy because he or she is a gypsy, at least they have to acknowledge their racism.
I’m not saying that racism an easy problem to solve, I’m just saying that hiding behind scientific language doesn’t change what we are actually doing.
LikeLike