There was a recent New York Times op-ed by Sonja Starr entitled Sentencing, by the Numbers (hat tip Jordan Ellenberg and Linda Brown) which described the widespread use – in 20 states so far and growing – of predictive models in sentencing.
The idea is to use a risk score to help inform sentencing of offenders. The risk is, I guess, supposed to tell us how likely the person is to commit another act in the future, although that’s not specified. From the article:
The basic problem is that the risk scores are not based on the defendant’s crime. They are primarily or wholly based on prior characteristics: criminal history (a legitimate criterion), but also factors unrelated to conduct. Specifics vary across states, but common factors include unemployment, marital status, age, education, finances, neighborhood, and family background, including family members’ criminal history.
I knew about the existence of such models, at least in the context of prisoners with mental disorders in England, but I didn’t know how widespread it had become here. This is a great example of a weapon of math destruction and I will be using this in my book.
A few comments:
- I’ll start with the good news. It is unconstitutional to use information such as family member’s criminal history against someone. Eric Holder is fighting against the use of such models.
- It is also presumably unconstitutional to jail someone longer for being poor, which is what this effectively does. The article has good examples of this.
- The modelers defend this crap as “scientific,” which is the worst abuse of science and mathematics imaginable.
- The people using this claim they only use it for as a way to mitigate sentencing, but letting a bunch of rich white people off easier because they are not considered “high risk” is tantamount to sentencing poor minorities more.
- It is a great example of confused causality. We could easily imagine a certain group that gets arrested more often for a given crime (poor black men, marijuana possession) just because the police have that practice for whatever reason (Stop & Frisk). Then model would then consider any such man at a higher risk of repeat offending, but that’s not because any particular person is actually more likely to do it, but because the police are more likely to arrest that person for it.
- It also creates a negative feedback loop on the most vulnerable population: the model will impose longer sentencing on the population it considers most risky, which will in turn make them even riskier in the future, if “length of time in prison previously” is used as an attribute in the model, which is surely is.
- Not to be cynical, but considering my post yesterday, I’m not sure how much momentum will be created to stop the use of such models, considering how discriminatory it is.
- Here’s an extreme example of preferential sentencing which already happens: rich dude Robert H Richards IV raped his 3-year-old daughter and didn’t go to jail because the judge ruled he “wouldn’t fare well in prison.”
- How great would it be if we used data and models to make sure rich people went to jail just as often and for just as long as poor people for the same crime, instead of the other way around?
I’ve been fascinated to learn all sorts of things about how McDonalds operates their business in the past few days, as news broke about a recent NLRB decision to allow certain people who work in McDonalds to file complaints about their workplace and name McDonalds as a joint employer.
That sounds incredibly dull, right? The idea of letting McDonalds workers name McDonalds as an employer? Let me tell you a bit more. And this is all common knowledge, but I thought I’d gather it here for those of you who haven’t been following the story.
Most of the McDonalds joints you go to are franchises – 90% in this country. That means the business is owned by a franchisee, a person who pays good money (details here) for the right to run a McDonalds and is constrained by a huge long list of rules about how they have to do it.
The franchise owner attends Hamburger University and gets trained in all sorts of things, like exactly how things should look in the store, how customers should be funneled through space (maps included), how long each thing should take, and how to treat employees. There’s a QSC Playbook they are given (Quality, Service, and Cleanliness) as well as minute descriptions of how to organize their teams and even the vocabulary words they should use to encourage workers (see page 24 of the Shift Management Guide I found online here).
McDonalds also installs a real-time surveillance system into each McDonalds, which can calculate the rate of revenue brought in at a given moment, as well as the rate of pay going out, and when the ratio of those two numbers reaches a certain lower bound threshold, they encourage franchise owners to ask people to leave or delay people from clocking in. Encourage, mind you, not require. They are not the employers or anything remotely like that, clearly.
Take a step back here. What is the business model of a franchise? And when did McDonalds stop being a burger joint?
The idea is this. When you own a restaurant you have to deal with all these people who work for you and you have to deal with their complaints, and they might not like the way you treat them and they might organize against you or sue you. In order to contain your risks, you franchise. That effectively removes all of those people except one, the franchise owner, with whom you have an air-tight contract, written by a huge team of lawyers, which basically says that you get to cancel the franchise agreement for any minor infraction (where they’d lose a bunch of investment money), but most importantly it means the people actually working in a given franchise work for that one person, not for you, so their pesky legal issues are kept away from you. It’s a way to box in the legal risk of the parent company.
Restaurants aren’t the only business to learn that it’s easier to sell and manage a brand than it is to sell and manage an actual product. Hotels have been doing this for a long time, and avoid complaints and legal issues stemming from the huge population of service workers in hotels, mostly minority women.
For a copy of the original complaint that gave the details of McDonald’s control over workers, read this. For a better feel for being a McDonalds worker, please read this recent Reuters blog post written by a McDonalds worker. And for a better feel for being a McDonald’s franchise owner, read this recent Washington Post letter from a long-time McDonalds franchise owner who thinks workers are being unfairly treated.
Does that sounds confusing, that a franchise owner would side with the employees? It shouldn’t.
By nature of the franchise contract, the money actually available to a franchise owner is whatever’s left over after they pay McDonalds for advertising, and buy all the equipment and food that McDonalds tells them to from the sources that they tell them to, and after they pay for insurance on everything and for rent on the property (which McDonalds typically owns). In other words the only variable they have to tweak is the employer pay, but if they pay a living wage then they lose money on their business. In fact when franchise owners complain about the profit stream, McDonalds tells them to pay their workers less. McDonalds essentially controls everything except one variable, but since it’s a closed system of equations, that means the franchise owners have to decide between paying their workers reasonably and going in the red.
That’s not to say, of course, that McDonalds as an enterprise is at risk of losing money. In fact the parent corporation is making good money ($1.4 billion per quarter if you include international revenue), by squeezing the franchises. If the franchise owners had more leverage to negotiate better contracts, they could siphon off more revenue and then – possibly – share it with workers.
So back to the ruling. If upheld, and there’s a good chance it won’t be but I’m feeling hopeful today, this decision will allow people to point at McDonalds the corporation when they are treated badly, and will potentially allow a workers’ union to form. Alternatively it might energize the franchise owners to negotiate more flexible contracts, which could allow them to pay their workers better directly.
There’s a CNN video news story explaining how the NYC Mayor’s Office of Data Analytics is working with private start-up Placemeter to count and categorize New Yorkers, often with the help of private citizens who install cameras in their windows. Here’s a screenshot from the Placemeter website:
You should watch the video and decide for yourself whether this is a good idea.
Personally, it disturbs me, but perhaps because of my priors on how much we can trust other people with our data, especially when it’s in private hands.
To be more precise, there is, in my opinion, a contradiction coming from the Placemeter representatives. On the one hand they try to make us feel safe by saying that, after gleaning a body count with their video tapes, they dump the data. But then they turn around and say that, in addition to counting people, they will also categorize people: gender, age, whether they are carrying a shopping bag or pushing strollers.
That’s what they are talking about anyway, but who knows what else? Race? Weight? Will they use face recognition software? Who will they sell such information to? At some point, after mining videos enough, it might not matter if they delete the footage afterwards.
Since they are a private company I don’t think such information on their data methodologies will be accessible to us via Freedom of Information Laws either. Or, let me put that another way. I hope that MODA sets up their contract so that such information is accessible via FOIL requests.
I’m super excited about the recent “mood study” that was done on Facebook. It constitutes a great case study on data experimentation that I’ll use for my Lede Program class when it starts mid-July. It was first brought to my attention by one of my Lede Program students, Timothy Sandoval.
My friend Ernest Davis at NYU has a page of handy links to big data articles, and at the bottom (for now) there are a bunch of links about this experiment. For example, this one by Zeynep Tufekci does a great job outlining the issues, and this one by John Grohol burrows into the research methods. Oh, and here’s the original research article that’s upset everyone.
It’s got everything a case study should have: ethical dilemmas, questionable methodology, sociological implications, and questionable claims, not to mention a whole bunch of media attention and dissection.
By the way, if I sound gleeful, it’s partly because I know this kind of experiment happens on a daily basis at a place like Facebook or Google. What’s special about this experiment isn’t that it happened, but that we get to see the data. And the response to the critiques might be, sadly, that we never get another chance like this, so we have to grab the opportunity while we can.
There’s been a movement to make primary and secondary education run more like a business. Just this week in California, a lawsuit funded by Silicon Valley entrepreneur David Welch led to a judge finding that student’s constitutional rights were being compromised by the tenure system for teachers in California.
The thinking is that tenure removes the possibility of getting rid of bad teachers, and that bad teachers are what is causing the achievement gap between poor kids and well-off kids. So if we get rid of bad teachers, which is easier after removing tenure, then no child will be “left behind.”
The problem is, there’s little evidence for this very real achievement gap problem as being caused by tenure, or even by teachers. So this is a huge waste of time.
As a thought experiment, let’s say we did away with tenure. This basically means that teachers could be fired at will, say through a bad teacher evaluation score.
An immediate consequence of this would be that many of the best teachers would get other jobs. You see, one of the appeals of teaching is getting a comfortable pension at retirement, but if you have no idea when you’re being dismissed, then it makes no sense to put in the 25 or 30 years to get that pension. Plus, what with all the crazy and random value-added teacher models out there, there’s no telling when your score will look accidentally bad one year and you’ll be summarily dismissed.
People with options and skills will seek other opportunities. After all, we wanted to make it more like a business, and that’s what happens when you remove incentives in business!
The problem is you’d still need teachers. So one possibility is to have teachers with middling salaries and no job security. That means lots of turnover among the better teachers as they get better offers. Another option is to pay teachers way more to offset the lack of security. Remember, the only reason teacher salaries have been low historically is that uber competent women like Laura Ingalls Wilder had no other options than being a teacher. I’m pretty sure I’d have been a teacher if I’d been born 150 years ago.
So we either have worse teachers or education doubles in price, both bad options. And, sadly, either way we aren’t actually addressing the underlying issue, which is that pesky achievement gap.
People who want to make schools more like businesses also enjoy measuring things, and one way they like measuring things is through standardized tests like achievement scores. They blame teachers for bad scores and they claim they’re being data-driven.
Here’s the thing though, if we want to be data-driven, let’s start to maybe blame poverty for bad scores instead:
I’m tempted to conclude that we should just go ahead and get rid of teacher tenure so we can wait a few years and still see no movement in the achievement gap. The problem with that approach is that we’ll see great teachers leave the profession and no progress on the actual root cause, which is very likely to be poverty and inequality, hopelessness and despair. Not sure we want to sacrifice a generation of students just to prove a point about causation.
On the other hand, given that David Welch has a lot of money and seems to be really excited by this fight, it looks like we might have no choice but to blame the teachers, get rid of their tenure, see a bunch of them leave, have a surprise teacher shortage, respond either by paying way more or reinstating tenure, and then only then finally gather the data that none of this has helped and very possibly made things worse.
I’m too busy this morning for a real post but I thought I’d share a few things I’m reading today.
- Matt Stoller just came out with a long review of Timmy Geithner’s book: The Con Artist Wing of the Democratic Party. I like this because it explains some of the weird politics around, for example, the Mexican currency crisis that I only vaguely knew about.
- New York Magazine has a long profile of Stevie Cohen of SAC Capital insider trading fame: The Taming of the Trading Monster.
- The power of Google’s algorithms can make or break smaller websites: On the Future of Metafilter. See also How Google Is Killing The Best Site On The Internet.
- There is no such thing as a slut.
Here’s one recommendation related to discrimination:
Expand Technical Expertise to Stop Discrimination. The detailed personal profiles held about many consumers, combined with automated, algorithm-driven decision-making, could lead—intentionally or inadvertently—to discriminatory outcomes, or what some are already calling “digital redlining.” The federal government’s lead civil rights and consumer protection agencies should expand their technical expertise to be able to identify practices and outcomes facilitated by big data analytics that have a discriminatory impact on protected classes, and develop a plan for investigating and resolving violations of law.
First, I’m very glad this has been acknowledged as an issue; it’s a big step forward from the big data congressional subcommittee meeting I attended last year for example, where the private-data-for-services fallacy was leaned on heavily.
So yes, a great first step. However, the above recommendation is clearly insufficient to the task at hand.
It’s one thing to expand one’s expertise – and I’d be more than happy to be a consultant for any of the above civil rights and consumer protection agencies, by the way – but it’s quite another to expect those groups to be able to effectively measure discrimination, never mind combat it.
Why? It’s just too easy to hide discrimination: the models are proprietary, and some of them are not even apparent; we often don’t even know we’re being modeled. And although the report brings up discriminatory pricing practices, it ignores redlining and reverse-redlining issues, which are even harder to track. How do you know if you haven’t been made an offer?
Once they have the required expertise, we will need laws that allow institutions like the CFPB to deeply investigate these secret models, which means forcing companies like Larry Summer’s Lending Club to give access to them, where the definition of “access” is tricky. That’s not going to happen just because the CFPB asks nicely.