mathbabe

Have you been keeping up with the drama around the US News & World Reports college ranking and law school ranking models?

Well, I have, mostly, and it’s in large part because of my friend Michael Thaddeus who gets a ton of credit for pointing out how Columbia was totally lying about their self-reported numbers, which has kicked off a huge amount of scrutiny of the model and self-scrutiny by the colleges to why they still engage with it.

The thing I want to talk about today is this article, which describes a “law dean rebellion” where a bunch of law schools quite rightly questioned the value of the rankings system and decided to get out, and how US News is begging them to stay and offering to rejigger the formula a little bit to appease them.

I’m particularly interested in this story because I devoted a large part of a chapter of my book to it. It was essentially the first Weapon of Math Destruction, if you will, because it was the first opaque scoring system that had widespread and devastating impact, in large part because it was crappy from the get go, but parents really cared about it in spite of that, and so colleges were forced to reckon with it, and they ended up gaming it so badly (see above) that everything got warped, including most importantly the lives of teenagers, and education got worse, and most expensive, and nobody won, except maybe the huge number of people who now work in administration.

Anyhoo, that’s all pretty well understood and many times demonstrated. But what I wanted to point out about this new story is just how perfectly it demonstrates my favorite refrain about algorithms, namely that they are opinions embedded in math.

Because, get this, here are the ways that US News wants to tweak its formula, according to the piece:

1. less weight for “reputation”
2. more weight for “do students do public service”
3. nothing about student debt
4. nothing about how the schools spend money

By contrast, here are the things that the deans are purported to want adjusted:

1. credit for diversity
2. credit for loan forgiveness and financial aid
3. credit for public service by students

As you can see, there is one thing in common but a lot not in common, and more importantly, these are all stabs in the dark as to how a given person might actually decide to value a given school. In other words, it’s all opinion, it’s always been opinion, and the US News folks are finding themselves old fashioned and flat footed because their opinions are archaic, undependable, and totally reliant on a public that just trusts numbers because they “seem scientific.”