mathbabe

There’s reason to believe facial recognition software is getting very accurate. According to a WSJ article by Laura Mills, Facial Recognition Software Advances Trigger Worries, a Russian company called NTechLab has built software that “correctly matches 73% of people to large photo database.” The stat comes from celebrities recognized in a database of a million pictures.

Now comes the creepy part. The company, headed by two 20-something Russian tech dudes, are not worried about the ethics of their algorithms. Here are their reasons:

1. Because it’s already too late to worry. In the words of one of the founders, “There is no private life.”
2. They don’t need to draw a line in the sand for who they give this technology to, because“we don’t receive requests from strange people.”
3. Also, the technology should be welcomed, rather than condemned, because according to the founders, “There is always a conflict between progress and some scared people,” he said. “But in any way, progress wins.”

Thanks for the assurance!

Let’s compare the above reasons to not worry to the below reasons we have to worry, which include:

1. The founders are in negotiations to sell their products to state-affiliated security firms from China and Turkey.
2. Moscow’s city government is planning to install NTechLab’s technology on security cameras around the city.
3. They were already involved in a scandal in which people used their software to identify and harass women who had allegedly acted in pornographic films online in Russia.

I’ve been busy recording podcasts recently, and I wanted to collect some of them together for your convenience and listening pleasure.

1. New Yorker Radio Hour (starting at 25:40)
2. Slate Money: Shadow Courts edition
3. EconTalk: Cathy O’Neil on Weapons of Math Destruction
4. Harvard Biz Review: When Not To Trust The Algorithm
5. Guardian Science Weekly: Weapons of math destruction: how big data and algorithms affect our lives

Please forgive me for posting constantly about the amazing press my book is getting, I’m milking this once-in-a-lifetime opportunity.

Because, HOLY SHIT!

Clay Shirky wrote a review of my book in the NY Times Book Review in the context of a Reader’s Guide To This Fall’s Big Book Awards. It has this attractive accompanying graphic:

Also, I talked with Russ Roberts a while back about my book and it’s now on EconTalk as a podcast.

Also, when I visited London last week I made a trip to the Guardian offices and recorded a Guardian Science podcast which also features Douglas Rushkoff.

I’m on my way to Boston by train today to for a book talk and Q&A at the Harvard Bookstore in Harvard Square at 7pm. More information here.

Please join if you’re around, I’d love to see you.

Also!

My book and my bluegrass band The TomTown Ramblers were featured in this week’s Talk of The Town in the New Yorker. I’m hoping this means we’ll end the day with more than 34 followers on Twitter.

For the record, we sent them pics of the band but they opted for this illustration instead. Very New Yorkerish.

Other recent mentions of my book:

1. Boston Globe‘s Kate Tuttle interviewed me about my book.
2. I was quoted in the WSJ by Christopher Mims in an article about Facebook and political microtargeting.
3. My book was referred to in this excellent New Yorker article about online discrimination by Tammy Kim.

I’ve been on book tour for nearly a month now, and I’ve come across a bunch of arguments pushing against my book’s theses. I welcome them, because I want to be informed. So far, though, I haven’t been convinced I made any egregious errors.

Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores, and sometimes the recidivism risk scores as well. Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.

If you don’t know what a VAM is, or what my critique about it is, take a look at this post, or read my book. The very short version is that it’s little better than a random number generator.

The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.

Sometimes it’s true that a single inconsistent or badly conceived ingredient in an overall score is diluted by the other stronger and fairer assessment constituents. But I’d argue that this is not the case for how teachers’ VAM scores work in their overall teacher evaluations.

Here’s what I learned by researching and talking to people who build teacher scores. That most of the other things they use – primarily scores derived from categorical evaluations by principals, teachers, and outsider observers – have very little variance. Almost all teachers are considered “acceptable” or “excellent” by those measurements, so they all turn into the same number or numbers when scored. That’s not a lot to work with, if the bottom 60% of teachers have essentially the same score, and you’re trying to locate the worst 2% of teachers.

The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.

But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.

In a statistical sense, even if you allow 50% or more of a given teacher’s score to consist of non-VAM information, the VAM score will still dominate the variance of a teacher’s score. Which is to say, the VAM score will comprise much more than 50% of the information that goes into the score.

An extreme version of this is to think about making the non-VAM 50% of a teacher’s score always exactly the same. Denote it by 50. When we take the population of teacher VAM scores and average them with 50, the population of teacher VAM scores are now between 25 and 75, instead of 0 and 100, but besides being squished into a smaller range, they haven’t changed with respect to each other. Their relative rankings, in particular, do not change. So whoever was unlucky enough to get a bad VAM score will still be on the bottom.

y=(x+50)/2

This holds true for other choices of “50” as well.

A word about recidivism risk scores. It’s true that judges use all sorts of information in determining a defendant’s sentencing, or bail, or parole. But if one of the most trusted and most statistically variant ones is flawed – and in this case racist – then a similar argument to the above could be made, and the conclusion would be as follows: the overall effect of using flawed recidivism risk scores is stronger, rather than weaker, than one might expect given its weighting. We have to be more worried about it, not less.

1. The Vagina Dispatches (must watch video)
2. Massive Yahoo data breach
3. Gaming the Mayo markets
4. Big data meets insurance, there’s an explosion
5. Wells Fargo employees were fired for whistleblowing
6. Low-value customers are NOT always right
7. Amazon does not give customers the best deal
8. Why the mainstream media is ignoring the nationwide prison strike
9. Gun rights are for white people
10. Drowning in systemic injustice

I also discussed a bunch of these topics on this week’s Slate Money.

I’m flying to London Sunday night to conduct my UK book tour. Here’s the schedule so far:

Cambridge University

Date: Tuesday, September 27th

Time: 12:30pm

Place: Faculty of Education, 184 Hills Road, Cambridge

More info: here

London’s How To Academy

Date: Tuesday, September 27th

Time: 6:45pm

Place: CNCFD- Condé Nast College of Fashion & Design, 16-17 Greek Street, Soho, London

More info: here

King’s College London

Date: Wednesday, September 28th

Time: 3pm

Place:  S-2.08, King’s College London, Strand, London

More info: here

In addition to the above, I’ll also be on BBC’s Today Programme on Tuesday morning, and I’ll be interviewed by Significance, the Royal Statistical Society & American Statistical Association magazine, the Guardian Science podcast, and Business Daily for BBC’s The World Service.

I haven’t been posting too often, in part because I’ve been traveling a lot on book tour, and also because I’ve been writing for other things and interviewing quite a bit. Today I wanted to share some of that stuff.

1. I wrote a Q&A for Jacobin called Welcome to the Black Box.
2. I wrote a piece for Slate called How Big Data Transformed Applying to College.
3. Times Higher Education chose my book as their reviewed Book of the Week and had a nice spread about it.

There may be more, and I’ll post them when I remember them.

Also, great news! My book is a best-seller in Canada! Those Canadians are just the smartest.

This article about recidivism risk algorithms in use in Philadelphia really bothers me (hat tip Meredith Broussard). Here’s the excerpt that gets my goat:

“As a Black male,” Cobb asked Penn statistician and resident expert Richard Berk, “should I be afraid of risk assessment tools?”

“No,” Berk said, without skipping a beat. “You gotta tell me a lot more about yourself. … At what age were you first arrested? What is the date of your most recent crime? What are you charged with?”

Let me translate that for you. Cobb is speaking as a black man, then Berk, who is a criminologist and statistician, responds to Cobb as an individual.

In other words, Cobb is asking whether black men are systematically discriminated against by this recidivism risk model. Berk answers that he, individually, might not be.

This is not a reasonable answer. It’s obviously true that any process, even discriminatory processes that have disparate impact on people of color, might have exceptions. They might not always discriminate. But when someone who is not a statistician asks whether black men should be worried, then the expert needs to interpret that appropriately – as a statistical question.

And maybe I’m overreacting – maybe that was an incomplete quote, and maybe Berk, who has been charged with building a risk tool for $100,000 for the city of Philadelphia, went on to say that risk tools in general are absolutely capable of systematically discriminating against black men.

Even so, it bothers me that he said “no” so quickly. The concern that Cobb brought up is absolutely warranted, and the correct answer would have been “yes, in general, that’s a valid concern.”

I’m glad that later on he admits that there’s a trade-off between fairness and accuracy, and that he shouldn’t be the one deciding how to make that trade-off. That’s true.

However, I’d hope a legal expert could have piped in at that moment to mention that we are constitutionally guaranteed fairness, so the trade-off between accuracy and fairness should not really up for discussion at all.

I don’t know about you, but every time I go into the bathroom at Penn Station I cry a little bit.

That’s because I remember the 1980’s version of them, and believe you me, they’re so much better now. I grew up in the Boston area but I visited a bunch in high school, which means I spent way too much time in the very few available public toilet facilities. So I can appreciate me some improved amenities.

They are relatively clean! They have toilet paper, consistently! There’s soap available next to working sinks! And, probably most importantly, it’s not a threatening experience with dirty needles all over the floor.

For that matter, while I’m on the theme, have you noticed how much nicer JFK is now compared to 1988? Maybe it’s because I’ve been flying JetBlue a lot, but that terminal is nothing like the broken-down middle school experience I remember not so fondly.

That’s all I have today, just gratitude and anti-nostalgia. And I’m sure there are lots of things we miss as well from those days of New York City, but right now I can’t think of any besides cheaper rent.

It’s been an amazing two weeks – or actually, holy crap, only 10 days – since my book launched.

I found out two days ago that my book made it onto the Longlist for the National Book Award in Nonfiction, along with 9 other books. I haven’t had time to read the other books, but I did want to mention that last year’s NBA Nonfiction winner was Ta-Nehisi Coates’s excellent Between The World And Me, which I highly recommend.

What’s exciting about being on this list is that it means the ideas in the book will get exposure. So many really excellent books never get read by many people, because of bad timing, or small marketing and publicity budgets, or just bad luck. I’m so lucky to have a book that’s been given an extremely generous amount of all of that.

This week I’ve been busy on the West Coast going to book events and giving talks. My last one is today at noon in Berkeley (820 Barrows Hall). I’ve gotten almost no sleep what with jetlag, weird traveling requirements, and pure adrenaline, but it’s been absolutely incredible.

It’s been especially fantastic to meet the people who come to these events, which so far have taken place in Seattle, the San Francisco area, and a couple in New York last week. It seems like almost every person has something to tell me, a story of algorithms they encounter at work, or that their friends do, or questions about how to get a job that they can feel proud of in data science. Some of them are lawyers offering to talk to me about FOIA law or the Privacy Act. Incredible.

Some people who have read the book already will tell me it really changed their perspective, and others will tell me they’ve been waiting years for this book to be written, because it echoes their experience and long-held skepticism.

What?! Do you guys know what that means? It means the book is working!

In any case I’m overwhelmed and grateful to be able to talk to all of them and to start and continue the conversation. It’s never been more timely, and although I had hoped to get the book out sooner, I actually ended up thinking the timing couldn’t be better.

There’s only one thing. I wish I could send a message back to myself four years ago, when I decided to write the book, or even better, to Christmas 2014, when I was convinced it was an unwritable book. I’d just want to send some encouragement, a signal that it would eventually cohere. Those were some dark days, as my family can attest to.

Luckily for me, I had good friends who kept me from losing all hope. Thank you, blog readers, and thank you friends, and thank you Jordan and Laura especially, you guys are the best!

One of the best things about having my book out (finally!) is that once people read it, or hear interviews or read blogs about it, they sometimes sending me more examples of creepy models. I have two examples today.

The first is a company called Joberate which scores current employees (with a “J-score”) on their risk of looking for a job (hat tip Marc Sobel). Kind of like an overbearing, nosy boss, but in automated and scaled digital form. They claim they’re not creepy because they use only publicly available data.

Next, we’ve got Litify (hat tip Nikolai Avteniev), an analytics firm in law that’s attempting to put automatic scoring into litigation finance. Litify advertises itself thus:

…Litify is led by an experienced executive team, including one of the world’s most influential and successful lawyers, well known VC’s and software visionaries. Litify will transform the way legal services are delivered connecting the firm and the client with new apps and will use artificial intelligence to predict the outcome of legal matters, changing the economy of law. Litify.com will become a household consumer name for getting legal assistance and make legal advice dramatically more accessible to the people…

What could possibly go wrong?

I’ll be giving a version of my book talk at the Alt Banking meeting today. Please come!

Here are the deets:

When: 2-3pm today

Where: Room 409 of the International Affairs Building, 118th and Amsterdam

We discussed my new book, Weapons of Math Destruction, on my Slate Money podcast this week. Take a listen!

Have you heard of the most recent outrage committed by a bank? Wells Fargo just got fined a total of $185 million for corrupt practices involving the accounts of depositors.

Specifically, a bunch of depositors were given accounts they didn’t sign up for, and then charged for via fees. Wells Fargo claims they have fired 5,300 low-level workers over the past five years for doing stuff like this.

But as many have pointed out, including Naked Capitalism, this is really not about low-level workers. It’s about ridiculous and unattainable sales quotas imposed on bankers, and then a complete disregard for the knock-on effects of those stupid quotas.

The fact that so much fraud went on so widely means that either the top dogs knew about it and didn’t care (I’m voting for this – after all the high pressure sales tactics were probably profitable overall even with this $185 fine) or that they had entirely insufficient controls and didn’t know about it. Either way they’re idiots, and it’s outrageous that only the underlings were fired, and not the management. For that matter the person who came up with rigid sales quotas without thinking for five minutes about what would happen next needs to get canned.

Oh wait, I just remembered: the lowest paid bank workers, who really work for very little money under tremendous pressure, have very little power. It’s time they form a union. This is not a new idea, but it’s never been more obvious.

I’ll be live on the Brian Lehrer Show today around 11:15am, talking about my book, maybe with live callers! You can listen to it at 93.9 if you’re in the New York City area, and you can stream it from wnyc.org if not.

Also, tonight I’ll be talking about my book with my buddy Felix Salmon at Barnes & Noble on the upper east side, 86th and Lexington. It starts at 7pm tonight, and more info is here.

The reviews are coming in of my new book, Weapons of Math Destruction (also available as an audiobook, which I read myself! (clip here)). Here are some of them:

1. Weapons of Math Destruction: invisible, ubiquitous algorithms are ruining millions of lives by Cory Doctorow from his blog Boingboing.net
2. Weapons of Math Destruction by Peter Woit from his blog, Not Even Wrong
3. Big Data Isn’t Just Watching You—It’s Making You Poorer by Pankaj Mehta on the site In These Times
4. Review: Weapons of Math Destruction by Evelyn Lamb in Scientific American
5. Math is racist: How data is driving inequality by Aimee Rawlins on CNN Money. Note: I would not have chosen this title, since I’m not claiming math is racist, but rather that some potentially discriminatory practices are being shielded by mathematics. I should note that journalists don’t always choose their own titles, and I think Aimee did a good job with the content of the article
6. On Cathy O’Neil’s Weapons of Math Destruction by Chris Hoofnagle, on his blog at the UC Berkeley Law School
7. Math Is Biased Against Women and the Poor, According to a Former Math Professor by Priya Rao on NYMag’s The Cut

There may be more I’m missing, please send me links! The coverage has been fantastic, and I’m super excited for the coming weeks and months as we finally get to discuss these issues.

Also, I’d like to urge you all to review my book on Amazon when you get a chance. The book is controversial and a few negative reviews can drag down the average pretty quickly. Having said that, please be completely honest of course!

Here’s my favorite graphic, from CNN:

Oh my god, people, today’s the day! I’m practically bursting with excitement and anxiety. I feel like throwing up all the time, but in a good way. I want to go into every bookstore I walk by, find my book, and throw up all over it. That would be so nice, right?

Also, I wanted to mention that Carrie Fisher, who is a SUPREME ROLE MODEL TO ME, has just started an advice column at the Guardian. How exciting is that?! So please, anyone who still mourns the loss of Aunt Pythia, go ahead and take a look, she’s just the best.

Also! I’m into this new report, and accompanying Medium piece, by Team Upturn on the subject of predictive policing. It explains the field in a comprehensive way, and offers a convincing critique as well.

Also! It turns out I’ll be in Berkeley next Friday, here’s the flier thanks to Professor Marion Fourcade:

I hope I see you there!

This morning I stumbled upon a New York Times article entitled How Tech Giants Are Devising Real Ethics for Artificial Intelligence. The basic idea, and my enormously enraged reaction to that idea, is perfectly captured in this one line:

… the basic intention is clear: to ensure that A.I. research is focused on benefiting people, not hurting them, according to four people involved in the creation of the industry partnership who are not authorized to speak about it publicly.

So we have no window into understanding how insiders – unnamed, but coming from enormously powerful platforms like Google, Amazon, Facebook, IBM, and Microsoft – think about benefit versus harm, about who gets harmed and how you measure that, and so on.

That’s not good enough. This should be an open, public discussion.

I’m unreasonably excited that Citi Bike has finally expanded to the area where I live, Columbia University. Here’s the situation:

Specifically, this means I can drop my kid off at school at 110th and Broadway and then bike downtown.

People, this is huge. It means I never have to get on the 1 train at rush hour again! Unless everyone else has the same plan as me, of course.