Readers, I’ve got two announcements today.

First, I’ll be giving a TED talk in April in Vancouver. And yes, for those of you remember, I haven’t always been the biggest fan of such things. But I’ve changed my mind/ sold out/ decided that it might just be great.

As a friend of mine explained to me, sometimes things get so douchey they come out the other side and are super cool. Also, I’m giving a talk in the section called Our Robotic Overlords, so that’s a very good sign.

Second, I’ve decided to undergo bariatric surgery. I’m jumping through the many insurance-qualifying hoops for now but if all goes well it will happen later this year, possibly as soon as July.

And… I’m planning to chronicle my journey on mathbabe. If that kind of thing doesn’t interest you, feel free to never come back, but if that kind of thing does interest you, then buckle up!

I’m not planning to keep myself to the subject of the bariatric surgery; in fact that’s just an excuse to think about a lot more, specifically:

• the nature of scientific understanding and how it does or does not percolate throughout society as a whole,
• how money and shame corrupt our understanding of scientific evidence,
• how bad data and bad technologies and biased academic publishing prevent us from learning optimally,
• the nature of individual choice, willpower, and control,
• my historical self-image as a dieter, a fat person, a woman, a feminist, and a thinker,
• how I gathered evidence and made this decision, and of course
• the process itself.

So I’m thinking kind of big and I’m going to have fun with it. Please feel free to comment, I’d love your help!

My newest Bloomberg View column:

How Data Can Make Immigrants Look Like Criminals

My newest piece on Bloomberg:

Bigger Data Isn’t Always Better Data

My newest Bloomberg View piece about how that FitBit could be bad for your health:

That Free Health Tracker Could Cost You

1. I wrote about how big data is undermining our understanding and faith in historical facts and in statistics in my newest Bloomberg column, Do You Trust Big Data? Try Googling the Holocaust
2. Last week this Vice piece came out, which I contributed to along with lots of writers I really admire like Astra Taylor, on how technology can be made to work for us: Man Versus Machine
3. My buddy Paul-Olivier Dehaye is on fire over at medium.com with his newest approach to disrupting the big data surveillance state. He now has devised a way to request your file from Cambridge Analytica, and I’m totally doing this: Quick guide to asking Cambridge Analytica for your data

New Bloomberg View column:

Meet Facebook, Your New Financial Regulator

Next Wednesday evening I’ll be talking data, democracy, and the news with the amazing Julia Angwin at the NYU Journalism School moderated by Robert Lee Hotz. More information here.

Please come! Or if you can’t come, you can watch the livestream.

How do you get rid of the influence of Steve Bannon’s whispering in Trump’s ear? The best strategy I’ve heard is to make Trump jealous of the attention. And one way to do that is to refer to Bannon as the president.

The hashtag #PostcardsToBannon blew up on Twitter yesterday, with all sorts of people posting pics of their postcards:

From Justin Hendrix via Twitter

In fact, it got so much attention that it was featured overnight on USA Today.

It’s a small act but it might make you feel great to do it.

I have a new fun piece over at Bloomberg this morning:

Donald Trump is the Singularity

My newest Bloomberg post is out, in response to this article about Cambridge Analytica:

Trump’s ‘Secret Sauce’ Is Just More Ketchup

This is a guest post by Elizabeth Hutchinson, an Associate Professor of Art History at Barnard College/Columbia University who supports social justice initiatives at work and in her community.  She is also a yarn whisperer who likes nothing better than knitting with Mathbabe.

If you are a regular reader of Mathbabe, you may already be putting your time, money and intellectual labor to work in support of organizations that defend the rights of vulnerable groups and our vulnerable environment (#BlackLivesMatter, Make the Road New York, Planned Parenthood, SURJ, 350.org, NYCStandswithStandingRock, and many others).

But if you are a New York City resident, here’s another practical thing you can do: apply for an ID NYC card.

ID NYC is a program established by the de Blasio administration in 2014 that allows city residents to obtain a photo identification without requiring the same government-generated documents required for a drivers license or passport. These residents then have a municipal ID that can help them open bank accounts, apply for library cards and gain access to other services as well as free membership to a range of NYC cultural institutions like the Museum of Modern Art.

In lieu of a Social Security card or equivalent document, applicants for the ID NYC could use non-U.S. government-generated forms of identification, including, among other things, a combination of a utility bill verifying a local address and a foreign passport or consular identification.

Even if you have a photo ID and a library card, here’s why you should get an ID NYC: this program is widely used by the undocumented immigrants in our midst, and the records of their applications are vulnerable to seizure by federal government authorities charged with expanding the pursuit of both undocumented and documented immigrants.

How is this so, you might ask, knowing that New York is a sanctuary city? Well, it is true that New York is committed to not aiding Immigration and Customs Enforcement (ICE) in a number of ways. For example, it has pledged not to use its city precincts or jails to house immigrants detained by Immigration and Customs Enforcement (though it does cooperate when ICE requests individuals already in NYC custody who were convicted of a serious felony) and to not share city agency information with federal immigration authorities.

Sanctuary Cities according to this site. For a more complete list click here.

The ID NYC program was set up to be in line with this stance: the law establishing the program ordered that the copies of documents used in applying for the ID be destroyed at the end of the first two years, or in December 2016, in the meantime only sharing them with law enforcement only through judicial subpoena (something that happened only a handful of times). However,  a case brought by Republican members of the State Assembly from Staten Island in December resulted in a ruling that all records be retained indefinitely.

After Trump’s election, Mayor de Blasio pledged to change the record keeping system and stop retaining copies of the applicants’ documents beginning in 2017. However, the city will continue to retain significant information about applicants, including their name, gender, address, birthdate, and the photo taken when the id was made.

The ID NYC program DOES NOT ask applicants about their immigration status. Nevertheless, because this program is well used by members of New York’s immigrant communities (according to the Gothamist, over a third of NYC residents are foreign-born), these applications could be used for fishing expeditions looking for our undocumented neighbors.

Yes, the Mayor has pledged to fight to keep this paperwork private. But we can’t be sure how the courts will act when push comes to shove.

The solution? Gum up the works.

Blast the program with lots and lots of applications from NYC residents so that any authority that does manage to subpoena applications has an immense archive to wade through. Estimates suggest that about 1 million people have applied for ID NYC to date. That leaves about 6.8 million New Yorkers who still can. (Yes, kids can apply, too, as long as they are 14.)

Applying is easy, though it will take you a little time. You start by making an appointment at one of the 25 enrollment centers. There’s a form to fill out (applications are available in more than 25 languages), that you can do ahead of time and print out or fill out when you get there. Bring along your documents. Once you check in, you wait for an agent to go over the application and take your picture and then you can arrange to receive the id in the mail or pick it up. I got mine at the Mid-Manhattan Library. I made the appointment about a month ahead of time, though there were appointments sooner, and waited less than an hour to see the agent. It was about as much hassle as mailing a package at the post office.

Maybe this isn’t the most effective form of resistance, but it is an easy one that may do some good.

I look forward to seeing you in the streets. And the public library. And MoMA.

To report incidents of discrimination or hate

• The Governor’s Office – 1-888-392-3644
• The Mayor’s Office of Immigrant Affairs 311 or 212-788-7654. Translation is available. You can also go to www1.nyc.gov for many other resources for NYC immigrants.

Additional Resources

• ImmigrationLawHelp.org – Helps low-income immigrants find legal help.
• National Immigration Law Center: Explains your rights, no matter who is president.
• New York Immigrant Coalition and
• Make the Road: Provide policy updates and resources to support immigrants in NYC
• New York Communities for Change
• Causa Justa/Just Cause

I was excited to join the protest at JFK Airport last night. Here’s some footage:

And here’s two nice pictures:

One of the cool things about the protest is how messages were sent and spread through the chants. In particular I learned about another planned protest today at Battery Park at 2pm, which I believe is being organized by immigrant rights group Make the Road.

More information available here.

By the way, in case you’ve heard that a judge put a stay on the Executive Order about immigrants, there are plenty of reasons to question that. It’s also possible that border patrol agents are not obeying those orders.

Hey all, my second column came out today on Bloomberg:

When Algorithms Come  for Our Children

Also, I reviewed a book called Data for the People by Andreas Weigend for Science Magazine. My review has a long name:

A tech insider’s data dreams will resonate with the like-minded but neglect issues of access and equality

Great news! I’m now a Bloomberg View columnist. My first column came out this morning, and it’s called If Fake News Fools You, It Can Fool Robots, Too. Please take a look and tell me what you think!

I finally got around to knitting my first pussyhat yesterday, during the inauguration. It took less than two hours because I was using super bulky yarn and because I had lots of anxious energy to tap into.

I got the yarn last Saturday, when I went to a Black Lives Matter march in the morning (you can see my butt multiple times in the embedded video) and then afterwards to Vogue Knitting Live in the Times Square Marriott Marquis.

Pink for a reason.

Fancy elevators.

And here’s the thing, I thought I was going to enjoy the juxtaposition of activist-to-insane hobbiest, but I was wrong – knitters were activists too! Here’s what I saw:

Pink yarn everywhere.

Pussyhats everywhere

Not only women of course! Alex looks dashing with his ombre pussyhat.

Karida Collins doesn’t have a pussyhat on but she’s still killing it.

Since last weekend, I’ve been seeing pussyhats everywhere. You go into a yarn store and here’s what you see.

Or you happen upon an airplane full of women heading to D.C. and here’s what you see.

I’m pretty sure half those women have knitting needles in their laps.

My favorite way to measure this phenomenon is directly, at the source. I am of course referring to Ravelry, the online social media website for knitters and crafters. The pussyhat project has spawned all sorts of creative ideas, of course.

The original pattern has thousands of associated projects

Lots of variations have been invented of course

Here’s a great example

Not particularly cat-like but I like it

Now that it’s happened, it’s obvious that knitters are a perfect community for activism. We’re friendly, community-oriented, and desperate for an opportunity to make something and give it away. Because it gives us an excuse to buy more yarn.

Anyhoo, I’m going to the Women’s March NYC today with mine, and I’m going to try to knit at least one more before I leave at 11am. See you there!

This is a continuation of a discussion I’ve been having with myself about the various definition of fairness in scoring systems. Yesterday I mentioned a recent paper entitled Inherent Trade-Offs in the Fair Determination of Risk Scores that has a proof of the following statement:

You cannot simultaneously ask for a model to be well-calibrated, to have equal false positive rates for blacks and whites, and to have equal false negative rates unless you are in the presence of equal “base rates” or a perfect predictor.

The good news is that you can ask for two out of three of these. Here’s a picture of a specific example of this, where I’ve simplified the situation so there are two groups of people being scores, B and W, and they each can be scored as either empty or full, and then the reality is that could either be empty or full. They have different “base rates,” which is to say that in reality, a different proportion of the B group is empty (70%) than the W group (50%). We insist, moreover, that the labeling scheme is “well-calibrated”, so the right proportion of them are labeled empty or full. I’ve drawn 10 “perfect representatives” from each group here:

In my picture, I’ve assumed there was some mislabeling – there’s a full in the empty bin and there are empties in the full bin. Because we are assuming the model is well-calibrated, every time we have one kind of mistake we have to make up for that mistake with exactly one of the other type. In the picture there’s exactly one of each mistake for both the W group and the B group, so that’s fine.

Quick calculation: in the picture above, the “false full rate”, which we can think of as the “false positive rate,” for B is 1/3 = 33% but the “false positive rate” for W is 1/5 = 20%, even though they each have only one mislabeled representative each.

Now it’s obvious that, theoretically, the scoring system could adjust the false positive rate for B to match that of W, which would mean having 3/5 of a representative be mislabeled. But again, that’d mean we would need only 3/5 of a representative be mislabeled in the empty bin as well.

That’s a false negative rate for B of 3/35 = 8.6% (note it used to be 1/7 = 14.3%). By contrast the false negative rate for A stays fixed at 1/5 = 20%.

If you think about it, what we’ve done is sacrificed some false negative rate balance for a perfect match on the false positive rate, while keeping the model well-calibrated.

Applying this to recidivism scores, we can ask for the high scores to reflect base rates for the populations, and we can ask for similar false positive rates for populations, but we cannot also ask for false negative rates to be equal. That might be better overall, though, because the harm that comes from unequal false positive rate – sending someone to jail for longer – is arguably more toxic than an unequal false negative rate, which means certain groups are let off the hook more often than the others.

By the way, I want to be clear that I don’t think recidivism risk algorithms should actually be the goal, summed up in this conversation I had with Tom Slee. I’m not even sure why their use is constitutional, to tell the truth. But given that they are in use, I think it makes sense to try to make them as good as possible, and to investigate what “good” means in this context.

First, I think I over-reacted to automated pricing models (thanks to my buddy Ernie Davis who made me think harder about this). I don’t think immediate reaction to price changes is necessarily odious. I do think it changes the dynamics of price optimization in weird ways, but upon reflection I don’t see how they’d necessarily be bad for the general consumer besides the fact that Amazon will sometimes have weird disruptions much like the flash crashes we’ve gotten used to on Wall Street.

Also, in terms of the question of “accuracy versus discrimination,” I’ve now read the research paper that I believe is under consideration, and it’s more nuanced than my recent blog posts would suggest (thanks to Solon Barocas for help on this one).

In particular, the 2011 paper I referred defines discrimination crudely, whereas this new article allows for different “base rates” of recidivism. To see the different, consider a model that assigns a high risk score 70% of the time to blacks and 50% to whites. Assume that, as a group, blacks recidivate at a 70% rate and whites at a 50% rate. The article I referred to would define this as discriminatory, but the newer paper refers to this as “well calibrated.”

Then the question the article tackles is, can you simultaneously ask for a model to be well-calibrated, to have equal false positive rates for blacks and whites, and to have equal false negative rates? The answer is no, at least not unless you are in the presence of equal “base rates” or a perfect predictor.

Some comments:

1. This is still unsurprising. The three above conditions are mathematical constraints, and there’s no reason to expect that you can simultaneously require a bunch of really different constraints. The authors do the math and show that intuition is correct.
2. Many of my comments still hold. The most important one is the question of why the base rates for blacks and whites are so different. If it’s because of police practice, at least in part, or overall increased surveillance of black communities, then I’d argue “well-calibrated” is insufficient.
3. We need to be putting the science into data science and examining questions like this. In other words, we cannot assume the data is somehow fixed in stone. All of this is a social construct.

This question has real urgency, by the way. New York Governor Cuomo announced yesterday the introduction of recidivism risk scoring systems to modernize bail hearings. This could be great if fewer people waste time in jail pending their hearings or trials, but if the people chosen to stay in prison are chosen on the basis that they’re poor or minority or both, that’s a problem.

There’s a fascinating article on the FT.com (hat tip Jordan Weissmann) today about how algorithms can achieve anti-competitive collusion. Entitled Policing the digital cartels and written by David J Lynch, it profiles a classic cinema poster seller that admitted to setting up algorithms for pricing with other poster sellers to keep prices high.

That sounds obviously illegal, and moreover it took work to accomplish. But not all such algorithmic collusion is necessarily so intentional. Here’s the critical paragraph which explains this issue:

As an example, he cites a German software application that tracks petrol-pump prices. Preliminary results suggest that the app discourages price-cutting by retailers, keeping prices higher than they otherwise would have been. As the algorithm instantly detects a petrol station price cut, allowing competitors to match the new price before consumers can shift to the discounter, there is no incentive for any vendor to cut in the first place.

We also don’t seem to have the legal tools to address this:

“Particularly in the case of artificial intelligence, there is no legal basis to attribute liability to a computer engineer for having programmed a machine that eventually ‘self-learned’ to co-ordinate prices with other machines.