The arXiv should be supported by the NSF

What the fuck is wrong with the NSF? Why isn’t it supporting the arXiv?

I have been offended and enraged recently to receive pleading emails from members of the hard-working Cornell University Library arXiv Team for money. As in, please give us $5.

This is a ridiculous state of affairs.

Right now arXiv, which hosts preprints from the fields of mathematics, computer science, physics, quantitative biology, quantitative finance, and statistics, plays an absolutely pivotal role in basic research in this country, especially given the expense and time-consuming journal publishing process.

It has an operating budget of less that $1 million per year, and is somehow left begging for personal donations, supplemented by small grants from the Simons Foundation.

If you look at the mission of the National Science Foundation, it’s first part is “to promote the progress of science.” Moreover, it has an annual budget of $7.5 billion. I cannot think of a better way for it to fulfill its mission than to support the maintenance and expansion of the arXiv.

Am I wrong about this? WTF??

Categories: Uncategorized

WMD Audiobook!

Today and yesterday I’m recording the audiobook version of my upcoming book, Weapons of Math Destruction, in a studio in the Random House building near Columbus Circle.

It’s hard work! I’m constantly having to retake sentences, either because I thought my tone was too flat (I hate flat audiobook readers!), or wasn’t emphasizing the right words, or because the words are just hard to say.

Speaking of which, I promise to never, ever write the phrase “assist statistics” in anything that might someday be read out loud, ever, anywhere. And also, you are hereby prohibited from reading this blogpost out loud.

I was pretty worried that the actual content would be bothersome to me – that I’d find tons of typos, or that things would have changed so much that the content is no longer relevant. So far, so good, though, at least to my eyes.

I’m happy with the book! Is that ok to say (not out loud!!)? I’m holding on to this delicious feeling until the nasty reviews come out. After that I’ll just cry inside at all times.

In the meantime, I’ve started a website for the book, including early reviews (i.e. blurbs, including from my buddy Jordan Ellenberg) and one actual review from Publisher’s Weekly, which I’m super happy with.

Categories: Uncategorized

On Being Lane Bryant Fat

There was an amazing This American Life episode that aired earlier this month called Tell Me I’m Fat, centering around 4 stories about how people have dealt with being fat and the obesity epidemic more generally (hat tip Becky Jaffe).

And I plan to respond to all of them in turn, but let me mention right off the top that I didn’t think I had much to learn about this topic, but I learned a lot about this topic from listening to this episode, which was both empathetic and deep.

The first story could have been about me, almost. In short, it was about a woman who spent a bunch of wasted time in her youth worrying about being fat, then eventually she realized she was always going to be fat, that she was sick of apologizing for it and going on diets that didn’t work, and she came to terms with being fat. She owns it. Good for her.

What especially made me nod along was her talking about how she’d prefer the descriptor “fat” than the alternative, “overweight,” which is both a useless euphemism and a judgment, that it was somehow a temporary problem that would soon be fixed. Fuck that.

Oh, and also, she works with Dan Savage, and she called him on his fat shaming. I have always wanted someone to do that.

The second story was super sad, about a woman who was fat at some point but lost a bunch of weight by taking diet pills – basically speed – and found love and a good job by slimming down. She is now married to a man who admitted on tape that he wouldn’t love her if she were fat. She has a job which she claims she needs to be skinny to keep. She’s still taking (black market) diet pills. I am absolutely terrified for her.

The third story was what hit me. It was the story of a very fat woman of color, talking about just how hard it is to be that large. I really do get a lot of what she’s saying, but the more I think about it the more I realize I don’t get it, actually. I mean, I’ve been to restaurants where the chairs have arms and define a butt size that is simply smaller than mine. I have needed to ask for another chair. I have been extremely uncomfortable in an airplane seat.

But I’ve never been unable to fly, nor have I worried about chairs breaking beneath me. This woman does worry about this, and researches restaurants before she goes in case she cannot be accommodated. It’s a different level of humiliation and isolation. Where I feel annoyed that subway seats are too small, she is truly removed from the realm of normal.

She has a name for people like me: Lane Bryant Fat. I’m the woman who, increasingly, can find cute clothes to wear, who can talk about being fit and fat, and who can find company in a larger and larger adult population of women of size 22 or thereabouts.

She’s right, I don’t feel like a freak anymore. When I go to Brooklyn, I actually feel very normal. Even when I was in Paris I didn’t stick out very much, which was certainly very different 20 years ago.

And she’s also right that Lane Bryant Fat women don’t really get here or care about her. When I pass by people as large as she is, I do not regularly relate to them. On a normal day, some little voice inside me, some mean part of me, says, at least I haven’t let myself go that much.

Considering how hard I know I’ve tried in the past to change, you’d think I would be more enlightened about this issue, but until I heard this radio segment, I had never examined my own, internal version of fat shaming. Shame on me.

The last segment was about the Oral Roberts University effort in the 1970’s, I believe, to make its students lose weight as a graduation requirement. This resonated with me deeply, because it was a large scale version of what went on within my home as a child. For a time as a tweenager, I wasn’t given my allowance unless I’d lost enough weight each week. It was cruel, humiliating, and it imbued me with a shame that lasted longer than I’d care to admit.

This was a breakthrough, this radio program. I am so very glad this conversation has begun, and I’m so very glad it included these multiple voices, but it’s really just the beginning.

For example, here’s the thing I’m grappling with right now. I’m living in fear of becoming (type II) diabetic. I’m absolutely high risk for it: my age, my genetics, and my weight all point to it. The only thing I have going for myself is that I exercise regularly, which reduces the risk, but not entirely. So I’m on the lookout, and I’d like to think I’m prepared.

But part of that preparation includes being willing to have gastric bypass surgery, which has become much safer and is an almost miracle cure for type II diabetes. It is, in fact, the treatment of choice according to some international experts.

But at the same time, it’s a diet surgery, and if I underwent the procedure, I could expect to lose a lot of weight. For someone who has spent 20 years establishing a (Lane Bryant) fat identity, it’s actually really confusing to imagine opting for the knife. I’d feel like a turncoat.

Which isn’t to say I’d refuse it. I’ve already checked that my insurance covers the surgery. For BMI up to 40, it covers it if diabetes is present. But given that my BMI is actually above that, I could get the surgery now, without needing to “be sick.” I’m confused by this, and I don’t think I’m alone.

So what about it, This American Life? More episodes, please!

Categories: Uncategorized

Why did the Brexit polls get it so wrong?

The Brexit vote was a huge deal, both politically and economically. Tons of polls have been telling us for weeks that’s it’d be a close contest, but since the murder of Jo Cox’s, they had mostly been pointing one way: namely, to a Remain win.

To be clear, lots of people said it was too close to call, but the bulk of yesterday’s evidence said that Remain would win by 52% to 48%, with a margin of error of around 2%. The actual results were the opposite, Remain lost by 48% to 52%.

Stock markets can also embed beliefs, and in this case they definitely seemed to think Britain would vote to remain in the EU. For that matter, there were plenty of betting markets that allowed people to bet directly on the vote, and as of yesterday the odds were steeply in favor of Remain. Even the early exit polls pointed to Remain.

So, why did all the polls get it so wrong? I have no more information that anyone else, but I have some purely unsubstantiated, backwards-looking guesses:

  1. Older people are much more likely to vote, and they also tended to vote Leave.
  2. People who voted to Leave cared more about the issue.
  3. People lie in polls, and given that the Leave campaign was being accused of racism, it’s maybe easier to lie towards Remain than the other way around. Also could be a reason that more “undecided” voters were secretly planning to vote Leave but didn’t want to say it out loud.
  4. People might have actually put money in the betting markets, including the financial markets, that have nothing to do with their belief of the outcome but rather represents a hedge for another position.
  5. As for the exit polls, they are easier to take in cities, where there are a lot of people, but where there also tend to be more “Remain” voters.

What do you think? Here’s some demographic info from the Guardian that may or may not help:

Screen Shot 2016-06-24 at 7.02.05 AM.png

Categories: Uncategorized

OIG Report: Broken Windows doesn’t work

The Office of the Inspector General for the New York Police Department (OIG-NYPD) issued a report yesterday which used statistical analysis to demonstrate that the “Broken Windows” theory of policing is flawed. From their Recommendations, page 72:

OIG-NYPD found no evidence that the drop in felony crime observed over the past six years was related to quality-of-life summonses or quality-of-life misdemeanor arrests. This suggests that there are other strategies that may be driving down crime. Between 2010 and 2015, quality-of-life enforcement rates – in particular, quality-of-life summons rates – have dramatically declined, but there has been no commensurate increase in felony crime. While the stagnant or declining felony crime rates observed in this six-year time frame may perhaps be attributable to NYPD’s other disorder reduction strategies, OIG-NYPD finds no evidence to suggest that crime control can be directly attributed to quality-of-life summonses and misdemeanor arrests. Whatever has contributed to the observed drop in felony crime remains an open question worthy of further analysis.

The report goes on to say that the NYPD should take a more data driven approach to deciding what’s actually working and what isn’t, and should “conduct an analysis to determine whether quality-of-life enforcement disproportionately impacts black and Hispanic residents, males aged 15-20, and NYCHA residents.”

Very happy about this report, it’s been a super long time coming. The NYPD has said the report is flawed, and will come back with a response within 90 days. I’m looking forward to that as well.

Categories: Uncategorized

Gerrymandering algorithms

I’ve been thinking about gerrymandering recently, specifically how to design algorithms to gerrymander and to detect gerrymandering.

Whence “Gerrymander”?

First thing’s first. According to wikipedia (and my friend Michael Thaddeus), the term “Gerrymander” is a mash-up of a dude named Elbridge Gerry and the word “salamander.” It was concocted when Gerry got made fun of for his crazy districting of Massachusetts back in 1812 to push out the power of the Federalists:

800px-The_Gerry-Mander_Edit

It’s true, this is depicted as a dragon. But believe me, someone thought it looked like a salamander.

 

How To Gerrymander

Think about it. In this crazy pseudo-democratic world of ours, we’re still voting locally and asking delegates to ride their horses to a centralized location to cast a vote for the group. The system was invented well before the internet, and it is a ridiculous and unnecessary artifact from the days when information didn’t travel well. In particular, it means you can manipulate voting at the local level, by gaming the definition of the district boundaries.

So, let’s imagine you’re in charge of drawing up districts, and you want to rig it for your party. That means you’d like your party to win as often as possible and lose as seldom as possible per district. If you think about it for a while, you’ll come up with the following observation: you should win by a thin margin but lose huge.

Theoretically that would mean building districts – a lot of them – that are 51% in your favor, and then other districts that are 100% against you.

In reality, you can’t count on anything these days, so you might want to create slightly wider margins, of maybe 55% your party, and there might be rules about how connected districts must be, so you’ll never achieve 100% loss districts.

 

How Not To Gerrymander

On the other side of the same question, we might ask ourselves, is there a better way? And the answer is, absolutely yes. Besides just counting all votes equally, we could draw up districts to contain similar numbers of voters and to be more or less “compact.”

If you don’t know what that really means, you can go look at the work of a computer nerd named Brian Olsen, who built a program to do just this.

Screen Shot 2016-06-22 at 7.43.40 AM

Before and after Brian Olsen gets his hands on Pennsylvania

 

Detecting Gerrymandering

The concept of compactness is pretty convincing, and has led some to define gerrymandering to be, in effect, a measurement of the compactness of districts. More formally, there’s a so-called “Gerrymander Score” that is defined as the ratio of the perimeter to the area of districts, with some fudge factor which allows for things like rivers and coastlines.

Another approach is a “Gerrymander statistical bias” test, namely the difference between the mean and the median. Here you take the results of an election by district, and you rank them from lowest to highest for your party. So there might be a district that only voted 4% for your party, and it might go on the left end, and on the other end the district that voted 95% for your party would be on the other end. Now look at the “middle” district, and see how much that district voted for your party. Say it’s 47%. Then, if your party won 55% of the vote overall in the state – the mean in this case is 55% – there’s a big difference between 55 and 47, and you can perhaps cry foul.

I mean, this seems like a pretty good test, since if you think back to what we would do to gerrymander, in the ideal world (for us) we’d get a bunch of districts with 45% for the other side and then a few with 99% for the other side, and the median would be 45% even if the other side had way more voters overall.

Problems With Gerrymandering Detection 

There’s a problem, though, which was detected by, among other people, political scientists Jowei Chen and Jonathan Rodden. Namely, if you run scenarios on non-gerrymandered, compact districts, you don’t get very “fair” results as defined by the above statistical bias test.

This is because, in reality, Democrats are more clustered than Republicans. Democrats are quite concentrated in cities and college towns, and then they are more sparse elsewhere. They, in essence, gerrymander themselves.

Said another way, if you build naive districts that are compact (so their Gerrymander Scores are good) then there will be automatic “Gerrymander statistical bias” problems. Oy vey.

Which is not to say that there isn’t actual effective and nasty Gerrymandering going on. There is, in North Carolina, Florida, Pennsylvania, Texas and Michigan for the Republicans and in California, Maryland and Illinois for the Democrats.

But what it means overall is that there’s no reason to believe we’ll ever get out of this stupid districting system, because it gives an inherent advantage to Republicans. So why would they agree to tossing it?

Categories: Uncategorized

Adding another layer of cynicism to the “smart” city revolution

There was an interesting essay by Jacob Silverman in the New York Times last week called Just How ‘Smart’ Do You Want Your Blender to Be? (hat tip Ernie Davis).

In it, Silverman makes a few really great points. First, that we are sold “smart” products like the Nest thermostat or the my.Flow tampon or cell phones, and all we really get is surveillance and a lack of control over our own stuff (because it’s called “hacking” if we try to fiddle with our phones). Plus he goes into the old English definition of smart – a verb meaning “causing sharp pain,” as in “that smarts!” – as another reason that we might not want a smart version of everything.

All true, but I think there’s a couple of important points missing in his narrative. Specifically, I don’t think he was being sufficiently cynical.

First of all, something very old-fashioned is going on. Namely, people are marketing things as “smart” because they want to claim they have new products so they can then claim they have a business.

In other words, I’m imagining 95% of the smart products were invented like this: hey, let’s figure out an old product we can add sensor technology to and then sell it like it’s a new product. Blenders? Tampons? Can we add sensors to tampons to figure out when the tampon is drenched in blood? WOULD PEOPLE PAY FOR THAT?

The answer is, generally, no fucking way, but there’s too much money in Silicon Valley for that answer to be heard. So these ridiculous companies keep making their ridiculous, unnecessary products.

The second and more cynical point is focused on the “smart city” fad. Silverman rightly points out that we now have free “smart” wifi kiosks all over NYC but we still don’t have public bathrooms. I’d add that we have “smart” tampons but still don’t make normal tampons available to, for example, homeless women or high school girls.

That’s totally fucked up in both cases, but it’s not because we are in love with the concept of “smart” technology. It’s because that’s where the money is.

Some entrepreneurs have decided that it’d be a smart shrewd investment to install a bunch of wifi kiosks in old telephone booths, grab everyone’s telephone data as they walk by, profile them, and then tailor advertisements to them. They expect to make a big profit from this investment, which you can infer reading the misleading answer to the question, “how is LinkNYC Funded?” on their website:

Screen Shot 2016-06-20 at 7.15.57 AM

Can we all agree that’s not what most people mean when they ask, how is this funded? This is more like an answer to the question, how will this make money? Please tell me if you can figure out who actually funds LinkNYC.

Let’s think, by contrast, about how public bathrooms work. They cost money to build and to maintain. And yes, having a good public bathroom system would prevent a lot of nasty things from happening, like a bunch of arrests of people for being poor and having no place to pee, not to mention poop on the street. It’s the right thing to do. But, since we don’t think we will ever get more than half a billion dollars in revenue from them, no thanks.

Similarly, it’s not profitable to give poor women tampons, it’s merely the right thing to do (and it boosts attendance rates of poor girls).

The smart city fad is a boondoggle, a way of giving public space access to private advertising companies in exchange for a few nickels and a surveillance state. It’s kind of ingenious, because it’s in a sense selling a commodity – public space, and the concept of anonymity within a large city – that we didn’t even know we could measure, never mind commoditize. And in the meantime, we cannot expect actual ongoing problems to be addressed. Because we’re too busy being smart.

Categories: Uncategorized
Follow

Get every new post delivered to your Inbox.

Join 3,877 other followers