mathbabe

If you’re wondering why I don’t write more blog posts, it’s because I’m writing for other stuff all the time now! But the good news is, once those things are published, I can talk about them on the blog.

1. I wrote a piece about the Facebook algorithm versus democracy for Nova. TL;DR: Facebook is winning.
2. Susan Landau and I wrote a letter to respond to a bad idea about how the FBI should use machine learning. Both were published on the LawFare blog.
3. The kind folks at Data & Society met up, read my book, and wrote a bunch of fascinating responses to it.
4. Nikhil Sonnad from Quartz published a nice interview with me yesterday and brought along a photographer.

Now that I’m not on Facebook, I have more time to listen to music. Here’s what I’ve got this morning. First, Fare Thee Well from the soundtrack to Llewyn Davis:

Second, my son’s favorite song to sing wherever he happens to be, a Bruno Mars song called Count on Me:

Finally, one of my favorite songs of my favorite band, First Day of My Life by Bright Eyes:

I don’t need to hear from all you people who never got on Facebook in the first place. I know you’re already smiling your smug smile. This story is not for you.

But hey, you people who are on Facebook way too much, let me tell you my story.

It’s pretty simple. I was like you, spending more time than I was comfortable with on Facebook. The truth is, I didn’t even go there on purpose. It was more like I’d find myself there, scrolling down in what can only be described as a fetid swamp of echo-chamber-y hyper partisan news, the same old disagreements about the same old topics. So many petitions.

I wasn’t happy but I didn’t really know how to control myself.

Then, something amazing happened. Facebook told me I’d need to change my password for some reason. Maybe someone had tried to log into my account? I’m not sure, I didn’t actually read their message. In any case, it meant that when went to the Facebook landing page, again without trying to, I’d find myself presented with a “choose a new password” menu.

And you know what I did? I simply refused to choose a new password.

Over the next week, I found myself on that page like maybe 10 times, or maybe 10 times a day, I’m not sure, it all happened very subconsciously. But I never chose a new password, and over time I stopped going there, and now I simply don’t go to Facebook, and I don’t miss it, and my life is better.

That’s not to say I don’t miss anything or anyone on Facebook. Sometimes I wonder how those friends are doing. Then I remember that they’re probably all still there, wondering how they got there.

I’m happy to report that my book was reviewed in the New York Review of Books along with Ariel Ezrachi and Maurice E. Stucke’s Virtual Competition: The Promise and Perils of the Algorithm-Driven Economy, by Sue Halpern.

The review is entitled They Have, Right Now, Another You and in it she calls my book “insightful and disturbing.” So I’m happy.

Yesterday I heard a segment on WNYC on the effort to decriminalize box cutters in New York State. I guess it’s up to Governor Cuomo to sign it into law.

During the segment we hear a Latino man who tells his story: he was found by cops to be in possession of a box cutter and spent 10 days in Rikers. He works in construction and having a box cutter is literally a requirement for his job. His point was that the law made it too easy for people with box cutters to end up unfairly in jail.

It made me wonder, who actually gets arrested for possession of box cutters? I’d really like to know. I’m guessing it’s not a random selection of “people with box cutters.” Indeed I’m pretty sure this is almost never a primary reason to arrest a white person at all, man or woman. It likely only happens to people after being stopped and frisked for no particular good reason, and that’s much more likely to happen to minority men. I could be wrong but I’d like to see those stats.

It’s part of a larger statistical question that I think we should tackle: what is the racial discrepancy in arrest rates for other crimes, versus the population that actually commits those other crimes? I know for pot possession it’s extremely biased against blacks:

On the other end of the spectrum, I’d guess murder arrests are pretty equally distributed by race relative to the murdering population. But there’s all kinds of crimes in between, and I’d like some idea of how racially biased the arrests all are. In the case of box cutters, I’m guessing the bias is even stronger than for pot possession.

If w had this data, a statistician could mock up a way to “account for” racial biases in police practices for a given crime record, like we do in polling or any other kind of statistical analysis.

Not that it’s easy to collect; this is honestly some of the most difficult “ground truth” data you can imagine, almost as hard as money in politics. Still, it’s interesting to think about.

Denial. This happened to all of us at first, even people who voted for him. We couldn’t believe it, we were living through cognitive dissonance. We’d wake up in the morning wondering why they were referring to inane tweets on NPR, suddenly realize at lunch time that the Consumer Financial Protection Bureau will probably never do anything significant again. Binge-watching West Wing helped me sustain this stage. They were so damn patriotic and good. Their integrity and well-meaning-ness were leaking onto everything, and although they didn’t have gay marriage, they were progressing towards it, not backing down from it.

Anger. This stage hit me a few days after the election, in spite of my West Wing strategy. It was a rainy, cold day, and everyone I saw on the street looked absolutely pissed. People were bumping into each other more than usual, partly because of the umbrella traffic, partly on purpose. It was dumb rage, as anger always is. Nobody understood what the point was of being there, they just wanted to get home, to eat muffins, to smoke a damn cigarette. I came very close to picking up smoking that day.

Bargaining. For many people, this stage is still happening. I want to snap them out of it, out of the idea that the recounts will work or that the electoral college system will be changed or that electoral college delegates will refuse to do their job. It’s not gonna happen people, and Jill Stein can please stop. And it’s not that I don’t want to recount stuff – why not? – it’s just that the dying hope that it will change the outcome is sad to witness.

Depression. The problem with calling it depression is that people who are realistic, rather than overly optimistic, seem depressed. I’ve got to admit, I was much more prepared for this than most New Yorkers I know. I think it’s because I’ve been in war mode since joining Occupy in 2011. I never thought Hillary would win, that she was a good candidate, or that people’s resentment and anger had been properly addressed. I’ve basically been here, poised for this moment, since Obama introduced HAMP as a shitty and insufficient way to address the financial crisis back in 2009. So you can call it depression, I just call it reality.

Acceptance. And by acceptance I do not mean “normalization.” By acceptance I mean it’s time to move forward, to build things and communities and organizations that will protect the most vulnerable in post-fact America. That could mean giving money, but it should also mean being an activist and coming up with good ideas, like these churches offering sanctuary to undocumented migrants. It also might mean occupying the democratic party – or for that matter, some other party – and reimagining it for the future. Acceptance is not passive, not in this case. Acceptance means accepting our roles as activists and getting shit done.

This morning I’m in the New York Times, having written a short opinion piece on the following Facebook-centered theme:

How to Stop the Spread of Fake News

My actual opinion is entitled Social Media Companies Like Facebook Need to Hire Human Editors.

Tell me what you think!

I had a great time this weekend in Miami, attending the delightful Miami Book Festival with other longlisters (and winners!) of the National Book Award. We each read for about 5 minutes. Here’s a picture of me perched on the edge of the stage Saturday afternoon, getting ready to read, with many cool non-fiction writers:

Before my reading the wonderful Karan Mahajan brought me to a graffiti art area called Wynwood Wall and we were amazed by spray painted walls:

I was supposed to go to a party after that but I made a detour to South Beach, hanging out with the amazing Jonathan Rabb at the Clevelander:

After about 3 mojitos and many many performance artists I fell asleep at about 8pm.

Conclusion: I’m not cool enough for cool things like Miami, but I had a great time.

There’s lots of post-election talk that Facebook played a large part in the election, despite Zuckerberg’s denials. Here are some the various theories going around:

1. People shared fake news on their walls, and sometimes Facebook’s “trending algorithm” also messed up and shared fake news. This fake news was created by Russia or by Eastern European teenagers and it distracts and confuses people and goes viral.
2. Political advertisements had deep influence through Facebook, and it worked for Trump even better than it worked for Clinton.
3. The echo chamber effect, called the “filter bubble,” made people hyper-partisan and the election became all about personality and conspiracy theories instead of actual policy stances. This has been confirmed by a recent experiment on swapping feeds.

If you ask me, I think “all of the above” is probably most accurate. The filter bubble effect is the underlying problem, and at its most extreme you see fake news and conspiracy theories, and a lot of middle ground of just plain misleading, decontextualized headlines that have a cumulative effect on your brain.

Here’s a theory I have about what’s happening and how we can stop it. I will call it “engagement proxy madness.”

It starts with human weakness. People might claim they want “real news” but they are actually very likely to click on garbage gossip rags with pictures of Kardashians or “like” memes that appeal to their already held beliefs.

From the perspective of Facebook, clicks and likes are proxies for interest. Since we click on crap so much, Facebook (and the rest of the online ecosystem) interprets that as a deep interest in crap, even if it’s actually simply exposing a weakness we wish we didn’t have.

Imagine you’re trying to cut down on sugar, because you’re pre-diabetic, but there are M&M’s literally everywhere you look, and every time you stress-eat an M&M, invisible nerds exclaim, “Aha! She actually wants M&M’s!” That’s what I’m talking about, but where you replace M&M’s with listicles.

This human weakness now combines with technological laziness. Since Facebook doesn’t have the interest, commercially or otherwise, to dig in deeper to what people really want in a longer-term sense, our Facebook environments eventually get filled with the media equivalent of junk food.

Also, since Facebook dominates the media advertising world, it creates feedback loops in which newspapers are stuck in the loop of creating junky clickbait stories so they can beg for crumbs of advertising revenue.

This is really a very old story, about how imperfect proxies, combined with influential models, lead to distortions that undermine the original goal. And here the goal was, originally, pretty good: to give people a Facebook feed filled with stuff they’d actually like to see. Instead they’re subjected to immature rants and conspiracy theories.

Of course, maybe I’m wrong. I have very little evidence that the above story is true beyond my experience of Facebook, which is increasingly echo chamber-y, and my observation of hyper-partisanship overall. It’s possible this was entirely caused by something else. I have an open mind if there were evidence that Facebook’s influence on this system is minor.

Unfortunately, Facebook’s data is private and so I cannot audit their algorithm for the effect as an interested observer. That’s why I’d like to be brought in as an outside auditor. The first step in addressing this problem is measuring it.

I already have a company, called ORCAA, which is set up for exactly this: auditing algorithms and quantitatively measuring effects. I’d love Facebook to be my first client.

As for how to address this problem if we conclude there is one: we improve the proxies.

This is an anonymous guest post.

I am a member of Cathy’s Occupy group, and like a lot of people, had a really bad week. By Sunday I thought I was feeling better. It seemed some of the sadness and shock had passed, and I was developing a resolve about how to move forward.

Then I had a really weird experience Sunday night. I came into the City to attend a black-tie event at the Waldorf in support of an organization I really like, even if it raises a lot of its money from the .001%.

As a labor lawyer and Occupier, it is not my crowd. But I usually find the event amusing. They serve sushi for cocktails and pour Makers Mark into wine glasses, like it is wine.

I walked in and immediately felt strange, actually felt really sick. It was like being in an historical re-enactment, precisely because everything was the same. I went for the Makers Mark early. It only made the disembodied feeling worse. Nothing, nothing, had changed from years past. The beautiful young women in the exquisite dresses were the same. The conversation among the supremely confident looking men seemed the same.

I was not the same.

I got another drink and went to my table. Then, like everyone else, I rose for the National Anthem.

I started feeling super weird though, because everyone else was carrying on so completely normally. I thought of kneeling like Collin Kaepernick, but figured my wife would kill me. Then came “My Country Tisethy” and the room just started swirling.

I sat down and took a breath. They started introducing the first honoree: Hank Greenberg. Yes, the guy in charge of AIG until shortly before it blew up the world economy. The guy who sued the government alleging that the $182 billion bailout his company got was on inadequately advantageous terms. In other words, one of the guys most responsible for elite behaviors that led to this most awful eruption of fear, resentment and hate, that led to last Tuesday.

And all I heard was the introduction of him as a “Great American”.

I walked briskly out of the room, then ran fast through the hotel halls and down Park Ave to my car. Where I sat, for I don’t know how long, and just cried. Cried like a fucking baby. Cried for having to look out my car window at what seemed now like an unfamiliar place, cried for the kid in the Bronx who doesn’t know yet about the threat people think he poses to “law and order,” cried for the family in some far off country that doesn’t know about the charade war coming their way to assuage an angry people losing its collective mind over broken empty promises, cried for all the people who, after I’m gone, will live on a chaotic planet my purposefully ignorant country cooked. Damn, I cried hard.

Then I stopped. And I felt a lot better.

It feels so weird to share this publicly, because it is really embarrassing. I really did all that. But I figured out what it was and wanted to say it out loud. It is moral injury. It is real. It hurts. It can make you cry. Don’t try to pretend otherwise. But also take solace that the only way to treat it is to do good anyway.

There are going to be a lot of opportunities. But we should not get out-imagined again, as I surely was. We should shoot really high this time, be really creative about the good we can do.

For example, if you think our national government will remain awful for a long time, you are probably right. So think locally and globally. What stops us from creating real “sanctuary cities, ” ones that are sanctuaries in such a wider sense, to all of the people he has declared hated or who otherwise just reject him? And why cant we make contact, seek advice, and give aid to the 99.9% of the world that is far more affected by this than us, and got no vote? Again, they are the ones who will otherwise get bombed when he starts dumb wars to distract from his mindless policies; and get drowned and fried when he turns up the temperature on the already sizzling planet.

And remember, 2017 is an election year in New York City. Yes, there is another election coming up which it would feel really good to WIN. Let’s demand better. The Left has never said “think nationally” … no, it is has always been Local and Global.

Feel the pain; it is real; cry; and then gather a stronger opposing force to treat it by occupying the spaces that remain up for the taking.

The Netherlands’ Vrij Nederland, written by Gerard Janssen:

Germany’s Die Tageszeitung, written by Ingo Arzt:

Israel’s Calcalist, written by Uri Pasovsky:

This is a guest post by Tom Adams, who spent over 20 years in the securitization business and now works as an attorney and consultant and expert witness on MBS, CDO and securitization related issues.

I don’t expect anyone to really come up with the perfect explanation for why Clinton lost and Trump won the presidential election.  But I do spend some time looking at these maps:

The first map is from RealtyTrac, and indicates the states with the largest foreclosure inventory in 2012. The second is a map of the key battleground states. In 2008 and 2012, Obama won these states. In 2016 Clinton lost them. There’s a lot of similarities between those two maps.

Even in the best economic environment, residential mortgage foreclosure is a long, messy process. The massive wave of foreclosures that hit these regions after the financial crisis had enormous consequences economically. They also had a tremendous, painful impact on the families and neighborhoods of the people affected, directly and indirectly by the foreclosures.

A rise in the number of suicides have been tide to the wave of foreclosures. Large swaths of neighborhoods were plagued by falling property values, blighted abandoned homes and a sense of uncertainty and, perhaps, doom. I often think about the effect the foreclosure crisis had on the children of affected families and the impact of children watching families, neighbors and classmates going through the painful process.

I was involved, to a small degree, with homeowners, activists and lawmakers that tried to deal with the issues and problems in the foreclosure crisis, some of which is documented in David Dayen’s excellent new book, “Chain of Title“. As Dayen documents, the government response to the issues was ultimately terribly unsatisfying and at best, had the effect of sweeping the issue under the carpet.

The consequences of the government’s response played out in this presidential election.

Clinton was aware of the problems caused by the wave of foreclosures: last fall the NY Times reported that the campaign was frustrated that the crisis had displaced so many homeowners that their database of voters was disrupted. Perhaps this is why the campaign’s get out the vote efforts in Michigan, Wisconsin, Minnesota and other states were much less effective than the campaign had hoped for. Some reports were that up to [25%] of the voters the campaign contacted were actually Republicans or potential Trump voters. In fairness, Clinton was probably concerned about the economic plight of affected homeowners and communities than she was about the technological issues it caused, but that was hardly the dominant campaign message.

How much of an impact would a compassionate outreach have had on these neighborhoods? It’s also worth remembering that the people hit by the foreclosure crisis were generally middle class – prior to the crisis they owned homes, held jobs, were members of the community. Where were they by the time the 2016 election came around?

Certainly, it’s a complicated issue and made more complicated by the fact that the Obama Administration didn’t cover themselves in accolades during the mess. But what if she had said something like this while campaigning in the battleground states:

“While I appreciate the efforts of the Obama administration to address the foreclosure crisis, the Home Affordable Modification Program simply has not provided the relief needed by many families. That is why I strongly support the creation of an Office of the Homeowner Advocate to help struggling families who have been wrongly denied assistance, or who have had difficulties navigating the extremely stressful system of avoiding foreclosure. The Office of the Homeowner Advocate will not only give Vermonters a strong voice in the process, but it will identify ways to make the HAMP program work better,”

That, unfortunately is a statement from Bernie Sanders, in 2010, rather from Clinton (Sanders continued to make it a focus of his primary efforts in 2016 as well).

Or perhaps Clinton could have spoken out in support of the frustrated community groups that sought to participate in the HUD auctions of distressed loans, only to lose out time and again to hedge funds, many of which were run by bankers who were directly involved in the financial crisis.

Maybe she was reluctant to get too involved in the issue because she tried to talk about it back in the 2008 primary and ended up being tagged as a too close to Wall Street. On several occasions in foreclosure states like Nevada, she seemed to cede the issue of the financial crisis  to Sanders and focused her efforts on minority outreach instead. But in states like Florida, where many homeowners remained underwater on the value of the homes and mortgages still in 2016, the issue appeared to still be on the minds of voters on the eve of the election.

Of course, it’s easy to second guess the campaign now. I, and many others, spend hours over several years trying to get the Obama Administration or state governments to improve their response to the foreclosure crisis. By 2016, many of the people I worked with back in 2011 to 2013 on housing issues were exhausted and frustrated. I can only imagine how the people living with the foreclosure crisis must have felt.

Still, a few thousand votes in three key states would have been enough to change the outcome of the election. And when you compare these maps, it’s hard not to see the lost opportunities.

This is a post by Eugene Stern, originally posted on his blog sensemadehere.wordpress.com.

Nate Silver got the election right.

Modeling this election was never about win probabilities (i.e., saying that Clinton is 98% likely to win, or 71% likely to win, or whatever). It was about finding a way to convey meaningful information about uncertainty and about what could happen. And, despite the not-so-great headline, this article by Nate Silver does a pretty impressive job.

First, let’s have a look at what not to do. This article by Sam Wang (Princeton Election Consortium) explains how you end up with a win probability of 98-99% for Clinton. First, he aggregates the state polls, and figures that if they’re right on average, then Clinton wins easily (with over 300 electoral votes I believe). Then he looks for a way to model the uncertainty. He asks, reasonably: what happens if the polls are all off by a given amount? And he answers the question, again reasonably: if Trump overperforms his polls by 2.6%, the election becomes a toss-up. If he overperforms by more, he’s likely to win.

But then you have to ask: how much could the polls be off by? And this is where Wang goes horribly wrong.

The uncertainty here is virtually impossible to model statistically. US presidential elections don’t happen that often, so there’s not much direct history, plus the challenges of polling are changing dramatically as fewer and fewer people are reachable via listed phone numbers. Wang does say that in the last three elections, the polls have been off by 1.3% (Bush 2004), 1.2% (Obama 2008), and 2.3% (Obama 2012). So polls being off by 2.6% doesn’t seem crazy at all.

For some inexplicable reason, however, Wang ignores what is right in front of his nose, picks a tiny standard error parameter out of the air, plugs it into his model, and basically says: well, the polls are very unlikely to be off by very much, so Clinton is 98-99% likely to win.

Always be wary of models, especially models of human behavior, that give probabilities of 98-99%. Always ask yourself: am I anywhere near 98-99% sure that my model is complete and accurate? If not, STOP, cross out your probabilities because they are meaningless, and start again.

How do you come up with a meaningful forecast, though? Once you accept that there’s genuine uncertainty in the most important parameter in your model, and that trying to assign a probability is likely to range from meaningless to flat-out wrong, how do you proceed?

Well, let’s look at what Silver does in this article. Instead of trying to estimate the volatility as Wang does (and as Silver also does on the front page of his web site, people just can’t help themselves), he gives a careful analysis of some possible specific scenarios. What are some good scenarios to pick? Well, maybe we should look at recent cases of when nationwide polls have been off. OK, can you think of any good examples? Hmm, I don’t know, maybe…

Aiiieeee!!!!

Look at the numbers in that Sun cover. Brexit (Leave) won by 4%, while the polls before the election were essentially tied, with Remain perhaps enjoying a slight lead. That’s a polling error of at least 4%. And the US poll numbers are very clear: if Trump overperforms his polls by 4%, he wins easily.

In financial modeling, where you often don’t have enough relevant history to build a good probabilistic model, this technique — pick some scenarios that seem important, play them through your model, and look at the outcomes — is called stress testing. Silver’s article does a really, really good job of it. He doesn’t pretend to know what’s going to happen (we can’t all be Michael Moore, you know), but he plays out the possibilities, makes the risks transparent, and puts you in a position to evaluate them. That is how you’re supposed to analyze situations with inherent uncertainty. And with the inherent uncertainty in our world increasing, to say the least, it’s a way of thinking that we all better start becoming really familiar with.

The models were plain as day. What the numbers were telling us was that if the polls were right, Clinton would win easily, but if they were underestimating Trump’s support by anywhere near a Brexit-like margin, Trump would win easily. Shouldn’t that have been the headline? Wouldn’t you have liked to have known that? Isn’t it way more informative than saying that Clinton is 98% or 71% likely to win based on some parameter someone plucked out of thin air?

We should have been going into this election terrified.

People voted for Trump because he was speaking to them about their pain, and making unreasonable promises about how great the future would be for them.

At the same time he was unforgivably awful to all sorts of subpopulations of Americans. The people who voted for him either embraced that hate or ignored it.

This means two things for the rest of us.

First, it means we need to help Trump voters smell their particular shit, which is going to be hard for them, because many of them actually trusted Trump’s promises. That means we document all the ways their expectations have been unmet in the next four years. We have to keep track of the inevitable blame game that Trump is so good at, where he will vilify random people when he fails to deliver his promises.

Second, it means we need to carefully watch all those people who were willing to embrace the hate; they have been empowered and could be truly dangerous, especially when the shit first gets smelled. Nor can we rely on those people who don’t think of themselves as racist but who ignored the hate. They are willing to remain passive in the face of hatred, exactly what we cannot do. People, we need to protect one another, and in particular we need to protect the most vulnerable among us.

How do we document and protect? It starts with citizen journalism. As individuals, we need to use our phones, our blogs, and our conversations as opportunities to speak clearly about what we witness.

We need to train ourselves to intervene when we see someone get singled out for their religion or the color of their skin. We all need to get off of the toxic echo chamber that is Facebook and engage with people in a coffee shop that we happen to meet. Who knows, we might disagree with them, but that shouldn’t stop us from communicating civilly. We need to travel away from our cities and interact with people outside our normal lives.

We need to support independent journalism (choose your two favorite) and civil rights groups (ACLU, Legal Defense Fund) so they can do their jobs, which are crucially important.

We also need to organize locally to do more. This means more than a protest march. It’s a long game, and it needs to be strategic. It needs to reimagine the Democratic party as well a strategy to empower unionization or some other form or forms of working class solidarity. In my Occupy group we’re going to watch this video soon to know what that might look like.

There’s real risk that if we don’t document and protect, we’ll have a disappointed and angry mob casting their anger and blame on minorities with impunity.

We can do this. We can smell the shit together.

Go bake your pie, your lasagna. Get your comfort food made, and check on the kids.

And then contribute to your favorite, most hard-hitting independent journalism organization, if you have money to spare. Look to the future, don’t dwell. Ignore conversations about what happened, about the mathematics of polling, of demographic nonsense. It’s time to prepare for whatever the hell is happening next. It’s up to us to focus, to value information over propaganda. Nobody else is going to do that for us.

Because we are all activists now.

This is a voting guide my son Aise put together for me. He’s not old enough to vote so he made it to influence my vote. I thought he did a nice job distilling some real information, so I got his permission to post it here.

Federal:

Presidential: Dan Vacek (Legal Marijuana Now)

New York Senate: Alex Merced (Libertarian)

• He wants to legalize all drugs
• He wants to have a very open border policy.
  • He says illegal immigration is like a black market, if you make something more or less legal, the black market will go away

New York House District 10: Jerrold Nadler (Democratic)

• He has been a reliable progressive democrat. One example of this was his voting against laws that would have helped along the tpp. He also voted to stop the expansion of military suspending and voted to keep the Iran deal together.
• His one opponent is Philip Rosenthal who favors entitlement reform, tearing up the iran agreement and whose website has the words on it “When America retreats, evil advances.”

State:

State Senate District 30: Bill Perkins (Democrat)

• He is running against Jon Girodes who was arrested for running a scam by taking people’s money to rent out an apartment and then not returning the money or giving them the apartment.
• He sponsored legislation to allow people 16 and over to donate an organ.
• He sponsored legislation to regulate emissions from cars.
• He sponsored legislation to give inmates translation services in parole hearings.
• He sponsored legislation to make it easier for the disabled receiving social security money to avoid rent hikes.

State Assembly District 69: Daniel O’Donnell (Democratic)

• He supported legislation to allow convicted felons to vote once they completed their sentence.
• He supported legislation to expand eligibility for “shock incarcerations.” A shock incarceration is when someone serves time in a treatment facility instead of prison.

Fake News

Facebook uses an algorithm to decide what you see. It’s proprietary but my guess it’s optimized to keep you on Facebook for as long as possible.

This wouldn’t be a problem but becomes one when we realize that people get their news from Facebook.

When you optimize to something, and when you ignore something else, that other thing can be expected to balloon beyond recognition. We’ve seen that with ballooning tuition for colleges because of the US News & World Report, for example.

In this example, the thing Facebook has ignored is “truth.” The result is a proliferation of fake news:

False Information

Beyond simply fake news, there’s tons of hyper partisan articles that make use of false information. These pseudo-news sites have popped up simply to exist on Facebook and to game the Facebook algorithm.

When Facebook started 12 years ago, there was a much healthier journalism industry. It’s now much less healthy, in no small part because of the ad dollars that now pour into Facebook. What will another 12 years bring? I’m worried that we won’t have real news anymore even if we search for it.

This is bad for democracy, because people are constantly being misinformed or hysterically informed. It’s pushing people further into their corners, or pushing them off of Facebook and politics entirely.

Stupid Polls

Finally, the polling conversations are out of hand. We tune into our favorite radio shows to hear about policy and instead we hear about poll numbers, or even worse, debates between poll watchers about whose poll is more accurate. That’s not news.

We have obsessed over the college educated white Iowan women’s vote for long enough, and we need to enter a new phase where we discuss actual issues. Leave the polling to campaigns. Mona said it best:

https://embed.theguardian.com/embed/video/commentisfree/video/2016/jun/01/political-polls-bad-for-democracy-heres-why-video

What can we do?

Here are some ideas that might help a little but won’t solve everything. Tell me yours.

1. Facebook absolutely must acknowledge its role in the spread of misinformation. They need to act as editors. This will take an army of workers, but there are plenty of journalists who are looking for jobs, and Facebook makes tons of money, so there’s no actual problem besides the will of Facebook.
2. Beyond that, Facebook needs to redesign its algorithm so that people don’t only see things they already agree with. This echo chamber (or “filter bubble,” as Eli Pariser described it in his 2011 book) has had a terrible effect on political partisanship. We’ve ended up thinking people who don’t agree with us are actually bad people. Facebook should redesign its platform so that we talk and listen to each other more.
3. We need to demand that media stop fixating on polls. If we can’t outlaw them, at the very least we can complain and move our attention to real information.

Tonight I’m taking my adorable husband with me to accept an award from the Special Libraries Association of New York called the PrivCo Spotlight Award. Here’s the description of the award and their reasoning in choosing me, from their website:

This award celebrates website founders and bloggers, curators of distinctive collections, solo librarians, mentors and teachers, conference organizers, and librarians typically working outside the traditional scope of SLA-NY award consideration. As a data scientist and author of the “MathBabe” blog, we feel Cathy O’Neil strongly embodies the spirit of SLA NY. Her book Weapons of Math Destruction was published in 2016 and has been nominated for the 2016 National Book Award for Nonfiction.

I’m a huge fan of librarians – they are, in my opinion, the original ethical data scientists – so I’m both honored and psyched for the award and for the chance to meet the kind people at SLA-NY as well as the other award winners.

There’s an article that’s been forwarded to me by a bunch of people (I think first by Becky Jaffe) by Anya Kamanetz entitled How One University Used Big Data To Boost Graduation Rates.

The article centers on an algorithm being used by Georgia State University to identify students in danger of dropping out of school. Once identified, the school pairs those wobbly students with advisers to try to help them succeed. From the article:

A GPS alert doesn’t put a student on academic probation or trigger any automatic consequence. Instead, it’s the catalyst for a conversation.

The system prompted 51,000 in-person meetings between students and advisers in the past 12 months. That’s three or four times more than was happening before, when meetings were largely up to the students.

The real work was in those face-to-face encounters, as students made plans with their advisers to get extra tutoring help, take a summer class or maybe switch majors.

I wrote a recent book about powerful, secret, destructive algorithms that I called WMD’s, short for Weapons of Math Destruction. And naturally, a bunch of people have written to me asking if I thought the algorithm from this article would qualify as a WMD.

In a word, no.

Here’s the thing. One of the hallmark characteristics of a WMD is that it punishes the poor, the unlucky, the sick, or the marginalized. This algorithm does the opposite – it offers them help.

Now, I’m not saying it’s perfect. There could easily be flaws in this model, and some people are not being offered help who really need it. That can be seen as a kind of injustice, if others are receiving that help. But that’s the worst case scenario, and it’s not exactly tragic, and it’s a mistake that might well be caught if the algorithm is trained over time and modified to new data.

According to the article, the new algorithmic advising system has resulted in quite a few pieces of really good news:

• Graduation rates are up 6 percentage points since 2013.
• Graduates are getting that degree an average half a semester sooner than before, saving an estimated $12 million in tuition.
• Low-income, first-generation and minority students have closed the graduation rate gap.
• And those same students are succeeding at higher rates in tough STEM majors.

But to be clear, the real “secret sauce” in this system is the extraordinary amount of advising that’s been given to the students. The algorithm just directed that work.

A final word. This algorithm, which identifies struggling students and helps them, is an example I often use in explaining that an algorithm is not inherently good or evil.

In other words, this same algorithm could be used for evil, to punish the badly off, and a similar one nearly was in the case of Mount St. Mary’s College in Virginia. I wrote about that case as well, in a post entitled The Mount St. Mary’s Story is just so terrible.