Ernie Davis: The Palantir of New Orleans

Did you hear that Palantir, the big data company founded by Peter Thiel, has been secretly building predictive policing algorithms for New Orleans?

Well, when my buddy Ernie Davis found out, he decided to write a poem. He’s generously allowed me to reproduce it here:

Six years ago, one of the world’s most secretive and powerful tech firms developed a contentious intelligence product in a city that has served as a neoliberal laboratory for everything from charter schools to radical housing reform since Hurricane Katrina. Because the program was never public, important questions about its basic functioning, risk for bias, and overall propriety were never answered.— Palantir has secretly been using New Orleans to test its predictive policing technology, Ali Winston, The Verge, Feb. 27, 2018.

In Eldamar, so long before
Our time, that none can tell the count in years,
The elven craftsman Féanor
Devised the seeing stones, the Palantirs.

The men of old, in seven towers,
Installed the stones that Féanor had wrought
And used their extrasensory powers
To see far off and to converse in thought.

But using a device whose might
Exceeds your wisdom risks a fearful fall.
The fates of Saruman the White
And Steward Denethor are known to all.


The enterprising Peter Thiel
Built Paypal and became a billionaire.
A man of business nonpareil
But arrogant as Féanor the Fair.

He scorned the college education
That piles useless knowledge in your head,
And so established a foundation
So youths could start up businesses instead.

He scorned the privileged elite,
Self-righteous, over-educated, smug,
And thus endorsed the loathsome cheat
Who honors every autocratic thug.

Since folks online are always willing
To publish on the web all they can tell
Thiel saw that he could make a killing
By mining it for content he can sell.

His team of workers then designed
The mightiest program they could engineer
To sift the data to be mined.
He named the company “The Palantir”.

The palantirs of Féanor
Could show what was long past and far away.
Thiel’s Palantir sees vastly more:
It knows right now what men will do some day.

It studies billions of relations
‘Twixt men as they develop over time.
And finds the key configurations
That augur the committing of a crime.


To prove, past reasonable doubt,
Who’s guilty of specific criminal acts
Requires reasoning about
An awful lot of pesky little facts.

Who was where and when and why?
What show the footprints, blood stains, DNA?
An inconvenient alibi
Can ruin any prosecutor’s day.

A human being is still needed
To comprehend these kinds of evidence
No AI program has succeeded
In mastering the basic common sense.

But building an AI detector
For criminal propensity’s no sweat.
You map a person to a vector
And classify it with a neural net.


New Orleans, fair but troubled Queen
Has not in full recovered from the blow
Dealt by Hurricane Katrine
In storm and flood, a dozen years ago.

Gangs that trafficked in the sale
Of heroin and methampetamines
Fought turf wars, and they left a trail
Of murder on the streets of New Orleans

James Carville, famed politico,
Lived in New Orleans and held it dear
And Carville also chanced to know
About the products built at Palantir.

Carville convinced the company
(He was a paid advisor at the time)
To share their software, all for free
To help N.O.P.D. to combat crime.

An altogether secret deal:
Only N.O.P.D. and the mayor,
The folks at Palantir and Thiel
Were any time informed or made aware.

“Fool!” Thus Saruman was named
For secret hid from Council long ago.
And should not those be likewise shamed
Who thought the city council need not know?

Policing with predictive code
The guardians of security delights,
But leads us on a risky road
Toward bias and ignoring civil rights.

Matalin, James Carville’s wife
Assures us all that we will be OK
As long as, in the course of life,
No cousin or acquaintance goes astray.

So Palantir in place remains
And now that we have heard of it, we must
Conclude that those who hold the reins
Of power have betrayed the public trust.

This is part of the collection Verses for the Information Age by Ernest Davis

Categories: Uncategorized

A natural experiment for Facebook

I wrote a Bloomberg View piece about something Facebook could probably do to determine if the Russian ads made a difference in the 2016 election:

Facebook Could Do a Lot More on Trump-Russia

The company could probably measure the effect of election-meddling if it wanted to.


For my complete list of Bloomberg View pieces, go here.

Categories: Uncategorized

Short rant about diets

I’m off of Twitter and Facebook, so my blog is starting to become my go-to rant spot once again.

This morning it’s Jane Brody that’s got me ranty, with her unbelievably incoherent take on exercise and diets:

More Fitness, Less Fatness

Let me give you my snarky summary of this piece:

  1. Diets don’t work.
  2. People are beginning to understand this.
  3. They’re starting to accept their bodies as they are and go on with their lives.
  4. This makes Jane Brody feel bad for them, because they really should feel bad for themselves.
  5. Turns out people should exercise instead of diet to be healthier.
  6. But Jane Brody suggests they should diet too, even though it doesn’t work.
  7. Jane Brody concludes by entitling the piece to imply exercise makes people lose weight even though there’s plenty of evidence that it doesn’t.
Categories: Uncategorized

I was feeling dark when I wrote this

My newest Bloomberg View piece is out:

Are You Poor? Here’s Your Virtual Hamster Cage

Technology could erase the limits of inequality.


See more of my Bloomberg View pieces here.

Categories: Uncategorized

My favorite kind of Amazon Q&A


Categories: Uncategorized

Workers Should Have Their Fingers Crossed for a Market Downturn

Who cares if the stock market tanks? No, really. I’m wondering who actually has a stake in the levels of the stock market.

The average person doesn’t have much savings, including retirement savings which is the standard way to have a direct stake in the market. In fact a majority of Americans, and more than that if you consider minorities, have less than $1000 put away for retirement. They might care about the few hundred dollars they have, but it’s really not much directly at stake, and it’s a long term abstract investment if it even exists.

For that matter, truly rich people have investment advisors that diversify their positions by using bonds, hedge funds, and so on to make their bet more market neutral. Plus, they have plenty of assets, so to the extent that the market goes down by a bit won’t overly concern them.

That leaves the well off but not rich people who are adequately long the market to care what it does, and still their stake is mostly via retirement savings. I’m not sure how much they represent as a percentage of the population, but it’s fair to say the average member of the population don’t really care about a market fall.

It’s been a long time since the market has been a good proxy for the economy as a whole. Thinking used to be that if corporations made more money, at least if it came from higher productivity, then some portion of that would be distributed to workers. But it was long ago that productivity decoupled from the median wage.

In fact, it’s become just the opposite: good news for workers means bad news for the market. That became clear recently when a substantial rise in wages led to a drop in the market. The argument went something along the lines of higher wages will cause inflation and then interest rates yadda yadda, but the bottom line is that shareholders have gotten used to keeping all the corporate profits.

Actually, this anti-correlation between the market and worker interests has actually been true for quite a while. The tax bill, which heavily privileges stockholders over wage earners, was slowly baked into the stock market as it became increasingly clear it would pass. In other words, good news for the market has meant bad news for workers for the past year and a half. It’s also why Davos loved Trump: he gave out goodies to rich people with the abstract promise that this will end up in the pockets of workers.

Of course, there are pieces of news that would be bad for both the workers and for the market, like a recession, and there are potential turns of events that would be good for everyone, like exciting new industries that hire lots of people. But for the foreseeable future, I’m thinking that workers should be cheering a tanking market.

Categories: Uncategorized

Guest post: What We Should Worry About When We Worry About Virtual Reality

This is a guest post by Eugene Stern, originally posted on his blog sensemadehere.

My friend Cathy O’Neil just sent me an article she wrote for the NY Times reviewing two books by technologists about virtual reality (VR). Part of her take was that neither book talked enough about ways that VR could be abused, and she speculated that worrying about VR was still mostly the provenance of science fiction writers (think Star Trek) rather than technologists.

I’m pretty comfortable around both sci-fi and technology, but you really don’t need to be an expert in either to worry about how VR could upend our lives and civilization. Just some sense of recent history is enough. If you think that massive computational power, the internet, and smartphones might have turned out to be a bit more than we bargained for, maybe it’s time to consider how amazingly well-positioned VR is to amplify some of the most troublesome aspects of the technology revolution:

Personalization. We’ve learned over the last quarter century that we don’t mind being monitored (cookies, GPS, Fitbits), just as long as some benefits (recommendations, special offers, traffic advice, a tailored Facebook feed, the ability to broadcast our 5.4 mile running route to all our friends) come from crunching the resulting data. Never mind who might be storing all that data or what they might be doing with it.

Now think about VR, which massively scales up both the amount of data and the ability to collect it. On one hand, VR is an immersive experience, generated by high dimensional data sets (indeed, one of the uses of VR is as a tool to allow us to navigate data sets that are otherwise too complex to make sense of; see here or here or here). On the other, VR is delivered through a device, which can be used to track eye movements, and VR technology to monitor other biometrics like heart rate, pulse, and  electrical activity in the brain is already on the way (see here or here). You’ve probably heard of Google’s A/B tests, which enable web designers to vary individual aspects of a web page and track how people respond. Now imagine such tests in VR space, targeted at each individual user, and able both to vary all kinds of stimuli affecting all the senses, and to measure all kinds of response. In a contest between your family, friends, and VR set over who knows you better, it’s hard to see the humans having a chance.

Addictiveness. By now it’s sort of a cliche to hear a technologist speak thoughtfully about how they won’t let their children near smartphones or Instagram until they’re in high school, or to read articles about internet use sprinkled with multiple mentions of dopamine. Won’t this all seem quaint in a few years, when internet porn gives way to (personalized!) VR sex, and your social network can deliver a full VR simulation of your crush’s reaction to the cute photo you just posted, not just a stylized thumbs-up or heart. Um, yeah, VR is going to make the virtual world way more addictive. “Why go into the outside world at all, it’s such a fright,” as Black Flag sang, to their televisions, and that was at least two whole generations of technology ago!

Marketing. I was born in the Soviet Union, which had no ads, and it always felt strange to me that our entire media landscape (or, today, our entire information landscape) was driven by companies inserting little messages meant to sell you things. For one thing, I was always a bit skeptical that advertising was actually worth it. Well, with VR, there’ll be no question, because we’ll be able to track the outcomes of ads so precisely: eyeballs widen, heart rate rises just a bit, electrical activity heightens in the buying center of the brain (which by this time we will have effectively mapped, using — what else — VR technology). Advertisers will know exactly which ads worked (so the economy will make sense!), and, with predictive analytics and the heavy volumes of data attached to VR, they’ll also know which ads will work, for any given person. And lots of them will, because VR’s ability to virtually sample any product you might imagine might make it the most effective advertising medium ever. If today we think about ads as delivering eyeballs and clicks, in the age of VR, they might be delivering (virtual) wallets directly.

Will users object? One more thing we’ve learned in the internet age is that people don’t seem to mind being targeted with ads across their entire virtual experience. Ad-based media still dominate, while raising revenues via direct subscription works for a few niche publications at best. The internet is funded by advertising. Why wouldn’t VR be?

Though VR seems expensive today — the domain of rich NFL teams needing to train quarterbacks to have split second reactions to thousands of different stimuli, as Cathy writes in her book review — from another point of view, it might actually be quite cheap. In the non-virtual world, you have to be rich to sit in the front row at midfield at the Super Bowl, or swim with tortoises in the Galapagos Islands, or climb Mount Everest. But, mass adoption of VR could be a great leveler in a way, making virtual versions of all of these accessible to the masses. Being marketed to may seem like a small price to pay to have these experiences, especially if the income gap between rich and poor grows as technology makes more and more segments of the economy winner-take-all. Sure, VR might enable advertisers to fully exploit you economically, to optimize and control all of your purchasing power — but so what, we haven’t been troubled yet whenever our technology asks us to give up control to gain comfort.

The scariest thing about VR might be that it could be more of the same, but on steroids. If we’ve shown no societal ability so far to confront technology addiction, data collection and surveillance, or media manipulation, what happens when VR technology renders all of these ten times more powerful? Be afraid, be very afraid.

Categories: Uncategorized