The Data For Good Exchange

I’m happy to announce that I’m on the Program Committee for Bloomberg’s Data For Good Exchange. This is a one day event, taking place on September 28th at the Bloomberg offices. It’s been scheduled to lead into the annual Strata NY conference which is run by O’Reilly.

In addition to the event, which will have speakers and keynotes by people like my buddy Jake Porway, there’s a competition for papers that contribute to the public good. The fact that I’m on the Program Committee means that I get to choose a bunch of papers which were submitted according to this call for papers, take a look at the contents, and rate them.

I’ve taken a quick look at the papers and they look pretty amazing. Stay tuned for more.

Categories: Uncategorized

As a futurist, I have lots of work to do

It’s time to get busy, people. I need to find futurist conferences to go to (and to speak at), I need to hobnob at cocktail parties. Now that I care deeply about predicting and shaping the future, I need to get on top of this shit.

As part of my research, I have stumbled upon Dylan Matthews’s brilliant Vox piece entitled I spent a weekend at Google talking with nerds about charity. I came away … worried. In a word, Matthews agrees with my post from yesterday.

He spent a weekend at an “Effective Altruism” (EA) conference at Google Mountain View, with many other “white male nerd(s) on the autism spectrum” and he came away with this observation:

In the beginning, EA was mostly about fighting global poverty. Now it’s becoming more and more about funding computer science research to forestall an artificial intelligence–provoked apocalypse. At the risk of overgeneralizing, the computer science majors have convinced each other that the best way to save the world is to do computer science research. Compared to that, multiple attendees said, global poverty is a “rounding error.”

This particular brand of futurism takes refuge in “existential threats” which they measure very carefully with lots of big powers of 10. They worship a certain extra-special white male nerd from Oxford named Nick Bostrom. From Matthews’ piece, where a majority of those at the conference were worrying about the risk robots taking over:

Even if we give this 10^54 estimate “a mere 1% chance of being correct,” Bostrom writes, “we find that the expected value of reducing existential risk by a mere one billionth of one billionth of one percentage point is worth a hundred billion times as much as a billion human lives.”

No, it doesn’t matter what that means. The point is that it’s a way of nerdifying the current messy world and thereby have an excuse for not improving things now.

Matthews sees through this all, in terms of their logic as well as their assumptions. Here’s his logical argument:

The problem is that you could use this logic to defend just about anything. Imagine that a wizard showed up and said, “Humans are about to go extinct unless you give me $10 to cast a magical spell.” Even if you only think there’s a, say, 0.00000000000000001 percent chance that he’s right, you should still, under this reasoning, give him the $10, because the expected value is that you’re saving 10^32 lives.

And here’s his critique on their assumptions:

…the AI crowd seems to be assuming that people who might exist in the future should be counted equally to people who definitely exist today.

Just in case you’re thinking that this stuff is too silly to be taken seriously, some of the people putting money into think tanks that worry about this crap include Elon Musk, Peter Thiel, and other Silicon Valley success stories. The money, and the Google location, adds to the self-congratulatory tone. An event organizer made this embarrassingly clear: “I really do believe that effective altruism could be the last social movement we ever need.”

From left: Daniel Dewey, Nick Bostrom, Elon Musk, Nick Soares, and Stuart Russell.

From left: Daniel Dewey, Nick Bostrom, Elon Musk, Nick Soares, and Stuart Russell. Taken from Vox by Anna Riedl.

I find this kind of reasoning very familiar, and here’s why. Anyone who’s worked at a hedge fund has heard far too many people with a similar “Bill Gates Life Plan”: first, amass asstons of money by hook or by crook, and then, and only then, deploy their personal plans for charity and world improvement.

In other words, this whole movement might simply be a way of applying a sheen of scientific objectivity and altruism to a vain and greedy impulse.

I’ve got my work cut out for me. Please tell me if you know of conferences or such which I can apply to.

Categories: Uncategorized

I am a futurist!

I’ve been thinking about the future a lot recently, so I’ve decided to throw my hat into the futurism ring. Don’t get me wrong, this wasn’t an easy decision. Nevertheless I think it’s the right one.

It all started with fretting over the present. Things seem to be unraveling, and I spend at quite some time each day worrying about stuff like our country’s oligarchy problem, our racist policing and justice systems, and the overall lack of good middle class jobs.

So far that’s just a list of our present woes, but any plan to address them needs to incorporate a hopeful plan for the future, right? So naturally I decided to look into “futurism,” which attempts to anticipate and guide our future plans.

Here’s the thing, though. As described beautifully in Rose Eveleth’s Atlantic piece, Futurism Needs More Women, the field is currently clogged with white North American men between the ages of 55 and 65 who talk optimistically about super cool future technology, and living longer, and uploading their brains, and so on.

It’s is not a particularly appealing pool to jump in on, but here I come anyway. And, being a world class cannonballer, I’m not afraid of making a splash.


So, there’s a big problem with futurism right now, which is that, on the whole, they pretty much entirely ignore social issues, which as you’ll notice are highly entwined with my top three concerns, politics, racism, and the end of work (otherwise put: the only way to compete with robots is to become one). I plan to change futurism; I’ll be the loudmouth at the futurism conference talking about other things and how we need to plan. I’ll make this shit real. I mean, after all, why should the white guys have all the fun of deciding what the future might look like?

Before I go on, let me explain why women (so far!) have been reluctant to join the futurism movement. We remain unimpressed with their major visions so far: live forever, become one with a machine, let technology solve all social problems. Here’s why.

Living forever/ the singularity. Women get their period when they’re young, then they go through menopause when they’re old, and then they die when they’re really old. I mean, oversimplification, but whatever; the point is they are firmly tied to their aging bodies and are well aware of the ticking biological clock, and not just the one for having babies. Personally I’m 43 and even though getting my period is a messy pain, at this point I am deeply nostalgic for my youth every time it happens.

By contrast, men grow pubes at the age of 14 and nothing ever seems to change again. They might have even forgotten they ever didn’t have them. In any case men are more likely to consider the idea of putting their brains in jars – hooked up to the internet, of course – as a reasonable approximation of their current state of existence. I feel sorry for them. Being reminded of death once a month makes it impossible to be so silly. Or at least much harder.

Technophilia. Men – especially futurists – seem to love technology, and fail to cast a critical eye on anything that seems remotely “innovative.” I call this the “I win” blindspot, whereby people who are generally rewarded in a system seem to think the system must be great. After all, if you’re the one creating the surveillance software or analyzing the surveillance footage or sensor data on say, long-haul trucks, you’re getting paid really well to promote “progress.” Not so much the story for the truckers being surveilled, but whatever, I guess they should have learned to code.

I mean, that’s just one small example, but I could go on for hours. And I think women, and for that matter anyone who isn’t a successful white dude, sees both sides. That’s why we’re not jumping at the chance to join the technophiliac bandwagon.

Anyhoo, futurism is unbearably narrow at the moment, but I don’t think that should stop me. In spite of my focus on social issues, I have the credentials required to worm my way into the conversation. In fact, I have a convincing explanation for why their approach so far – into the probability of various future trends – is fatally flawed. Namely, they’ve got too few variables. They focus on technological change without taking into account human beings. So their Monte Carlo engine, if you will, of possibly futures ends up with only tweaks that they allow in their tiny little list of possibilities. I plan to add to that list. And given that they seek to influence policy as well as the individual’s forward-looking self, this list might matter.

Please congratulate me! I cannot wait to meet Ray Kurzweil in person and congratulate him on all his rings.

Categories: Uncategorized

Aunt Pythia’s advice

Aunt Pythia asked a few days ago whether her advice would be better dispense in video format, and there was near consensus: no indeed.

You have spoken with one voice, loud and clear! And that is why Aunt Pythia has readers, dear readers, and not viewers. She toasts to you.

Holy crap I want a mimosa.

Holy crap I want a mimosa.

But readers, please read this next line carefully, not all is well. As of today, Aunt Pythia only has enough questions for one more week of her advice column.

That’s right! Aunt Pythia is starving for ethical conundrums! She’s thirsty for romantic entanglements and she’s eager to ponder, muse, and ruminate on your deepest and darkest quandaries. Let her help! Please please please:

ask Aunt Pythia any question at all at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Aunt Pythia,

Would you be willing to share your recipe for those identity crisis crepes? They look delicious and very helpful.

Handling Undeniable Nagging Gripes Requires Yummies


Why of course. I use a modified Joy of Cooking recipe – modified because I use salted butter and 2% milk, and the recipe book usually calls for unsalted butter and whole milk. I also triple the recipe to feed my kids and the neighbor kids, which I happily present. Mix in a large bowl:

  • 2 and 1/4 cups white flour
  • 1 slightly rounded teaspoon of salt
  • 1 flat tablespoon baking powder
  • 1/4 cup or so of powdered sugar (I just shake a bit into the bowl)

Then add:

  • 3 cups of milk
  • 6 eggs
  • a large dash of vanilla

Mix everything until it’s relatively smooth. Next, find a nonstick pan (or two if you’re ambitious) and put a generous pat of butter on the pan on medium heat. Spread the butter around to coat the entire pan, and when it’s frothy add a ladle spoon of batter, spreading it out over the whole pan by tipping the pan this way and that. Turn it over as soon as the spatula lets you, and cook on the other side for about the same amount of time (maybe 3 minutes for each side). Then put your finished crepe on a platter and continue. Makes about 9 crepes.

I serve the pile of crepes on a table set with cut-up fruit, nutella, jam, syrup, and powdered sugar. When I’m feeling Dutch I also offer bacon and eggs and I call them “pannekoeken” instead of crepes.

To make them “identity crisis” specific, simply use extra nutella at the end and pair with mimosas.

Aunt Pythia


Aunt Pythia,

How do I convince myself, in the face of half a lifetime of evidence to the contrary, that there are women who want to date me and might even eventually want to sleep with me?

Forty And Increasingly Lonely

Dear Forty,

I actually have quite a bit of experience giving advice in this realm, but not knowing anything more about you is going to severely limit my advice. So, if you were here with me I’d ask you a bunch of questions about your habits, attitude, and previous attempts. I’ll do my best to give you general advice though.

First, make sure you exercise regularly. This doesn’t make you lose weight, contrary to popular marketing belief, but it gets you out of the house, wards against depression, makes you feel good in your body, and forces you to take regular showers. All good things.

Second, figure out how to meet people. A lot of people, preferably in a female-dominated setting. I suggest joining a class at your local community college on cooking or pottery or meditation. Really nice people go to such classes, and they are often open to meeting new people. If you have the inclination, go to church, or even better, choir. There are basically no straight men in choir, and those that there are get snatched up.

Third, examine your self-confidence. Figure out mysterious and compelling things about yourself and practice making them even cooler. About half of self-confidence is the belief that other people will want to spend time with you, so practice being a good listener and asking polite and encouraging questions. Don’t forget to flatter people (when it is deserved and not creepy), and figure out how to accept compliments graciously as well.

Finally, ask people out a LOT. Make it a habit to put yourself out there, in a non-threatening way, pretty much every time you actually want to see someone a second time. Sometimes it will work, other times it won’t, but it’s the only way you’ll ever start a relationship. And it doesn’t have to be romantic, either: asking someone out to coffee to continue a conversation is something that people do, and you should be sure you do it whenever you feel like it.

Good luck!

Aunt Pythia

p.s. if you have more specific questions, feel free to email me personally. My email address is on my “About” page.


Aunt Pythia,

Is there any part of these arguments with merit?


Dear K,

I actually feel dumber for having read – well, skimmed actually – that article. Good news is he gave himself away early with the word “shrill”; after that I knew he was a woman hater.

The only positive I came away with is that I might want to dye my armpit hair blue to match my head hair.

Aunt Pythia


Dear Aunt Pythia,

So, Mathbabe says that some smarts and math skills are essential for being a data scientist. In particular, mathbabe says if one lacks the quantitative prowess to invert a matrix, then they do not have the math aptitude to be a decent data scientist. Does someone have to be able to get the concept instantly and effortlessly when they see it for the first time?

I was a humanities (history, specifically) major in college and I currently work in education, and I want to pursue an MS in statistics. I can invert a matrix pretty comfortably now, but it did take some effort (study group, office hours) to figure out how to do it when I encountered the concept for the first time in a linear algebra class. I am necessarily aiming to be a data scientist, per se. I see data as a promising and powerful tool for advancing problems I really care about, and I want to be able to meaningfully interact with people who analyze data to understand what they have done and make sense of what it can and cannot do.

Depressed in the Suburbs

Dear Depressed,

Just to be crystal clear, I don’t actually think everyone needs to go around practicing how to actually invert a matrix. Personally I’ve memorized the inversion of a 2 by 2 matrix, but if I were to invert a 3 by 3 matrix I’d have to derive the formula.

The real purpose I have in talking about matrix inversion is to point out the computational fragility of inverting a “nearly uninvertible” matrix, namely a matrix whose determinant is very close to 0.

Why, you might ask, would I have to worry about this? Well, for two reasons. First, when you’re dealing with real world data, everything is an approximation of truth. That means that if you have two vectors that are theoretically pointing in the same direction, they will only very approximately do so when the computation is worked out. For the same reason, when you have a matrix which theoretically should have dependent rows or columns, when you actually calculate the determinant, it will not be zero, but simply a very very small number, say 10^{-14}.

Next, when you invert a matrix, you do a bunch of things and then divide by the determinant at the end. Of course, you’re not supposed to “invert” an uninvertible matrix, but you of course can invert a matrix that has incredibly small but non-zero determinant. What you end up with is garbage.

OK, here’s why I’m telling you all this. Because the data scientist’s job is mostly to figure out why their model is fucking up massively. Models never work the first, second, or 17th time they are run, so you’d better be good at understanding what’s going wrong. One thing that often goes wrong is trying to invert a matrix that is not invertible, but it doesn’t manifest that way as the above story explains. So the data scientist has to start with ridiculous garbage answers, and backtrack to the actual problem, and knowing something about how a matrix is inverted is critical in this story.

Of course, matrix inversion isn’t the only example of the mathematical detective work inherent in a data scientist’s job. It’s kind of a metaphor for what you might end up doing as a data scientist. But it’s also a good place to start.

Anyway, none of this stuff is easy or effortless, so throw away that misconception immediately. I’m sure that someone with general intelligence can learn this stuff. I just think that there’s plenty of stuff they’d actually need to know.

Good luck!

Aunt Pythia


Readers? Aunt Pythia loves you so much. She wants to hear from you – she needs to hear from you – and then tell you what for in a most indulgent way. Will you help her do that?

Please, pleeeeease ask her a question. She will take it seriously and answer it if she can.

Click here for a form for later or just do it now:

Categories: Aunt Pythia

Mathematrucker’s take on the current state of long-haul truck driving

This is a guest post by mathematrucker.

There are a lot of pros and cons to being an over-the-road (OTR) truck driver, namely, one who spends weeks or even months at a time on the road. The pros can outweigh the cons for those like me who enjoy long highway trips. But this may be about to change.

A huge issue right now is surveillance. Inward-facing cameras that keep a constant watch on the driver may soon become the norm. Swift Transportation (the largest carrier in the U.S.) began installing them in all its company-owned trucks a few months ago.

Most OTR drivers are allowed to drive up to eleven hours per work shift and seventy hours every eight days. Their actual driving hours frequently reach these limits. That’s a lot of time to be in front of a running camera, never knowing for sure who might be watching you. If these cameras become widespread they are sure to cause many drivers (including me) to look for different work.

ELDs and Hours of Service Rules

One surveillance tool that is already well established is the electronic logging device (ELD). The Federal Motor Carrier Safety Administration (FMCSA, the regulatory body that covers interstate trucking) recently sent a rule to the White House for final review mandating that all trucks use them.

Federal hours-of-service regulations require drivers to maintain an up-to-date log book. Unlike paper logs, ELDs effectively prevent drivers from driving way past the legal limits. But they can also severely hinder drivers from driving short distances when they need to.

Most OTR drivers are paid by the mile—the more miles they drive, the more money they make. This provides a strong incentive to use all eleven driving hours per work shift. With paper logs, if a driver needs to exceed the limit by a few minutes to get to a safe place to sleep (versus stopping after say ten hours, possibly sacrificing some pay), they can. With ELDs this same scenario might force the driver into choosing between (1) sacrificing pay, (2) sacrificing overnight safety by stopping wherever, or (3) recording a logging violation to get to the safe place.

But many ELDs offer a fourth option as well: gaming the device (without tampering with it). Carriers can and usually do program some flexibility into ELDs. For example, trucks might be allowed to travel up to one mile below 10 MPH without the current duty status going to line 3 (driving), with resets occurring every time it does go to line 3. (Anyone catching a whiff of loophole here may want to hold their nose before reading the next sentence…) Many ELDs update and display the current duty status every second, but only record it at the top of every minute.

If the truck is not on a freeway, the driver can easily game such a device by alternately accelerating to around 45 MPH and decelerating to below 10 MPH once every minute, perhaps signaling a phantom turn to help avoid notice. To keep from decelerating too late, a smartphone can be used to watch the time to the second. Though no logging violation gets recorded, this technique does leave a trail of evidence on the device that might be noticed in an audit, but audits are infrequent. The alternative—recording a logging violation—will be detected immediately.

The FMCSA’s proposed mandate should require that ELDs record duty status by the second. It probably doesn’t.

The hours-of-service rules themselves are far from perfect. For one thing they do too little to prevent employers from depriving drivers of sleep. Ones who sleep nights are routinely directed to drive all night. Such orders went into my personal “go ahead, fire me if you need to” (for refusing) category many years ago, but this shouldn’t have been necessary.

The hours-of-service rules never said anything about time of day until a new rule was introduced in 2013 requiring two 1 AM to 5 AM periods in every thirty-four-hour rest break (such breaks reset hours driven to zero). Strong industry resistance caused this rule to be suspended in December 2014.

Of course, the problem of fatigue at the wheel will finally be solved when automation replaces truck drivers. Some studies predict it will happen soon. Anyone under the age of thirty (forty?) should take this into consideration if they are thinking about becoming a trucker.

Per Diem Pay

Due to a corporate tax strategy that has gained wide acceptance in recent years, income figures reported nowadays for OTR truckers are probably considerably lower than actual—perhaps by as much as twenty percent.

For many years the tax code has supplied OTR truck drivers with a surprisingly generous deduction called the standard meal allowance. For the past several years it has been $47.20 (80% of $59) per day on the road. Multiplied by 300 days (a typical number), this equals $14,160. The driver’s log book suffices to document how many days were spent on the road.

This sweet tax deduction sours into something called per diem pay when employers decide to get in the middle and “reimburse” drivers instead. They get the money to pay for this by reducing wages. Some companies report this nontaxable pay in Box 14 “Other” on the W‑2 but many do not report it at all; they are not required to.

Per diem pay is bad for drivers and good for companies. Companies mainly benefit from the reduction in payroll taxes. Per diem can also subtly reduce driver vacation pay per week: many companies pay 1/52 of the previous year’s earned income. Drivers also get socked with an “administrative fee” when per diem pay is used. I’ve seen this fee as high as 3% of gross income. The other downsides to per diem are too numerous to go into here. More info can be found in this article.

The reason I left my longtime, relatively well-paying OTR job five years ago wasn’t so much because my employer switched over to using per diem pay, it was more because of a deceptive “more pay in your pocket” ad campaign it foisted on drivers while per diem was still optional. Most drivers were not fooled by it. After spouting the same nonsense over and over for nearly two years, the company finally made per diem mandatory in late 2009.

Rather than resign immediately, I decided to stay on until the following June when my annual vacation pay would accrue. Without knowing it at the time, this decision would also bring me the million-mile safe driving award (during calendar years in which I had no preventable accidents, my paid miles at this company added up to more than a million by 2010). The purpose of these awards is mainly to promote the company image, but they do also look good on the resume, so I gladly accepted.

Closing Remarks

Returning to the theme of pros and cons, I close with a few pros:

  • It doesn’t take an inordinate amount of training to get behind the wheel of a truck. Many say more training should be required—and they are probably right—but as of now it doesn’t take that much. Viewed strictly within the context of job-seeking my years of formal math education have served little to no useful purpose to date. By contrast, in 1994 after just a three-week training course plus three more weeks of paid, on-the-road training, I was earning a modest living doing something that didn’t even seem like work (it still doesn’t).
  • One of the standard pros people cite is you don’t have a boss breathing down your neck. You do have one telling you where to go and when, but yes, the physical distance helps. Unfortunately inward-facing cameras threaten to obliterate the no-boss-breathing advantage.
  • It’s real easy to get out of jury duty if you’re an OTR driver and you don’t want to go.
Categories: Uncategorized

Uber drivers’ collective action problem

I’ve been enjoying thinking about ways for Uber drivers to game the surge pricing algorithm at Uber. I don’t know how it works, exactly, but I’m going to imagine that it’s along these lines:

  • there are well-defined neighborhoods in a city. This seems to be corroborated by the way the Uber app works for both drivers and riders.
  • in a given neighborhood, there are two groups: people asking for a ride who haven’t yet been picked up, and drivers looking to give a ride.
  • If the number of riders is 5 more than the number of drivers, then it becomes a “surge zone” for some amount of time, say 30 minutes.

Of course, I made up the numbers 5 and 30, but I’m guessing it’s more or less of this form, and those particular values don’t matter for the rest of the discussion anyway.

So here the thing, Uber wants to keep their riders happy, but to do that they actually tend to want to avoid creating surge situations, since surge situations usually imply riders wait longer and pay more. On the other hand, Uber drivers prefer surges, since they get paid more, and sometimes much more.

That means Uber drivers have a great incentive to game the system and create artificial surges. One way they can do this is by waiting outside an area that might become surge, wait for it to become surge, and then go into that area and swoop up a rider.

But it would make a lot more sense for drivers to work together to do this. Imagine what would happen if all the drivers agreed to sit together in some central location, wait for surge pricing somewhere, and then assign people in order to go get those riders. Pretty much all the rides would become surge. Again, that wouldn’t make the riders happy, but it would benefit the drivers.

All they’d need to coordinate this is something like a walkie talkie system. Or an app. And oh, wait, such a thing already exists, and it’s called Blinkr (hat tip Alex Rosenblat). Instead of congregating in the same place, though, they had an even simpler idea, namely to turn off their Uber app, thus decreasing the local supply of drivers, then wait for surge pricing.

It’s something like an Uber strike, and it requires coordination, but I don’t think it’s illegal, right? I mean, Uber can’t fire them for doing this, since they aren’t employees, right?

Categories: Uncategorized

Occupy Summer School is in the New Yorker!

I’m super proud to say that Alex Carp, a journalist who was present for more than half of Occupy Summer School last month, did a fantastic job of writing up the OSS experiment for the New Yorker’s Talk Of The Town column.

Here is Alex’s New Yorker Piece, called “Protest U,” please enjoy!

We also got coverage from the German Public Radio, which includes a picture:

I'm the blue-haired person on the left.

I’m the blue-haired person on the left.

Anyone who speaks German can go ahead and tell me if it’s a nice piece.

Categories: Uncategorized

Get every new post delivered to your Inbox.

Join 3,604 other followers