Going to Haiti next week

I’m off to Haiti next week, for a week, with my buddie and bandmate Jamie Kingston. I was trying to figure out what to do with the blog while I was gone, and so I asked sometimes-guest blogger Becky Jaffe to cover for me (some of you may remember her Hip Hop’s Cambrian Explosion series which to this day gets traffic) but by the time I’d explained my trip, she’d decided to come along too! Which is awesome. We’re staying at the Hotel Oloffson in Port au Prince:

It has a room called "the Mick Jagger room".

It has a room called “the Mick Jagger room”.

So two things. First, if you know of fun stuff to do in the Port au Prince area, please tell me. I tend to like talking to people, and music and crafts, and Becky and Jamie are more into nature and insects.

Second, if you have a lovely or inspiring suggestion for what should happen to mathbabe next week while we’re away, please tell me!

Categories: Becky Jaffe, musing

“Hand To Mouth” and the rationality of the poor

Here’s one thing that you do as a mathematician a lot: change the assumptions and see how wildly the conclusions change. You usually start with lots of assumptions, and then see how things change when they are taken away one by one: what if the ring isn’t commutative? What if it doesn’t have a “1″?

Of course, it’s easy enough to believe that we can no longer prove the same theorems when we don’t start with the same kinds of mathematical set-ups. But this kind of thing can also apply to non-mathematical scenarios as well.

So, for example, I’ve long thought that the “marshmallow” experiment is nearly universally misunderstood: kids wait for the marshmallow for exactly as long as it makes sense to them to wait. If they’ve been brought up in an environment where delayed gratification pays off, and where the rules don’t change in the meantime, and where they trust a complete stranger to tell them the truth, they wait, and otherwise they don’t – why would they? But since the researchers grew up in places where it made sense to go to grad school, and where they respect authority and authority is watching out for them, and where the rules once explained didn’t change, they never think about those assumptions. They just conclude that these kids have no will power.

Similarly, this GoodBooksRadio interview with Linda Tirado is excellent in explaining the rational behavior of poor people:

Tirado just came out with a book called Hand To Mouth: Living in Bootstrap America and was discussing it with Dr. John Cook, who was a fantastic interviewer. You might have come across Tirado’s writing – her essay on poverty that went viral, or the backlash against that essay. She’s clearly a tough cookie, a great writer, and an articulate speaker.

Among the things she explains is why poor people eat McDonalds food (it’s fast, cheap, and filling), why they don’t get much stuff done (their lives are filled with logistics), why they make bad decisions (stress), and, what’s possibly the most important, how much harder work it is to be poor than it is to be rich. She defines someone as “rich” if they don’t lease their furniture.

I’m looking forward to reading her book. As the Financial Times review says, “Hand to Mouth – written with scorching flair – should be read by every person lucky enough to have a disposable income.”

Categories: economics

Aunt Pythia’s advice

Well, hello and good morning! Glad you all could make it onto Aunt Pythia’s magic bus today! I’ve redecorated to celebrate Daylight Savings Time (or rather, the end of it):

We can all get in! Squeeeeeze!!!

We can all get in! Squeeeeeze!!!

Daylight savings time has made Aunt Pythia very happy today, because it means an extra hour for me to focus on you, you and your problems, which is what Aunt Pythia loves to do, at least on Saturday mornings, and at least when they involve sex or math (or ideally, both).

By the way, to investigate and demolish the myths around Daylight Savings Time, check out this fantastic and scientific video (best line, “waking up is like sneezing”).

Before we dig in to this week’s juicy questions, Aunt Pythia has an unusual request. Do you remember a couple of weeks ago, when I dragged my family apple picking? Well it turns out that a bushel of apples is A LOT OF APPLES, and I’m really very sick of apples, apple pies (current count: 9 pies made in the past 2 weeks), and apple sauce. If anyone wants some apples, swing on by and I’ll hook you up. Please. Oh, and also:

please think of something interesting, reasonable, and non-apple related

to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

I am pretty much fine with pornography, aside from instances in which women are blatantly coerced or otherwise not participating of free will.

My question pertains to the increasing prevalence of extreme porn and how it impacts real relationships. As with our news and so on, everything has become click bait. Remember back when lesbian porn was risque? Now if a girl isn’t sucking a dick that was just in her ass the previous minute, it’s considered sorta boring. Next thing you know, men imagine women should be doing all these things in real life.

Trying to frame a question here, how does one be generally supportive of the existence of pornography and also help men understand that she is “not doing that” without coming off like a prude? Moreover, when encountering such a man, is it better to just tell them to fuck off entirely? I cannot imagine that any man who is obsessed with the idea to jizz in my eyeball can have actual respect for me as a human being.

Wondering Tolerant Female

Dear WTF,

Great question!

But before we go there, how can you be sure someone isn’t being coerced? I can’t, so I prefer the animated kind of pornography, preferably Japanese, because those Japanese animators are totally perverted and awesome, and then there’s really nobody being coerced. Perhaps TMI about Aunt Pythia, but since I didn’t tell you which of the hundreds of subgenres of Japanese anime I’m into, you really don’t know much – trust me.

Now, on to your actual question. I agree that the realm of “normal sex” has moved by more than a few notches recently. When I was in high school, there was no internet, so we actually had to steal our parent’s dirty magazines and VCR tapes – lots of them – to figure stuff out. Come to think of it, at least where I came from, it really wasn’t hard to come across porn, and moreover I remember it being insanely misogynistic and violent, almost always involving rape of a clearly drugged-up woman. From that vantage point the weird, rape-dominated scenes from the 1980′s have been replaced by weird but consensual extreme positions of today, and I’m personally glad to make that trade.

I’m not a historian of porn, though, I so I might be getting this all wrong, and yes of course I know there’s lots of very extreme stuff available nowadays as well.

In terms of respect for someone as a human being, I’m not sure we’re speaking the same language. There’s nothing logically inconsistent with thoroughly objectivizing a sexual partner during a sex act and then having a mutually respectful and thoughtful conversation about free will fifteen minutes later. It’s all about what you’ve agree to, and what’s fun for you.

So in other words, if you don’t want to be doing this stuff, that’s fine and you shouldn’t agree to it, but measure someone’s respect for you by how they bring up the question, not whether they want to do it. In other words, the man who doesn’t respect you is the man who pushes this stuff on you without consultation, or who makes it your problem that you’re not into it.

Good luck!

Aunt Pythia


Dear Aunt Pythia,

The Empire Builder (Amtrak) takes around 46 hours to reach Seattle from Chicago (if it’s on time). Besides the amazing scenery, the trip offers the possibility of scintillating conversations with strangers in the Dining Car. A flight between the two cities, on the other hand, will take just 5 hours. It would be much cheaper, but would otherwise be a nondescript experience. While air travel is the pragmatic choice, the rail option underscores the point that sometimes the journey is as interesting as the destination. As I teach an informal math course to some colleagues, I often find that we are conditioned to find the shortest path to the answer. In many “toy problems” that we discuss in class, it is the path to the answer that is relevant to the real world. The actual answer is relatively inconsequential. Should math be taught differently so that it is more akin to train travel than flying? If so, what would you recommend to make math teaching more contemplative? And would these approaches be scalable, i.e., work in structured courses with larger enrollment?

Obsessive Correlator

Dear OC,

This question is also great, and has a much smaller chance of having been stolen from Savage Love.

I have often fantasized about taking a sleeper train across the country, with my whole family in tow, and meeting people in the dining car and having fascinating conversations. I’ve even priced it out, and it’s expensive but not impossible. I got the idea from a mathematician who had traveled with his family on the Orient Express in the 1980′s, which is even more fantastical (and expensive). Can you imagine getting on a train in Paris and getting off in Hong Kong? How cool would that be?


Even the angle of Europe on this map is mysterious.

Back to your question. Why yes, I think a meandering route through mathematics would be wonderful, and is sadly almost never done. We are so obsessed with skills-based accomplishments, we rarely spend time on why we’re doing something or how someone could have come up with it in the first place.

One of my few regrets of leaving Barnard is that I never had a chance to run a freshman seminar course on mathematics that I’d planned in the style of the Pythagorean Society (minus some of their crazy rules like “not picking up that which has fallen”).

It is my earnest belief that every person engaged in learning mathematics is themselves a mathematician, rediscovering and rejoicing in the mathematics that has been understood by our culture for hundreds of years but by us as individuals for no time at all. We should all be treated as philosopher queens in this process, and so my idea was to do that in a wifi-blocked room, focusing on the questions we pose and how we pose them and what patterns we might find and why we’d care (or not!) about them beyond their intrinsic beauty.

Sounds great! I still wish I could do that. And I also hope that other people do that.

So here’s the thing. Most people think of mathematicians as super lazy, and there’s of course something to that; lots of mathematical breakthroughs are essentially proven shortcuts to long-ass calculations that go something like, “we have a collection of things, and some of them have this cool property that makes them easy to understand, and now I’ve proven that all of them actually have that cool property.”

But at the same time, mathematicians are also the most inefficient people in the world, because they get entirely focused on abstract rules and scenarios that almost never have a concrete application to anything, and they think about the patterns they notice for hours. They are all about the meandering path, in other words. It’s not a bad life, but it does take time.

Finally, to your question: when it’s a small group, consider yourself a facilitator rather than a teacher. Ask questions and get people involved in the discovery. Make a silent pact with yourself that you won’t explain anything directly, that you will only issue hints, and try to emphasize the beauty and truth in everyone’s contributions. With a larger class it’s much harder, but sometimes you can get the right atmosphere and then have people work in groups.

Good luck!

Aunt Pythia


Dear Aunt Pythia,

I’m a senior male professor in a STEM department. Here’s my question. What, if anything, should I say about romantic relationships between faculty members and graduate students? In particular, what action should I take concerning a professor who has dated at least three graduate students in our department? There is no formal rule at our university against faculty/student dating, as long as the faculty member has no direct supervisory relationship with the student. What’s more, there is a senior faculty member who is married to a woman he started dating when she was a graduate student here, which makes it awkward to denounce such relationships in general. And I know that Aunt Pythia herself is married to someone she met when she was a grad student and he was faculty!

So you could argue it’s none of my business. But you could also argue it’s rotten to put our grad students in a position of feeling like they’re a captive dating pool for the single faculty members. I know that our graduate students are aware of the serial dating; no grad student has directly told me that they find it threatening or off-putting, but another faculty member (a junior woman) has told me that she thinks it’s bad for the department.

What do you think, Aunt Pythia? Talk to the serial dater himself? Talk to the department chair? Or butt out and say nothing to anyone?

Tenured Professor at a Singles Bar

Dear TPaaSB,

It’s true, I was a grad student and my husband was a post-doc in the same department, but I’d argue that’s a bit different from his being a professor. Even so, I’d probably have dated him even if he had been, so there’s that as well.

I’m not sure how much anyone can do about this, to be honest. You can make rules but then people will probably break them. Not sure if that’s better.

On the one hand, you want graduate students to feel safe and not sexualized in their role as learners, and having the feeling that you might be “next in line” for this professor isn’t helping. It’s particularly unhelpful that he’s dated three, because it is starting to seem like he is both incapable of finding women outside the department and bad at relationships. Or maybe the women all dumped him, who knows. But yes, I agree that this guy is making things weird.

On the other hand, there’s a moment in your life as a man or a woman that you decide it’s time to look around for a life partner, and if you’re a 23-year-old woman who wants kids, like I was, then the men your age simply burst out crying in your presence from the pressure of commitment, and you end up looking for older, more stable, and more mature men that aren’t intimidated by your brains and your life plans. You could look outside the department, but the problem is you spend almost all your time in the department and there are all these yummy smart nerd boys who look great in homemade sweaters would look great in your homemade sweater, so whatareyagonnado.

In terms of advice for you, I’m going to say to keep quiet, unless you feel like the guy is actually predatory or is fucking with these women’s egos or chances of graduating. If that’s true, then talk to him and voice your concerns.

Readers, if you disagree, by all means chime in.

Aunt Pythia


Dear Aunt Pythia,

I just met a twenty-something hot girl online (only ten years or so younger than me); her interests (her words) are work, martinis, and rough sex (not necessarily in that order). We’ve met once so far, and it was everything I could have hoped for — spitting, faceslapping, and some dirty talk. But I know that the best way to great sex is though pleasing your partner, so for the next time we meet I want to step up my game. She said she was game for anything, so I want to be creative without crossing any boundaries. She was happy last time to be called a slut and a whore, but maybe there’s something more original as far as dirty talk goes (for example, how can I make it more interactive by forcing her to respond in some way?) Also, she’s a gorgeous BBW (mmmm), so (given the context) is calling her a fat whore a good idea? My impression is that she is naturally very confident and outspoken, so I am imagining her fantasies stem from a positive rather than negative aspect of her personality, but I really don’t know…

Naughty and Salacious; Tenured Young.

Dear NaSTY,

Aahhh, the triple fantasy of spitting, faceslapping, and dirty talk. You’re living the dream, buddy, there’s no doubt. I mean, if you’re into that kind of thing. Which you obviously are.

As far as whether she wants you to call her a “fat whore,” my guess is she wouldn’t be offended if you tried it. It’s not like fat people don’t know they’re fat! We get told it every day of our lives, so turning it around and making it a good thing (erm, in this context I think it qualifies as a good thing) might be fun!

If you’re worried about it, ask her before the next tryst, “hey honey, would it be ok if I call you a fat whore during sex? I find your body exquisite and it turns me on to talk about it.”

Auntie P


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Click here for a form.


Categories: Aunt Pythia

Tailored political ads threaten democracy

Not sure if you saw this recent New York Times article on the new data-driven political ad machines. Consider for example, the 2013 Virginia Governor campaign won by Terry McAuliffe:

…the McAuliffe campaign invested heavily in both the data and the creative sides to ensure it could target key voters with specialized messages. Over the course of the campaign, he said, it reached out to 18 to 20 targeted voter groups, with nearly 4,000 Facebook ads, more than 300 banner display ads, and roughly three dozen different pre-roll ads — the ads seen before a video plays — on television and online.

Now I want you to close your eyes and imagine what kind of numbers we will see for the current races, not to mention the upcoming presidential election.

What’s crazy to me about the Times article is that it never questions the implications of this movement. The biggest problem, it seems, is that the analytics have surpassed the creative work of making ads: there are too many segments of populations to tailor the political message to, and not enough marketers to massage those particular messages for each particular segment. I’m guessing that there will be more money and more marketers in the presidential campaign, though.

Translation: politicians can and will send different messages to individuals on Facebook, depending on what they think we want to hear. Not that politicians follow through with all their promises now – they don’t, of course – but imagine what they will say when they can make a different promise to each group. We will all be voting for slightly different versions of a given story. We won’t even know when the politician is being true to their word – which word?

This isn’t the first manifestation of different messages to different groups, of course. Romney’s famous “47%” speech was a famous example of tailored messaging to super rich donors. But on the other hand, it was secretly recorded by a bartender working the event. There will be no such bartenders around when people read their emails and see ads on Facebook.

I’m not the only person worried about this. For example, ProPublica studied this in Obama’s last campaign (see this description). But given the scale of the big data political ad operations now in place, there’s no way they – or anyone, really – can keep track of everything going on.

There are lots of ways that “big data” is threatening democracy. Most of the time, it’s by removing open discussions of how we make decisions and giving them to anonymous and inaccessible quants; think evidence-based sentencing or value-added modeling for teachers. But this political campaign ads is a more direct attack on the concept of a well-informed public choosing their leader.

Categories: data science, modeling, rant

Core Econ: a free economics textbook

Today I want to tell you guys about core-econ.org, a free (although you do have to register) textbook my buddy Suresh Naidu is using this semester to teach out of and is also contributing to, along with a bunch of other economists.


This was obviously not taken in New York.

It’s super cool, and I wish a class like that had been available when I was an undergrad. In fact I took an economics course at UC Berkeley and it was a bad experience – I couldn’t figure out why anyone would think that people behaved according to arbitrary mathematical rules. There was no discussion of whether the assumptions were valid, no data to back it up. I decided that anybody who kept going had to be either religious or willing to say anything for money.

Not much has changed, and that means that Econ 101 is a terrible gateway for the subject, letting in people who are mostly kind of weird. This is a shame because, later on in graduate level economics, there really is no reason to use toy models of society without argument and without data; the sky’s the limit when you get through the bullshit at the beginning. The goal of the Core Econ project is to give students a taste for the good stuff early; the subtitle on the webpage is teaching economics as if the last three decades happened.

What does that mean? Let’s take a look at the first few chapters of the curriculum (the full list is here):

  1. The capitalist revolution
  2. Innovation and the transition from stagnation to rapid growth
  3. Scarcity, work and progress
  4. Strategy, altruism and cooperation
  5. Property, contract and power
  6. The firm and its employees
  7. The firm and its customers

Once you register, you can download a given chapter in pdf form. So I did that for Chapter 6, The firm and its employees, and here’s a screenshot of the first page:

Still dry but at least real.

Still dry but at least real.

The chapter immediately dives into a discussion of Apple and Foxconn. Interesting! Topical! Like, it might actually help you understand the newspaper!! Can you imagine that?

The project is still in beta version, so give it some time to smooth out the rough edges, but I’m pretty excited about it already. It has super high production values and will squarely compete with the standard textbooks and curriculums, which is a good thing, both because it’s good stuff and because it’s free.

The war against taxes (and the unmarried)

The American Enterprise Institute, conservative think-tank, is releasing a report today. It’s called For richer, for poorer: How family structures economic success in America, and there is also an event in DC today from 9:30am til 12:15pm that will be livestreamed. The report takes a look at statistics for various races and income levels at how marriage is associated with increased hours works and income, for men especially.

It uses a technique called the “fixed-effects model,” and since I’d never studied that I took a look at it on the wikipedia page, and in this worked-out example on Josh Blumenstock’s webpage of massage prices in various cities, and in this example, on Richard William’s webpage, where it’s also a logit model, for girls in and out of poverty.

The critical thing to know about fixed effects models is that we need more than one snapshot of an object of interest – in this case a person who is or isn’t married – in order to use that person as a control against themselves. So in 1990 Person A is 18 and unmarried, but in 2000 he is 28 and married, and makes way more money. Similarly, in 1990 Person B is 18 and unmarried, but in 2000 he is 28 and still unmarried, and makes more money but not quite as much more money as Person A.

The AEI report cannot claim causality – and even notes as much on page 8 of their report – so instead they talk about a bunch of “suggested causal relationships” between marriage and income. But really what they are seeing is that, as men get more hours at work, they also tend to get married. Not sure why the married thing would cause the hours, though. As women get married, they tend to work fewer hours. I’m guessing this is because pregnancy causes both.

The AEI report concludes, rightly, that people who get married, and come from homes where there were married parents, make more money. But that doesn’t mean we can “prescribe” marriage to a population and expect to see that effect. Causality is a bitch.

On the other hand, that’s not what the AEI says we should do. Instead, the AEI is recommending (what else?) tax breaks to encourage people to get married. Most bizarre of their suggestions, at least to me, is to expand tax benefits for single, childless adults to “increase their marriageability.” What? Isn’t that also an incentive to stay single and childless?

What I’m worried about is that this report will be cleverly marketed, using the phrase “fixed effects,” to make it seem like they have indeed proven “mathematically” that individuals, yet again, are to be blamed for the structural failure of our nation’s work problems, and if they would only get married already we’d all be ok and have great jobs. All problems will be solved by tax breaks.

Categories: economics, modeling, rant

Guest post: Clustering and predicting NYC taxi activity

This is a guest post by Deepak Subburam, a data scientist who works at Tessellate.


from NYCTaxi.info

Greetings fellow Mathbabers! At Cathy’s invitation, I am writing here about NYCTaxi.info, a public service web app my co-founder and I have developed. It overlays on a Google map around you estimated taxi activity, as expected number of passenger pickups and dropoffs this current hour. We modeled these estimates from the recently released 2013 NYC taxi trips dataset comprising 173 million trips, the same dataset that Cathy’s post last week on deanonymization referenced. Our work will not help you stalk your favorite NYC celebrity, but guide your search for a taxi and maybe save some commute time. My writeup below shall take you through the four broad stages our work proceeded through: data extraction and cleaning , clustering, modeling, and visualization.

We extract three columns from the data: the longitude and latitude GPS coordinates of the passenger pickup or dropoff location, and the timestamp. We make no distinction between pickups and dropoffs, since both of these events imply an available taxicab at that location. The data was generally clean, with a very small fraction of a percent of coordinates looking bad, e.g. in the middle of the Hudson River. These coordinate errors get screened out by the clustering step that follows.

We cluster the pickup and dropoff locations into areas of high density, i.e. where many pickups and dropoffs happen, to determine where on the map it is worth making and displaying estimates of taxi activity. We rolled our own algorithm, a variation on heatmap generation, after finding existing clustering algorithms such as K-means unsuitable—we are seeking centroids of areas of high density rather than cluster membership per se. See figure below which shows the cluster centers as identified by our algorithm on a square-mile patch of Manhattan. The axes represent the longitude and latitude of the area; the small blue crosses a random sample of pickups and dropoffs; and the red numbers the identified cluster centers, in descending order of activity.

Taxi activity clusters

We then model taxi activity at each cluster. We discretize time into hourly intervals—for each cluster, we sum all pickups and dropoffs that occur each hour in 2013. So our datapoints now are triples of the form [<cluster>, <hour>, <activity>], with <hour> being some hour in 2013 and <activity> being the number of pickups and dropoffs that occurred in hour <hour> in cluster <cluster>. We then regress each <activity> against neighboring clusters’ and neighboring times’ <activity> values. This regression serves to smooth estimates across time and space, smoothing out effects of special events or weather in the prior year that don’t repeat this year. It required some tricky choices on arranging and aligning the various data elements; not technically difficult or maybe even interesting, but nevertheless likely better part of an hour at a whiteboard to explain. In other words, typical data science. We then extrapolate these predictions to 2014, by mapping each hour in 2014 to the most similar hour in 2013. So we now have a prediction at each cluster location, for each hour in 2014, the number of passenger pickups and dropoffs.

We display these predictions by overlaying them on a Google maps at the corresponding cluster locations. We round <activity> to values like 20, 30 to avoid giving users number dyslexia. We color the labels based on these values, using the black body radiation color temperatures for the color scale, as that is one of two color scales where the ordering of change is perceptually intuitive.

If you live in New York, we hope you find NYCTaxi.info useful. Regardless, we look forward to receiving any comments.


Get every new post delivered to your Inbox.

Join 2,045 other followers