Author Archive

The Lede Program has awesome faculty

A few weeks ago I mentioned that I’m the Program Director for the new Lede Program at the Columbia Graduate School of Journalism. I’m super excited to announce that I’ve found amazing faculty for the summer part of the program, including:

  1. Jonathan Soma, who will be the primary instructor for Basic Computing and for Algorithms
  2. Dennis Tenen, who will be helping Soma in the first half of the summer with Basic Computing
  3. Chris Wiggins, who will be helping Soma in the second half of the summer with Algorithms
  4. An amazing primary instructor for Databases who I will announce soon,
  5. Matthew Jones, who will help that amazing yet-to-be-announced instructor in Data and Databases
  6. Three amazing TA’s: Charles Berret, Sophie Chou, and Josh Vekhter (who doesn’t have a website!).

I’m planning to teach The Platform with the help of a bunch of generous guest lecturers (please make suggestions or offer your services!).

Applications are open now, and we’re hoping to get amazing students to enjoy these amazing faculty and the truly innovative plan they have for the summer (and I don’t use the word “innovative” lightly!). We’ve already gotten some super strong applications and made a couple offers of admission.

Also, I was very pleased yesterday to see a blogpost I wrote about the genesis and the goals of the program be published in PBS’s MediaShift.

Finally, it turns out I’m a key influencer, according to The Big Roundtable.

The US political system serves special interests and the rich

A paper written by Martin Gilens and Benjamin Page and entitled Testing Theories of American Politics: Elites, Interest Groups, and Average Citizens has been recently released and reported on (h/t Michael Crimmins) that studies who has influence on policy in the United States.

Here’s an excerpt from the abstract of the paper:

Multivariate analysis indicates that economic elites and organized groups representing business interests have substantial independent impacts on U.S. government policy, while average citizens and mass-based interest groups have little or no independent influence.

A word about “little or no independent influence”: the above should be interpreted to mean that average citizens and mass-based groups only win when their interests align with economic elites, which happens sometimes, or business interests, which rarely happens. It doesn’t mean that average citizens and mass-based interest groups never ever get what they want.

There’s actually a lot more to the abstract, about abstract concepts of political influence, but I’m ignoring that to get to the data and the model.

The data

The found lots of polls on specific issues that were yes/no and included information about income to determine what poor people (10th percentile) thought about a specific issue, what an average (median income) person thought, and what a wealthy (90th percentile) person thought. They independently corroborated that their definition of wealthy was highly correlated, in terms of opinion, to other stronger (98th percentile) definitions. In fact they make the case that using 90th percentile instead of 98th actually underestimates the influence of wealthy people.

For the sake of interest groups and their opinions on public policy, they had a list of 43 interest groups (consisting of 29 business groups, 11 mass-based groups, and 3 others) that they considered “powerful” and they used domain expertise to estimate how many would oppose or be in favor of a given issue, and more or less took the difference, although they actually did something a bit fancier to reduce the influence of outliers:

Net Interest Group Alignment = ln(# Strongly Favor + [0.5 * # Somewhat Favor] + 1) – ln(#
Strongly Oppose + [0.5 * # Somewhat Oppose] + 1).

Finally, they pored over records to see what policy changes were actually made in the 4 year period after the polls.


The different groups had opinions that were sometimes highly correlated:

Screen Shot 2014-04-17 at 6.40.59 AM

Note the low correlation between mass public interest groups (like unions, pro-life, NRA, etc) and average citizens’ preferences and the negative correlation between business interests and elites’ preferences.

Next they did three bivariate regressions, measuring the influence of each of the groups separately, as well as one including all three, and got the following:

Screen Shot 2014-04-17 at 6.46.55 AM

This is where we get our conclusion that average citizens don’t have independent influence, because of this near-zero coefficient in Model 4. But note that if we ignore elites and interest groups, we do have 0.64 in Model 1, which indicates that preferences of the average citizens are correlated with outcomes.

The overall conclusion is that policy changes are determined by the elites and the interest groups.

We can divide the interest groups into business versus mass-based and check out how the influence is divided between the four defined groups:

Screen Shot 2014-04-17 at 6.52.19 AM


This stuff might depend a lot on various choices the modelers made as well as their proxies. It doesn’t pick up on smaller special interest groups. It doesn’t account for all possible sources of influence and so on. I’d love to see it redone with other choices. But I’m impressed anyway with all the work they put into this.

I’ll let the authors have the last word:

What do our findings say about democracy in America? They certainly constitute troubling news for advocates of “populistic” democracy, who want governments to respond primarily or exclusively to the policy preferences of their citizens. In the United States, our findings indicate, the majority does not rule — at least not in the causal sense of actually determining policy outcomes. When a majority of citizens disagrees with economic elites and/or with organized interests, they generally lose. Moreover, because of the strong status quo bias built into the U.S. political system, even when fairly large majorities of Americans favor policy change, they generally do not get it.

Categories: #OWS, modeling

How recently have you experienced democracy? #OWS

A few weeks ago Omar Freilla came to talk to my Occupy group. Omar is a founder of the Green Worker Cooperatives and shared his experience as an organizer.

He is a well-spoken guy and talked passionately about forming community cooperatives, where workers “have a direct role in decision-making and a share of all profits, build community wealth and help make a democratic economy real.”

At one point in his presentation, Omar asked us was how recently we’d “experienced democracy.”

On the face of it I didn’t think it was a fair question, especially when he compared it to experiencing anger or happiness. After all, democracy isn’t an emotion and I can’t experience democracy, say, by myself in a room, but of course I can conjure up emotions by myself in a room, especially if I have a laptop, wifi, and Netflix to help me.

But since his visit, I have to admit I have dwelled on that question and it’s become more and more reasonable in my mind, although I made two decisions on how to interpret it.

First of all, I chose to interpret it not as a formal gesture of democracy, like asking how recently have you voted in a formal election. Instead, it’s a local decision-making process question: how recently has your vote mattered in a local decision that affects a group?

Second, it’s not really about me. It’s about looking around and deciding who around me gets to participate in democratic decisions and who doesn’t.

For example, it might be at work. Although I personally get to make a lot of decisions at work, that fact clearly separates me from tons of people who simply get told what to do by some kind of authority. And there is an important distinction between people who have a manager but get to make decisions internal to their projects and people who have every decision laid out for them.

And that latter workplace anti-democratic situation is, I imagine, maximally soul-crushing, and is the audience that Omar is worried about and is reaching out to. And that’s why his question turns out to be a really good question after all.

I also consider democracy inside my own family. Since I’m the mom of the family, I tend to make more decisions that affect my little group than other people, but now I’m more sensitive to sharing that power there when I can. Turns out my kids love making decisions, it makes them gleeful in fact, even if it’s just what to eat for dinner. And they make good decisions too, which I’m consistently proud of.

My final example is Occupy, which is by construction a direct democracy, and I know how good participating and experiencing democracy actually feels there, and it’s a big part of why it works.

What about you? How recently have you experienced democracy?

Categories: #OWS

Let’s experiment more

What is an experiment?

The gold standard in scientific fields is the randomized experiment. That’s when you have some “treatment” you want to impose on some population and you want to know if that treatment has positive or negative effects. In a randomized experiment, you randomly divide a population into a “treatment” group and a “control group” and give the treatment only to the first group. Sometimes you do nothing to the control group, sometimes you give them some other treatment or a placebo. Before you do the experiment, of course, you have to carefully define the population and the treatment, including how long it lasts and what you are looking out for.

Example in medicine

So for example, in medicine, you might take a bunch of people at risk of heart attacks and ask some of them – a randomized subpopulation – to take aspirin once a day. Note that doesn’t mean they all will take an aspirin every day, since plenty of people forget to do what they’re told to do, and even what they intend to do. And you might have people in the other group who happen to take aspirin every day even though they’re in the other group.

Also, part of the experiment has to be well-defined lengths and outcomes of the experiment: after, say, 10 years, you want to see how many people in each group have a) had heart attacks and b) died.

Now you’re starting to see that, in order for such an experiment to yield useful information, you’d better make sure the average age of each subpopulation is about the same, which should be true if they were truly randomized, and that there are plenty of people in each subpopulation, or else the results will be statistically useless.

One last thing. There are ethics in medicine, which make experiments like the one above fraught. Namely, if you have a really good reason to think one treatment (“take aspirin once a day”) is better than another (“nothing”), then you’re not allowed to do it. Instead you’d have to compare two treatments that are thought to be about equal. This of course means that, in general, you need even more people in the experiment, and it gets super expensive and long.

So, experiments are hard in medicine. But they don’t have to be hard outside of medicine! Why aren’t we doing more of them when we can?

Swedish work experiment

Let’s move on to the Swedes, who according to this article (h/t Suresh Naidu) are experimenting in their own government offices on whether working 6 hours a day instead of 8 hours a day is a good idea. They are using two different departments in their municipal council to act as their “treatment group” (6 hours a day for them) and their “control group” (the usual 8 hours a day for them).

And although you might think that the people in the control group would object to unethical treatment, it’s not the same thing: nobody thinks your life is at stake for working a regular number of hours.

The idea there is that people waste their last couple of hours at work and generally become inefficient, so maybe knowing you only have 6 hours of work a day will improve the overall office. Another possibility, of course, is that people will still waste their last couple of hours of work and get 4 hours instead of 6 hours of work done. That’s what the experiment hopes to measure, in addition to (hopefully!) whether people dig it and are healthier as a result.

Non-example in business: HR

Before I get too excited I want to mention the problems that arise with experiments that you cannot control, which is most of the time if you don’t plan ahead.

Some of you probably ran into an article from the Wall Street Journal, entitled Companies Say No to Having an HR Department. It’s about how some companies decided that HR is a huge waste of money and decided to get rid of everyone in that department, even big companies.

On the one hand, you’d think this is a perfect experiment: compare companies that have HR departments against companies that don’t. And you could do that, of course, but you wouldn’t be measuring the effect of an HR department. Instead, you’d be measuring the effect of a company culture that doesn’t value things like HR.

So, for example, I would never work in a company that doesn’t value HR, because, as a woman, I am very aware of the fact that women get sexually harassed by their bosses and have essentially nobody to complain to except HR. But if you read the article, it becomes clear that the companies that get rid of HR don’t think from the perspective of the harassed underling but instead from the perspective of the boss who needs help firing people. From the article:

When co-workers can’t stand each other or employees aren’t clicking with their managers, Mr. Segal expects them to work it out themselves. “We ask senior leaders to recognize any potential chemistry issues” early on, he said, and move people to different teams if those issues can’t be resolved quickly.

Former Klick employees applaud the creative thinking that drives its culture, but say they sometimes felt like they were on their own there. Neville Thomas, a program director at Klick until 2013, occasionally had to discipline or terminate his direct reports. Without an HR team, he said, he worried about liability.

“There’s no HR department to coach you,” he said. “When you have an HR person, you have a point of contact that’s confidential.”

Why does it matter that it’s not random?

Here’s the crucial difference between a randomized experiment and a non-randomized experiment. In a randomized experiment, you are setting up and testing a causal relationship, but in a non-randomized experiment like the HR companies versus the no-HR companies, you are simply observing cultural differences without getting at root causes.

So if I notice that, at the non-HR companies, they get sued for sexual harassment a lot – which was indeed mentioned in the article as happening at Outback Steakhouse, a non-HR company – is that because they don’t have an HR team or because they have a culture which doesn’t value HR? We can’t tell. We can only observe it.

Money in politics experiment

Here’s an awesome example of a randomized experiment to understand who gets access to policy makers. In an article entitled A new experiment shows how money buys access to Congressan experiment was conducted by two political science graduate students, David Broockman and Josh Kalla, which they described as follows:

In the study, a political group attempting to build support for a bill before Congress tried to schedule meetings between local campaign contributors and Members of Congress in 191 congressional districts. However, the organization randomly assigned whether it informed legislators’ offices that individuals who would attend the meetings were “local campaign donors” or “local constituents.”

The letters were identical except for those two words, but the results were drastically different, as shown by the following graphic:


Conducting your own experiments with e.g. Mechanical Turk

You know how you can conduct experiments? Through an Amazon service called Mechanical Turk. It’s really not expensive and you can get a bunch of people to fill out surveys, or do tasks, or some combination, and you can design careful experiments and modify them and rerun them at your whim. You decide in advance how many people you want and how much to pay them.

So for example, that’s how then-Wall Street Journal journalist Julia Angwin, in 2012, investigated the weird appearance of Obama results interspersed between other search results, but not a similar appearance of Romney results, after users indicated party affiliation.


We already have a good idea of how to design and conduct useful and important experiments, and we already have good tools to do them. Other, even better tools are being developed right now to improve our abilities to conduct faster and more automated experiments.

If we think about what we can learn from these tools and some creative energy into design, we should all be incredibly impatient and excited. And we should also think of this as an argumentation technique: if we are arguing about whether a certain method or policy works versus another method or policy, can we set up a transparent and reproducible experiment to test it? Let’s start making science apply to our lives.

Categories: data journalism, modeling, rant

People who obsessively exercise are boring

I’m not saying anything you don’t know already. I’m just stating the obvious: people who obsessively exercise are super boring. They talk all the time about their times, and their workout progress, and their aching muscles, and it’s like you don’t even have to be there, you could just replace yourself with a gadget that listens, nods, and then says encouraging things like, “Way to go!” at the very end. Excruciating.

Look, don’t get me wrong. I’ve gone through bouts of obsessive exercise myself, and those bouts sometimes were pretty lengthy. And no, it didn’t ever make me skinny, just incredibly fit. I remember I trained for a sprint triathlon once, and man was I fit by the time it finally happened in the spring on 2004.

But then, when I got to the starting line, and there I was wishing I could reorder the events so the the beginning swim would 5 kilometers and the run at the end were a quarter mile – I’ve never been much of a runner – and I just looked around at myself and everyone else there, and I wondered how I’d become so incredibly boring and self-obsessed that I had paid good money and driven miles and miles just to obsessively exercise in front of other people.

What was going on with me? I became increasingly disgusted by my own boringness throughout the race. I think the worst part was how many people said “You go, girl!” when I jogged by. They were trying to encourage the fat girl, I get it, but it made it even more obvious that I was doing something that I honestly didn’t need to be getting public response to.

Look, I’m not against exercise, and I love doing it, or at least I love having done it because it makes you feel good, and I encourage everyone to be fit and happy. But I’m serious when I say I will no longer tolerate hanging out with people who obsess over it and want to talk to me about their obsession. Too frigging boring, people!

So if someone mentions that they went biking over this gorgeous spring weekend, then awesome, I’ll be happy for them. But if they want to talk about which bike they used, and what their time around Central Park was, and how they’re training for this or that event, then no. I will tell them “sorry but can we talk about something not incredibly boring now?”

Why do I mention this today? Because I finally figured out what my hostility towards the Quantified Self crowd is, and it’s this same thing. All those gadgets and doodads are essentially props to pull out and use to have that same boring conversation that I’ve already refused to give into. So please, don’t show me your sleep tracker or your step monitor and expect me to care. I don’t care.

And don’t get me wrong – again – I know some people will benefit from that kind of thing. And some people actually have illnesses or physical therapy and exercise and particularly quantified exercise might particularly help them keep track of their health! I get it!

But let’s face it, most people are not doing this for health. They are doing it for some other weird, narcissistic and anxiety-shielding coping-mechanistic self-competitive (or outright competitive) reason. And again, I’m not hating on them exactly, because I get it, and I’ve been there. But I don’t want to talk about it with them.

Categories: rant

Aunt Pythia’s advice

Readers, readers!

Aunt Pythia rarely does this, and really has never ever made up a question, but she absolutely needs to share a couple of things that nobody even came close to asking about this week. And yes, they’re about sex, or at least about genitals. Please skip this next section, and possibly all Aunt Pythia columns ever, if you are not interested in vulgar discussions of dirty topics. And yes, she’s taking this as an excuse to use the plural forms.


First, it might not surprise people to learn that Aunt Pythia talks about sex with her girlfriends. A lot. And she’s noticed a pattern in those conversations: some women can orgasm during intercourse, others cannot. And when I say cannot, I don’t mean cannot without help from, say, hands or a toy (Aunt Pythia does not often meet women who are in the alleged 15% of the population who cannot orgasm). I mean women who cannot through intercourse alone, with only a partner’s pelvic bone as tool.

Such an intriguing and natural categorization of women does not leave Aunt Pythia cold! She wants to get to the bottom of this! Why is this not one of the most basic conundrums of human existence, she wonders? Well, it is, and it has been (possibly! hopefully!) solved, in this article. It’s all about measuring distances, and once you say it like that, it makes perfect sense. Mystery revealed.


Aunt Pythia has been thinking a lot about objectification lately, and wondering why it happens so much more to women. One slightly philosophical approach to this question is understanding the extent to which men’s genitals are, to put it delicately, ripe for objectification versus, say, women’s breasts.

And the thing is, dear readers, they are not. Not as a general rule. And that is why, in my opinion, this video is so deeply radical. Warning! Please do not watch this video unless you are ready to have your mind blown! And please don’t say I didn’t warn you about this video.

Addendum: I feel like there should be a phrase invented for that video. Something like, hey man, don’t make me robocop your dick, or hey dude, don’t robocop your own dick. Suggestions welcome.

Back to our regularly scheduled advice column

After enjoying (or not!) today’s Aunt Pythia post and advice:

think of something to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

Inspired by the PDF Hackathon, I was wondering if anyone has scraped any of the sites where things like NECAP and MCAS scores are posted ala Wikileaks. Are these things publicly available, and I just haven’t looked in the right places?.

I’m working on a letter to the chair of a committee in the state legislature that is considering a bill that would exempt teachers rated “highly effective” or “effective” from being evaluated the next three years. The link you posted to the six articles on VAM in New York have provided me with a lot of material. Mandatory inclusion of test scores in teacher evaluations is scheduled to be implemented this year in RI and I’m hoping that if we can drag our feet in RI until the negative effects in early adoption states become widely known maybe we can avoid making some of those mistakes. Unfortunately our education commissioner is one of Jeb Bush’s infamous “chiefs for change” and has a lot of clout.

Confused in RI

Dear Confused,

First, I feel your pain. And I’m not sure which link of mine you were referring to but it might as well be Eugene Stern’s guest post on his fears of a VAM-like model being implemented in New Jersey, which he wrote about a year ago, when it was being pushed by Chris Christie (I wonder what happened to that initiative? Eugene?). In fact please feel free to write your letter with many of the same points Eugene was making.

Second, there’s no way we want student test scores leaked ever, and for this purposes especially. The models themselves are secret, and the amount of details and information necessary to reconstitute even one teacher’s VAM score would expose many innocent people to scrutiny that they don’t deserve and is outright illegal and rightly so.

I’m afraid you will have to play this out politically, not using data. The whole point of it is that they control the data and are using the model as an opaque and threatening weapon.

Aunt Pythia


Dear Aunt Pythia,

My best friend and I used to talk often on the phone (like once a week for an hour, over the course of 5 years) but gradually over the past year, the frequency and length has declined, and also now she never initiates the calls or texts. She’s pretty busy – she is overworked at a job she hates, and has had various boyfriends, and recently moved in with one, and hangs out with friends who live close to her (I don’t). A few times lately she has failed to return my phone calls.

I am a really lonely guy … I have tried basically the kitchen sink at finding a girlfriend, but haven’t been successful yet and so she is who I typically turn to for emotional support. Anyways, I love talking with her, but often times lately I have been feeling sad because I feel so ignored by her, and I kind of feel that I am not valued as a friend. Do you think I should I stop initiating calls and texts? Not because I want to stop being friends with her, but because it’s a bit unfair of me to ask for so much support from her, and I should try to finding an actual girlfriend instead?

Lonely in New York(?!)

Dear Lonely,

I love it when my job is easy and the tough answer has already been provided. Yes, you need to go find your own girlfriend and stop asking one friend for that much support and love. She probably loves you and knows you need to disconnect from her in order to connect with others. Once you have a real live woman to talk to in New York I bet she’ll be psyched for you and more willing to return your (less frequent and less pathetic) calls.

Plus, according to this interactive love map created by Jonathan Soma, you should count your statistically lucky stars that there are 5 extra single women per 100 single men in New York.

And by the way, you definitely should start hanging out with friends first. Making friends is easier than finding a girlfriend and often helps.

Auntie P


Dear Aunt Pythia,

I’m an undergraduate woman at a well-regarded (and presumably well-endowed) university in the South, in fact it once had the winning Putnam team and the winning NCAA basketball team in the same year, or some such nonsense.

Tuition is exorbitant, and in order to afford it, I have turned to some financially rewarding but controversial, um, artistic endeavors, let’s say, in which I exhibit my natural beauty and enhance my acting skills.

Some say these gigs are demeaning to or objectify women, others say they are a freedom of expression of things our culture tends to repress. Anyways, what would your advice to me be? Should I stay in the industry?


Dear BK,

Let’s think about this pragmatically and long-term. Besides the risk of STD’s, which I assume you’re taking care of, I think the main question is the extent to which you are putting your future goals at risk by taking these gigs now, rather than a more standard, presumably lower-paying evening job.

So, how easily identifiable will you be to later potential employers? And what kind of employment are you interested in? In the extreme case, where you want to be a politician and you’re a huge internet porn star, I think we have a problem. On the other extreme, where you want to be a lab scientist and you are stripping at a local bar with no recordings, I don’t think there’s a major risk.

Your actual case is probably in the middle, and although I don’t know what it is, I’d venture to guess that you also don’t know exactly what it is, because you’re probably 19 or 20 or so and don’t know exactly what you’re doing with your life.

So my advice for staying or leaving hinges on this concern, that your options are being at least somewhat narrowed without you really knowing it. Or even if your options are not explicitly being narrowed, I’m hoping you gird yourself for assholes and assumptions, and I hope you can push back.

And when I say that, I’m also saying “fuck this society” under my breath. We are super prudish in the best of times and at less good time we are freaky and hostile towards women. That’s not your fault.

As to whether the industry is demeaning and objectifying, I’d say it’s objectifying almost by definition, and whether it’s demeaning is probably something you will not be able to fully answer until you live with the ramifications of having been in it. So get back to me in 10 years and tell me what you think.

Good luck,

Aunt Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

Envy, greed, and the American Dream #OWS

I was sent this Falkenblog post entitled Why Envy Dominates Greed a while back (hat tip David Murrell). The post suggests an interesting thought experiment which I’d like to discuss this morning.

Namely, it asks us to examine the extent to which our economic assumption that “everyone is working in their own self-interest” can be replaced by the assumption that “everyone is working to improve their relative ranking” and whether you’d get more clarity from economics that way.

I’ve done myself the favor of ignoring everything author Eric Falkenstein actually says about the economic theory, because he’s focusing on investing in the stock market, which honestly only a minority of people ever do even once. Even so I’d like to consider this idea of envy versus greed and try to make sense of it.

First of all, I do think that a certain kind of relativity combined with proximity is deeply important to humans. When members of my Occupy group talk about living on $2 a day while sleeping at homeless shelters in New York City, surrounded by men in suits with chauffeurs, it is very relevant that the privations described are combined with with a deep sense of humiliation of their understanding of their relative position. These are highly intelligent people who know how things look and they feel it keenly.

Similarly, when I think about poor people in other countries, it’s a different level of destitution than we see here, and yet it doesn’t make me want to drop everything and work in India. There’s something about proximity that we all respond to, and which has been well examined by social scientists.

Going back to my New York friend: is that envy being displayed, exactly? I don’t think so. I think it’s something more like dispossession and despair. And it’s honestly something I believe our natures would rather avoid, but sometimes just slaps us in our face, especially in places like New York City.

I’m not throwing envy out altogether. In fact, I do think envy is strongly at work, but only at a local level. I am working at Columbia now, so it’s natural and proper that I am envious of my colleague’s slightly-larger office. I ignore the stuff I don’t see like how the trustees are chosen and treated. A person in a given town is envious of their neighbor’s house or car or job or wife, but they don’t think about what’s happening in a different neighborhood. In fact they might obsess over such things. It happens. But again, it’s local.

Evidence that people only think very locally about wealth and inequality is everywhere; so when people are polled and asked to describe income or wealth inequality, they always think it’s much less skewed than it is. Why? I’ll guess. It’s because they extrapolate from their very local experience, where there the outliers are not so very outlying at all. It’s a safe kind of assumption that doesn’t boil the blood.

So envy is there, it’s powerful, but it biases us enormously. If anything, I’m starting to think envy is something to distract us from something more dangerous, which is that sense of privation and dispossession, which runs deeper and is more anarchic. By contrast, envy seems like a myopic feeling that keeps us acting safely inside the system, where if we follow the rules but we’re a little bit better at them, we will get that bigger office or bigger car.

In the end, I reject envy as a unifying glue that describes our world, at least in times of severe inequality like now. It just doesn’t address the growing hostility that I’m sensing, which is that second kind of feeling, which exists beyond envy.

Moreover, I think the assumption that everyone is feeling something as small as envy, or rather the projection of envy onto the entire population, is damaging.

So, for example, there was an New York Times Op-Ed recently entitled Capitalize Workers! that suggested we get more people involved for saving for their retirement by investing in the stock market with “minimum pensions”.

I think the idea here is that everyone wants a piece of that amazing stock market return. But if you think about where people actually are financially, it’s such a weirdly out-of-touch plan, the idea that everyone is a Wall Street trader or wants to be.

For most people I meet and talk to, at this point retirement is not at all about the thrill of risk-taking, but rather the avoidance of risk altogether. If you asked those people, they’d rather just have their Social Security benefits doubled. They are not trying to take their chances to double their money, but rather trying to eke out a retirement without severe pain.

Why is this happening? Why are the authors of this piece, who both work at the think tank Third Way,  making such bizarre assumptions about how poor people want to retire? My first guess was that they are just working with the funds on Wall Street who would reap (even more) profits if more people invested.

But another less suspicious possibility is given by my above observation. Namely, they are projecting their myopic envy, that makes sense in their world, onto the poor and middle class worrying about retirement.

In their neighborhood, the way envy works is about trading and making big gains with extra money, but of course to do that you have to have extra money to start out with. In other words, the distance between the authors and the people they claim to be trying to help is too large for their system of envy to translate meaningfully.


Categories: #OWS, musing

Does OpenSSL bug prove that open source code doesn’t work?

By now most of you have read about the major bug that was found in OpenSSL, an open source security software toolkit. The bug itself is called the Heartbleed Bug, and there’s lots of information about it and how to fix it here. People are super upset about this, and lots of questions remain.

For example, was it intentionally undermined? Has the NSA deliberately inserted weaknesses into this as well? It seems like the jury is out right now, but if I’m the guy who put in the bug, I’m changing my name and going undercover just in case.

Next, how widely was the weakness exploited? If you’re super worried about stuff, or if you are a particular target of attack, the answer is probably “widely.” The frustrating thing is that there’s seemingly no way to measure or test that assumption, since the attackers would leave no trace.

Here’s what I find interesting the most interesting question: what will the long-term reaction be to open source software? People might think that open source code is a bust after this. They will complain that something like this should never have been allowed to happen – that the whole point of open software is that people should be checking this stuff as it comes in – and it never would have happened if there were people getting paid to test the software.

First of all, it did work as intended, even though it took two years instead of two days like people might have wanted. And maybe this shouldn’t have happened like it did, but I suspect that people will learn this particular lesson really well as of now.

But in general terms, bugs are everywhere. Think about Knight Capital’s trading debacle or the ObamaCare website, just two famous recent problems with large-scale coding projects that aren’t open source.

Even when people are paid to fix bugs, they fix the kind of bugs that cause the software to stop a lot sooner than the kind of bug that doesn’t make anything explode, lets people see information they shouldn’t see, and leaves no trace. So for every Knight’s Capital there are tons of other bugs in software that continue to exist.

In other words it’s more a question of who knows about the bugs and who can exploit them. And of course, whether those weaknesses will ever be exposed to the public at all.

It would be great to see the OpenSSL bug story become, over time, a success story. This would mean that, on the one hand the nerds becoming more vigilant in checking vitally important code, and learning to think like assholes, but also the public would need to acknowledge how freaking hard it is to program.

Categories: musing, open source tools

Let’s stop talking about HFT for a little while

It’s unusual that I find myself in the position of defending Wall Street activities, but here goes.

I just don’t think HFT is that big of a deal relative to other Wall Street evils. I have written a couple of times about HFT and I’m not a huge fan, and I don’t buy the “liquidity is good and more liquidity is better” argument: at some point enough is enough. I do think that day-to-day investors have largely benefitted from it but that people whose money is in massive funds which are regularly traded have seen their money get skimmed every month. Overall it’s a smallish negative tax on the average person, I’d expect.

Here’s why HFT deserves some of our hatred: there’s way too much human resources going into this stuff and it’s embarrassing, what with the laying of cables and blasting through mountains and such. And it’s a great sociological look into the absolutely greed-led mindset of the Wall Street trader, but honestly I think we already had that. It’s really business as usual at a microscopic scale, and nobody should really be surprised to learn that people will do anything to make money that’s technically possible and technically legal, and that they will brag about how they’re making the world a better place while they do it. Same old same old.

So I’m not saying HFT is awesome and we should encourage more of it. I’m all for thinking about how to slow down trading to once a second and make it “more fair” for more players (although that’s hard to do even as a thought experiment), or taxing transaction to make things slow down by themselves, which would be easy.

But here’s the thing, it’s not some huge awful thing we should focus on, even though Michael Lewis is a really good and engaging writer.

You wanna focus on something? Let’s talk about money laundering in HSBC and now Citi that is not under control. Let’s talk about ongoing mortgage fraud and robo-signing and the ongoing bailout/ taxpayer subsidy and people still losing their homes, and the poor still being the targets of illegal and predatory loans, and Too-Big-To-Fail getting worse, and the direct line between the bailout and the broken pension promises for civil servants and the overall price list for fraud that has been built.

Let’s talk about the people who created the underlying fraud still at work in places like Bank of America, and how few masterminds have gone to jail and how the SEC and the Obama administration has made that happen through inaction and passivity and how Congress is sitting on its hands because of the money coming in from lobbyists. Let’s talk about the increasing distance between the justice system for the poor and the justice system for the rich in this country.


Tell me what I missed.

The HFT noise is misplaced and a distraction from the ongoing real story.

Categories: finance, rant

An Interview And A Notebook

Interview on Junk Charts

Yesterday I was featured on Kaiser Fung’s Junk Charts blog in an interview where he kindly refers to me as a “Numbersense Pro”. Previous to this week, my strongest connection with Kaiser Fung was through Andrew Gelman’s meta-review of my review and Kaiser’s review of Nate Silver’s book The Signal And The Noise.

iPython Notebook in Data Journalism

Speaking of Nate Silver, Brian Keegan, a quantitative social scientist from Northeastern University, recently built a very cool iPython notebook (hat tip Ben Zaitlen), replete with a blog post in markdown on the need for openness in journalism (also available here), which revisited a fivethirtyeight article originally written by Walt Hickey on the subject of women in film. Keegan’s notebook is truly a model of open data journalism, and the underlying analysis is also interesting, so I hope you have time to read it.

Defining poverty #OWS

I am always amazed by my Occupy group, and yesterday’s meeting was no exception. We decided to look into redefining the poverty line, and although the conversation took a moving and deeply philosophical turn, I’ll probably only have time to talk about the nuts and bolts of formulas this morning.

The poverty line, or technically speaking the “poverty threshold,” is the same as it was in 1964 when it was invented except for being adjusted for inflation via the CPI.

In the early 1960′s, it was noted that poor families spent about a third of their money on food. To build an “objective” measure of poverty, then, they decided to measure the cost of an “economic food budget” for a family of that size and then multiply that cost by 3.

Does that make sense anymore?

Well, no. Food has gotten a lot cheaper since 1964, and other stuff hasn’t. According to the following chart, which I got from The Atlantic, poor families now spend about one sixth of their money on food:

Rich people spend even less on food.

Rich people spend even less on food.

Now if you think about it, the formula should be more like “economic food budget” * 6, which would effectively double all the numbers.

Does this matter? Well, yes. Various programs like Medicare and Medicaid determine eligibility based on poverty. Also, the U.S. census measures poverty in our country using this yardstick. If we double those numbers we will be seeing a huge surge in the official numbers.

Not that we’d be capturing everyone even then. The truth is, in some locations, like New York, rent is so high that the formula would likely be needing even more adjustment. Although food is expensive too, so maybe the base “economic food budget” would simply need adjusting.

As usual the key questions are, what are we accomplishing with such a formula, and who is “we”?

Categories: #OWS, modeling, statistics

Aunt Pythia’s advice

Aunt Pythia is psyched to be writing today after missing a couple of days of regular posts. Aunt Pythia loves you people and understands how much you rely on her ridiculous advice, so she really goes out of her way to get up on Saturdays, stretch out on the couch in her underwear and armed only with a laptop and copious coffee, and spout utter nonsense. She knows you love it to, and want it to continue indefinitely. So please, after enjoying today’s bilge:

think of something to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

I am an undergraduate at a liberal arts college applying to REUs. If I don’t get into any, I won’t have any opportunities to do research before applying to PhD programs. Would that mean I won’t get into grad school either? What options do I have to prove I am research-ready?

Possibly Not Research Ready

Dear PNRP,

I’m just blown away by the list of REU’s that have sprung up since I was a wee lass. I mean, I went to one, it was at Duluth and run by the incomparable Joe Gallian, but I’m more or less sure it was the only one around back then. He’s been doing it since 1977, and although I wasn’t at the very first one, I was early enough so that all the participants names could fit on one shirt. Holy crap there’s a picture of me from this page at my REU:

Screen Shot 2014-04-05 at 6.20.45 AM

Man, we played a lot of bridge that summer.

OK sorry for the nostalgic stream of consciousness. I will now attempt to answer your question.

First of all, given that very few people used to do REU’s before grad school, I obviously don’t think that it’s strictly necessary. On the other hand, given how many now exist, I’m guessing it’s become a common if not vital signaling device for getting into grad school (readers, weigh in!). It’s also probably gotten easier to get into. Definitely apply to many of them.

If you somehow didn’t apply to enough and it’s too late and you don’t get in anywhere, don’t despair. Look around for a teacher at your school or a nearby school, or even online, that is willing to do a reading course with you and develop some kind of senior thesis type project, or open problem to solve.

I feel that I need to add that most of these programs don’t actually ask you to solve open problems. It’s more like a peek at graduate school math and a mindset of research rather than an expectation that you will publish a paper. I know because I’ve taught at a couple since my college years.

Good luck!

Aunt Pythia


Dear Aunt Pythia,

What kind of skills/classes do I need to break into data science as an undergrad? It seems like a really interesting field and I don’t know whether I’m qualified to jump into an internship or not. Currently a math major without any stats classes under my belt.

Data Internship Youngster

Dear DIY,

First steps: take a CS class in a scripting language like python, take probability and statistics, and read my recent book or at least skim it at the bookstore.

Second steps, if you have time: take classes on machine learning, Bayesian statistics, and ethics.

Third steps, if you have even more time: take an advanced programming class, an optimization or information theory class, and become an anthropologist.

In the meantime, keep an eye on the curriculums for the industry data science camps not popping up everywhere, for example at the Microsoft Research Center.

Good luck!

Aunt Pythia


Dear Pythia,

Thank you for answering my question about “fairness” rankings by mentioning the Gini coefficient and upward mobility study, both of which I found interesting and hadn’t seen before.

Though obviously major, money wasn’t the only thing I had in mind. Judicial systems that imprison unfairly – possibly due to unfair laws to begin with – unfair job and housing discrimination, unfair environmental conditions and situations (Bhopal comes to mind), reasonable access to medical care, or lack thereof – all of these could be tossed into a fairness score as well.

I guess that in the end though, “fairness” may be a little too vague and subjective a term to be attached to any meaningful objective ranking. Fortunately the world already has lots of watchdog organizations that observe and report on objectively measurable facets of human life. OWS is one such organization.

Thanks again,


First of all, that’s Aunt Pythia to you. Har har.

Actually, even though it doesn’t appear that you’ve asked another question, I want to thank you for giving me an opening to my favorite recent rant.

In the context of my weekly Occupy meetings, I’ve been thinking more and more about the outrageous prison system in our own country and the multitude of mostly minority young men in that system. It’s a truly disgusting and predatory big business. As one of my co-occupiers said, if you’re too poor for us to make money off of you directly, we will throw you in prison and make money off of your incarceration.

Which brings me to your idea of measuring that kind of unfairness, even within our own country, and indeed even within the city of New York. Here’s the idea I’ve been tossing around inside my head.

It’s been long tossed around that the rate of marijuana use is similar for whites and blacks but blacks are going to jail way more for possession, essentially because of Stop & Frisk. In other words, blacks are more likely to get caught and to not have a fancy lawyer to get them out of trouble when they find themselves in trouble.

It brings up a host of questions, but I’ll focus on one: what is the relative chance that someone can get away with a mistake?

In other words, think about it this way.  We all make mistakes, and young men (and women) are especially impulsive and judgment-lacking. So instead of asking whether they make mistakes, ask instead what the chances are that such mistakes will land them in jail or prison. I feel like those probabilities might be a good start at what you’re getting at. Do you agree?


Aunt Pythia


Dear Aunt P,

My question may appear a little blunt but it’s one that’s troubled me for ages and I can’t think of any other way to ask it, so here goes: Does clitoris size reflect sexuality?

I mean, might larger be associated with more dominant or further along the hetero/homo-sexual scale, for example?

My follow-up question is, how would one go about assessing this? No, I don’t mean you to say ‘with warm hands and a micrometer’ but a suggestion of the mathematic parameters and procedures.

Jenny Taylor

Dear …umm… Jenny,

I’m going to say no. I have the following reasons for this answer, with exactly zero evidence gathered and assessed. Namely, it’s patently untrue of penises, which we all think about all the time in this society, so why should it be true of clitorides? Yes, that’s the plural of clitoris, I looked it up.

Now it’s true that a given woman’s clitoris ebbs and flows depending on how sexually stimulated she is, but other than that I think you just assume randomness.

As far as follow up, I’m gonna have to say: none needed, but if you want to turn this into a weird pick-up line at a bar then the more power to you.

Aunt Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

What Monsanto and college funds have in common


I recently read this letter to the editor written by Catharine Hill, the President of Vassar College, explaining why reducing family contributions in college tuition and fees isn’t a good idea. It was in response to this Op-Ed by Steve Cohen about the onerous “E.F.C.” system.

Let me dumb down the debate a bit for the sake of simplicity. Steve is on one side basically saying, “college costs too damn much, it didn’t used to cost this much!” and Catharine is on the other side saying, “colleges need to compete! If you’re not willing to pay then someone else will!”

Here’s the thing, there’s an arms race of colleges driving up costs. In some perverse combination of US News & World Reports model gaming and in responding to the Federal loan support incentive system, not to mention political decisions methodically removing funding from state colleges, college costs have been wildly rising.

And when you have an arms race, as I’ve learned from Tom Slee, the only solution is an armistice. In this case an armistice would translate into something like an agreement among colleges to set a maximum and reasonable tuition and fee structure. Sounds good, right? But an armistice won’t happen if the players in question are benefitting from the arms race. In this case parents are suffering but colleges are largely benefitting.


This recent Salon article detailing the big data approach that Monsanto is taking to their massive agricultural empire is in the same boat.

The idea is that Monsanto has bought up a bunch of big data firms and satellite firms to perform predictive analytics on a massive scale for farming. And they are offering farmers who are already internal to the Monsanto empire the chance to benefit from their models.

Farmers are skeptical of using the models, because they are worried about how much data Monsanto will be able to collect about them if they do.

But here’s the thing, farmers: Monsanto already has all your data, and will have it forever, due to their surveillance. They will know exactly what you plant, where, and how densely.

And what they are offering you is probably actually a benefit to you, but of course the more important thing for them is that they are explicitly creating an arms race between Monsanto farmers and non-Monsanto farmers.

In other words, if they give Monsanto farmers a extra boost, it will lead other farmers into the conclusion that, without such a boost, they won’t be able to keep up, and they will be forced into the Monsanto system by economic necessity.

Again an arms race, and again no armistice in sight, since Monsanto is doing this deliberately towards their profit bottom line. Assuming their models are good, the only way to avoid this for non-Monsanto farmers is to build their own predictive models, but clearly that would require enormous investment.

Categories: arms race, modeling

Navigating sexism does not mean accepting sexism

Not enough time for a full post this morning, but I’d like people to read a New York Times article ironically entitled Moving Past Gender Barriers to Negotiate a Raise (hat tip Laura Strausfeld). It has amazing and awful tidbits like the following:

“It’s totally unfair because we don’t require the same thing of men. But if women want to be successful in this domain, they need to pay attention to this.”

If you read on you realize that what they mean by “pay attention to” is “roll over and conform to stereotypes”. Super gross, and fuck that.

I feel like this is a more subtle, New York Times version of Susan Patton’s terrible advice for young women in snaring husbands. What happened to the feminists?!!

Categories: rant

Lobbyists have another reason to dominate public commenting #OWS

Before I begin this morning’s rant, I need to mention that, as I’ve taken on a new job recently and I’m still trying to write a book, I’m expecting to not be able to blog as regularly as I have been. It pains me to say it but my posts will become more intermittent until this book is finished. I’ll miss you more than you’ll miss me!

On to today’s bullshit modeling idea, which was sent to me by both Linda Brown and Michael Crimmins. It’s a new model built in part by the former chief economist for the Commodity Futures Trading Commission (CFTC) Andrei Kirilenko, who is now a finance professor at Sloan. In case you don’t know, the CFTC is the regulator in charge of futures and swaps.

I’ll excerpt this New York Times article which describes the model:

The algorithm, he says, uncovers key word clusters to measure “regulatory sentiment” as pro-regulation, anti-regulation or neutral, on a scale from -1 to +1, with zero being neutral.

If the number assigned to a final rule is different from the proposed one and closer to the number assigned to all the public comments, then it can be inferred that the agency has taken the public’s views into account, he says.

Some comments:

  1. I know really smart people that use similar sentiment algorithms on word clusters. I have no beef with the underlying NLP algorithm.
  2. What I do have a problem with is the apparent assumption that the “the number assigned to all the public comments” makes any sense, and in particular whether it takes into account “the public’s view”.
  3. It sounds like the algorithm dumps all the public comment letters into a pot and mixes it together to get an overall score. The problem with this is that the industry insiders and their lobbyists overwhelm public commenting systems.
  4. For example, go take a look at the list of public letters for the Volcker Rule. It’s not unlike this graphic on the meetings of the regulators on the Volcker Rule:reg_volcker
  5. Besides dominating the sheer number of letters, I’ll bet the length of each letter is also much longer on average for such parties with very fancy lawyers.
  6. Now think about how the NLP algorithm will deal with this in a big pot: it will be dominated by the language of the pro-industry insiders.
  7. Moreover, if such a model were to be directly used, say to check that public commenting letters were written in a given case, lobbyists would have even more reason to overwhelm public commenting systems.

The take-away is that this is an amazing example of a so-called objective mathematical model set up to legitimize the watering down of financial regulation by lobbyists.


Update: I’m willing to admit I might have spoken too soon. I look forward to reading the paper on this algorithm and taking a deeper look instead of relying on a newspaper.

Categories: #OWS, finance, modeling, rant

Aunt Pythia’s advice

Aunt Pythia is so very pleased to bring you more of her pearls of wisdom this nearly-believably-spring morning.

In celebration of above-freezing temperature, she’s extra cheerful and she welcomes the clouds and drizzle. After all, late March showers bring late April flowers, or something like that! Let there be blooming and cleansing!

And please, after you enjoy Aunt Pythia’s wisdom, and possibly after you clean out the front closet, please don’t forget to:

think of something to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

What the hell is goin’ on with Bitcoin? Will it survive into the future (or something else akin to it) or is it ultimately doomed???

Bitcoin Boogie

p.s. – I hope you realize you’ll probably have more success explaining quantum mechanics to me than Bitcoin.

Dear BB,

I promise not to try to explain Bitcoin’s underlying algorithms. But I think I can still answer your questions.

First of all, Bitcoin has been in the news lately in bad or confusing ways, first with the exchange (Mt. Gox) that went bankrupt, and second because regulators and institutional authorities are having trouble figuring out what Bitcoins are.

Even so, think of these hiccups as growing pains, according to Coinbase co-founder and former Goldman Sachs foreign exchange trader Fred Ehrsam, quoted as saying inspiring things like:

I would go to the bathroom and trade bitcoin on my smartphone and then return to my real desk to do my real job trading real currency.

If you don’t know about it, Coinbase is the “digital wallet” company that you’d probably sign up with if you wanted Bitcoins and you weren’t a huge nerd or a criminal willing to do things on the technical downlow: it makes owning Bitcoins easy, like signing up for a normal checking account.

And they are seeing lots of people joining, and they just got Overstock to accept Bitcoins as payment. So Ehrsam and people like him are pretty positive, and you never know.

Between you and me, though, I think the biggest competitor out there is Google, which has plans to allow people to share money over gmail (hat tip Suresh Naidu). Instead of paying heavy fees, you – guess what – tell Google about your checking accounts and other financial information. I see this potentially competing with banks, Apple, and of course PayPal, which sucks.

I hope that helps!

Aunt Pythia


Dear Aunt Pythia,

I am originally from a country where it’s normal to be sentimental. I am easily moved to tears and worry that this annoys others around me. Of course I can take counter-measures, for example I try to steady myself if the music is becoming emotional or before viewing some breathtakingly beautiful scenery, or, when news about a disaster or a sad film is being shown on the television I discreetly leave the room before it affects me.

I would like to be strong enough to withstand what appears to provoke no reaction in people here. Do you have any suggestion?

Too Sensitive

Dear Too,

I hear you, I’m a huge cryer too.  I blame the Irish side of my family.

What I do is I playfully prepare people I’m around, for their own comfort, and especially when they are not familiar with this side of me. So when I feel some sentimentality coming on, I’ll announce, “Hey I’m about to totally cry, because that’s what I do! Please bear with me and please ignore the tears, I’ll be OK in 10 minutes or less.” and then I’ll laugh, usually out of embarrassment.

That way they will know I realize it’s about me, not them, and that they’re not responsible to comfort me in any way. It works great, and it’s easy for me to do because I’m an extrovert. If you’re shy, it’s going to be harder, but the alternative is often that you have to explain yourself while you’re crying, which I think is worse.

Good luck!

Auntie P


Dear Aunt Pythia.

I am but a humble traveler trying to win you over with a Firefly reference and desperately seeking your advice.

Come July, I will find myself in New York for a week. I will be in need of a place to stay and some things to do while I’m visiting your fine city.

I have been looking on airbnb for a place to stay over a hotel or a hostel but am overwhelmed by all the options. Do I stay in Brooklyn, or Lower Manhattan? Harlem or the Upper West Side. I am a young data analyst from New Zealand, what do I know of New York neighborhoods?

And then there is the sightseeing, do I go and tick off all the tourist spots or are there better things for me to do with my time? Do you know any secret spots filled with good food, great coffee and devoid of the fanny-pack wearing, obnoxiously-photographing tourist hordes?


Seeking Habitation In New York

P.S. In New Zealand we call fanny-packs ‘bum-bags’. A fanny in NZ is something entirely different!


I don’t know from Firefly, sorry. But I’ll answer you anyway and let readers add their opinions.

I’d suggest you stay in a different neighborhood every night or two. That way you get to see more of New York, and any annoyance is short-lived. Most of your time will be spent traveling from place to place, so pack light. Make sure at least one night is in Astoria, Queens, which is just cool and kind of the epitome of the melting pot.

The reason I suggest this is that, for me, official tourist destinations are incredibly boring and expensive for what they offer (and what they offer is bum-bag bearing tourists, which you can already see in NZ anyway). I mean, if you think you’ll regret not going to the top of the Empire State Building, then by all means go, but go 10 minutes before they open and depart quickly.

Authentic sight-seeing in New York City consists, in my opinion, of walking through neighborhoods and checking out bars and restaurants and the local cultural gathering places. Look for live music in each neighborhood you stay in, if you like that sort of thing. Or if you are into food, make a plan for a foodie tour of each neighborhood. Yum!

Aunt Pythia


Dear Aunt Pythia,

In searching online dating profile in New York City (I live nearby), I encounter a bunch of profiles of finance professionals working in, say, investment banking. After reading your blog, I have become convinced that people who work in banking

1) are morally bankrupt
1.5) are swindlers
2) are not very thoughtful in regards to the concerns of the 99%
3) are greedy
4) are arrogant … they think they are the best and the brightest, and point to the fake wealth they created to justify their salary
5) are overworked, stressed out at work, and their job is slowly killing them physically and emotionally
6) have expectations of a lavish lifestyle (nothing wrong with that, just not for me…I can’t compete, and perhaps mo money mo problems)

Am I right or am I right? Should I even bother expressing an interest in these profiles?

Just Pondering

Dear Just,

There are two questions here, which I’d like to pose separately.

First, are investment bankers are morally bankrupt swindlers who ignore lesser folk and hate their jobs?

Second, how do optimize my chances of finding love – or at least great sex with a tolerable partner – on an online dating site?

The answer to the first questions is, of course not. There are plenty of people in finance and even in investment banking that are perfectly nice and even sensitive and empathetic. On the other hand, there is some story explaining why they’re there, and it often exposes a weird side to them. On the other other hand, who here doesn’t have a weird side? On the whole I’d say, never disqualify someone on one attribute, especially if they otherwise seem great and you find yourself liking them at a basic human level.

The answer to the second question is a lot trickier, though, and is related to the first in the following sense: if you are playing the numbers – which is all you can do on these websites – then you might well decide to avoid investment bankers. After all, you only have so much time and some many free Friday nights, and you want to optimize for best chance of liking someone. All you have is demographic information like their job and age, and even if you gather more information through emails, you might first want to filter out red flags, and you might find “investment banker” to be a red flag.

As an aside, I would love someone to do a quantitative and qualitative investigation to see how people have changed their dating and mating habits through online dating. It seems like the most profound area of the internet affecting cultural practices.

My bottomline suggestion is to try to find a date through a friend of a friend. Good luck!

Aunt Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!


Categories: Aunt Pythia

Interview with a high school principal on the math Common Core

In my third effort to understand the Common Core State Standards (CC) for math, I interviewed an old college friend Kiri Soares, who is the principal and co-founder of the Urban Assembly Institute of Math and Science for Young Women. Here’s a transcript of the interview which took place earlier this month. My words are in italics below.


How are high school math teachers in New York City currently evaluated?

Teachers are now evaluated on 2 things:

  1. First, measures of teacher practice, which are based on observations, in turn based on some rubric. Right now it’s the Danielson Rubric. This is a qualitative measure. In fact it is essentially an old method with a new name.
  2. Second, measures of student learning, that is supposed to be “objective”. Overall it is worth 40% of the teacher’s score but it is separated into two 20% parts, where teachers choose the methodology of one part and principals choose the other. Some stuff is chosen for principals by the city. Any time there is a state test we have to choose it. In terms of the teachers’ choices, there are two ways to get evaluated: goals or growth. Goals are based on a given kid, and the teachers can guess they will get a certain slightly lower score or higher score for whatever reason. Otherwise, it’s a growth-based score. Teachers can also choose from an array of assessments (state tests, performance tests, and third party exams). They can also choose the cohort (their own kids/ the grade/the school). The city also chose performance tasks in some instances.

Can you give me a concrete example of what a teacher would choose as a goal?

At the beginning of year you give diagnostic tests to students in your subject. Based on what a given kid scored in September, you extrapolate a guess for their performance in the June test. So if a kid has a disrupted homelife you might guess lower. Teacher’s goal setting is based on these teachers’ guesses.

So in other words, this is really just a measurement of how well teachers guess?

Well they are given a baseline and teachers set goals relative to that, but yes. And they are expected to make those guesses in November, possibly well before homelife is disrupted. It definitely makes things more complicated. And things are pretty complicated. Let me say a bit more.

The first three weeks of school are all testing. We test math, social studies, science, and English in every grade, and overall it depending on teacher/principal selections it can take up to 6 weeks, although not in a given subject. Foreign language and gym teachers also getting measured, by the way, based on those other tests. These early tests are diagnostic tests.

Moreover, they are new types of tests, which are called performance-based assessments, and they are based on writing samples with prompts. They are theoretically better quality because they go deeper, the aren’t just bubble standardized tests, but of course they had no pre-existing baseline (like the state tests) and thus had to be administered as diagnostic. Even so, we are still trying to predict growth based on them, which is confusing since we don’t know how to predict performance on new tests. Also don’t even know how we can consistently grade such essay-based tests- despite “norming protocols”, which is yet another source of uncertainty.

How many weeks per year is there testing of students?

The last half of June is gone, a week in January, and 2-3 weeks in the high school in the beginning per subject. That’s a minimum of 5 weeks per subject per year, out of a total of 40 weeks. So one eighth of teacher time is spent administering tests. But if you think about it, for the teachers, it’s even more. They have to grade these tests too.

I’ve been studying the rhetoric around the CC. So far I’ve listened to Diane Ravitch stuff, and to Bill McCallum, the lead writer of the math CC. They have very different views. McCallum distinguished three things, which when they are separated like that, Ravitch doesn’t make sense.

Namely, he separates standards, curriculum, and testing. People complain about testing and say that CC standards make testing easier, and we already have too much testing, so CC is a bad thing. But McCallum makes this point: good standards also make good testing easier.

What do you think? Do teachers see those as three different things? Or is it a package deal, where all three things rolled into one in terms of how they’re presented?

It’s much easier to think of those three things as vertices of a triangle. We cannot make them completely isolated, because they are interrelated.

So, we cannot make the CC good without curriculum and assessment, since there’s a feedback loop. Similarly, we cannot have aligned curriculum without good standards and assessment, and we cannot have good tests without good standards and curriculum. The standards have existed forever. The common core is an attempt to create a set of nationwide standards. For example, without a coherent national curriculum it might seem OK to teach creationism in place of evolution in some states. Should that be OK?

CC is attempting to address this, in our global economy, but it hasn’t even approached science for clear political reasons. Math and English are the least political subjects so they started with those. This is a long time coming, and people often think CC refers to everything but so far it’s really only 40% of a kid’s day. Social studies CC standards are actually out right now, but they are very new.

Next, the massive machine of curriculum starts getting into play, as does the testing. I have CC standards and the CC-aligned test, but not curriculum.

Next, you’re throwing into the picture teacher evaluation aligned to CC tests. Teachers are freaking out now – they’re thinking, my curriculum hasn’t been CC-aligned for many years, what do I do now? By the way, importantly, none of the high school curriculum in NY State is actually CC-aligned now. DOE recommendations for the middle school happened last year, and DOE people will probably recommend this year for high school, since they went into talks with publication houses last year to negotiate CC curriculum materials.

The real problem is this: we’ve created these new standards to make things more difficult and more challenging without recognizing where kids are in the present moment. If I’m a former 5th grader, and the old standards were expecting something from me that I got used to, and it wasn’t very much, and now I’m in 6th grade, and there are all these raised expectations, and there’s no gap attention.

Bottomline, everybody is freaking out – teachers, students, and parents.

Last year was the first CC-aligned ELA and math tests. Everybody failed. They rolled out the test before any CC curriculum.

From the point of view of NYC teachers, this seems like a terrorizing regime, doesn’t it?

Yes, because the CC roll-out is rigidly tied to the tests, which are in turn rigidly tied to evaluations of teachers. So the teachers are worried they are automatically going to get a “failure” on that vector.

Another way of saying this is that, if teacher evaluations were taken out of the mix, we’d have a very different roll-out environment. But as it is, teachers are hugely anxious about the possibility that their kids might fail both the city and state tests, and that would give the teacher an automatic “failure” no matter how good their teacher observations are.

So if I’m a special ed teacher of a bunch of kids reading at 4th and 5th grade level even through they’re in 7th grade, I’m particularly worried with the introduction of the new and unknown CC-aligned tests.

So is that really what will happen? Will all these teachers get failing evaluation scores?

That’s the big question mark. I doubt it there will be massive failure though. I think given that the scores were so clustered in the middle/low muddle last year, they are going to add a curve and not allow so many students to fail.

So what you’re pointing out is that they can just redefine failure?

Exactly. It doesn’t actually make sense to fail everyone. Probably 75% of the kids got 2′s or 1′s out of a 4 point scale. What does failure mean when everyone fails? It just means the test was too hard, or that what the kids were being taught was not relevant to the test.

Let’s dig down to the the three topics. As far as you’ve heard from the teachers, what’s good and bad about CC?

My teachers are used to the CC. We’ve rolled out standards-based grading three years ago, so our math and ELA teachers were well adjusted, and our other subject teachers were familiar. The biggest change is what used to be 9th grade math is now expected of the 8th grade. And the biggest complaint I’ve heard is that it’s too much stuff – nobody can teach all that. But that’s always been true about every set of standards.

Did they get rid of anything?

Not sure, because I don’t know what the elementary level CC standards did. There was lots of shuffling in the middle school, and lots of emphasis on algebra and algebraic thinking. Maybe they moved data and stats to earlier grades.

So I believe that my teachers in particular were more prepared. In other schools, where teachers weren’t explicitly being asked to align themselves to standards, it was a huge shock. For them, it used to be solely about Regents, and also Regents exams are very predictable and consistent, so it was pretty smooth sailing.

Let’s move on to curriculum. You mentioned there is no CC-aligned curriculum in NY. I also heard NY state has recently come out against the CC, did you hear that?

Well what I heard is that they previously said they this year’s 9th graders (class of 2017) would be held accountable but now the class of 2022 will be. So they’ve shifted accountability to the future.

What does accountability mean in this context?

It means graduation requirements. You need to pass 5 Regents exams to graduate, and right now there are two versions of some of those exams: one CC-aligned, one old-school. The question is who has to pass the CC-aligned versions to graduate. Now the current 9th grade could take either the CC-aligned or “regular” Regents in math.

I’m going to ask my 9th grade students to take both so we can gather information, even though it means giving them 3 extra hours of tests. Most of my kids pass 2 Regents in 9th grade, 2 in 10th, and 3 in 11th, and then they’re supposed to be done. They only take those Regents tests in senior year that they didn’t pass earlier.

What are the good and bad things about testing?

What’s bad is how much time is lost, as we’ve already said. And also, it’s incredibly stressful. You and I went to school and we had one big college test that was stressful, namely the SAT. In terms of us finishing high school, that was it. For these kids it’s test, test, test, test. I don’t think it’s actually improved the quality of college students across the country. 20 years ago NY was the only one that had extra tests except California achievement tests, which I guess we sometimes took as well.

Another way to say it is that we did take some tests but it didn’t take 5 weeks.

And it wasn’t high stakes for the teacher!

Let’s go straight there: what are the good/bad things for the teachers with all these tests?

Well it definitely makes the teachers more accountable. Even teachers think this: there is a cadre of protected teachers in the city, and the principals didn’t want to take the time to get rid of them, so they’d excess them out of the schools, and they would stay in the system.

Now with testing it has become much more the principal’s responsibility to get rid of bad teachers. The number of floating teachers is going down.

How did they get rid of the floaters?

A lot of different ways. They made them go into the schools, take interviews, they made their quality of life not great, and a lot if them left or retired or found jobs. Principals took up the mantle as well, and they started to do due diligence.

Sounds like the incentive system for over-worked principals was wrong.

Yes, although the reason it became easier for the principals is because now we have data. So if you’re coming in as ineffective and I also have attendance data and observation data, I can add my observational data (subjective albeit rubric based) and do something.

If I may be more skeptical, it sounds like this data gathering was used as a weapon against teachers. There were probably lots of good teachers that have bad numbers attached to them that could get fired if someone wanted them to be fired.

Correct, except those good teachers generally have principals who protect them.

You could give everyone a bad number and then fire the people you want, right?


Is that the goal?

Under Bloomberg it was.

Is there anything else you want to mention? 

I think testing needs to be dialed down but not disappear. Education is a bi-polar pendulum and it never stops in the middle. We’re on an extreme but let’s not get rid of everything. There is a place for testing.

Let’s get our CC standards, curriculum, and testing reasonable and college-aligned and let’s keep it reasonable. Let’s do it with standards across states and let’s make sure it makes sense.

Also, there are some new tests coming out, called PARCC assessments, that are adaptive tests aligned to the CC. They are supposed to replace Regents down the line and be national.

Here’s what bothers me about that. It’s even harder to investigate the experience of the student with adaptive tests.

I’m not sure there’s enough technology to actually do this anyway very soon. For example, we were given $10,000 for 500 student. That’s not going to go far unless it takes 2 weeks to administer the test. But we are investing in our technology this year. For example, I’m looking forward  to buying textbooks and get my updates pushed instead of having to buy new books every year.

Last question. They are redoing the SAT because rich kids are doing so much better. Are they just trying to get in on the test prep game? Because, here’s the thing, there’s no test that can’t be gamed that’s also easy to grade. It’s gotta depend on the letters and grades. We keep trying to shortcut that.

Listen, this is what I tell the kids. What’s going to matter to you is the letter of recommendation, so don’t be an jerk to your fellow students or to the teachers. Next, are you going to be able to meet the minimum requirements? That’s what the SAT is good for. It defines a lower bound.

Is it a good lower bound though?

Well, I define the lower bound as 1000 in total. My kids can target that. It’s a reasonable low bar.

To what extent do your students – mostly inner-city, black girls interested in math and science – suffer under the wholly gamed SAT system?

It serves to give them a point of self-reference with the rest of the country. You have to understand, they, like most kids in the nation, don’t have a conception of themselves outside of their own experience. The SAT serves that purpose. My kids, like many others, have the dream of Ivy League minus the understanding of where they actually stand.

So you’re saying their estimates of their chances are too high?

Yes, oftentimes. They are the big fish in a well-defined pond. At the very least, The SAT helps give them perspective.

Thanks so much for your time Kiri.

The Lede Program: An Introduction to Data Practices

It’s been tough to blog what with jetlag and a new job, and continuing digestive issues stemming from my recent trip, which has prevented me from drinking coffee. It really isn’t until something like this happens that I realize how very much I depend on caffeine for my early morning blogging. I really cherish that addiction like a child. Don’t tell my other kids.

Speaking of my new job, the website for the Lede Program: An Introduction to Data Practices is now live, as is the application. Very cool.

We’re holding information sessions about the program next Monday and Thursday at 1:00pm at the Stabile Center, on the first floor of the Journalism School which is in Pulitzer Hall. Please join us and please spread the news.

We are also still looking for teachers for the program, and we’ve fixed the summer classes, which will be:

  1. Basic computing,
  2. Data and databases,
  3. Algorithms, and
  4. The platform

I’m really excited about all of these but probably most about the last one, where we will investigate biases inherent in data, systems, and platforms and how they affect our understanding of objective truth. Please tell me if you know someone who might be great for teaching any of these, they are intense, seven week classes (either from end of may to mid-July or from mid-July to the end of August) which will meet 3 hours twice a week each.

Categories: data journalism, education

Tia Pythia’s advice

Aunt Pythia is coming to you from Costa Rica, where she’s been on vacation all week and is officially 100% sunburnt, relaxed, and happy, except for the occasional digestive issue.

To commemorate the occasion and location she’s temporarily changed her name to “Tia Pythia”, but don’t worry, you can expect consistent obnoxious and over-the-top advice coming from her. She hasn’t lost her edge, even in 95 degree heat!

After you enjoy her column (and the copious fruity drinks!) today please don’t forget to:

think of something to ask Tia Pythia at the bottom of the page!

By the way, if you don’t know what the hell Tia Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Tia Pythia,

I am a graduate student still early in my career, at a university that I am quite happy with (people, subject areas good; geography tolerable for 5 years). However, my (would-be, as-of-yet-unofficial) adviser is moving to a more prestigious, if less outwardly friendly, math department (the way it was described to me is, my current institution is solidly “tier n” while they are considering moving to a “tier n-1″ school). They have offered to bring me with them, but I am nervous about a)whether I could cut it at a more competitive place and b)even if I could, whether 3-4 years of relative misery is worth a “more prestigious” degree.

I’m very excited about the research that my adviser is doing and the field in general, and the prospect of more favorable geography along with a higher “payoff” (in terms of where my degree is from) is attractive, but I’m still seeking sage advice in case I haven’t thought of it in a certain way.

Also, a more direct question: could you have done everything you’ve done since getting your phd if it hadn’t been from Harvard, but from some middle-of-the-road school?

With love,

Future Anxiety Revealed, Troubling Situation


Nice sign-off, and it kind of makes up for a super long letter.

As I’ve written about recently, not all graduate school experiences are the same. Even so, Harvard wasn’t known as the most friendly department and I made it work for me, partly because of the location and the fact that I could make friends outside math. It helped that I grew up in the area and knew people like Nancy from Fair Foods and crucial information like where the best yarn stores were.

I’d suggest visiting the place and seeing if it can work. And importantly, try to make it work. Having an exciting advisor you trust is crucial to the graduate school experience, so I would definitely do my best if I were you to stick with him or her.

In terms of prestige, I definitely think it helps me personally, but I’m never sure how much of that is because I’m a woman – it definitely still seems true that you have to be top-notch to impress people if you’re female whereas men often get the benefit of the doubt.

Also it depends on whether you’re talking to someone inside math or outside math, because outsiders don’t have a definite sense of ranking and also don’t usually care too much. So it also depends on what you want to do with your life after school.

Good luck!

Tia Pythia


Dear Aunt Pythia,

I’ve never had a girlfriend. I’ve worked for a long time (years) to get one. It hasn’t been a complete waste of time from a social skills and self-improvement point of view, but I haven’t gotten anywhere, and the lack of success has taken its toll.

Logically, I tell myself that it has no bearing on my worth as a human being (compare Isaac Newton and Charlie Sheen for example), that I should enjoy being young and single, and I don’t get as depressed about it as I once did, but we all know that logic isn’t everything. And to clarify, I know I don’t need or am entitled to a girlfriend (I know a lot of guys in my situation do), and my life is satisfying – or getting there – on my own.

I fear that when I’m older, I’ll look back on these years-my most sexually fertile, as well as my most “fun” ones, and see barrenness, when others see great memories with lovers. And I’ll be constantly reminded that they do. I just feel so… tired, or deadened sometimes when thinking about it. What can I do about this?

Draußen vor der Tür

Dear Draußen,

Pardon me for cutting about 85% of your letter, it was just too long. I’m in a short question – short answer kind of mood this morning. Something about the last day of a vacation. And I didn’t cut out the part about Charlie Sheen, because honestly I don’t get it and I’m wondering if readers do and could comment on a possible interpretation.

Look, I have sympathy for your situation. As a guy in physics (part of what I cut), I’m sure you spend most of your time around other guys. It must be tough to meet nice women.

But at the same time, I guess I’m wondering what it is you’ve been doing to try to meet women, and importantly how you’ve talked to them when you’ve met them. From the 85% of your letter that I cut, I can tell you spend a lot of time thinking about yourself and what you should do with your life.

But to be honest, unless someone is already your friend, they probably don’t care about that stuff. At all. If I meet someone who starts talking about that stuff, I find a quick reason to depart.

You need to make sure you have opinions about other things besides yourself. Like, do you read the paper? What do you think about Ukraine? Or the new SAT? Make sure you are not too self-involved and that you have truly interesting opinions and things to say before you meet women. Even better: have ideals. Have plans to fix problems. That’s interesting! That’s possibly even sexy!

Another idea: try reading How Not to Be a Dick: An Everyday Etiquette Guide (hat tip Becky Jaffe). And I’m not saying that because I think you’re a dick (although I’m also not saying you aren’t a dick!) but because it has lots of great points about communication and making sure you’re coming across well. I know I learned something reading it!

One last thing: it doesn’t have to be work. Find something you like doing that lots of women also like doing, and go enjoy yourself. Joy is extremely catchy. Worst case you make some new friends.

Good luck,

Tia Pythia


Dear Aunt Pythia,

What do nerdy women talk about at lunch?

I’m a woman who recently started working in an office after a few years of working at home. It’s an educational technology company, and I’m in the unique and fortunate position of being both an educator and a software tester in training.

Most days when I go to the communal kitchen to heat my lunch, many of the women from the education team are engaging in loud discussions about their kids. I’ve started noticing that most of the men educators and people from other departments bury themselves in a book or take their food back to their desks. As a non-parent, I’m sometimes curious about life on the other side, but there’s such a thing as too much information. Today after yet another round of hearing about children’s eating habits, sleep habits, etc, I took my plate and headed for the tech zone for more stimulating conversation.

Any suggestions on things I can ask my coworkers with kids to steer the topic in another direction? I would like to get to know them better and pick their brains on their career paths and aspirations. Or am I better off spending more time with the mostly male tech geeks and absorbing their lingo?

Lunch Uncomfortable Need Conversation Help


First, congrats on the new job, it sounds cool.

Second, to be honest I have never encountered this problem personally because I’m such a freaking loudmouth. I pretty much just barge into conversations and change the topic if I’m bored. I often even tell people they’re really boring and need to spice up their conversation, preferably with sex. Nobody ever seems to complain that they want to talk more about their sleeping or non-sleeping kids. Not sure they like me, but whatevs.

I mean, I don’t literally interrupt the conversation, because I’m not totally rude. But I’ll wait for a good moment and just jump in with something off-topic like the new SAT or Putin or whether House of Cards is too cynical or not cynical enough.

My advice: come prepared with a short list of 4 non-parenting but general topics and see how they fly. I’m guessing they are themselves just bored and talking about that stuff out of habit and will welcome new blood.

Also, engage the men as well, especially if you go with sex.

Good luck!

Tia Pythia


Dear Aunt Pythia,

An acquaintance has started sending emails urging his “friends” to call upon their political leaders to oppose immigration reform. The first time, I assumed his message was spam and deleted it. I replied to the second message saying that I thought his account had been hacked. He replied that he had indeed sent it, explaining his position. After another message, which he forgot to BCC, one of the recipients replied with a well-reasoned rebuttal. The spammer’s response was to remove that person from his contact list.

Of course, everyone is entitled to their opinion, but I was initially confused because:
1. I met this person through an organization that celebrates cultural diversity.
2. His wife and stepson are immigrants.

His reasons for opposing immigration reform:
1. He was forced into early retirement because his employer went out of business.
2. Big business is profiting off cheap illegal labor, taking better paying jobs from Americans.
3. “Those people” are migrating northward, taking over, etc. (His wife is from a country bordering Europe, and the immigrants he opposes come from other places.)

So, the question is, should I ignore the emails, ask him to stop, or attempt to find common ground? I would generally ignore such spam, but I consider his wife a good friend. What kinds of holes can I point out in his argument, like the many forms of corporate greed?

Stop Propagating Antagonizing Messages

Dear SPAM,

Awesome sign-off. Both topical and sensical. Seriously, you should hold a master class in these motherfuckers.

In terms of your quandary, I’ll tell you what I’d do. I’d ask him politely to take you off the email list, and I’d never discuss it again with him. I’d continue to be friends with his wife.

Here’s why. He’s gotten it into his head that he lost his job for an abstract political reason. In doing so, he’s made it incredibly personal, and no amount of factual evidence is going to change his stance. You are not going to change it either.

Maybe at some point something will change it, but it will be emotional and deeply personal to him, not something you can effect.

Better yet, just build a filter to send his emails to trash and never think about it again.

Tia Pythia


Dear Tia Pythia,

From your remark of blowing off steam at a conference I remembered this article. Have you read it? It’s very informative and fun.

If conferences like JMM were to have bowls of condoms at the end of the tables where you pick up your badge do you think people would get the idea and pocket a hand full, then use them?

Open Relationships Rock

Dear ORR,

Wait, what article? That’s super unfair.

Tia Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

Billionaire money and academic freedom

If you haven’t seen this recent New York Times article by William Broad, entitled Billionaires With Big Ideas Are Privatizing American Sciencethen go take a look. It generalizes to all of scientific research my recent post entitled Billionaire Money in Mathematics.

My favorite part of Broad’s article is the caption of the video at the top, which sums it up nicely:

Funding the Future: As government financing of basic science research has plunged, private donors have filled the void, raising questions about the future of research for the public good.

In his article Broad makes a bunch of great points.

First, the fact that rich people generally ask for research into topics they care about (“personal setting of priorities”) to the detriment of basic research. They want flashy stuff, bang for their buck.

Second, academics interested in getting funding from these rich people have to learn to market themselves. From the article:

The availability of so much well-financed ambition has created a new kind of dating game. In what is becoming a common narrative, researchers like to describe how they begged the federal science establishment for funds, were brushed aside and turned instead to the welcoming arms of philanthropists. To help scientists bond quickly with potential benefactors, a cottage industry has emerged, offering workshops, personal coaching, role-playing exercises and the production of video appeals.

If you think about it, the two issues above are kind of wrapped up together. Flashy academic content goes hand in hand with flashy marketing. Let’s say goodbye to the true nerd who doesn’t button up their cardigan correctly. And I don’t know about you but I like those nerds. My mom is one of them.

This morning I thought of another way to express this issue, from the point of view of the individual scientist or mathematician, that might have profound resonance where the above just sounds annoying.

Namely, I believe that academic freedom itself is at stake. Let me explain.

I’m the last person who would defend our current tenure system. It’s awful for women, especially those who want kids, and it often breeds a kind of arrogant laziness post-tenure. Even so, there are good things about it, and one of them is academic freedom.

And although theoretically you can have academic freedom without tenure, it is certainly easier with it (example from this piece: “In Oklahoma, a number of state legislators attempted to have Anita Hill fired from her university position because of her testimony before the U.S. Senate. If not for tenure, professors could be attacked every time there’s a change in the wind.”).

But as we’ve seen recently, tenure-track positions are quickly declining in number, even as the number of teaching positions is growing. This is the academic analog of how we’ve seen job growth in the US but it’s majority shitty jobs. And as I’ve predicted already, this trend is surely going to continue as we scale education through MOOCs.

The dwindling tenured positions means there are increasing number of people trying to do research dependent upon outside grants and funding, and without the safety net of tenure. These people often lose their jobs when their funding flags, as we’ve recently seen at Columbia.

Now let’s put these two trends together. We’ve got fewer and fewer tenure jobs, which are precariously dependent on outside funding, and we’ve got rich people funding their own tastes and proclivities.

Where does academic freedom shake out in that picture? I’m going to say nowhere.

Categories: education, math, math education

Get every new post delivered to your Inbox.

Join 885 other followers