People hate me, I must be doing something right

September 30, 2014 32 comments

Not sure if you’ve seen this recent New York Times article entitled Learning to Love Criticism, but go ahead and read it if you haven’t. The key figures:

…76 percent of the negative feedback given to women included some kind of personality criticism, such as comments that the woman was “abrasive,” “judgmental” or “strident.” Only 2 percent of men’s critical reviews included negative personality comments.

This is so true! I re-re-learned this recently (again) when I started podcasting on Slate and the iTunes reviews of the show included attacks on me personally. For example: “Felix is great but Cathy is just annoying… and is not very interesting on anything” as well as “The only problem seems to be Cathy O’Neill who doesn’t have anything to contribute to the conversation…”

By contrast the men on the show, Jordan and Felix, are never personally attacked, although Felix is sometimes criticized for interrupting people, mostly me. In other words, I have some fans too. I am divisive.

So, what’s going on here?

Well, I have a thick skin already, partly from blogging and partly from being in men’s fields all my life, and partly just because I’m an alpha female. So what that means is that I know that it’s not really about me when people anonymously complain that I’m annoying or dumb. To be honest, when I see something like that, which isn’t a specific criticism that might help me get better but is rather a vague attack on my character, I immediately discount it as sexism if not misogyny, and I feel pity for the women in that guy’s life. Sometimes I also feel pity for the guy too, because he’s stunted and that’s sad.

But there’s one other thing I conclude when I piss people off: that I’m getting under their skin, which means what I’m saying is getting out there, to a wider audience than just people who already agree with me, and if that guy hates me then maybe 100 other people are listening and not quite hating me. They might even be agreeing with me. They might even be changing their minds about some things because of my arguments.

So, I realize this sounds twisted, but when people hate me, I feel like I must be doing something right.

One other thing I’ll say, which the article brings up. It is a luxury indeed to be a woman who can afford to be hated. I am not at risk, or at least I don’t feel at all at risk, when other people hate me. They are entitled to hate me, and I don’t need to bother myself about getting them to like me. It’s a deep and wonderful fact about our civilization that I can say that, and I am very glad to be living here and now, where I can be a provocative and opinionated intellectual woman.

Fuck yes! Let’s do this, people! Let’s have ideas and argue about them and disagree! It’s what freedom is all about.

Categories: musing, statistics

Chameleon models

September 29, 2014 13 comments

Here’s an interesting paper I’m reading this morning (hat tip Suresh Naidu) entitled Chameleons: The Misuse of Theoretical Models in Finance and Economics written by Paul Pfleiderer. The paper introduces the useful concept of chameleon models, defined in the following diagram:

Screen Shot 2014-09-29 at 8.46.46 AM


Pfleiderer provides some examples of chameleon models, and also takes on the Milton Friedman argument that we shouldn’t judge a model by its assumptions but rather by its predictions (personally I think this is largely dependent on the way a model is used; the larger the stakes, the more the assumptions matter).

I like the term, and I think I might use it. I also like the point he makes that it’s really about usage. Most models are harmless until they are used as political weapons. Even the value-added teacher model could be used to identify school systems that need support, although in the current climate of distorted data due to teaching to the test and cheating, I think the signal is probably very slight.

Categories: economics, modeling

Aunt Pythia’s advice

September 27, 2014 4 comments

Holy crap, peoples!

Aunt Pythia just counted up her readers’ questions and found super high quality (yay!) combined with super small quantity (boo!), a non-ideal situation. Do you know that there are currently fewer than two weeks’ worth of questions in the bin?! That means that next week might be extra short if nobody comes up with (sexual, optionally true) dilemmas between now and next Saturday.

It’s a situation!! If things don’t change Aunt Pythia will be forced to:

  1. Make up questions. Aunt Pythia has never done this but desperate times call for desperate measures.
  2. Force good friends to submit questions. Aunt Pythia has totally done this but it aint pretty.
  3. Answer questions that have been submitted to other advice columns. Aunt Pythia is actually kind of into this idea. Like, there are plenty of Dan Savage answers she disagrees with, although she loves the guy, obv.

So seriously consider mixing that shit up and:

Don’t forget to submit stolen question from old Dan Savage columns,

especially if you are one of Aunt Pythia’s good friends.

Got it? Good! Love ya!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

I have a question on your recent post entitled Gillian Tett gets it very wrong on racial profiling, when you say:

Specifically, this means that the fact that black Americans are nearly four times as likely as whites to be arrested on charges of marijuana possession even though the two groups use the drug at similar rates would be seen by such a model (or rather, by the people who deploy the model) as a fact of nature that is neutral and true. But it is in fact a direct consequence of systemic racism.”

Racism is not the only interpretation of data. Another possible explanation is that black Americans are less educated and cannot hide marijuana from the cops as well as whites. So it is a correlate, not a cause. Had president Obama done something about education in the US I don’t think we’d see such terrible racial disparity.


Dear NYCN,

Not sure why this is an Aunt Pythia question instead of a comment on that post, but let me respond by, a “WHAA?”. Clearly black kids are much more educated about cops than the average white kids. Are you kidding?

But you do bring up a great point: white kids smoke pot in their own rooms in suburbia, and it’s harder for them to get caught. Black kids maybe don’t have privacy, so they end up doing more pot smoking in public, which means they get caught more. But obviously both whites and blacks walk around with pot in their pockets, so at the end of the day there’s serious racial bias.

This reminds me that I heard a group of Stuyvesant parents met with cops in the Stuy neighborhood and tried to make a deal that, when their kids were caught smoking pot in the nearby park, the cops would just bring them back to school rather than arresting them. Imagine that deal being made in Harlem.

Aunt Pythia


Dear Aunt Pythia,

I saw you asked some good questions at David Madigan’s IDSE Colloquium event this week. I thought it was a really compelling talk (and that Madigan is dreamy…). What were you reactions?

Anyways, when thinking about colloquium in the shower, I started to think about the word. It’s interesting that it’s essentially the same word as colloquial, yet to me they have opposite meanings. Do you think there’s truth to a colloquium really being colloquial?

Curious At the Colloquium

Dear Curious,

First of all, Madigan is awesome and I saw an earlier version of that talk before, in fact I wrote it up in Doing Data Science (Chapter 12). And yes, he’s indeed dreamy, a rare man of integrity. I am a groupie of his, and I don’t mind admitting it. After the talk I gave him a hug and felt a tingle.

Great question about the word colloquium. According to this online Oxford Dictionary, it basically just means “talk together”. Similarly, colloquial just means “conversational”. It makes sense. I wish more things were that informal combined with great.

I just got back from a Day of Data at Yale and I met a guy from the NIH, a really cool motorcycle-driving scientist in fact, and I told him all about Madigan. So I hope that helps the word get out too.

One question, what exactly were you doing in the shower whilst thinking about the talk? Just curious.

Aunt Pythia


Dear Aunt Pythia,

I’m returning back to teaching math after a long illness. I’m starting slow with a modest calculus course at a modest university, somewhere along the north atlantic coast. The budget crisis has taken its toll on the department. The students are barely prepared (serious gaps in algebra and trig.) I have 3 contact hours per week, no discussion sections, no graders.

I’m finding myself with a strange dilemma: should I cut lecturing down to a minimum and rely heavily on the book and youTube videos, while using most of class time for problem-solving and giving insightful examples, or should I go the other way: lecture and relegate homework and quizzes to the online platform that comes with the book.

Please help! I feel like I’m trying to tutor in the midst of lecture and run out of time every time.

Shy and Confounded


Dear Shy,

Given that there are serious gaps in their knowledge, I’d probably try to do at least a few worked-out examples with the students during class to make sure they can handle the mechanics of the solutions in addition to the conceptual ideas you’re presenting.

So maybe that means a 20 minute “review” of the new idea of the day, and then 40 minutes devoted to working out examples, with lots of interaction from the students so you can see what their gaps are and then make announcements about “things to remember”, basically showing them how to do stuff from algebra or trig.

Also consider asking them what is most useful for them to learn the stuff most efficiently in the three hours you have together. And finally, keep in mind that the quiet ones will probably be the ones that feel most behind, so make sure you don’t just listen to the loud people! Maybe a survey monkey?

But I definitely like your idea of offering them lots of online resources to get practice with this stuff if they are having trouble. I definitely think they should be encouraged to do that as well. Keep track of what works so next semester you have something to build on.

Good luck!

Aunt Pythia


Dear Aunt Pythia,

I’m just starting a tenure track job at a well-known place. Another more prestigious university is currently considering giving me a tenured full professorship. At what point do I mention this? I don’t want to mention it too early, because of course it might turn out nothing happens. But it does also seems like an opportunity for a market correction. Variation: How would a tall handsome man handle this?

Wanting Info on Negotiating Contract Extension


I don’t think you can mention it until the offer is firm. I don’t think a tall man would either, however handsome.

The real question is, how do you handle the negotiation between the two places once they are both actively recruiting you? Some people would try to start a bidding war, some wouldn’t. But since I’m one of those people who wouldn’t, I’m not the right person to ask about how to do that. If you want advice about that, write back and I’ll get my good friend who is the king of bidding wars to weigh in.

Aunt Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

Women not represented in clinical trials

September 26, 2014 13 comments

This recent NYTimes article entitled Health Researchers Will Get $10.1 Million to Counter Gender Bias in Studies spelled out a huge problem that kind of blows me away as a statistician (and as a woman!).

Namely, they have recently decided over at the NIH, which funds medical research in this country, that we should probably check to see how women’s health are affected by drugs, and not just men’s. They’ve decided to give “extra money” to study this special group, namely females.

Here’s the bizarre and telling explanation for why most studies have focused on men and excluded women:

Traditionally many investigators have worked only with male lab animals, concerned that the hormonal cycles of female animals would add variability and skew study results.

Let’s break down that explanation, which I’ve confirmed with a medical researcher is consistent with the culture.

If you are afraid that women’s data would “skew study results,” that means you think the “true result” is the result that works for men. Because adding women’s data would add noise to the true signal, that of the men’s data. What?! It’s an outrageous perspective. Let’s take another look at this reasoning, from the article:

Scientists often prefer single-sex studies because “it reduces variability, and makes it easier to detect the effect that you’re studying,” said Abraham A. Palmer, an associate professor of human genetics at the University of Chicago. “The downside is that if there is a difference between male and female, they’re not going to know about it.”

Ummm… yeah. So instead of testing the effect on women, we just go ahead and optimize stuff for men and let women just go ahead and suffer the side effects of the treatment we didn’t bother to study. After all, women only comprise 50.8% of the population, they won’t mind.

This is even true for migraines, where 2/3rds of migraine sufferers are women.

One reason they like to exclude women: they have periods, and they even sometimes get pregnant, which is confusing for people who like to have clean statistics (on men’s health). In fact my research contact says that traditionally, this bias towards men in clinical trials was said to protect women because they “could get pregnant” and then they’d be in a clinical trial while pregnant. OK.

I’d like to hear more about who is and who isn’t in clinical trials, and why.

Categories: modeling, news, rant, statistics

The business of public education

September 25, 2014 25 comments

I’ve been writing my book, and I’m on chapter 4 right now, which is tentatively entitled Feedback Loops In Education. I’m studying the enormous changes in primary and secondary education that have occurred since the “data-driven” educational reform movement started with No Child Left Behind in 2001.

Here’s the issue I’m having writing this chapter. Things have really changed in the last 13 years, it’s really incredible how much money and politics – and not education – are involved. In fact I’m finding it difficult to write the chapter without sounding like a wingnut conspiracy theorist. Because that’s how freaking nuts things are right now.

On the one hand you have the people who believe in the promise of educational data. They are often pro-charter schools, anti-tenure, anti-union, pro-testing, and are possibly personally benefitting from collecting data about children and then sold to commercial interests. Privacy laws are things to bypass for these people, and the way they think about it is that they are going to improve education with all this amazing data they’re collecting. Because, you know, it’s big data, so it has to be awesome. They see No Child Left Behind and Race To The Top as business opportunities.

On the other hand you have people who do not believe in the promise of educational data. They believe in public education, and are maybe even teachers themselves. They see no proven benefits of testing, or data collection and privacy issues for students, and they often worry about job security, and public shaming and finger-pointing, and the long term consequences on children and teachers of this circus of profit-seeking “educational” reformers. Not to mention that none of this recent stuff is addressing the very real problems we have.

As it currently stands, I’m pretty much part of the second group. There just aren’t enough data skeptics in the first group to warrant my respect, and there’s way too much money and secrecy around testing and “value-added models.” And the politics of the anti-tenure case are ugly and I say that even though I don’t think teacher union leaders are doing themselves many favors.

But here’s the thing, it’s not like there could never be well-considered educational experiments that use data and have strict privacy measures in place, the results of which are not saved to individual records but are lessons learned for educators, and, it goes without saying, are strictly non-commercial. There is a place for testing, but not as a punitive measure but rather as a way of finding where there are problems and devoting resources to it. The current landscape, however, is so split and so acrimonious, it’s kind of impossible to imagine something reasonable happening.

It’s too bad, this stuff is important.

When your genetic information is held against you

September 23, 2014 16 comments

My friend Jan Zilinsky recently sent me this blogpost from the NeuroCritic which investigates the repercussions of having biomarkers held against individuals.

In this case, the biomarker was in the brain and indicated a propensity for taking financial risks. Or maybe it didn’t really – the case wasn’t closed – but that was the idea, and the people behind the research mentioned three times in 8 pages that policy makers might want to use already available brain scans to figure out which populations or individuals would be at risk. Here’s an excerpt from their paper:

Our finding suggests the existence of a simple biomarker for risk attitude, at least in the midlife [sic] population we examined in the northeastern United States. …  If generalized to other groups, this finding will also imply that individual risk attitudes could, at least to some extent, be measured in many existing medical brain scans, potentially offering a tool for policy makers seeking to characterize the risk attitudes of populations.

The way the researchers did their tests was, as usual, to have them play artificial games of chance and see how different people strategized, and how their brains were different.

Here’s another article I found on biomarkers and risk for psychosis, here’s one on biomarkers and risk for PTSD.

Studies like this are common and I don’t see a reason they won’t become even more common. The question is how we’re going to use them. Here’s a nasty way I could imagine they get used: when you apply for a job, you fill in a questionnaire that puts you into a category, and then people can see what biomarkers are typical for that category, and what the related health risks look like, and then they can decide whether to hire you. Not getting hired doesn’t say anything about your behaviors, just what happens with “people like you”.

I’m largely sidestepping the issue of accuracy. It’s quite likely that, at an individual level, many such predictions will be inaccurate but could still be used by commercial interests – and even be profitable – even so.

In the best case scenario, we would use such knowledge strictly to help people stay healthy. In the worst case, we have a system whereby people are judged by their biomarkers and not their behavior. If there were ever a case for regulation, I think this is it.

Categories: data science, modeling

Aunt Pythia’s advice

September 20, 2014 6 comments

It was a long week! Very emotional!

And to top it all off, last night Aunt Pythia and her sweetie and some besties went to see – what else? – Ivo Van Hove’s adaptation of Ingmar Bergman’s Scenes From A Marriage. Aunt Pythia’s review of this deeply felt, Swedish introspection and investigation into the darkest corners of marital communication, and lack thereof, can be summarized in three words:

more sex, please.

Sadly, that may be the exact review you will give Aunt Pythia’s column today, although keep in mind she’s done her best to foster sex-related questions, and moreover she generously doles out sex-related advice, even when it isn’t called for.

So please have pity on her, and of course don’t forget to:

please think of something (sexual) to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

I’m applying to some math PhD programs this fall. Some of the applications ask me to specify faculty members at that university whom I would like to work with, and I’ve also been given the general advice to reach out to professors at various schools in order to get my name out there and increase my chances of admission. I have a couple of questions about this:

1) I feel like professors must be inundated with these emails from applicants, and that this would be a really annoying aspect of being a professor. How can I be minimally annoying?

2) I feel like professors must know that students (including myself) are angling for admission offers and not necessarily driven by the pure motive of academic interest. I’m not suggesting that I would lie to or try to manipulate someone whose work I wasn’t interested in, but the truth is I have never before gone around contacting mathematicians who have published interesting papers, so it feels disingenuous to do so only now that I hope to gain something. Is there any way to do this without feeling dishonest? Also, should I be explicit about my intention to name-drop them on the application, or should I pretend my motives are less self-serving?

3) Although I have some general ideas about areas of math that interest me (e.g. Representation Theory), I don’t have a really specific idea about the kind of research or thesis I will do– and because I’m just starting out, I don’t have the background to understand the papers and research on these professors’ CV’s. Should I just contact people in Algebra or whatever field I’m thinking about, or do I need to decide that “I want to contribute to your research on specific esoteric topic X” or whatever?

Although I think I have a reasonably solid application in terms of GPA, test scores, and letters of recommendation, I have essentially no research background or professional networking. So I really would like to do whatever I can to bolster my chances of getting into a program. Any advice you can give would be much appreciated. Even if that advice is simply to forget about sending annoying requests to strangers and just apply with what I have.

Getting Responses About Doctorate

Dear GRAD,

Here’s the thing, people like to take students. So if you express interest in working with them, they will like it, while they will of course also know it’s partly because you want to get into grad school, but that’s okay and normal. Of course there are some people that already have too many students, or actually don’t like taking students, so if you are ignored don’t take it personally. But in general it’s a flattering introduction, and people like to be flattered.

Plus, at the end of the day math is a community of people, and the sooner you start getting to know the people the better. So I’d suggest you really do reach out to people and take a look around at their papers and do your best to understand the gist of them. Ideally you would be able to meet them in person, say at a visit to the department or something, but barring that introducing yourself over email is fine, as long as it’s not a form letter.

Tell them about what you’ve read of their work, what interests you, and mention that you’re graduating now and applying to grad schools. Not offensive. And good luck!

Aunt Pythia


Dear Aunt Pythia,

I’m currently a math postdoc planning to transition to data science/something similar. The decision to leave academia hasn’t been easy, and one thing making it hard is that I really enjoy teaching. I particularly like teaching probability/stats/data analysis and I think the data journalism program you’re running is really cool! I’m wondering if you have any thoughts on (i) is it possible to stay involved in education in some form as a non-academic mathematician and (ii) if so, what to do to create these opportunities? I don’t plan to spend time on this early in my industry career as I need to establish myself professionally, but I hope to have opportunities to share what I love with others at some point down the line.

Pursuing A New Direction Actually


The sign-offs are killing it today.

OK so I agree, the worst part about leaving academic math for industry is that you don’t get to teach, and teaching is super fun. I’ve made do with going to math camp every now and then to get a dose of teaching, and more recently I worked at the Lede Program in data journalism, which allowed me to teach as well.

Suggestion: tutoring? Taking a few weeks to work at summer programs? Signing up to teach night classes? Becoming an adjunct at a local university and teaching whatever? All these things are possible.

There are also quite a few data science training programs springing up around the country that you might be able to work at, so take a look at that as well. Good luck!

Aunt Pythia


Dear Aunt Pythia,

Aunt Pythia starts this recent column by saying “Aunt Pythia kind of blew her load, so to speak, on the sex questions last week”.

But on MY PC, there is no update between 9th and 30th August, so my question is “Where is the 23rd August sexfest?”

Seeks Titillating Internet Material

Dear STIM,

Here it is. I got there by going to the front page and searching for “Aunt Pythia sex”. It’s really not that difficult, but I can understand why you might have been distracted. Plus, thank you for letting me link to that, it’s saving an otherwise sexless column.


Aunt Pythia


Dear Aunt Pythia,

I am a particle physics grad student who knows embarrassingly little about statistical analysis. For me, a significance of 5 sigma means a discovery, and 3 sigma stuff is ‘interesting’ (but almost always goes away with more data).

A while ago, I came upon this article. I am sure you heard/read about it. It basically says elite male-run labs hire female postdocs at 36%, while elite female-run labs hire female postdocs at 47% while the female postdocs are 39% of the pool. This is presented as “Male PIs don’t usually hire female postdocs”.

I was very confused when I read this, because to me male PIs were hiring at a level close to the average number of female postdocs available. As you can imagine, the female-run lab number is higher because there are ~20% female PIs in their data. So, that skews the numbers. They also give some significance (p-value) for their results, but how robust is the p-value? Or, what is the significant result here? Please give me a lecture on this!

Significance Is Greatly Mind Abusing


Physicists are kind of spoiled for data. They often just collect way more data than other people can, and their experiments don’t typically affect the results nearly as much, nor are they as messy, as you see in human experiments.

Anyway, a few points.

  1. I don’t understand your argument for why the female numbers are naturally skewed, unless you’re saying that there are so few data points that the averages tend to be far away from the expected average, which is true, but it could have just as well been below average, at least theoretically. Correct me if I’m mistaken.
  2. Not knowing more about this field, I don’t know the answer to a bunch of important questions I would ask. For example, do some fields expect you to work very long hours which would be tough for young mothers? Or are some fields for other reasons more friendly to women, for example if the hours were flexible, or if the wages were more transparent, or if the leaders of the field were more welcoming? All sorts of reasons that women might bunch together in certain fields and thus in certain labs.
  3. Most importantly, this paper seems to think there’s a natural experiment going on, but there almost never is. There are almost always confounding factors such as the above.
  4. So, if we really wanted to say men are less willing to hire women, we’d need to set up a randomized experiment and send a bunch of resumes that differ only in the gender of the applicant, and see what happens next.
  5. Having said all that, I didn’t actually read the paper, so I might be overly skeptical of the results. I have pancakes to make pretty quick so there’s a constraint in place here.
  6. In any case every time a randomized experiment has been performed, to my knowledge, there has been systemic sexism in place. So I wouldn’t be surprised if there is actual sexism at work here, even if I’m not convinced this is proof of it.
  7. Finally, you should take a look at t-tests, which you probably already know about, but here’s the reason: you can never get a 5-sigma results when your n is small. In other words, your test result, no matter what you do, is a function both of the amount of sexism that exists in a given lab and the number of labs you are evaluating, and you can’t do much about the latter even if the former is substantial.

I hope that helps!

Aunt Pythia


Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

Get every new post delivered to your Inbox.

Join 1,734 other followers