Home > guest post, journalism, math, math education, statistics, women in math > Guest post: Be more careful with the vagina stats in teaching

Guest post: Be more careful with the vagina stats in teaching

February 20, 2015

This is a guest post by Courtney Gibbons, an assistant professor of mathematics at Hamilton College. You can see her teaching evaluations on ratemyprofessor.com. She would like you to note that she’s been tagged as “hilarious.” Twice.

Lately, my social media has been blowing up with stories about gender bias in higher ed, especially course evaluations.   As a 30-something, female math professor, I’m personally invested in this kind of issue.  So I’m gratified when I read about well-designed studies that highlight the “vagina tax” in teaching (I didn’t coin this phrase, but I wish I had).

These kinds of studies bring the conversation about bias to the table in a way that academics can understand. We can geek out on experimental design, the fact that the research is peer-reviewed and therefore passes some basic legitimacy tests.

Indeed, the conversation finally moves out of the realm of folklore, where we have “known” for some time that students expect women to be nurturing in addition to managing the class, while men just need to keep class on track.

Let me reiterate: as a young woman in academia, I want deans and chairs and presidents to take these observed phenomena seriously when evaluating their professors. I want to talk to my colleagues and my students about these issues. Eventually, I’d like to “fix” them, or at least game them to my advantage. (Just kidding.  I’d rather fix them.)

However, let me speak as a mathematician for a minute here: bad interpretations of data don’t advance the cause. There’s beautiful link-bait out there that justifies its conclusions on the flimsy “hey, look at this chart” understanding of big data. Benjamin M. Schmidt created a really beautiful tool to visualize data he scraped from the website ratemyprofessor.com through a process that he sketches on his blog. The best criticisms and caveats come from Schmidt himself.

What I want to examine is the response to the tool, both in the media and among my colleagues.  USAToday, HuffPo, and other sites have linked to it, citing it as yet more evidence to support the folklore: students see men as “geniuses” and women as “bossy.” It looks like they found some screenshots (or took a few) and decided to interpret them as provocatively as possible. After playing with the tool for a few minutes, which wasn’t even hard enough to qualify as sleuthing, I came to a very different conclusion.

If you look at the ratings for “genius”  and then break them down further to look at positive and negative reviews separately, it occurs predominantly in negative reviews. I found a few specific reviews, and they read, “you have to be a genius to pass” or along those lines.

[Don’t take my word for it — search google for:

rate my professors “you have to be a genius”‘

and you’ll see how students use the word “genius” in reviews of professors. The first page of hits is pretty much all men.]

Here’s the breakdown for “genius”:


So yes, the data shows that students are using the word “genius” in more evaluations of men than women. But there’s not a lot to conclude from this; we can’t tell from the data if the student is praising the professor or damning him. All we can see that it’s a word that occurs in negative reviews more often than positive ones. From the data, we don’t even know if it refers to the professor or not.  


Similar results occur with “brilliant”:


Now check out “bossy” and negative reviews:


Okay, wow, look at how far to the right those orange dots are… and now look at the x-axis.  We’re talking about fewer than 5 uses per million words of text.  Not exactly significant compared to some of the other searches you can do.


I thought that the phrase “terrible teacher” was more illuminating, because it’s more likely in reference to the subject of the review, and we’ve got some meaningful occurrences:

And yes, there is a gender imbalance, but it's not as great as I had feared. I'm more worried about the disciplinary break down, actually. Check out math -- we have the worst teachers, but we spread it out across genders, with men ranking 187 uses of "terrible teacher" per million words; women score 192. Compare to psychology, where profs receive a score of 110.  Ouch.

And yes, there is a gender imbalance, but it’s not as great as I had feared. I’m more worried about the disciplinary break down, actually. Check out math — we have the worst teachers, but we spread it out across genders, with men ranking 187 uses of “terrible teacher” per million words; women score 192. Compare to psychology, where profs receive a score of 110.  Ouch.


Who’s doing this reporting, and why aren’t we reading these reports more critically?  Journalists, get your shit together and report data responsibly.  Academics, be a little more skeptical of stories that simply post screenshots of a chart coupled with inciting prose from conclusions drawn, badly, from hastily scanned data.

Is this tool useless? No. Is it fun to futz around with? Yes.

Is it being reported and understood well? Resounding no!

I think even our students would agree with me: that’s just f*cked up.

  1. February 20, 2015 at 8:15 am

    “Journalists, get your shit together and report data responsibly.”

    Good luck with that ever happening! Much of journalism will always go for quick catchy headlines or stories, and as a business model the need for eyeballs or clicks will drive it. But glad to see someone giving the critical take on this widely-cited “tool.” Agreed, it’s fun to play with… so long as you do so with a huge grain-of-salt.


  2. February 20, 2015 at 8:42 am

    Possibly the best guest post ever. Also I love the visualization tool. Thanks Courtney and thanks Cathy.


  3. February 20, 2015 at 8:55 am

    “Check out math — we have the worst teachers, but we spread it out across genders, with men ranking 187 uses of “terrible teacher” per million words; women score 192. Compare to psychology, where profs receive a score of 110. Ouch.”

    So if a student evaluation says someone is a terrible teacher it means they are a terrible teacher? It couldn’t be that the math course is a required one the student does not want to take? (There pretty much aren’t any psychology requirements everybody has.) It couldn’t be that the math and science require more time and effort than the higher rated courses?

    I see that English is a single subject. I bet that is English Composition was broken out separately, English Composition being another of those requirements that students tend to dislike, it would have high ‘terrible teacher’ numbers too.


  4. rob
    February 20, 2015 at 12:10 pm

    The conflict between the “bossy” and “terrible teacher” stats and the low usage of “bossy” strongly imply that “bossy” is a gender-specific epithet, not a bias. The comparable epithet might be a masculine-specific epithet like “dick,” schmuck” or “a-hole” which should also appear less frequently than “terrible teacher” and be weighted towards males.

    My take on the “bossy” controversy: the use of the word is not evidence of gender bias, but a linguistic accommodation. When a male employer behaves in a pushy, insensitive and commanding manner, employees call him “a-hole,” “SOB,” “dick,” “dickhead,” “schmuck,” “jerk,” “putz,” “bastard.” The overflowing wealth of these epithets is a testimony to the frustration of the male employer-employee relationship. It should surprise no one that employees have found an epithet for female employers.

    What should surprise is that “bossy” is not a taboo vulgar “cuss” word. Contrary to assumption, the use of “bossy” seems to be an attempt to be more cautious towards women — it is a euphemism for “bitch,” the closer counterpart to “a-hole.” It suggests that the application of vulgarity to men is freer than toward women, although there are equally stinging epithets for men as well that are avoided in circumstance comparable to those in which “bitch” would be avoided (“faggot,” “pussy,” “cocksucker” — it’s possible to view “schmuck,” for example, as a euphemism for one of these). The reasons are probably complex: why are there so many masculine-specific epithets that are blunt (like “dick”) but not stinging (like “faggot”)? Why so few for women? Is there a “never hit a woman” politeness effect in using “bossy” to avoid “bitch”? Euphemism and euideism are not well understood, I think


  5. crgibbons
    February 21, 2015 at 6:04 am

    Larry, I agree with you. My comment was tongue in cheek. I doubt you can use this tool to draw conclusions about professors in particular disciplines. If it has a use, I think you can learn more about the population of students who use RMP.

    Rob, that’s some interesting reading. I had never seen the word “euideism” before. I do know that RMP scrubs profanity, which could lead to students falling back on some of these epithets. I didn’t search for “bitch” et al. Did you? Does RMP let it through?


  6. rob
    February 21, 2015 at 3:01 pm

    What a fun tool, Courtney — I could spend the whole day there! Turns out “jerk” is the preferred slide for male profs, and around thirty times as frequent as “bossy.” One of the commenters mentions that profs overall are 58% male, but I can’t find any info on the sex differential on Rate My Prof.

    “Euideism” hasn’t caught on, but it’s needed to distinguish acceptable-word replacements (euphemisms – from “phemi,” to say) from acceptable-idea replacements that behave differently in the structure of language and tend to have a more appealing affect.


  1. February 27, 2015 at 5:12 pm
Comments are closed.
%d bloggers like this: