Home > data science, math education, news > Politics of teacher pay disguised as data science

Politics of teacher pay disguised as data science

January 4, 2012

I am super riled up about this report coming out of the Heritage Foundation. It’s part of a general trend of disguising a political agenda as data science. For some reason, this seems especially true in education.

The report claims to prove that public school teachers are overpaid. As proof of its true political goals, let me highlight a screen shot of the “summary” page (which has no technical details of the methods in the paper):

I’m sorry, but are you pre-writing my tweets for me now? Are you seriously suggesting that you have investigated the issue of public school teacher pay in an unbiased and professional manner with those pre-written tweets, Heritage Foundation?

If you read the report, which I haven’t had time to really do yet, you will notice how few equations there are, and how many words. I’m not saying that you need equations to explain math, but it sure helps when your goal is to be precise.

And I’d also like to say, shame on you, New York Times, for your coverage of this. You allow the voices of the authors, from the American Enterprise Institute and the Heritage Foundation, as well as another political voice from the Reason Foundation. But you didn’t ask a data scientist to look at the underlying method.

The truth is, you can make the numbers say whatever you want, and good data scientists (or quants, or statisticians) know this. The stuff they write in their report is almost certainly not the whole story, and it’s obviously politically motivated. I’d love to be hired to research their research and see what kind of similar results they’ve left out of the final paper.

  1. JSE
    January 4, 2012 at 10:26 am

    The Heritage Foundation has no incentive to get things right in-house — and the same goes for advocacy groups with opposite political views, to be honest. People with math knowledge can help by being willing to do quantitative pushback in public. I did one of these for Heritage a few years ago in Slate, about their white paper on abstinence education: http://www.slate.com/articles/life/do_the_math/2005/07/sex_and_significance.single.html

    And you should take care of this one!


  2. January 4, 2012 at 10:40 am

    I think that you make an excellent point. It could really shake things up if these foundations “studies” were subjected to a critical scientific review which was given as much publicity as the study itself.

    When I imagine this scenario, I note my own cynicism. In a very short time, there would probably also be a fair number of politically biased data scientists lending their voices to the fray with accusations flying about their affiliations. But the conversation would at least be changed. Numbers would perhaps themselves no longer be sufficient evidence to make your point, but would need to be backed up with the method used to generate them. I wonder what the tweets would look like then.


  3. Mike
    January 4, 2012 at 12:00 pm

    Many of the highlighted comments in the NYTimes discussion get to the core complaint I have about this study. In NYC, it’s fairly common knowledge among teachers that the public system pays better than the private schools for the most part. This is true in salary and benefits. The “folk knowledge” pay differential in 2004, before I left for grad school, was believed to be ~$10,000 a year, which, assuming a similar magnitude benefits differential that increases with seniority, is compatible with the 50% higher lifetime compensation reported by Heritage (I’d’ve guesstimated 25%).

    It’s also known that the private schools usually have better working conditions—fewer students, more autonomy in the classroom, more involved parents, less language and cultural diversity, and freedom from many of the onerous and ineffective certification requirements. (There’s a bitch-fest for you: wanna see some really elaborate cargo-cult science? Hang out at NYU-Steinhardt for a week, or 2 years like I did…)

    While it is true that older teachers are often not shy amongst themselves about how they’re staying on for a few more years to maximize their benefits, very few people get into the profession for that reason. I’m pretty sure if you walked up to a private school teacher and told them point-blank, “You’re receiving only 2/3 the lifetime financial compensation you’d receive in the public system!” He or she would say, “Yep!” and go back doing their job however they see fit.

    I think it’s not too hard to infer the implications from Heritage. Go back to my sentence about working conditions in private schools. If you want public schools to look like private schools, you’re asking for even more local control with the inescapable consequence of increased socioeconomic segregation in schools.

    It’s been very frustrating to me for a decade now that we almost always discuss education from the point of view of economics, whether school costs or the role of training future employees. Almost entirely missing is that the last 100 years of universal compulsory schooling has been the primary engine of assimilation and is foundational to the fact that most of us identify as “Americans”. One of the few useful things I learned in Ed school is that, in terms of language, dominant culture, and employment, there is very little functional meaning to the term “third generation immigrant” in the US. In contrast, the Chinese in Bangkok have been there for 400 years and a Turk is forever a Turk in Germany.

    The irony is that the “traditional American values” that Heritage (the term is from their about page) strives to conserve have traditionally been forged in large urban public school districts.


    Anyway, thanks for the comment box! Thought I was gonna write a paragraph for my first comment on your blog. By the way, I love your blog. I’m now a physicist working in computational neuroscience, and am about to finish my PhD (hence the long blog comment instead of page 99 of my thesis. I swear 100 pages is like the sound barrier, with the turbulence and the whatnot.) But, I’m in the market for a new direction after graduation and you’re posts have definitely broadened my search space.


  4. January 4, 2012 at 3:24 pm

    If being a policy wonk actually paid well and people could tell the difference between good work and bad, it would be a great job.


  5. January 4, 2012 at 3:28 pm

    I have a few issues with this post. First, the criticism about this particular study could be leveled to any study funded by any think tank, from the lowly ones, to the more prestigious ones, which have near-academic status (e.g. Brookings or Hoover). But indeed, most social scientists have a political bias. Piketty advised Segolene Goyal. Does it invalidate his study on inequality in America? Rogoff is a republican. Should one dismiss his work on debit crises? I think the best reaction is not to dismiss any study, or any author for that sake, on the basis of their political opinion, even if we dislike their pre-made tweets (which may have been prepared by editors that have nothing to do with the authors, by the way). Instead, the paper should be judged on its own merit. Even if we know we’ll disagree, a good paper can sharpen and challenge our prior convictions.

    Second, it’s not a prerequisite for a good paper, even an empirical one, to have plenty of formulas. I can produce examples, but I think this not controversial.

    Third, I think it’s a mistake, especially for another empiricist, to criticize a paper before having read it carefully. “You can make the numbers say whatever you want” is not a cvalid criticism that applies a priori because we *really* don’t like their conclusions.

    Incidentally, I don’t believe that, only because a paper contains data, then “a data scientist” should be consulted about it. The person I’d like to be consulted on the subject is someone with deep subject-specific knowledge and a reputation for impartiality and original research. In this case, I’d seek the opinion of Diane Ravitch.


  6. January 4, 2012 at 5:24 pm

    I think that abuses of this sort have a long history: lies, damned lies and statistics. The problem is in large part that not enough people have the intellectual tools necessary to question partial research when peppered with statistical allusion. Statistics would probably be more useful to many people than calculus. We need teachers to sort this out (but they won’t be able to do so if we keep cutting their wages) However, even more important than the ability to understand the maths is the ability to question in such a way as to disentangle assumptions from conclusions and to see how the former impinge on the latter.

    Actually I suspect that a big omission from the calculation is that they assume that teachers should be paid according to how expert they are in the subject being taught (they indicate this in the abstract). I disagree with this premise. Although expertise certainly helps, ultimately the quality most required of teachers are to inspire their pupils to want to know about the subject taught. The teachers only need sufficient subject expertise to allow the pupils to get started and to kick-start the pupils’ self-education. The qualities teachers need are essentially leadership qualities, so they should be compared to leaders rather than do-ers. (OK, the last sentence is slightly tongue in cheek in order to emphasise my first point; the validity of any statistical argument depends on the validity of the underlying assumptions at least as much as the validity of the statistical analysis).

    I also admit that I neither have the time nor the inclination to really study the paper.

    On the other hand, the use of statistics as political polemic can be a social good. Your arguments elsewhere on stop and search are an illustration. The work of people like Florence Nightingale and Richard Doll illustrate this well, especially since we can view them with the hindsight of history. And since use and abuse are two sides of the same coin, life is like peanut butter: it comes in both crunchy and smooth.


  7. Rebecca Goldin
    January 6, 2012 at 4:09 pm

    SO much to say about the Heritage Foundation analysis, and your overall conclusion about politics hiding behind the data. Obviously they are out to attack public school teachers, and this seems to be favored now not only by organizations on the right, but also those on the left.

    Take Obama’s desire to “hold teachers accountable” which has brought in Value Added Models and other rather obscuring (no longer so obscure!) statistical models to try to assess what a teacher does for a kid. It’s not much better than random noise in the end, that determines who is good versus who is not; someone judged as “good” is fairly likely to be judged as “poor” the next year. Everyone who has seen good teaching knows that you can’t pull it out of a the data on how well students do/improve on standardized tests plus a lot of student demographics, yet that’s what’s put into VAMs. All states applying to federal grants for education are required to institute “reforms” that include high-stakes evaluation of teachers using hard core data. Yet the data are so rife with problems, and are unable to describe what makes for good teaching, no matter how much clever statistical manipulation you do. The obscurity of it, though, is what makes it appealing to politicians and bureaucrats alike — it must be better than nothing, right? (I say wrong, because lots of great future teachers may well prefer to do something that doesn’t have an unfair assessment component, with the consequences being you could lose your job!)

    With regard to Heritage, there are a number of issues, regardless of whether their samples are unbiased or whether they have an agenda in deciphering these numbers. Like all professions, it is hard to tell what the “market” is for teachers without including the public in this market. While I wonder what “cognitive ability” measurement they used to say that “The wage gap between teachers and non-teachers disappears when both groups are matched on an objective measure of cognitive ability rather than on years of education,” I would agree that we don’t attract the top talent into teaching for the years of graduate education. I can’t speak for subjects outside of math, but we all know that there are “graduate courses” specifically designed for mathematics school teachers, and these are MUCH less mathematically sophisticated than the ones designed for MS students in math. If anything, this speaks to me of the market being “right on”, that is to say that the pay for a teacher is attracting more-or-less the same amount that person would get elsewhere. SO, what sort of pay would attract PhDs in math to teach high school? I am involved with Math for America-DC, trying to get talented math majors to move into teaching in an inner city environment and YES more money does help attract them. That said, the biggest barriers to improving the situation (especially in math) only starts with money. Training for teachers is key, and increasing our educational standards for teachers is part of this.

    So for Heritage who believes in “free market principles” you have to ask, Why is it that it’s so hard to get great teachers if the money is so good?

    Lastly, there seems to me no mention of the elephant in the room: teaching K-12 is generally compatible with playing a very large role with family life. When kids are out of school, teachers are too. When kids have vacation, so do teachers. The pay is also comparably lower, i.e. teachers are not paid in the summer. Presumably Heritage is not comparing a 10 month teaching salary to a 12 month non-teaching salary — yet for both people who work 12 and people who work 10 months, this is an extremely important part of the equation. If you make $50,000 as a teacher, the equivalent 12 month salary would be $60,000. But for some people, they would be willing work 12 months for just $55,000 because that marginal benefit of $5,000 is so important to their finances. For others, they would be willing to go down to $40,000 to get their summers off. This “marginal” calculation is not just linear — and there is no reason to believe it averages to linear over all people. This in my mind totally invalidates most comparisons of teaching with not teaching. I know teachers who would not work at all if they couldn’t be home after school with their kids. I know other teachers who feel they can barely afford to teach, but they do it because they love it. I also know many, many people who would have considered teaching in high school had it not been so low-paying and so under-appreciated culturally (I count myself in this lot, and would have definitely become a high school teacher had I not found a university job). Fundamentally, it seems to me that the Heritage is undervaluing the fact that people tend to perceive risk minimally (so they under-value the ‘benefit’ of job security, leading to Heritage saying it’s worth more than what people searching for jobs believe it’s worth). Heritage is also undervaluing the fact that teachers are generally speaking not in it for the money. The most “materialistic” reason I have heard for going into teaching from anyone, ever, is that it’s compatible with having kids. In contrast, I’ve heard doctors, lawyers, business people, etc say how they are in it for the money/prestige/ride. So while Heritage marks these folk as overpaid, it’s not overpaid enough to get hardly anyone (except perhaps those close to retirement) in it just to get the big ol’ pay-off.

    And incidentally, I have basically heard EVERY worker with a pension set up to pay them a certain percentage of their maximum salary say that they are sticking it out in order to maximize retirement. This is not an indictment of teaching, but rather a consequence of having “golden handcuffs” of any type, where there is a large financial incentive to make particular job decisions the final few years.


  8. FrankJones
    January 7, 2012 at 5:04 am

    It’s obviously politics disguised as science, but unfortunately it is not wrong.

    Your perspective as new yorkers may paint a different picture. Perhaps in New York people who have PhDs and would otherwise be working on Wall Street actually choose to go the noble route and become public school teachers instead? I think this is the wrong way to look at this. Where I live, teaching is one of the few jobs outside of minimum wage. It’s more like people would either teach or are stuck working at Wal-Mart. I think this is a more realistic perspective, nationwide.

    Here is an “outstate” perspective on some of these points:

    * Workers who switch from non-teaching jobs to teaching jobs receive a wage increase of roughly 9 percent. *

    Where I live, all the other jobs are minimum wage unless they are government jobs or jobs related to medicine (doctor, nurse, etc). Getting a teaching job is doubling your salary.

    *Pension programs for public-school teachers are significantly more generous than the typical private sector retirement plan*

    Again, aside from government jobs and medical jobs, teachers are the only people in town who even have pensions. My gym teacher retired at age 55. So here’s a guy who got paid 20 bucks an hour to shout “panty waist” to young boys for 30 years on the taxpayer’s dime and now has about 20 years of subsidized golf to look forward to.

    *Most teachers accrue generous retiree health benefits as they work*

    Sounding like a broken record, but again, here teaching is one of the few employers with health benefits.

    *Job security for teachers is considerably greater than in comparable professions*

    Absolutely no job security for most people in town doing anything, but a teacher has to get caught assaulting a student to get fired, and even then it takes a long drawn out process and they still get their pension…

    These observations may not be such an indictment of public school teaching as it is illustrative of how truly awful everyone in the private sector has it.


  1. January 11, 2012 at 7:07 am
Comments are closed.
%d bloggers like this: