### Archive

Archive for the ‘education’ Category

## Duke deans drop the ball on scientific misconduct

Former Duke University cancer researcher Anil Potti was found guilty of research misconduct yesterday by the federal Office of Research Integrity (ORI), after a multi-year investigation. You can read the story in Science, for example. His punishment is that he won’t do research without government-sponsored supervision for the next five years. Not exactly stiff.

This article also covers the ORI decision, and describes some of the people who suffered from poor cancer treatment because of his lies. Here’s an excerpt:

Shoffner, who had Stage 3 breast cancer, said she still has side effects from the wrong chemotherapy given to her in the Duke trial. Her joints were damaged, she said, and she suffered blood clots that prevent her from having knee surgery now. Of the eight patients who sued, Shoffner said, she is one of two survivors.

What’s interesting to me this morning is that both articles above mention the same reason for the initial investigation in his work. Namely, that he had padded his resume, pretending to be a Rhodes Scholar when he wasn’t. That fact was reported by a website called Cancer Letter in 2010.

But here’s the thing, back in 2008 a 3rd-year medical student named Bradford Perez sent the deans at Duke (according to Cancer Letter) a letter explaining that Potti’s lab was fabricating results. And for those of you who can read nerd, please go ahead and read his letter, it is extremely convincing. An excerpt:

Fifty-nine cell line samples with mRNA expression data from NCI-60 with associated radiation sensitivity were split in half to designate sensitive and resistant phenotypes. Then in developing the model, only those samples which fit the model best in cross validation were included. Over half of the original samples were removed. It is very possible that using these methods two samples with very little if any difference in radiation sensitivity could be in separate phenotypic categories. This was an incredibly biased approach which does little more than give the appearance of a successful cross validation.

Instead of taking up the matter seriously, the deans pressured Perez to keep quiet. And nothing more happened for two more years.

The good news: Bradford Perez seems to have gotten a perfectly good job.

The bad news: the deans at Duke suck. Unfortunately I don’t know exactly which deans and what their job titles are, but still: why are they not under investigation? What would deans have to do – or not do – to get in trouble? Is there any kind of accountability here?

## Gender And The Harvard Math Department

This is a guest post by Meena Boppana, a junior at Harvard and former president of the Harvard Undergraduate Math Association (HUMA). Meena is passionate about addressing the gender gap in math and has co-lead initiatives including the Harvard math survey and the founding of the Harvard student group Gender Inclusivity in Math (GIIM).

I arrived at Harvard in 2012 head-over-heels in love with math. Encouraged to think mathematically since I was four years old by my feminist mathematician dad, I had even given a TEDx talk in high school declaring my love for the subject. I was certainly qualified and excited enough to be a math major.

Which is why, three years later, I think about how it is that virtually all my female friends with insanely strong math backgrounds (e.g. math competition stars) decided not to major in math (I chose computer science). This year, there were no female students in Math 55a, the most intense freshman math class, and only two female students graduating with a primary concentration in math. There are also a total of zero tenured women faculty in Harvard math.

So, I decided to do some statistical sleuthing and co-directed a survey of Harvard undergraduates in math. I was inspired by the work of Nancy Hopkins and other pioneering female scientists at MIT, who quantified gender inequities at the Institute – even measuring the square footage of their offices – and sparked real change. We got a 1/3 response rate among all math concentrators at Harvard, with 150 people in total (including related STEM concentrations) filling it out.

The main finding of our survey analysis is that the dearth of women in Harvard math is far more than a “pipeline issue” stemming from high school. So, the tale that women are coming in to Harvard knowing less math and consequently not majoring in math is missing much of the picture. Women are dropping out of math during their years at Harvard, with female math majors writing theses and continuing on to graduate school at far lower rates than their male math major counterparts.

And it’s a cultural issue. Our survey indicated that many women would like to be involved in the math department and aren’t, most women feel uncomfortable as a result of the gender gap, and women feel uncomfortable in math department common spaces.

The simple act of talking about the gender gap has opened the floodgates to great conversations. I had always assumed that because no one was talking about the gender gap, no one cared. But after organizing a panel on gender in the math department which drew 150 people with a roughly equal gender split and students and faculty alike, I realized that my classmates of all genders feel more disempowered than apathetic.

The situation is bad, but certainly not hopeless. Together with a male freshman math major, I am founding a Harvard student group called Gender Inclusivity in Math (GIIM). The club has the two-fold goal of increasing community among women in math, including dinners, retreats, and a women speaker series, and also addressing the gender gap in the math department, continuing the trend of surveys and gender in math discussions. The inclusion of male allies is central to our club mission, and the support from male allies at the student and faculty level that we have received makes me optimistic about the will for change.

Ultimately, it is my continued love for math which has driven me to take action. Mathematics is too beautiful and important to lose 50 percent (or much more when considering racial and class-based inequities) of the potential population of math lovers.

## Fingers crossed – book coming out next May

As it turns out, it takes a while to write a book, and then another few months to publish it.

I’m very excited today to tentatively announce that my book, which is tentatively entitled Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, will be published in May 2016, in time to appear on summer reading lists and well before the election.

Fuck yeah! I’m so excited.

p.s. Fight for 15 is happening now.

## The achievement gap: whose problem is it?

On Monday night I went to see Boston College professor Henry Braun speak about the Value-Added Model for teachers (VAM) at Teachers College, right here in my hood (hat tip Sendhil Revuluri).

I wrote about VAM recently, and I’m not a fan, so I was excited for the event. Here’s the poster from Monday:

The room was not entirely filled with anti-VAM activists such as myself, even though it was an informed audience. In fact one of the people I found myself talking to before the talk started mentioned that he’d worked on Wall Street, where they “culled” 10% of the workforce regularly – during downsizing phases – and how fantastic it was, how it kept standards high.

I mentioned that the question is, who gets decide which 10% and why, and he responded that it was all about profit, naturally. Being an easily provoked person, I found myself saying, well right, that’s the definition of success for Wall Street, and we can see how that’s turned out for everyone. He stared blankly at me.

I told that story because it irks me, still, how utterly unscathed individuals feel, who were or are part of the Wall Street culture. They don’t see any lesson to learn from that whole mess.

But even more than that, the same mindset which served the country so poorly is now somehow being held up as a success story, and applied to other fields like public education.

That brings me to the talk itself. Professor Braun did a very good job of explaining the VAM, and the inconsistencies, and the smallish correlations and unaccountable black box nature of the test.

But he then did more: he drew up a (necessarily vague) picture of the entire process by which a teacher is “assessed,” of which VAM plays a varying role, and he asked some important questions: how does this process affect the teaching profession? Does the scrutiny of each teacher in this way make students learn more? Does it make bad teachers get better? Does it make good teachers stay in the profession?

Great questions, but he didn’t even stop there. He went on to point something out that I’d never directly considered. Namely, why do we think individual responsibility – i.e. finger pointing at individual teachers – is going to improve the overall system? Here he suggested that there’s been a huge split in the profession between those who want to improve educational systems and those who want to assess teachers (and think that will “close the achievement gap”). The people who want to improve education talk about increasing communication between teachers in a school or between schools in a district, and they talk about improving and strengthening communities and cultures of learning.

By contrast the “assess the teachers” crowd is convinced that holding teachers individually accountable for the achievement of their students is the only possible approach. Fuck the school culture, fuck communicating with other teachers in the school. Fuck differences in curriculum or having old books or not having enough books due to unequal funding.

It got me thinking, especially since I read that book last week, The New Prophets of Capitalism (review here). That book explained how hollow Oprah’s urging to live a perfect life is to people whose situations are beyond their control. The problem with Oprah’s reasoning is that it ignores real systemic problems and issues that radically affect certain parts of the population and make it much harder to take her advice. It’s context free in a world where context is more and more meaningful.

So, whose problem is the achievement gap? Is it owned in tiny pieces by every teacher who dares to enter the profession? Is it owned by schools or school systems? Or is it owned by all of us, by the country as a whole? And if it is, how are we going to start working together to solve it?

Categories: education, modeling

## Everyone hates college administrators

If you were wondering why I didn’t blog yesterday, which you probably weren’t (confession: I don’t read other peoples’ blogs and I don’t listen to any podcasts. So I would never, ever ask anyone to read my blog or listen to my podcast), it was because I was completely confused and irritated by this NYTimes opinion piece on the rising cost of college, written by University of Colorado Law Professor Paul Campos.

I really think the Times needs to either have footnotes or hyperlinks in their opinion pieces, because this guy was playing so fast and loose with his numbers that I had really no idea what he was talking about most of the time. That’s saying something considering that this, the cost of college and its causes, is something I have spent many hours thinking about and researching.

So what happened was, I didn’t have time to completely formulate my opposition to why his reasoning was muddled and confusing. I spent way too much time trying to figure out where he was getting his data. Waste of time.

Good news, though, my Slate Money co-host Jordan Weissman has done all that work for us, in his piece aptly entitled The New York Times Offers One of the Worst Explanations You’ll Read of Why College Is So Expensive. Who says procrastination doesn’t work?

As usual, if you’ve ever listened to my podcast (and this isn’t a request for you to do so!), I don’t agree completely with Jordan. However, my delta of agreement with Jordan is very manageable compared to the delta of disagreement I had with Campos. Basically I would quibble with laying any of the blame at the feet of instructors, but since he barely does that, let’s just go with his awesome take-down.

Take-down of what? Well, Campos basically hates college administrators, and pretends there’s no other problems in the world except them. It’s a mistake that he doesn’t have to make.

I mean really, who doesn’t hate college administrators? As a former college administrator myself, I know it’s universal; I certainly hated myself the entire time.

But that doesn’t mean there’s no other factors at all. Reduced public money for colleges is in fact a huge problem, especially when you pair it with the increased federal aid money going to students at corrupt for-profit colleges. Corinthian obtained \$1.4 billion in federal grant and loan dollars in 2010 alone, more than the 10 University of California campuses combined for that same year. This system is in terrible need of repair.

Instead of simply hating on college admin, or rather, in addition to hating on admin, can we start thinking about an alternative no-frills state college system that is truly affordable and gives honest and basic instructions without trying to compete on the US News & World Reports stage?

Categories: education, rant

## How many NYC are arbitrarily punished by the VAM? About 578 per year.

There’s been an important update in the thought experiment I started yesterday. Namely, a reader (revuluri) has provided me with a link to show how many teachers are considered “ineffective,” which was my shorthand for scoring either third or fourth in the four categories.

According to page 5 of this document, that percentage was 16% in 2011-2012, 17% in 2012-2013, and 16% in 2013-2014. We’ll take this to mean that the true cutoff is about 16.3%. Using my formula from yesterday, that means that after 4 years, about

$1- (0.837^4 + 4 \cdot 0.837^3 \cdot 0.163) = 0.127,$

or 12.7% of teachers going up for tenure in the new system will be arbitrarily denied tenure based only on their VAM score.

How many people is that in a given year? Well, this document explains that in 2000, 9,000 teachers were hired and in 2008, 6,000 teachers were hired. I’ll assume my best guess for “teachers hired” in a given year is something between those two numbers, but I’ll also assume it’s closer to the latter since it is more recent information. Say 7,000 new teachers per year.

Of course, not all of them go up for tenure. There’s attrition. Say 35% of those teachers leave before the tenure decision is made (also guessing from this document). That leave us with about 4,550 teachers going up for tenure each year, and 12.7% of them is 578 people.

So, according to my crude estimates, about 578 people will be denied tenure simply based on this random number generator we call VAM. And as my reader said, this says nothing about the hard-to-measure damage done to all the good teachers trying to teach their kids but having to deal with this standardized testing nonsense. It’s a wonder anyone is willing to work here.

Please comment if you have updated numbers for anything here.

Categories: education, math

## The arbitrary punishment of New York teacher evaluations

The Value-Added Model for teachers (VAM), currently in use all over the country, is a terrible scoring system, as I’ve described before. It is approximately a random number generator.

Even so, it’s still in use, mostly because it wields power over the teacher unions. Let me explain why I say this.

Cuomo’s new budget negotiations with the teacher’s union came up with the following rules around teacher tenure, as I understand them (readers, correct me if I’m wrong):

1. It will take at least 4 years to get tenure,
2. A teacher must get at least 3 “effective” or “highly effective” ratings in those three years,
3. A teacher’s yearly rating depends directly on their VAM score: they are not allowed to get an “effective” or “highly effective” rating if their VAM score comes out as “ineffective.”

Now, I’m ignoring everything else about the system, because I want to distill the effect of VAM.

Let’s think through the math of how likely it is that you’d be denied tenure based only on this random number generator. We will assume only that you otherwise get good ratings from your principal and outside observations. Indeed, Cuomo’s big complaint is that 98% of teachers get good ratings, so this is a safe assumption.

My analysis depends on what qualifies as an “ineffective” VAM score, i.e. what the cutoff is. For now, let’s assume that 30% of teachers receive “ineffective” in a given year, because it has to be some number. Later on we’ll see how things change if that assumption is changed.

That means that 30% of the time, a teacher will not be able to receive an “effective” score, no matter how else they behave, and no matter what their principals or outside observations report for a given year.

Think of it as a biased coin flip, and 30% of the time – for any teacher and for any year – it lands on “ineffective”, and 70% of the time it lands on “effective.” We will ignore the other categories because they don’t matter.

How about if you look over a four year period? To avoid getting any “ineffective” coin flips, you’d need to get “effective” every year, which would happen 0.70^4 = 24% of the time. In other words, 76% of the time, you’d get at least one “ineffective” rating just by chance.

But remember, you don’t need to get an “effective” rating for all four years, you are allowed one “ineffective rating.” The chances of exactly one “ineffective” coin flip and three “effective” flips is 4 (1-0.70) 0.70^3 =  41%.

Adding those two scenarios together, it means that 65% of the time, over a four year period, you’d get sufficient VAM scores to receive tenure. But it also means that 35% of the time you wouldn’t, through no fault of your own.

This is the political power of a terrible scoring system. More than a third of teachers are being arbitrarily chosen to be punished by this opaque and unaccountable test.

Let’s go back to my assumption, that 30% of teachers are deemed “ineffective.” Maybe I got this wrong. It directly impacts my numbers above. If the overall probability of being deemed “effective” is p, then the overall chance of getting sufficient VAM scores will be $p^4 + 4 p^3 (1-p).$

So if I got it totally wrong, and 98% of teachers are described as effective by the VAM model, this would mean almost all teachers get sufficient VAM scores.

On the other hand, remember that the reason VAM is being pushed so hard by people is that they don’t like it when evaluations systems think too many people are effective. In fact, they’d rather see arbitrary and random evaluation than see most people get through unscathed.

In other words, it is definitely more than 2% of teachers that are called “ineffective,” but I don’t know the true cutoff.

If anyone knows the true cutoff, please tell me so I can compute anew the percentage of teachers that are arbitrarily being kept from tenure.

Categories: education, rant, statistics