Home > Becky Jaffe, math education, modeling, rant > “Here and Now” is shilling for the College Board

“Here and Now” is shilling for the College Board

September 30, 2013

Did you think public radio doesn’t have advertising? Think again.

Last week Here and Now’s host Jeremy Hobson set up College Board’s James Montoya for a perfect advertisement regarding a story on SAT scores going down. The transcript and recording are here (hat tip Becky Jaffe).

To set it up, they talk about how GPA’s are going up on average over the country but how, at the same time, the average SAT score went down last year.

Somehow the interpretation of this is that there’s grade inflation and that kids must be in need of more test prep because they’re dumber.

What is the College Board?

You might think, especially if you listen to this interview, that the college board is a thoughtful non-profit dedicated to getting kids prepared for college.

Make no mistake about it: the College Board is a big business, and much of their money comes from selling test prep stuff on top of administering tests. Here are a couple of things you might want to know about College Board through its wikipedia page:

Consumer rights organization Americans for Educational Testing Reform (AETR) has criticized College Board for violating its non-profit status through excessive profits and exorbitant executive compensation; nineteen of its executives make more than $300,000 per year, with CEO Gaston Caperton earning $1.3 million in 2009 (including deferred compensation).[10][11] AETR also claims that College Board is acting unethically by selling test preparation materials, directly lobbying legislators and government officials, and refusing to acknowledge test-taker rights.[12]

Anyhoo, let’s just say it this way: College Board has the ability to create an “emergency” about SAT scores, by say changing the test or making it harder, and then the only “reasonable response” is to pay for yet more test prep. And somehow Here and Now’s host Jeremy Hobson didn’t see this coming at all.

The interview

Here’s an excerpt:

HOBSON: It also suggests, when you look at the year-over-year scores, the averages, that things are getting worse, not better, because if I look at, for example, in critical reading in 2006, the average being 503, and now it’s 496. Same deal in math and writing. They’ve gone down.

MONTOYA: Well, at the same time that we have seen the scores go down, what’s very interesting is that we have seen the average GPAs reported going up. So, for example, when we look at SAT test takers this year, 48 percent reported having a GPA in the A range compared to 45 percent last year, compared to 44 percent in 2011, I think, suggesting that there simply have to be more rigor in core courses.

HOBSON: Well, and maybe that there’s grade inflation going on.

MONTOYA: Well, clearly, that there is grade inflation. There is no question about that. And it’s one of the reasons why standardized test scores are so important in the admission office. I know that, as a former dean of admission, test scores help gauge the meaning of a GPA, particularly given the fact that nearly half of all SAT takers are reporting a GPA in the A range.

Just to be super clear about the shilling, here’s Hobson a bit later in the interview:

HOBSON: Well – and we should say that your report noted – since you mentioned practice – that as is the case with the ACT, the students who take the rigorous prep courses do better on the SAT.

What does it really mean when SAT scores go down?

Here’s the thing. SAT scores are fucked with ALL THE TIME. Traditionally, they had to make SAT’s harder since people were getting better at them. As test-makers, they want a good bell curve, so they need to adjust the test as the population changes and as their habits of test prep change.

The result is that SAT tests are different every year, so just saying that the scores went down from year to year is meaningless. Even if the same group of kids took those two different tests in the same year, they’d have different scores.

Also, according to my friend Becky who works with kids preparing for the SAT, they really did make substantial changes recently in the math section, changing the function notation, which makes it much harder for kids to parse the questions. In other words, they switched something around to give kids reason to pay for more test prep.

Important: this has nothing to do with their knowledge, it has to do with their training for this specific test.

If you want to understand the issues outside of math, take for example the essay. According to this critique, the number one criterion for essay grade is length. Length trumps clarity of expression, relevance of the supporting arguments to the thesis, mechanics, and all other elements of quality writing. As my friend Becky says:

I have coached high school students on the SAT for years and have found time and again, much to my chagrin, that students receive top scores for long essays even if they are desultory, tangent-filled and riddled with sentence fragments, run-ons, and spelling errors.

Similarly, I have consistently seen students receive low scores for shorter essays that are thoughtful and sophisticated, logical and coherent, stylish and articulate.

As long as the number one criterion for receiving a high score on the SAT essay is length, students will be confused as to what constitutes successful college writing and scoring well on the written portion of the exam will remain essentially meaningless. High-scoring students will have to unlearn the strategies that led to success on the SAT essay and relearn the fundamentals of written expression in a college writing class.

If the College Board (the makers of the SAT) is so concerned about the dumbing down of American children, they should examine their own role in lowering and distorting the standards for written expression.

Conclusion

Two things. First, shame on College Board and James Montoya for acting like SAT scores are somehow beacons of truth without acknowledging the fiddling that goes on time and time again by his company. And second, shame on Here and Now and Jemery Hobson for being utterly naive and buying in entirely to this scare tactic.

  1. September 30, 2013 at 7:10 am

    The testing-industrial complex is serious business. We were just warned in NY that the new Common Core tests were going be much more expensive. Guess that means we’ll see even smaller budgets for classroom instruction.

    The issue of essay-length and score on the SAT was famously tackled by a 14-year Stuyvesant student who collected data from his friends and ran a regression.

    http://abcnews.go.com/GMA/ConsumerNews/teen-student-finds-longer-sat-essay-equals-score/story?id=12061494

    • September 30, 2013 at 7:43 pm

      As the grading on the essays is extremely subjective and graded by teachers who are probably paid for piece-work, the scores are not indicative of student abilities. But an 800 on the math portion is quite indicative when compared with a 400, or even a 600. So I wouldn’t necessarily throw out the baby with the bath water. Of course, there are errors on the SAT/GRE/GMAT/LSAT exams, but not enough to make them useless.

      • September 30, 2013 at 7:46 pm

        But Abe, I didn’t say they were useless. I said public radio shouldn’t be free advertising for the College Board. They have enough advertising.

        • September 30, 2013 at 7:55 pm

          Cathy, I was replying to Patrick, rather than to your exposition on “free advertising.” I don’t listen to NPR, but my wife, the nuclear engineer turned physics teacher, makes me watch the 7 pm news on PBS (what used to be McNeil-Lehrer). They seem to have plenty of advertising for their sponsors. One might wonder if the College Board is an NPR sponsor, but I hear your point. As to Patrick’s comment on Common Core, I am waiting for my wife to articulate and discuss her viewpoint on this.

        • September 30, 2013 at 8:01 pm

          oops.

  2. Guest2
    September 30, 2013 at 9:10 am

    We need to especially consider the inability of folks like the College Board to assimilate and appropriately respond to the POV of OWS.

    The future will judge the College Board harshly, especially in regard to the ongoing betrayal (continuing to ramp up admissions, and tuition, as the college premium vanishes along with jobs students are trained for, etc.) as immoral and unethical. Schools were, after all, established for the common good. No longer true.

    The college board has long been a cheerleader for the college-industrial complex. No impartiality here and now; the bias is completely one sided. Frontline actually did a series on this. Check out the links between eugenics, testing, and racist and elitist social models.

    http://www.pbs.org/wgbh/pages/frontline/shows/sats/where/history.html

    Also, Harold Wechsler, The Qualified Student: A History of Selective College Admission in America.

  3. September 30, 2013 at 1:39 pm

    I can believe the College Board has mixed incentives, and *could* create an arbitrary downward-trend in year-over-year scores if desired. The report should have at least entertained the idea that the decline is “score deflation” to mirror the “grade inflation”.

    But, score comparability between years can (and I believe usually is) a goal of test design. It’s achieved by the use of similar or identical questions between years (and tests within the same year), including sections on the test which are primarily for that purpose.

    Similarly, since all takers in the same year don’t face exactly the same questions, even same-year scores are adjusted to achieve some desired level of comparability. So “friend Becky’s” impression that the math test got harder question can be true, but even with test-takers missing more questions, their scores won’t necessarily go down – if the designed-in controls indicate that last year’s takers would have missed the new-style questions, too. (That is, these scores are far from a simple mapping of percentage-right to final-score.)

    (Also, the BI writeup linked from the text, _people are getting better at them_, offered in support of the idea that “they had to make SAT’s harder”, does not in fact say they’ve ever made them harder, and in fact its graphs mainly show score rises over the years. That’s in apparent contradiction of both Hobson’s claim that objective scores are declining, and your claim that tests/scoring-standards have historically been toughened.)

    If the College Board’s test designers have been directed to pursue cross-period comparability, and are properly monitored/rewarded for that goal, then your allegation that inter-year averages are manipulated for marketing purposes is quite unfair. On the other hand, if College Board does have its thumbs on the scale like that, there should be a paper-trail or whistleblower that can reveal where their practice deviates from proper cross-period controls. It’s worth asking the question, but not assuming the conclusion.

    • September 30, 2013 at 1:49 pm

      Fair enough, but given that the incentives are there and that they didn’t mention them, I’m suspicious.

    • Guest2
      October 1, 2013 at 8:44 am

      No mention either of the “Item Response” black-box, used to reconstruct for testing purposes just what the thought process SHOULD BE for answering these questions. No mention at all.

      http://en.wikipedia.org/wiki/Item_response_theory

  4. M. Winslow
    September 30, 2013 at 7:54 pm

    While some test makers may be trying their best to make fair and comparable exams year to year, the fact that the whole process is literally a black box should draw some red flags. My thought has always been that the analysis of the tests should be more transparent. Moreover, while the College Board and the SAT are bad, consider the Educational Testing Services (ETS) and their suite of tests (GREs, TOEFL, etc). Even though ETS is non-profit, their subsidiaries are NOT. That is, the computer test center where you *take* the test is quite the for-profit entity. I never understood how a non-profit could own for-profit entities and still be called a “non-profit”. But I don’t know anything about corporate structure, so maybe it is all supposed to make sense. Then again, maybe it really is a scam.

    Anyway, I am glad someone is keeping a critical eye on this sort of thing ;-)

  5. Billikin
    October 3, 2013 at 9:20 pm

    What? No mention of the Flynn effect? The SAT is pretty much an IQ test. People, including high school students, are smarter in terms of IQ than ever. Why all the hand wringing?

  1. October 1, 2013 at 1:30 am
  2. October 3, 2013 at 4:38 pm
Comments are closed.
Follow

Get every new post delivered to your Inbox.

Join 1,807 other followers

%d bloggers like this: