Nick Kristof is not Smarter than an 8th Grader
This is a post by Eugene Stern, originally posted on his blog sensemadehere.wordpress.com.
About a week ago, Nick Kristof published this op-ed in the New York Times. Entitled Are You Smarter than an 8th Grader, the piece discusses American kids’ underperformance in math compared with students from other countries, as measured by standardized test results. Kristof goes over several questions from the 2011 TIMSS (Trends in International Mathematics and Science Study) test administered to 8th graders, and highlights how American students did worse than students from Iran, Indonesia, Ghana, Palestine, Turkey, and Armenia, as well as traditional high performers like Singapore. “We all know Johnny can’t read,” says Kristof, in that finger-wagging way perfected by the current cohort of New York Times op-ed columnists; “it appears that Johnny is even worse at counting.”
The trouble with this narrative is that it’s utterly, demonstrably false.
My friend Jordan Ellenberg pointed me to this blog post, which highlights the problem. In spite of Kristof’s alarmism, it turns out that American eighth graders actually did quite well on the 2011 TIMSS. You can see the complete results here. Out of 42 countries tested, the US placed 9th. If you look at the scores by country, you’ll see a large gap between the top 5 (Korea, Singapore, Taiwan, Hong Kong, and Japan) and everyone else. After that gap comes Russia, in 6th place, then another gap, then a group of 9 closely bunched countries: Israel, Finland, the US, England, Hungary, Australia, Slovenia, Lithuania, and Italy. Those made up, more or less, the top third of all the countries that took the test. Our performance isn’t mind-blowing, but it’s not terrible either. So what the hell is Kristof talking about?
You’ll find the answer here, in a list of 88 publicly released questions from the test (not all questions were published, but this appears to be a representative sample). For each question, a performance breakdown by country is given. When I went through the questions, I found that the US placed in the top third (top 14 out of 42 countries) on 45 of them, the middle third on 39, and the bottom third on 4. This seems typical of the kind of variance usually seen on standardized tests. US kids did particularly well on statistics, data interpretation, and estimation, which have all gotten more emphasis in the math curriculum lately. For example, 80% of US eighth graders answered this question correctly:
Which of these is the best estimate of (7.21 × 3.86) / 10.09?
(A) (7 × 3) / 10 (B) (7 × 4) / 10 (C) (7 × 3) / 11 (D) (7 × 4) / 11
More American kids knew that the correct answer was (B) than Russians, Finns, Japanese, English, or Israelis. Nice job, kids! And let’s give your teachers some credit too!
But Kristof isn’t willing to do either. He has a narrative of American underperformance in mind, and if the overall test results don’t fit his story, he’ll just go and find some results that do! Thus, the examples in his column. Kristof literally went and picked the two questions out of 88 on which the US did the worst, and highlighted those in the column. (He gives a third example too, a question in which the US was in the middle of the pack, but the pack did poorly, so the US’s absolute score looks bad.) And, presto! — instead of a story about kids learning stuff and doing decently on a test, we have yet another hysterical screed about Americans “struggling to compete with citizens of other countries.”
Kristof gives no suggestions for what we can actually do better, by the way. But he does offer this helpful advice:
Numeracy isn’t a sign of geekiness, but a basic requirement for intelligent discussions of public policy. Without it, politicians routinely get away with using statistics, as Mark Twain supposedly observed, the way a drunk uses a lamppost: for support rather than illumination.
So do op-ed columnists, apparently.
mathbabe 
And in case you missed it, a few years ago Kristof hailed the virtues of VAM, which his editors and Cuomo love! http://www.nytimes.com/2012/01/12/opinion/kristof-the-value-of-teachers.html?emc=tnt&tntemail0=y
LikeLike
I have found many articles by columnists and others in the NY Times and other newspapers to be inaccurate. Before publishing his column, NY Times editors should have shown a mathematician such as Cathy the article. I have found columns about Israel such as those by Tom Friedman totally misleading as well.
It is true politicians and others misuse statistics and math either deliberately or unknowingly but if there were appropriate editors to correct the mistakes in articles or in other media then this would not be so much of a problem.
Since the US is a leader in engineering and technology, the overall average score in math is not a problem for the nation in these areas. In fact, there is a surplus of STEM people.
A reason why the US is not doing as well as it could is that highly educated mothers are having few children compared with lesser educated moms because of career and lack of support for career moms in the US and because of the very, very high cost of education. University education and professional education is very expensive as is sending children to private schools grades 1-12 which may be necessary in places such as NYC.
If you want the scores to increase, these families need to be having more children. This means making child rearing easier for professional moms (and dads) and lowering the cost of education for their children.
LikeLike
Apparently we need scientific translators even for arithmetic. Where are these scientific translators? Isn’t everyone a scientific translator, to the best of her ability? If not, why not?
Kristof makes me gag. But aside from him, the problem of scientific translators makes me sad because I’m a librarian. In principle, librarians can make excellent scientific translators. Yet librarians’ training in “information literacy instruction” actually discourages them from this. Librarianship literature tells librarians that they should teach students only to look for “authoritative sources,” and leave it at that.
No one has a bumper sticker saying, “Don’t question authority.” So I suspect that some librarians are not confined by these stifling professional boundaries. But librarians who go further will tend to be viewed with suspicion by colleagues, since they will seem to cast doubt on their colleagues’ work.
Since journalists’ perspective and journalists’ position between experts and the public is roughly the same as that of librarians, I suspect that a similar pressures arise in journalism. But I need to know more about journalistic education and practices than I do.
LikeLike
An article was just published in the NYTimes relating to educated women and children. An unintended consequence of women becoming more educated is that they tend to have fewer children and thus the proportion of children who will do well on math tests and other tests will go down. From the article:
“Among mothers with an advanced degree, 23 percent have only one child, and only 8 percent have four or more.
But among mothers who did not complete high school, 13 percent have only one child, and 26 percent have four or more.”
LikeLike
It is sad that the NYT is looked upon by many Leftists as the go to source for information. To make that choice, one needs to accept that propaganda is the go to source.
LikeLike
One of Bob Somerby’s (The Daily Howler) continuing
concerns is the cherry-picking of data of comparative
international test performance data. His latest posting
(http://dailyhowler.blogspot.com/2015/05/supplemental-professor-brenowitz-plays.html)
also addresses the Kristof op-ed.
LikeLike
you realize that you are , like, wrong about this, and wrong in a nasty way by citing TDH, whic makes all sorts of nasty accusation without any cites ?
however, if you can read, go look at Fig1 here
http://www.ets.org/s/research/30079/overview.html
and then apologize to Mr Kristof
LikeLike
I second that.
Kristof did pick a couple of problems from one test, but immediately linked to the report you reference, which is the source of his alarmism, and which supports his position.
The TDH screed, ironically, cherrypicks through Kristof’s column in an effort to make him look like a hack.
LikeLike
I didn’t go into detail about the ETS report in the post, but to my mind, it makes Kristof look even sloppier, or even more deliberately misleading. First off, the ETS report is about PIAAC, which is a survey of ADULTS, not schoolkids. How adults do on a math survey brings in a lot of factors beside school education. Second, the PIAAC is given in 22 countries, all in what’s traditionally considered the first world, or about HALF the 42 for which we have TIMSS data. Kristof is focusing on 8th graders — just look at his title, and the bulk of his column, and this tweet (https://twitter.com/NickKristof/status/592314195628003328): “My column notes that US kids lag the world in basic math”. He is implying that American 8th graders (“kids”) did worse (“lag”) than kids from Iran and Ghana (“the world”) in basic math. And NONE of that is true.
LikeLike
That US secondary school students do less well than those in some other advanced countries on international tests is not news. The same thing was true in the 1960s if not earlier as well. Low scorers tend to be minority and poor kids. A not widely reported fact is that over the past several years they have scored better and better on such tests, gaining a grade level or so in some areas. (Sorry, I don’t have reference handy, but it should not be difficult to check this.) IOW, our secondary schools are doing a good job educating minority and poor children. Sudden improvement would be unrealistic.
LikeLike
The fact that minority scores have improved is arguable. Here’s an article from 538 with some charts on the issue. Certainly any changes have been very small. Also: Looking at those tables we see noticeable gains in the 1980-1990 period, but then basically stagnant after that point (at least for those aged 9+).
http://fivethirtyeight.com/features/black-and-hispanic-students-are-making-meaningful-gains-but-its-hard-to-tell/
Of course, I totally agree that all signs are that the major factors are poverty and rough living situations. I do wish this country could get its act together and help people with that, have nationally-supported school systems, etc.
LikeLike
Thanks for the reference, Dan. May I suggest taking another look at the article?
The headline is confusing, IMO. It sounds like minority students are making gains, but it is hard to tell because they are small gains. But as the article explains, it is hard to tell **when you only look at the overall averages** because the proportion of minority students is increasing.
LikeLike