mathbabe

1. The life and death of a metric

There’s a problem, or at least potentially a problem. Someone figures out how to measure the problem. The measurement isn’t perfect, and everyone admits that, but nevertheless nobody argues against using it, since knowing something is better than knowing nothing.

That metric is used a few times, and people get used to hearing it, and they mostly forget what its limitations were. Moreover, they start assigning blame to it. Instead of seeing a “bad score” as something the indicates a need for more resources and support, it becomes a moral failing: take responsibility for your terrible score and do something about it.

Stakes get high. People are measured, judged, and rewarded or punished based on their score. They start focusing on improving their score at all costs, and the small imperfections of the scoring system in the first place are magnified and distorted. Cheating happens too.

Before long, it’s all about the score, at least until dissenting voices point out that all this focus on the score hasn’t actually addressed the initial problem. In fact, it’s gotten worse over time. The focus on this metric is given up by some, held on to by others who have found other uses for it, and everyone starts looking around for a new metric to solve the initial problem.

2. Example: education reform

A few decades ago we decided to look into the international competitiveness of our nation’s children. We developed tests to see how much people in different states and different schools knew about certain things. This wasn’t a perfect process, to be sure, since the curriculums varied from state to state and school to school, but it did yield results, and they were numeric, so people trusted them.

The argument for doing this was convincing – how could anyone argue against wanting to know where we stand on education? With this knowledge in hand, surely it would be easier to know where there were struggling schools and give them help.

But instead of coming to the aid of the school systems that needed help, we ended up punishing them, blaming the principals and teachers for the problems. The fact that the scores were extremely correlated to poverty was explained away by saying our teachers had “given up on poor students.”

Thus began the era of high stakes testing for students and teachers, where teachers were “held accountable” for their students’ progress on standardized tests, if not their scores. It’s purely punitive, and a far cry from the original purpose of helping out those who need it. We haven’t equalized funding for schools, for example.

Moreover, it hasn’t helped the students and schools which are struggling. In fact scores overall seem to be going up for everyone, but the rich students, to put it bluntly, are improving faster than the poor ones. And in the meantime everyone is getting sick of all the tests.

3. What’s next in educational reform?

Recent research has shown there might be a new explanation for why some kids do well when others don’t, and it’s all about “character,” or “grit.” Kids who have it in abundance seem resilient in the face of failure, and they thrive even in tough situations.

Hallelujah! Now we can try to develop kids’ grit quotients, or better yet, we can hold teachers accountable for doing so. The only problem with this plan is that it’s actually kind of hard to consistently measure grit.

Until now. Angela Duckworth, who is one of the social scientists that has been studying grit in children and who recently wrote an opinion piece on the subject, has helped develop a scorecard, based on self-assessment, so that kids can be scored.

In her piece, Duckworth warns us that the scoring system isn’t perfect, and she has even said it’s a bad idea to use this scoring system for accountability; it could undermine the very thing we’re trying to promote.

But she’s also said that kids can improve their scores, with help from teachers, and that high scores are good signs for progress in behavioral and academic achievement.

Let’s not go there.