Home > Uncategorized > Grit metrics for kids: let’s not

Grit metrics for kids: let’s not

March 28, 2016

1. The life and death of a metric

There’s a problem, or at least potentially a problem. Someone figures out how to measure the problem. The measurement isn’t perfect, and everyone admits that, but nevertheless nobody argues against using it, since knowing something is better than knowing nothing.

That metric is used a few times, and people get used to hearing it, and they mostly forget what its limitations were. Moreover, they start assigning blame to it. Instead of seeing a “bad score” as something the indicates a need for more resources and support, it becomes a moral failing: take responsibility for your terrible score and do something about it.

Stakes get high. People are measured, judged, and rewarded or punished based on their score. They start focusing on improving their score at all costs, and the small imperfections of the scoring system in the first place are magnified and distorted. Cheating happens too.

Before long, it’s all about the score, at least until dissenting voices point out that all this focus on the score hasn’t actually addressed the initial problem. In fact, it’s gotten worse over time. The focus on this metric is given up by some, held on to by others who have found other uses for it, and everyone starts looking around for a new metric to solve the initial problem.

2. Example: education reform

A few decades ago we decided to look into the international competitiveness of our nation’s children. We developed tests to see how much people in different states and different schools knew about certain things. This wasn’t a perfect process, to be sure, since the curriculums varied from state to state and school to school, but it did yield results, and they were numeric, so people trusted them.

The argument for doing this was convincing – how could anyone argue against wanting to know where we stand on education? With this knowledge in hand, surely it would be easier to know where there were struggling schools and give them help.

But instead of coming to the aid of the school systems that needed help, we ended up punishing them, blaming the principals and teachers for the problems. The fact that the scores were extremely correlated to poverty was explained away by saying our teachers had “given up on poor students.”

Thus began the era of high stakes testing for students and teachers, where teachers were “held accountable” for their students’ progress on standardized tests, if not their scores. It’s purely punitive, and a far cry from the original purpose of helping out those who need it. We haven’t equalized funding for schools, for example.

Moreover, it hasn’t helped the students and schools which are struggling. In fact scores overall seem to be going up for everyone, but the rich students, to put it bluntly, are improving faster than the poor ones. And in the meantime everyone is getting sick of all the tests.

3. What’s next in educational reform?

Recent research has shown there might be a new explanation for why some kids do well when others don’t, and it’s all about “character,” or “grit.” Kids who have it in abundance seem resilient in the face of failure, and they thrive even in tough situations.

Hallelujah! Now we can try to develop kids’ grit quotients, or better yet, we can hold teachers accountable for doing so. The only problem with this plan is that it’s actually kind of hard to consistently measure grit.

Until now. Angela Duckworth, who is one of the social scientists that has been studying grit in children and who recently wrote an opinion piece on the subject, has helped develop a scorecard, based on self-assessment, so that kids can be scored.

In her piece, Duckworth warns us that the scoring system isn’t perfect, and she has even said it’s a bad idea to use this scoring system for accountability; it could undermine the very thing we’re trying to promote.

But she’s also said that kids can improve their scores, with help from teachers, and that high scores are good signs for progress in behavioral and academic achievement.

Let’s not go there.

Categories: Uncategorized
  1. March 28, 2016 at 8:44 am

    They don’t teach grit in ruling class schools.
    Grit is the virtue of the slave —
    Grit your teeth and bear the yoke —
    Pick and grin, the butt of every joke.
    Pick and grin and jump when they poke.

    Liked by 1 person

  2. mlachans
    March 28, 2016 at 8:52 am

    The history of “grit” is actually a little worse than that. In fact its fair to say that non-cognitive skills are currently the “dark matter” of education research.

    In the 1950s, 60s, and 70s, much of the education agenda was driven entirely by IQ because IQ/g has a more-or-less stable and robust rank relationship with wages, crime, and other social ills. This meant that if a program improved your IQ (like iodine treatments and prenatal nutrition), you could make reasonably OK forecasts of the benefits/costs of the program as soon as the program was complete by administering IQ tests before and after the program was complete or doing so for a “matched” group.

    Cut to the early childhood education movement of the 1960s, and you see wild claims being made about five point (1/3 of a standard deviation) gains in IQ. When these gains faded-out and the kids converged back to the IQ levels of non-treated children, pessimists were quick to claim that the program had failed (as in The Bell Curve (1994)). However, upon re-evaluation, people noticed that those who had access to early childhood education programs were much less likely to be involved in crime, needed less special education, had more stable marriages, more likely to attend college, and so on…

    Enter non-cognitive skills (e.g., grit). Do we really know what these skills are? No. Can we list them? No. Can we “measure” them? No. But all measurable, statistically significant gains from a program not explained by “cognitive skills” are *by definition* explained by non-cognitive skills in the education literature. So if a program delivered the goods, and we can’t see any cognitive skill increase, it must have been unobservable, not-really-measurable, non-cognitive skill increases.

    Voila, mystery solved?


  3. March 28, 2016 at 10:18 am
  4. March 28, 2016 at 10:20 am

    Thank you Cathy for addressing “Excess Scoring”. That’s what I started calling it a while back and I think your observations are right on the mark. How far do you go with scoring consumers? It doesn’t stop when those scores can be sold for profit. In healthcare to avoid HIPAA issues, a score gets sold instead of actual data. It’s a full blown racket and of course it accelerates inequality in the process with flawed data used to “score” people and they never get a chance as they are already “scored” into the toilet. Insurance companies live off of this and create risk assessment scores like they are going out of style as a way of denying access.


    What goes on at the drug store or with your pharmacy benefit manager every time you get a prescription filled with “medication adherence prediction scores” is a real racket. Look at what Express Scripts does with metrics that have zero to do with taking medications. If one is a male, seeing a female doctor, get a ding. If your income does not meet a certain level, another ding. If you have kids in the home, yet another ding as they will distract you from taking your meds. I talk with pharmacists on this and they hate it as now with Medicare there’s software called Equipp where Medicare patients, ones they have never seen already have a “predicted” score and if one pays cash and doesn’t use credit cards and there’s no available data to pull from a series of databases, you are now an outlier.

    The scoring methodologies leave a lot to be questioned and of course it’s all proprietary with nobody getting another look to see how this works, so you know buried in the compiled code there’s race and all kinds of other metrics in there…scary. It’s all about money as all these scores get sold and they make money. Insurers buy them and take a look at the Goldman Sachs of health insurance, United, they are buying up pharmacy benefit managers like crazy as they can use those scores with denying or allowing access. The scores became so bad that CMS had to sanction Cigna for denying drugs and the contracted workers of the PBM Catamaran, the PBM, didn’t even know why they were denying drugs, but some score on a screen said so. Scoring indeed is done in excess, and it works to further inequality everywhere. The correlations become too spurious as well as they keep building more metrics on top. So if you don’t fit the “scoring” patterns, you’re an undesirable outlier now..


    So when you get your next prescription, there’s a secret medication adherence score being created on all of us and it’s a secret and you can’t get it..makes it even worse. Excess Scoring of consumers is indeed a huge problem.


  5. March 28, 2016 at 12:30 pm

    Yeah. The sole focus on grades, for instance, became a prominent feature and we see it all the time here. In fact, many students are merely looking for a quick fix before the exams, so the purpose of learning and continuing development is hampered.

    On the topic of grit, this is also one of the characteristics outlined by several world-class peak performers in their respective area of expertise. In the end, intelligence can only carry you so far, if there is no drive underlying the pursuit of excellence. For some individual this drive could be it’s ego-driven (which is not necessarily a bad thing) and for other it’s more a ritual. And this phenomenon is known under different names: mental resilience, “rage” of master, killer instinct, etc, so it’s not only restricted to the community of social scientists.


  6. A
    March 28, 2016 at 3:40 pm

    Reading Duckworth’s piece she says throughout she is against using it for high-stakes assessment (as you mention in your summary, too). But if the “let’s not go there” refers to teachers helping students improve their scores, and you are against that? What is your proposal? Just let kids be? (That might be a fine proposal).


  7. LKT
    March 28, 2016 at 4:19 pm

    Reminds me of the science news cycle from PhD comics



  8. Bas
    March 28, 2016 at 4:38 pm

    It smacks to me also of an arm of the effort to smear the public school system in order to privatize education. And we know how that has turned out–great for Venture Capital, terrible for the kids.


  9. March 28, 2016 at 4:58 pm

    ESSA, the law that replaced NCLB, requires states to devise and use “measures of school quality and student success” that are NOT standardized tests… and if you think these new measures will be statistically valid and worthwhile I’ve got a free certificate for you to attend a seminar offered at Trump Institute… and those who want to believe that poverty is an inconsequential obstacle that can be overcome by “grit” and “grit” can be measured with a cheap, easy to administer pencil-and-paper assessment will be very happy to use that metric as a way to support their belief…


  10. Savonarola
    April 1, 2016 at 3:36 am

    Personally, I worry that when they get to scoring kids on these “character” things it makes life even worse for those with neurological issues. On the one hand, an autistic kid who is successfully mainstreamed is off the chart on “grit”: they are managing in an environment that strains them on every level, and living through it and learning anyway. On the other hand, this child presents as very rigid and incapable of flexing around problems that seem miniscule to neurotypicals. I have learned to be very careful about on whom any measure is normed. In a modern heterogenous classroom, there are a lot of kids who are not the norm, and the results might not be valid for them – but from the outside, they seem valid. With educational outcomes, we’re shorting a kid on knowledge. With character issues, do we wind up branding them as people in a self-fulfilling way?


  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: