How to think statistically (about dieting)
There are lots of ways to get statistical thinking wrong, not so many ways to get it right. Here’s a series of examples from wrong to right:
- I did this, and it’s not a “diet,” it’s a lifestyle change, and it works for me!
- I know people who live or interact with the world in a certain way, and it seems to work for them! After all, French women are thin. We should all do what they do.
- There was a study of volunteers, and for the people who stayed in the study to the end, they lost weight doing such and such lifestyle change!
- There was a study of volunteers, and they tracked people down who tried to leave the study, and the average weight gain was still real, among the people they found!
- There was a study of doctors giving advice or enrolling people in programs to help overweight people lose weight, and 97% of people lost no weight and plenty of people gained weight, maybe even more than half.
What I’d love is for people to understand how much difference there is between a personal experience (1) and advice we’d have on public health (5).
Here’s the golden standard: if you can come up with something to tell Medicare about how to have a population of morbidly obese people become a population of regular weight people, then you win. Otherwise, if you’re tempted to tell me about a lifestyle change that worked for you, please don’t, because that’s not statistical.
Also, I’d like a word about the theory that with enough discipline and willpower, anyone can lose weight. I think it’s fair to say I have discipline and willpower. In fact, I’m a fucking poster child for them. I wrote a Ph.D. as one of few women in a male-dominated field. I wrote a book or two. I’ve had three kids and I’ve never struck one of them in anger. In fact I’m pretty nice to people most of the time, even though I’m relatively often filled with rage at the unfairness of the world. That’s hard. It takes willpower.
I even ran a sprint triathlon at 275 pounds, really fast, which took months of ridiculous training. Also, I know all about healthy habits, I don’t eat “emotionally,” just when I’m hungry, and I love brussel sprouts and other healthy foods. I just get really fucking hungry, often.
Readers, I’m the fucking center of the disciple in willpower universe over here.
Given all of that, if anything I’d argue my willpower is one reason I’m so heavy. When I was 22 or so, I went on a fat-free diet, on the advice of my doctor, that fucked me up; I lost 30 pounds but then gained something like 75. I think I messed up my insulin resistance. In fact I believe that also happened to me on my first starvation diet when I was 14.
I’m guessing I’d be thinner if I’d had less willpower, in other words. I wouldn’t be better off, though, because I kind of like my books and my Ph.D. and my kids who don’t fear their parents.
Anyway, from now on let’s talk statistically, shall we?
mathbabe 
I understand that willpower is not a magical infinite resource. Diets have to be realistic, you can’t expect people to adhere to some long-term regimes on “motivation” alone.
The issue is, when a study comes out saying “diets don’t work”, it’s misleading. It leaves the general public with the impression that if you follow the diet perfectly, it won’t work. That sounds like a surprising new finding. And for some short-term diets like the fat-free one you mention, it is exactly how it should be reported. But what the “diet X doesn’t work” finding can also mean is, “this diet is too akratic/hard to follow”.
If you’re framing this solely as selling a study to Medicare, then I’d say this – medicare can be expected to read the details, so calling your study “100% of adherents fail to follow diet X” is both more descriptive and less clickbaity. Journalists and other summarizers who then change this to “diet X doesn’t work” are introducing a lot of ambiguity, and IMO even changing the naive reading – the first thing that comes to mind when I read “X doesn’t work” is literally “X doesn’t work if you do it”, NOT “X is impossible to do”.
LikeLiked by 2 people
Well, now we’re talking semantics. I’m so used to thinking about dieting as an effort on the part of an individual to lose weight, that I’m at ease with saying “diets don’t work,” because those efforts consistently fail. If you narrowly define a diet to be the set of rules you’re supposed to follow, then you’d be correct, and I should say “diets work but nobody can follow diets.”
I don’t think it’s a meaningful disagreement, though. I’m going to keep mine because it’s shorter.
LikeLiked by 1 person
So if there’s something that worked for me but I don’t have a double-blind study on a large sample over time then you don’t wanna hear about it?
LikeLike
Yes that’s right. Thanks so much for asking.
Cathy
LikeLiked by 1 person
Fair enough!
LikeLike
Anecdotes are most relevant when they treat something demonstrably out of the ordinary statistically speaking. If someone’s personal experience is that a certain regimen lead to secured weight loss, then that is relevant to the discussion, precisely because there is so much evidence that losing weight is difficult. Perhaps that anecdote signals the way to do something differently and productively.
LikeLike
No, it’s just noise.
LikeLike
I’m a little late to comment because I’ve been searching for a blog post I stumbled across some years ago where someone had gathered together line charts showing the weight changes over time for a whole bunch of diet studies. I just can’t find it.
The interesting thing that showed up there if you focused on the longer studies (ie 6 months – 2 years) was a pattern where people before the diet would start out at a slowly increasing trend. Then, when the diet started they would lose weight, first rapidly and then increasingly slowly until it bottomed out. After bottoming out they would start gaining weight rapidly until that also slowed down. At the end they would end up at a similar trend they started out with somewhere around where they would have been if they hadn’t dieted at all.
In a couple of diets they would actually end up above their original trend. They would have been better of not dieting at all. This is not an uncommon phenomena at all. I’ve seen the joke “the easiest way to gain 25 lbs is to lose 20!“ throw around by obesity researchers because the phenomenon is so common.
LikeLike
You pretty much nailed it. Developing insulin resistance will mess with a person’s weight & attempts to “control” it. No, I can’t cite particular studies. But I’ve been reading a lot about this issue and how it’s coming to affect the “conventional wisdom” about weight.
So what I’d like to offer is thanks for putting this issue out there in a logical way – rather than the emotional way it’s presented in the popular press.
LikeLike
Perhaps it would be helpful to think of this topic like we do smoking. If the question is, “How do I quit smoking?” then the answer is going to be personal. It’ll be “just don’t do it anymore” and may be followed with some tips on how to make that easier. If the question is “How do we reduce smoking in the population?” then I think you could learn a lot of lessons about the last 30 years of public policy aimed at reducing smoking and apply it to weight-related questions.
What have we done that has been effective in reducing smoking? Can we apply this to obesity?
LikeLike
I think about it that way too. Problem is success rates for smoking cessation programs are much much better than for dieting. Even so some people don’t seem to be able to quit. Maybe we can learn from them?
LikeLike
This is brilliant. I am here for all of this. Thank you so much for saying this. Thank you.
LikeLike
Hi Cathy, I am not a physician nor am I a nutritionist. I am a mother whose child was diagnosed with multiple autoimmune diseases who has different types of allergies that causes inflammation( everything from CTD to JRA). From the past few years, I learned that my DD’s inflammation and weight gain was caused by certain foods( & life styles). My own theory is that all types of “fat” on a person’s body are not created equal. In fact, what is perceived as fat, can be rid of by eliminating foods and adding others. I’ve been able to help a few friends by redirecting them to different foods. And no, I never, ever recommend anything that can be referred to as “diet food”. Eating is really important to me. In fact, the first thing I tell people who ask me for help is to thrown their scales and full length mirrors away. You were nice enough to help me a few years with my mathy daughter by making a few recommendations. If you need another point of view and some input, please feel free to contact me any time.
LikeLike
I recently read “The Case Against Sugar” by Gary Taubes. In it, he claimed to look at populations before and after the introduction of refined sugar into the diet of those populations. I’ve read many things before and since about his approach and the validity of the conclusions, and there’s controversy about what he states. However, what I liked was that he didn’t pursue the usual diets-as-willpower argument or look at individual progress when discussing the issue of weight gain. I’m interested if you’ve seen the book or have looked at/pursued a similar approach when studying the problem of why we, as a population, are getting fatter.
LikeLike
I was working as a statistician for a small bio-tech company and my very smart and experienced boss (non-statistician) told me: “If in order to prove something you have to use statistics then you’re in trouble. Good results in life science speak for themselves”. Meaning that if you understand the biological mechanism behind the phenomenon you need very “simple numbers” to back your claims.
I know its not in vogue but instead of talking about the genetics, statistics and psychology of weight loss we should be taking about the less fashionable endocrinology. The answers have been there since the 1950’s (!)
The IHMC (look them up on youtube) have amazing lectures going over the bad science that has been done on this for the past 70 years…
LikeLike