Women not represented in clinical trials
This recent NYTimes article entitled Health Researchers Will Get $10.1 Million to Counter Gender Bias in Studies spelled out a huge problem that kind of blows me away as a statistician (and as a woman!).
Namely, they have recently decided over at the NIH, which funds medical research in this country, that we should probably check to see how women’s health are affected by drugs, and not just men’s. They’ve decided to give “extra money” to study this special group, namely females.
Here’s the bizarre and telling explanation for why most studies have focused on men and excluded women:
Traditionally many investigators have worked only with male lab animals, concerned that the hormonal cycles of female animals would add variability and skew study results.
Let’s break down that explanation, which I’ve confirmed with a medical researcher is consistent with the culture.
If you are afraid that women’s data would “skew study results,” that means you think the “true result” is the result that works for men. Because adding women’s data would add noise to the true signal, that of the men’s data. What?! It’s an outrageous perspective. Let’s take another look at this reasoning, from the article:
Scientists often prefer single-sex studies because “it reduces variability, and makes it easier to detect the effect that you’re studying,” said Abraham A. Palmer, an associate professor of human genetics at the University of Chicago. “The downside is that if there is a difference between male and female, they’re not going to know about it.”
Ummm… yeah. So instead of testing the effect on women, we just go ahead and optimize stuff for men and let women just go ahead and suffer the side effects of the treatment we didn’t bother to study. After all, women only comprise 50.8% of the population, they won’t mind.
This is even true for migraines, where 2/3rds of migraine sufferers are women.
One reason they like to exclude women: they have periods, and they even sometimes get pregnant, which is confusing for people who like to have clean statistics (on men’s health). In fact my research contact says that traditionally, this bias towards men in clinical trials was said to protect women because they “could get pregnant” and then they’d be in a clinical trial while pregnant. OK.
I’d like to hear more about who is and who isn’t in clinical trials, and why.
mathbabe 
If it were not so serious, it would be hilarious. Of course you don’t want to do drug tests on pregnant women, you might find out what drugs work for pregnant women!
Taking a step back, is this an unfortunate side effect of many individual scientists taking decisions to work on easier things they can probably get results from rather than harder things they are less likely to get results from?
LikeLike
Mouse studies are done with genetically identical mice, to eliminate genetic noise. Now that is also very far removed from reality and it also the very beginning of studies (and the cheapest). If it cant make it pass genetically identical mice there is no point pursuing the theory. many drugs/experiments are tried, most never make it pass mouse studies and hardly any make it to the drug store. There are many levels beyond mouse studies, and the NIH knows this, that is why they have allocated 10 million (will that even fund 1 female mouse study? probably not). This a stupid issue, played up by ignorant politicians, to fan the flames of the “war on women”. And again, the NIH is not serious when it only commits 10 million, you are being played.
LikeLike
@Jonathan Kirby
I don’t think it’s quite fair to malign scientists for “making decisions to work on easier things they can probably get results from rather than harder things they are less likely to get results from”. Scientists don’t just have a giant bag of money that they can reach into to fund their research; they write grant proposals to funding agencies in which they explain what questions they want to answer, why they’re important, how they’re going to answer them, demonstrate that they will have success. Over the last few decades science funding agencies have become increasingly risk averse, preferring to fund science with a high probability of success, but lower reduced impact (with success defined as getting the predict/expected outcome) instead of science with a much lower probability of success but the opportunity to be game changing. One of my supervisors said the problem with doing innovative new science is that when you’re that far ahead of the curve your research is being reviewed by your ‘inferiors instead of your peers’, who in many cases have competing research agendas. Science is definitely an ego-drive elbows out field.
@mathbabe: To me it seems even worse than ‘The downside is that if there is a difference between male and female, they’re not going to know about it’. When they say it would ‘skew the results” it suggests that they have an a priori expectation that things will behave differently in women than in men, and so we’re going to avoid that problem all together. As a statistician and scientist I become alarmed when experiments are so narrowly defined that they become almost tautological, because in those cases you only learn something if you don’t get the expected outcome.
LikeLike
My dad is a neurobiologist who, before he retired, studied things like drugs for treating depression. When I was about 12 years old, in the early 80s, he told me that they only study men because women’s menstrual cycles would interfere with the results. I immediately replied, “Won’t women be having their menstrual cycles when they take the drugs?”
I can hardly believe that anyone still makes this claim.
LikeLike
HUGE ECONOMIC DISADVANTAGE – Men Not Represented in Set of People With Reproductive Freedom – WAR ON MEN.
Oddly, Women, While Complaining Healthcare System Not Fair, Have Much Longer Lifespan.
If I want to measure the impact of a drug on some variable, the larger the variation of that variable without the drug, the larger the sample size needed to accurately quantify the change caused by the drug. The larger the sample size needed, the larger the cost of the study. Adding a more variable population subtype for reasons of political correctness may not, in general, be smart, notwithstanding that in particular cases, skewing the sample towards women may be smart.
Non-tribal researchers are probably the way to go.
LikeLike
Gah. This is sloppy journalism by the NYT. They say two things about clinical trials:
1. Researchers will use the funding to include more participants, generally women, in clinical trials
2. Women are not adequately represented in many clinical trials for drugs and medical devices
1 is great, assuming the money is enough to make a difference. 2 is probably true, and if so it’s a problem that needs to be addressed. But it’s not true that we don’t test treatments and devices that are going to be used on actual humans on women. Clinical trials are HIGHLY regulated, there are a ton of screening procedures, and by the time you get to Phase III you have thousands of participants. I suspect that the shortage of women is in Phase I (20-80 people) and maybe Phase 2 (100-300). I don’t know that, since the article doesn’t go into detail. But I’m damn sure that women aren’t being prescribed treatment that has only been adequately tested on men.
The rest of the article is about studies, generally on lab animals. There aren’t enough studies being done that investigate gender differences. Okay, cool, we should fix that. But the article is written in a way that makes it easy to conflate the two issues*, we end up with scary, clickbait nonsense like ‘Treatments aren’t being tested on women!’ instead of a description of the actual problem. Why don’t we have enough women in clinical trials? None of the explanations in the article work, because those are all about animal studies. That would make a fascinating story, but it’s not the one we get to read.
*As it seems to me like you have, Cathy. No offense intended! It happens, particularly when science journalists aren’t doing a good job.
LikeLike
I’m a big fan of the blog, but not this entry. Clinical trials should include women and to a sufficient N that if an effect is seen only in women or men the drug maker can prove efficacy on that group. Currently the largest cause of failure for new medications in phase I-III is lack of efficacy, not toxicity. If drug makers are able to show sex based efficacy they would not pass that up. If clinical trials do not include enough women to show statistically significant efficacy in sex divided cohorts then by all means they should be improved.
However, the jump to animal models is spurious. As I mentioned the prime reason for compound attrition during clinical trials is efficacy and that is directly a consequence of our animal models. A compound which shows activity in a mouse model, and later a non-human primate model, frequently shows no efficacy above placebo in human. That is because our animal models are still poor. They are extremely noisy and many times being developed while we are using them. Luckily we don’t experiment on prisoners, so animal models are our only way of generating confidence in a compound’s mechanism of action. Testing a drug on female mice does not result in a better prediction of efficacy in human females.
Disclaimer: I work for a large pharmaceutical company in early drug discover, particularly in data analysis (which is why I love this blog).
LikeLike
Here’s the thing–it sounds so right that we shouldn’t exclude females (of any species) from medical research. But doesn’t the importance of doing this depend on the frequency with which the effects of a treatment will differ in men and women? Sure, there are some examples–not too many, actually. But if 99.9% of all treatments would work the same in men and women, and if it takes a lot more resources to study both genders, does it really make sense to study both genders routinely? To make an (admittedly imperfect) analogy, we know that some new drugs will end up having rare but serious side effects–but we don’t insist that every new drug be studied in tens of thousands of people to make sure we find such effects, we understand that we may have to find out some of these things after products are out there being used.
Now, I don’t know what percentage of treatments work differently in men and women. If it were substantial, like 20-30%, then it would clearly be important to study routinely. But if it’s very low (as many scientists believe) then it’s not so obvious that we need to routinely invest the resources to fully evaluate gender differences. Unfortunately, to even raise this question opens someone up to being accused of saying it’s not important to find out what works in women.
LikeLike
Except, how do you know empirically the rate of difference in gender response until you systematically test both genders? What people — scientists or not “believe” is not a basis for empirical research.
LikeLike
How do we know empirically the rate of difference in response by race? Or by ethnic heritage? Or by any of a huge number of genetic factors? Shall we systematically test all treatments in large enough numbers to see if there are different treatment effects in any such categories? We have examples of different treatment effects by race; if we are going to test for sex differences systematically, why not race differences? the numbers that would be needed to test treatments in this way would become huge. Should we settle for evaluating 90% fewer treatments in order to systematically test for effects by race and sex, without good evidence that such differences will emerge in a reasonable proportion of cases?
LikeLike
Like Guillermo I also work in large pharma in early discovery doing computational biology. I agree largely with what he says, and would add that it is not true that drugs are not tested on female gender. In addition, there is a great deal of interest in understanding the role that sex differences can play in disease, and this is particularly true in disease which show greater incidence based on gender (eg certain auto immune diseases are much more prevalent in women) and for sure clinical trials are designed to reflect this. In diseases (eg many cancers) where there is no obvious gender bias, the data is pretty evenly split between men & women.
I would also be very surprised if the pharmacological effects of a drug are not tested for sex based differences, since the FDA requires that their off target effects also be tested for before a drug is approved. It is true that the role of gender in disease is poorly understood, but I don’t think this because of gender bias, but mostly because the science is really hard (and fascinating-there is a whole very active area of research about how one copy of the X chromosome is silenced in women, and whether there are similar silencing mechanisms elsewhere, at the genome level;however the role of hormones in regulation and disease are not well understood).
LikeLike
Cathy, great post. Commenters, interesting things to think about.
Yeah, I’m just taking this as another case of the NYTimes sensationalizing a bit on the gender bias issue. Although, I always chalked up the bias to be less about variability and more about liability (if perhaps the drug later turned out to increase birth defects à la Thalidomide).
LikeLike
Cathy,
What phase drug trial are you concerned about? The early phases were the volunteers can get killed/permantly damaged (e.g. sorry about that kidney/liver/uterus) or the later phases where you work on the target population (which includes women) to show efficacy (like birth control)? We all remember thalidomide, too.
I’ve done trials involving live viruses and external viricides & we didn’t use women. Not just because thet get pregnant, but because pregnant women miscarry or give birth to all sorts of malformed/stillborn children, which is a huge legal liability. Finally, don’t forget that an increasing number of you are barren (chlamydia, PID, etc.) If that’s the case, women and their lawyers will be looking for some deep pockets to blame.
So its not because of lack of interest, but simply because of the risks and associated costs.
LikeLike