Crowdsourcing a Theranos test?
Have you been following the Theranos debacle? The WSJ reported twice last week on this much-hyped Silicon Valley company which is trying to “disrupt” the blood test industry but seems to be stumbling on fraudulent methods. The company has fancy investors and even fancier board members (former Secretaries of State George Schultz and Henry Kissinger and former Secretary of Defense William Perry) and was valued early last week at $9 billion. I’m pretty sure its value has gone way down since then. Since WSJ is behind the paywall, take a look at this summary on Forbes or this one from Wired.
Well, does the blood test technology work or not? It’s frustratingly hard to know. Theranos CEO and founder, Stanford dropout, and black-turtleneck-wearing Elizabeth Holmes claims (for example at the end of this Mad Money interview from last week) there have been multiple tests of their methods against standard (more expensive) tests that require more blood. But she doesn’t provide them to the public for scrutiny. So that’s unsettling.
Here’s my idea. We crowdsource the answer to this question. It’s not a random sample but that’s ok, because we already have “ground truth” in the form of standard tests. We just want to compare Theranos blood test results against them. This guy did it already:
On June 29th I went to the Hematology lab at Stanford for routine CBC and Metabolites numbers. As I walked back to Palo Alto, I stopped by my doctor’s office, got an order, went to the Theranos office at Walgreens on University Avenue in Palo Alto and got a CBC test.
Taken one hour apart, the Stanford and Theranos HCT numbers differ by about 7%: 44.1 Theranos vs. Stanford 41.1. For platelets, the difference is even wider: Theranos 430 vs. Stanford 320
Intrigued, I got a new order and went back to Theranos the following day, on June 30th. Theranos numbers were markedly different 24 hours later: HCT 40.6; PLT 375
Just to make sure, I went back to Stanford for a second test today July 1st: Stanford HCT 41.7; PLT 297
I find the price and convenience of Theranos services attractive, but I worry about the reliability of the important HCT number. What is the confidence interval in your measurement? + or – 1 point? + or – 5 points? I do get a phlebotomy at 45. How should I look at your June 29th 44.1 HCT number?
I’m curious to hear more about your methodology, standards and quality controls and would like to give you an opportunity to respond before I write a Monday Note on the broader topic of lab exams and other healthcare mysteries.
More people should do this, preferably on the same day! Within an hour of each other, too, if possible. It’s in the public interest. We just need to set up an app or something to let people upload their results with some kind of verification method so we know it’s not spammy.
Or else we just ignore Theranos entirely, because it’s gotta be a fraud given the way they’re acting. Here’s a convincing argument from the comment section of the above first person account, someone who calls themselves Skeptical Owl:
You are the CEO of a company that has been working on a revolutionary, disruptive technology for a decade or so. This technology is so amazing that, based on price and customer experience, you can capture most of the (very large) existing market as soon as you enter it. Armed with all of these advantages, you choose to avoid allowing scientists or regulators to validate your technology, enter the marketplace through a single partner (Walgreens) at a glacial pace, and conduct most of your business using existing technologies that are not your revolutionary product. Are you choosing this strategy because your technology doesn’t actually work, because you are incompetent, or because you hate capitalism? Bonus question: if the technology doesn’t work, why is your board a Who’s Who of the military-industrial complex instead of a group of scientists who can help?
Update: I just received this email from a Theranos PR firm:
We read your coverage of Theranos with interest, and wanted to share with you that – because there has been a lot of inaccurate information in the media to date – we have posted detailed information on our technology, finger-stick test, accuracy, and conversations with The Wall Street Journal on our website: https://www.theranos.com/news/posts/custom/theranos-facts
We hope you will take the time to review the information we have posted online, and look forward to engaging with you in the future.
Peyton Burgess, on behalf of Theranos