Home > data science, math education, modeling, statistics > I totally trust experts, actually

I totally trust experts, actually

December 31, 2012

I lied yesterday, as a friend at my Occupy meeting pointed out to me last night.

I made it seem like I look into every model before trusting it, and of course that’s not true. I eat food grown and prepared by other people daily. I go on airplanes and buses all the time, trusting that they will work and that they will be driven safely. I still have my money in a bank, and I also hire an accountant and sign my tax forms without reading them. So I’m a hypocrite, big-time.

There’s another thing I should clear up: I’m not claiming I understand everything about climate research just because I talked to an expert for 2 or 3 hours. I am certainly not an expert, nor am I planning to become one. Even so, I did learn a lot, and the research I undertook was incredibly useful to me.

So, for example, my father is a climate change denier, and I have heard him give a list of scientific facts to argue against climate change. I asked my expert to counter-argue these points, and he did so. I also asked him to explain the underlying model at a high level, which he did.

My conclusion wasn’t that I’ve looked carefully into the model and it’s right, because that’s not possible in such a short time. My conclusion was that this guy is trustworthy and uses logical argument, which he’s happy to share with interested people, and moreover he manages to defend against deniers without being intellectually defensive. In the end, I’m trusting him, an expert.

On the other hand, if I met another person with a totally different conclusion, who also impressed me as intellectually honest and curious, then I’d definitely listen to that guy too, and I’d be willing to change my mind.

So I do imbue models and theories with a limited amount of trust depending on how much sense they makes to me. I think that’s reasonable, and it’s in line with my advocacy of scientific interpreters. Obviously not all scientific interpreters would be telling the same story, but that’s not important – in fact it’s vital that they don’t, because it is a privilege to be allowed to listen to the different sides and be engaged in the debate.

If I sat down with an expert for a whole day, like my friend Jordan suggests, to determine if they were “right” on an issue where there’s argument among experts, then I’d fail, but even understanding what they were arguing about would be worthwhile and educational.

Let me say this another way: experts argue about what they don’t agree on, of course, since it would be silly for them to talk about what they do agree on. But it’s their commonality that we, the laypeople, are missing. And that commonality is often so well understood that we could understand it rather quickly if it was willingly explained to us. That would be a huge step.

So I wasn’t lying after all, if I am allowed to define the “it” that I did get at in the two hours with an expert. When I say I understood it, I didn’t mean everything, I meant a much larger chunk of the approach and method than I’d had before, and enough to evoke (limited) trust.

Something I haven’t addressed, which I need to think about more (please help!), is the question of what subjects require active skepticism. On of my commenters, Paul Stevens, brought this up:

… For me, lay people means John Q Public – public opinion because public opinion can shape policy. In practice, this only matters for a select few issues, such as climate change or science education. There is no impact to a lay person not understanding / believing in the Higgs particle for example.

  1. December 31, 2012 at 8:36 am

    The answer is: every idea requires at least a skeptic or two to analyze the idea deeply, but not everyone needs to be skeptical of every idea.

    Critique of ideas should be much more common in our society than it is today, but we also need to be comfortable listening to the critiques and advice of others who have spent more time thinking about a particular idea than we have.

    I find it frustrating, for example, that everyone considers themselves an expert on learning and what schools should do or not do, and virtually no ones gives educators credit for the enormous amount of thinking about learning that we do. I’d like a little less critique of our perspective on learning, thank you, but at the same time, I’d like a whole lot more thinking and critique about the structures of schools themselves since it seems that everyone (I’m exaggerating) accepts themselves as an expert on learning without thinking too deeply about the structure of the institutions where this learning is supposed to take place.

    • Mark Gallagher
      December 31, 2012 at 8:40 pm

      Hmmmm. So it’s 1795, and while the world’s experts claim the earth is thousands of years old and catastrophically formed, William Smith is a simple surveyor who claims there’s a fossil record evincing a much slower, gradual process. Now, is William Smith an “expert”, or just a casual observer?

      • December 31, 2012 at 11:41 pm

        Isn’t William Smith playing the role of the Skeptic in this case, who’s job it is to raise questions about the experts? Without the Skeptic, we get stuck in our ideas, but if everyone is skeptical, then no progress can be made because no one can agree with each other.

    • will
      January 18, 2013 at 2:44 pm

      I think you’re on the right track in focusing on the structure of schools which is entirely different from the corpus of knowledge in both book/street smarts that educators possess on the “theory of” or “best practices of” Teaching a student/class.

      To make an analogy in the flavor of the blog’s theme: The public isn’t unimpressed with the amount of work that goes into quantitative risk models, but with the incentives that drive the behavior of the financial institutions as a whole.

      In education, I think the central critique is public K-12 ‘s willingness to make itself insulated from the demands of its stakeholders – students and families. And here, parents should indeed be considered experts ,on par with the teacher, in the education of their child.

      • January 18, 2013 at 6:03 pm

        Except that parents quite often make choices about the education of their children which are neither in the best interest of their children or of larger society as a whole.

        For example (true story, btw), If I choose to ensure that my child cannot get enough sleep to be successful at school by partying hard every night, loudly, in the living room where they are supposed to sleep, then I am certainly not making a choice in the best interest of my child (or greater society at large), but in the current model, this is considered the prerogative of the parent.

        I’m find and dandy with parents making all sorts of rational decisions about the welfare of their children, it is the irrational decisions that worry me.

  2. December 31, 2012 at 11:14 am

    As some former public officials have been known to say in their more undemented moments, “Trust, but verify.” Our entire set of socioeconomic systems depends on both ends of such a dictum – whether its the delicate balancing of fractional-reserve banking, self-compliant taxation, safety of food-like substances, sanity of gun possession, safety of easily self-administered psychoactive chemicals, or a blizzard of innovative, opportunistic financial instruments.

    These systems must remain manageable as trusted public utilities, but forever under authority-questioning skepticism of legions of non-experts contributing their germs of truth resulting in objective, practical, and effective regulation.

  3. December 31, 2012 at 1:00 pm

    Totally agree with Lou’s “Trust, but verify.” But who has time to verify everything!

    That said, way too many vulnerable people are signing tax returns without a clue as to what is in them. A responsible tax professional’s responsibilities ought to include helping clients read through their returns. (Not every word of boilerplate text, obviously, but highlighting and summarizing the key issues on it.)

    After all the taxpayer is signing a statement that says “Under penalty of perjury, I declare that I have examined this return and to the best of my knowledge and belief, it is true, correct, and complete.” But many people take this about as seriously as the garden variety “I have read agreed to the TOS” to some random Internet service provider.

    Cathy’s statement about signing tax returns without reading them inspired a post on my blog for IRS-certified volunteer tax preparers (including my students who are preparing for an IRS certification exam I’ll give next Monday.)

    http://bedbuffalos.blogspot.com/2012/12/under-penalty-of-perjury.html

  4. December 31, 2012 at 6:37 pm

    An interesting discussion of “warranted belief” in its quantitative form, and I am trying to situate it in the age of identity politics. What sort of identity supports the project of trusting people with common interests because their models are plausible? Whatever sort it is that draws us into this discussion. And what sort of identity distrusts people precisely because their trust follows models rather than ordinary human loyalties? That may be a fair description of coal country Romney voters who fear the power of those who can tax carbon and hate those who would do so with more thought for species in the Amazon than for struggling towns of miners in their own mountains. Ever since the upset election victory by Silver, Wang, et al so upset those who voted for Rove, Morris, et al we have noted this divide, and often gloated over it. We have not yet understood it; I think this discussion is useful.

  5. lemon bun
    January 1, 2013 at 5:51 pm

    I like to criticize TED, but I’m an avid comsumer or that kind of shows.

  6. January 2, 2013 at 7:44 am

    As usual, I’ll just link to previous posts at Gelman’s blog about the same subject:

    http://andrewgelman.com/2009/12/how_do_i_form_m/

    http://andrewgelman.com/2012/05/selection-bias/

  7. January 2, 2013 at 8:26 am

    global climate change denying father – just tell him about Krakatoa and how it cooled the earth and snowed Europe during the summer time – a freak storm that inspired Mary Shelly to write the first climate change novel call Frankenstein – that is a sort of climate change – when man can create a zombie – Global Cooling is a reality therefore Global Warming is a reality.

  8. DayHay
    January 10, 2013 at 1:48 pm

    Global climate vs. global warming? Your brush is very broad.
    Pick the graphs on the web that you agree with on temp over time and C02.
    Post up and then let’s see what the data says. Whose graph is correct, etc.
    Maybe talk about climate models, their output, their error bars.
    Then we can have a discussion on whether we need a $20/ton C02 tax as Obama has already floated. I deny we need to spend (waste) trillions worldwide on C02 abatement, I do not deny that temperatures fluctuate here on earth.

  9. January 11, 2013 at 7:28 pm

    I have learned to evaluate models on the basis of 3 criteria:
    1. Is it descriptive? Almost any model passes this criteria. Indeed, even critically flawed ones like “Thunder is caused by Angry God(s)”
    2. Is it predictive? A higher test to pass, When multiple models exist which can predict equally, choose that which is simpler. (Borrowing from Occam’s Razor)
    3. Is it prescriptive? Can I influence and then predict that influence over a range of outcomes. A yet higher test to pass.

    Generally I never say I understand something until I have written a computer program that implements a model and have tested it to my own satisfaction. This technique has successful dispelled arguments in the past. Arguments which I have sometimes conceded because my model has failed *criteria 3″.

  10. March 27, 2013 at 10:56 am

    I agree that skepticism is a good thing to have. For the average person who isn’t an expert in a specific field, it’s hard to know who is telling the whole story. Statistics and graphs can be manipulated and also are politically influenced – esp w the climate change debate. It’s hard to know who is giving a complete truth. How many times have you read articles about how a food is good for you, a year later it’s bad, then it’s good again?

  1. January 2, 2013 at 8:40 am
Comments are closed.
Follow

Get every new post delivered to your Inbox.

Join 1,728 other followers

%d bloggers like this: