Home > finance > Emanuel Derman’s Models.Behaving.Badly.

Emanuel Derman’s Models.Behaving.Badly.

October 25, 2011

This morning I want to talk about Emanuel Derman’s beautifully written and wise new book “Models. Behaving. Badly.”, available in some book stores now and on Amazon starting tomorrow.

It is in some sense an expanded version of this essay he wrote in January 2009 with Paul Wilmott. I particularly like the end of the essay where they present the “Modeler’s Hippocratic Oath”:

  • I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
  • Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
  • I will never sacrifice reality for elegance without explaining why I have done so.
  • Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
  • I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

This was written in direct reaction to the financial crisis of 2008, clearly, but I think data scientists should all be asked to sign at the bottom of the page. As I’ve said before, financial modelers are just data scientists working in the most sophisticated (under some metrics) subfield of data science; just as they have ridiculous powers of profit using their methods, data scientists in other fields have ridiculous powers as well, sometimes in ways that affect people even more directly than money. Modelers absolutely need to be aware of and wary of these powers. This oath is an excellent step towards that.

In the book, Derman sets up a dichotomy between models and theories. For him, theories are stand-alone descriptions of how things are, whereas models are relative descriptions of how things work, by analogy. He also differentiates between the models (and theories) and the way that humans ascribe truth to them, which is to me the most profound and important message of his book. I’ll discuss below.

His examples of theories come mostly from physics: he has a really beautiful explanation of the evolution of the theory of electro-magnetism, for example, which actually explains how people can sometimes develop theories using temporary models. One idea that emerges is that, sometimes, models work out so well that they eventually become part of the theory. The obvious love that Derman has for physics (which he trained in as a young man) shines through this entire part of the book, and it’s beautiful and intimate reading.

Another example Derman gives of a theory was Spinoza’s theory of human emotion, wherein the basic objects were pleasure and pain, and everything was a derivative of those. For example, love is defined as “pleasure associated with an external object,” pity as “pain arising from another’s hurt.” My favorite: to Spinoza, cruelty is “the desire to inflict pain on someone you love.”

To Derman, Spinoza’s theory is a theory, even though it’s not mathematical, and even though it may not be even “true” in the sense that you could just as well have an alternative theory (although it may not be as beautiful). It is a theory, then, because it describes a mini universe of existence without depending on an assumed external frame of reference. It describes emotions themselves rather than comparing them to something else.

What then is a model? It is something that tried to explain (and predict) the behavior of something through analogy or proxy; its accuracy depends on external conditions. He talks about the Black-Scholes option pricing model as a good model in that, in its purest form, it is actually a model for the price of something depending on the abstract concept of “risk”. Then the fact that it can be misused is due more to the fact that people incorrectly proxy risk itself as described by some brownian motion somewhere. With a better model of risk, then, we’d be happy to use Black-Scholes.

Of course Black-Scholes, or rather its use, is not what caused the financial crisis. It was rather the Efficient Market Model and its corollary the Capital Asset Pricing Model that he considers much more dangerous (I agree). He describes these models very clearly, for the uninitiated, and talk about their ubiquitous use in the vocabulary of finance (and how money mangers describe their Sharpe ratios, which is a ratio of their (past) return and their perceived risk, as if they are meaningful statistics).

The basic mistake people make in ascribing power to their financial models, he says, is that they depend on human beings and their actions to be as predictable as physics. Electrons, it turns out, are much more predictable than people with money on the line.

He also goes into a beautiful riff on how there is, but shouldn’t be, a “Fundamental Theorem of Finance.” Not only is it not fundamental (and not even understandable), but there shouldn’t even be such a thing, because it depends on a model which isn’t true, and cannot deserve the name “theorem”, and moreover can only serve as a false sense of security. This kind of mathematical idolatry, he believes, is at the very root of the problem which led us into the financial crisis. Agreed.

Another aspect of the book I want to bring up, because I find it fascinating, is the way we model things in our lives. As Derman correctly points out, we each model our futures; he describes growing up in South Africa during Apartheid and his involvement with a youth Zionist movement, and how he felt pressure to model his life in a very precise way from the youth leaders there, to move to Israel and work as a laborer (examples among many of how sometimes people blithely model other peoples’ lives as well as their own). He then went on to talk about studying physics, moving to finance and his early desire to find a “theory of everything” in finance.

It may be reasonable to say that, until we die, we are on an endless quest for the perfect model for ourselves. In other words, it’s not only that we use models for our internal lives, but we intensely desire models as well – they give us pleasure, they alleviate our worries and stress. We have trouble letting go of our internal models, even when they don’t work (or we even ignore their failure completely; this was described in a recent New York Times article on confidence). When we model the person we love loving us back, it gives us enormous strength and hope. We model our future success: getting tenure, a promotion, or a child. We model our gods.

But our models are not always realistic, and they don’t always account for bad conditions; just look at the Eurozone for one huge example of this. Sometimes our beloved doesn’t care, and sometimes we don’t get tenure.

The most essential question for me then is: how do we react in that moment when we realize our model is bad? Otherwise put, how do we disagree with ourselves? It’s an excruciating moment that we can learn from – do we take away from a moment like that only the pain? Or do we grab hold of it as an opportunity for growth? Can I train myself to be the kind of person who learns from her broken models?

A related question, which I will take up in another post, is whether the people in finance (or mathematics, or data science) are people who react well to their models failing.

Categories: finance
  1. Johann Hibschman
    October 25, 2011 at 9:06 am

    Thanks for the review, that sounds like an interesting book. How graphics-heavy is it? I don’t want to add another physical book to my already-strained bookshelf, but Kindle books really only deal well with pure flowing text.

    It’s a hard thing. I was modeling MBS through the 2008 crisis (prepayments, though, not defaults), and it was a strange thing to watch from the inside. My stuff did okay, although there was a long period in there where my best answer was to say that we’re in a tight-credit environment unlike any in the recent data, so the best we could do is to tweak things to fit the present and impose our best guess as to when things’ll change. I don’t think anyone had a terribly good answer. (Deane Y and Andy Kalotay at least had something qualitatively different.)

    There is a similar dynamic in physics. I did astrophysics theory, and there you could see a lot of people abstract out models into something tractable, but that no longer really got the dynamics they were looking for. Often, the crucial moments were the entry and exit from the tractable steady-state. I can see the appeal of mathematical physics. It’s wonderfully fun to play with symplectic manifolds and their friends, but it’s very hard to trim a real physical situation into cases where the math applies. Is an assumption of sphericity a good approximation, or are the asymmetries the entire source of the effect?

    So much of this boils down to having good taste in modeling, having the intuition to trim out the irrelevant, and then finding ways to test your assumptions, or at least to get a feel for the magnitude and nature of the effects you see if you relax some of the simplifications.


  2. Michael Yates
    October 25, 2011 at 9:16 am

    re: financial modelers being the most sophisticated. When Cray began making supercomputers, there were three customers: nuclear bomb modelers, weather modelers, and oil company seismic data crunchers. I don’t recall any financial modelers.


  3. Mekon
    October 25, 2011 at 9:52 am

    I haven’t seen the book yet, but the two-page manifesto is excellent. If you take away anything from it, take away this (which echoes George Box):

    “The most important question about any financial model is how wrong it is likely to be, and how useful it is despite its assumptions.”


  4. October 25, 2011 at 10:38 am

    The “Modeler’s Hippocratic Oath” is already taped up on my office door and the book is in my iPad Kindle app. Even if I disagree with some of the points..haven’t read the book yet, this is a really important message for data researchers. It is too easy to get lost in our data (because that’s fun) and forget the bigger picture. Thanks!


  1. October 27, 2011 at 9:16 am
  2. November 14, 2011 at 11:37 am
  3. January 23, 2012 at 7:01 am
Comments are closed.
%d bloggers like this: