Home > finance > Quants, Models, and the Blame Game

Quants, Models, and the Blame Game

June 22, 2012

This is crossposted from Naked Capitalism.

Recently a paper came out written by Donald MacKensie and Taylor Spears. It’s about the role of the Gaussian Copula model in the credit crisis, and it’s partly in reaction to Felix Salmon’s article in Wired from February 2009. Both Felix Salmon and Lisa Pollack have written responses to this paper, and they’re quite entertaining and worth a read.

Without going into too many details about the underlying models, which I might do in another post, I wanted to spend some time appreciating this paper for bringing up two issues that I believe far too few people give notice to:

  1. The politics of being a quant. The pressures on a quant inside an investment bank, a ratings agency, on a trading desk, or for that matter in a risk group are real and need to be understood.
  2. The narrative of blame. Who gets blamed when a model fails? For that matter, who is responsible for making sure it works at all?

Politics

In the paper, they discuss the concept of a “model dope,” which is a rhetorical device helping you imagine an idiot who ‘unthinkingly believes in the output of the model’. The paper explains that, as far as they could tell, there were no such actual people, that the quants they interviewed all knew the model was and is flawed and overly simplistic.

I completely believe this, and I think it wouldn’t surprise any quant who’s worked in the industry. Quants are the guys who get metaphorically paraded out in front of the bank, with their Ph.D. hanging out as a kind of badge, but when they get back to work are put back in the mines. It’s a trader’s world, or a salesman’s world, and nobody asks the quants for their nuanced opinion on the validity of basing billions of dollars in transactions on these models if the P&L looks good.

Let me say it this way: how many places employing quants to create risk or hedging models have their quants actually in charge of stuff? Very, very few is the answer. The quants are not in charge, they rarely have real power, and as soon as they produce something semi-functional and useful, they no longer own that thing – it’s been taken away from them and is owned by the real power brokers.

Which is not to say the guys in power don’t kind of understand the stuff- they do, they’re smart, but they’re not typically wedded to the idea of intellectual integrity. They typically understand it well enough to see how it can be gamed.

So I don’t think it was the quants that were promoting the wide use of the Gaussian Copula model. In the paper, they explain that it happened for essentially political reasons:

First, it was easy to talk about, since an entire correlation matrix of default was boiled down to one number, “base correlation”:

“If traders in one bank … had to ‘talk using a model’ to traders in a different bank that used a different copula, the Gaussian copula was the most convenient Esperanto: the common denominator that made communication easy.”

 Next, it allowed traders to book P&L on the same day they made a trade:

“The most important role of a correlation model, another quant told the first author in January 2007, is as ‘a device for being able to book P&L’,”

Next, once it was widely used, it had staying power just because it was difficult to explain something else, not to mention difficult to admit the current model’s flaws:

“Here, the fact that the Gaussian copula base correlation model was a market standard provided a considerable incentive to keep using it, because it avoided having to persuade accountants and auditors within the bank and auditors outside it of the virtues of a different approach.”

Putting that stuff together, we can see that the mere, lowly quant’s objection, if there was one, that the model sucked was the least of the considerations of the powers-that-were:

“From the viewpoint of both communication and remuneration, therefore, the Gaussian copula was hard to discard.”

Blame

As to the question of blame, that’s also all about power. Just because the objections of quants were likely ignored doesn’t mean we can’t blame them after the fact – that’s another useful thing about quants, since they even admit the models were overly simplistic. Easy fall guys.

By the way, I’m not saying that quants rebelled against the misuse of their models, that they tried their best to warn the public of the known flaws of the Gaussian Copula or any other model for that matter. In fact I don’t know of many quants who did stand up to these assholes, partly because they were paid really well not to, and partly because they were not the alpha males in the place.

I’m just trying to point out that blame can get kind of murky. If a quant comes up with a model and says up front, hey this is just a sketch of something, it’s not totally realistic, but it’s better than nothing, and then the investment bank ignored the quant’s misgivings and bets the house on the model, who is responsible for the resulting risk?

In other words, I’d love the quants to grow some balls, but it’s going to take a major revolution in the power structure for that to be enough.

A Question

The two issues of politics and blame raise for me a larger question in reference to modeling. Namely, why and how to models develop?

[This is a cultural question, and separate from the standard (and interesting) questions you usually hear people ask of a model:

  1. What does the model claims to do?
  2. How well it works with real data?, and
  3. If it is widely employed, how the model affects the market itself?]
Here are some examples of why I think models are built.
  • To simplify a businessman’s day. Instead of reading out results from 5 trading desks, we want to dumb it down to one single number, so we employ a modeler to come in and do their best to summarize with one P&L number and one risk number. In other words, it’s the modeler’s job to turn a report into a sound bite. Of course the problem with that model genesis is that it doesn’t necessarily make sense to combine a bunch of numbers into one number. Sometimes the world is actually complex and needs to be understood with a nuanced view. Sometimes a sound bite isn’t enough.
  • To sound incredibly smart – in other words, pure spin doctoring. I encounter this more in tech than in finance, where there are enough model-savvy people that you can’t be quite as blithe about hiding bullshit in a model. But this is real in finance too, and I think is used to confuse regulators all the time.
  • To dissect, or attempt to dissect, various kinds of ‘unintentional risk’ from ‘intentional positions’. This is the single most dangerous kind of model, because on paper it can look so good, and can seem to work for so long. Credit default swaps can be thought of as manifestation of this goal – an attempt to separate default risk from holding-a-bond risk. The problem we face is that our models are never really that good, or even testable, and there are unintended consequences of these new-fangled contracts that sometimes cause catastrophic events.
  • Of course, in quant shops like D.E. Shaw or RenTech or Citadel, there are also quants who try to predict the market or trade superfast on currencies, which is different from the stuff I’ve been talking about which mostly deals with hedging and risk, with different kinds of corresponding risks.
Categories: finance
  1. Deane
    June 22, 2012 at 10:50 am

    I’ve been arguing for some time now for the need of some kind of “professional code of conduct” for mathematicians, analogous to the Hippocratic Oath. It won’t cure the problem, but if an individual quant can stand behind the ethical and moral standards of his or her profession, as defined (and constantly clarified) by the appropriate professional society (American Mathematical Society?), it makes it easier for that person (if he or she wants) to refuse to do something that that person knows is wrong. The Hippocratic Oath does not stop incompetent or dishonest doctors but it *does* make it easier for the honest ones to stand up for their principles.

    • June 22, 2012 at 12:20 pm

      I do not believe the American Mathematical Society is the professional organization most closely related to quants. In fact the American Mathematical Society does already have a committee on the profession which considers issues of professional ethics and which periodically produces reports. Moreover, the Notices of the American Mathematical Society quite often has articles about ethical issues / professional issues facing mathematicians. I do not believe it is appropriate to take aim at the mathematics profession because of problems in quantitative finance any more than it is appropriate to take aim at mathematicians when your calculator stops working.

    • Tom Adshead
      June 26, 2012 at 2:29 am

      Paul Wilmott and Emmanuel Derman have done this:
      http://www.wilmott.com/blogs/eman/index.cfm/2009/1/8/The-Financial-Modelers-Manifesto

      As the article says, the problem really is that quant results tend to be wilfully selectively used by those with political power – I remember a head of sales at one employer asking me to put less data in the presentation that I wrote for him because “it constrains my freedom to make up numbers”

      • July 1, 2012 at 11:48 am

        I was once told that my recommendations tended to be “one sided”. There was no suggestion that I failed to present all relevant information or weigh the major pros and cons of all options to be considered. The criticism was that the recommendations themselves were one sided.

        Translated into tranparent (as opposed to opaque) surrealism, thi reads as: “I want to do something stupid and/or scummy, but don’t want to own it. So I want you to recommend that stupid/scummy thing, so that, if things go awry, I can blame you, even though only I had the relevant decision-making authority. I’d also like you to leave out selected background information, so I can honestly tell people you failed to tell me about this stuff. Should said stupid/scummy thing appear to work out well, however, I reserve the right to take all credit and assail your competence even as I keep you on the payroll and run desperately to you the next time there’s a problem to solve.”

        Quick poll: How many of you, when paying for “expert” recommendations value recommendations that are not one sided. The day I decide I want my career to end before lunchtime, I will turn in a report that includes the following.

        RECOMMENDATION: I could go either way on this, and so could you probably. I don’t know. It’s really tough to say, and, you know…Just because I’ve provided you more than enough background information on the basis of which you can accept or reject my recommendation, I wouldn’t want to bias you in favor of one course of action or other.

  2. June 22, 2012 at 12:21 pm

    May I try to make this simple, as Einstein said ,”Make it simple.”
    If “A” is false, anything (“X”) that is attached to “A” must be a false conclution.
    Any model (conclusion ) that attempts to predict a future event should have a disclosure
    attached that it is presented by a “fool”, albeit maybe a famous stock guru, Noble prize winning economist, or any other human (except those used for Divine Intervention)
    Perhaps this is why I love my tag “Justaluckyfool”-(Definition) “Anyone that attempts to predict a future event is a fool, if by chance correctly then they are just a lucky fool,
    albeit still a fool.” (A remembered quote that I can not find its source)
    “All solutions for future events must contain means for change”
    Our forefathers allowed for that and and established that we the people govern this nation…
    “in order to form a more perfect union.”
    Please read ,challenge, and improve,…

    http://bit.ly/MlQWNs

    then after editing, improving, help make the world a better place, even if for a short time.

    • July 1, 2012 at 11:53 am

      Correct. Look both ways before crossing the street, so as to predict, within a reasonable margin of certainty, the likelihood of avoiding being hit by a vehicle. Do not, however, have so much confidence in what you saw when you looked both ways, that you feel comfortable wearing a blindfold while crossing…I think I just figured out why vehicle traffic is prohibited on so many of the streets of lower Manhattan.

  3. June 22, 2012 at 12:27 pm

    P.S. Some good reading, believe to be on point.

    The Smartest Guys in the Room | The Daily Capitalistdailycapitalist.com/2008/11/17/the-smartest-guys-in-the-room/

  4. William Dunn
    June 22, 2012 at 6:08 pm

    re justaluckyfool 6/22/1221:

    Too broad a generalization…

    We infer the future from the past all the time; I couldn’t drive to work, catch a baseball, or walk without some kind of future-predicting capability. Alas, I don’t know how to write an abstract model specification of those situations so I can’t easily replicate that common-sense predictive capability for situations involving less well-defined ordering principles than those governed by the laws of physics.

    All statements about reality are approximations, ie, they are “false.” And yet, somehow, we seem to generate sorta-true (read not-false) conclusions. Another one of the mysteries of life: how do we DO that? Conclusions derived from these reality-based samples can be true; Galileo did a pretty good job of inferring a (predictive) model of gravity from rolling balls and crude timing devices. All his observations were “false” — yet the conclusions were “true.”

    Your aphorisms are catchy, but both reality and model-building are more complicated than you suggest.

    • June 22, 2012 at 8:42 pm

      Thank you for your challenge and as I had stated, “to improve”.
      OK, yes “WE infer..” but what is inferred should state that “past performance does not guarantee future results” That a child who learns after a few steps to walk only has a probability of future success. To state that person will walk is wrong because “unintented consequences” (which I do not wish to memtion) could change that childs future result.
      As for a financial model for value or price it would simply be “All future prices or values must be either higher, lower or the same.” Any one that says differently is a fool, and if you believe them hope that they are “just a lucky fool”
      Put this aside and ,please, improve, “Private banks making loans bearing compound interest at any rate for a long period of time (Rule 72) unimpeded would evenually
      own all thw worlds assets and any possible future assets.”
      Please read “justaluckyfool” and Michael Hudson’s work “Compound interest”

      Please, Mathbabe, I will in the future try not to reply to any comments made in my reply.
      Hopefully anyone that wishes to challenge or improve would comment on my site.
      Thank you for your understanding.
      We can only hope “to form a more perfect union” and I wish you success in your efforts, keep writing.

  5. DC
    June 22, 2012 at 8:24 pm

    If I may humbly submit a suggestion to the alpha-(fe)male quant wannabes: quants calling the other people “assholes” is not going to go very far in getting the respect you suggest they should have.

    • macy
      June 25, 2012 at 5:08 pm

      maybe it will , maybe it will

      • DC
        June 25, 2012 at 8:11 pm

        asshole

      • DC
        June 25, 2012 at 8:12 pm

        so, how did that work out, do you have more respect for me now?

        • FogOfWar
          July 2, 2012 at 9:42 pm

          You’ve never worked on a trading floor, have you?

          Just sayin’…

        • July 2, 2012 at 9:55 pm

          no but I’ve been an account manager, which makes me an official gimp.

  6. HawkeyeTheNoo
    June 25, 2012 at 7:04 am

    The political context of quant roles in a organisation is spot-on from my 20+ years in investment banking, some of it spent managing quants.
    Quants as a breed (and they are different: some of them have indeed been ‘special’) tend to regard themselves as above the fray and common herd and wear their (largely) self-imposed isolation as a badge of pride, so they are themselves contributors to the problem.

  7. mark
    June 25, 2012 at 11:34 am

    I think the same may be said of macroeconomic models. Good post.

  8. July 1, 2012 at 11:25 am

    Quants are SME’s. I don’t know quants, but I know SME’s, having been one of these creatures whose acronym, when spoken, sounds like the name of an absurd creature in a Dr. Seuss book…Very appropriate…General rule is the more muzzled they are and the closer they are kept to the current Masters of the Universe (my office was in an exec’s bottom desk drawer), the smarter they are (“smart” here meaning “needed but scary”. I checked, and, like all SME’s, I actually am made of plutonium). SME’s who are the spokespeople for the worst ideas are, in all likelihood, those most opposed to them and most in favor of the best and/or apparently most immediately profitable ideas floating around, the ones they created and for which the exec takes credit (bad choice of word). By the way, “profit” or gain takes many forms, so this goes on in NPO’s and government quite a bit. A major use for SME’s is that, if they get assertive in pointing out the shortcomings of models, their warnings are ignored and they are branded as “difficult”, “disgruntled” and, if necessary to avoid the acknowledgement of reality, “mentally unstable”. This, in turn, calls into question the level of basic social adjustment and mental stability of anyone questioning said model…I wonder if there’s a way to test my theory that one can estimate when the shit will hit the fan by watching behavior of and/or toward SME’s. If you have any stake in any entity and notice that previously placated and docile SME’s are suddenly sending out resumes, becoming competent at networking, quitting outright, being badmouthed in vague or specific terms, becoming notably more or less vocal than in the past, becoming “difficult”….my advise is to immediately sell, run for cover, and lawyer up.

  9. July 1, 2012 at 11:34 am

    For the record, I wish to add to my previous comment that at least one previous employer is willing to state that I am difficult, disgruntled, poorly socially adjusted, and mentally unstable–but only in relation to a handful of hot-button technical topics. I have been busying myself, of late, by sending out resumes and becoming competent at networking.

  10. Bobito
    July 4, 2012 at 4:28 am

    About blame: those who go work as quants in the financial industry are responsible for what that industry does in the same way that those who work designing propulsion mechanisms for arms manufacturers are responsible for what that industry does. Someone smart enough and ambitious enough to land this sort of job cannot pretend to have no idea what the institution for which he or she works does, and what its goals and ways of achieving them are, and shares in the responsibility for them.

    I’d have had an easier life and a lot more money if I had chosen, like a lot of my friends, to go work in finance. I chose against it precisely because I don’t like what these institutions do. They chose for it because for them the challenges and material rewards offered outweighed such considerations. It’s reasonable to make such a decision, but when one does, one assumes responsibility for what the institution for which one works does. Many of those who make such a choice have no problem with this – but some seem not to be honest with themselves about what they are doing and its societal effects.

    Working for Goldman, the NSA, or your favorite arms manufacturer may offer interesting problems, good pay, and a sense of doing something for the society – but it may also be socially compromised in fundamental ways – either way it should be done with the eyes open, and too many do it with the eyes shut.

  11. Emmett
    July 22, 2012 at 9:21 pm

    Much of what has been said here applies elsewhere, read Tufte on institutional failure and dumbing down to meet management goals on the Challenger program here:
    http://www.aaronsw.com/weblog/000931
    http://www.onlineethics.org/CMS/profpractice/exempindex/RB-intro/RepMisrep.aspx
    In both cases, management who had climbed the ladder from engineer (or quant) to a position of authority refused to look at data or acknowledge they even knew how anymore.

    And I no more hold quants responsible for their management’s decisions than the man in the moon.

    Wall Street has grossly failed before and after the invention of the quant. With some regularity.

    There are any number of regulators I’d like to pillory though. Actually a good time to bring back keel-hauling, but I said that back when it was Savings and Loans.

  1. June 24, 2012 at 10:10 am
  2. June 25, 2012 at 3:09 am
  3. June 25, 2012 at 9:35 am
  4. June 25, 2012 at 8:16 pm
  5. June 30, 2012 at 4:02 am
  6. September 22, 2012 at 7:33 pm
Comments are closed.
Follow

Get every new post delivered to your Inbox.

Join 976 other followers

%d bloggers like this: