Home > data science, modeling, rant > The complexity feedback loop of modeling

The complexity feedback loop of modeling

January 8, 2013

Yesterday I was interviewed by a tech journalist about the concept of feedback loops in consumer-facing modeling. We ended up talking for a while about the death spiral of modeling, a term I coined for the tendency of certain public-facing models, like credit scoring models, to have such strong effects on people that they arguable create the future rather than forecast it. Of course this is generally presented from the perspective of the winners of this effect, but I care more about who is being forecast to fail.

Another feedback loop that we talked about was one that consumers have basically inheriting from the financial system, namely the “complexity feedback loop”.

In the example she and I discussed, which had to do with consumer-facing financial planning software, the complexity feedback loop refers to the fact that we are urged, as consumers, to keep track of our finances one way or another, including our cash flows, which leads to us worrying that we won’t be able to meet our obligations, which leads to us getting convinced we need to buy some kind of insurance (like overdraft insurance), which in turn has a bunch of complicated conditions on it.

The end result is increased complexity along with an increasing need for a complicated model to keep track of finances – in other words, a feedback loop.

Of course this sounds a lot like what happened in finance, where derivatives were invented to help disperse unwanted risk, but in turn complicated the portfolios so much that nobody understand them anymore, so we have endless discussions about how to measure the risk of the instruments that were created to remove risk.

The complexity feedback loop is generalizable outside of the realm of money as well.

In general models take certain things into account and ignore others, by their nature; models are simplified versions of the world, especially when they involve human behavior. So certain risks, or effects, are sufficiently small that the original model simply doesn’t see them – it may not even collect the data to measure it at all. Sometimes this omission is intentional, sometimes it isn’t.

But once the model is widely used, then the underlying approximation to the world is in some sense assumed, and then the remaining discrepancy is what we need to start modeling: the previously invisible becomes visible, and important. This leads to a second model tacked onto the first, or a modified version of the first. In either case it’s more complicated as it becomes more widely used.

This is not unlike saying that we’ve seen more vegetarian options on menus as restauranteurs realize they are losing out on a subpopulation of diners by ignoring their needs. From this example we can see that the complexity feedback loop can be good or bad, depending on your perspective. I think it’s something we should at least be aware of, as we increasingly interact with and depend on models.

Categories: data science, modeling, rant
  1. Zathras
    January 8, 2013 at 9:32 am

    I think there is a simpler feedback loop here. You start with a simple model. It is easy to understand, gets buy-in and achieves success. Now people think “continuous improvement” and say, let’s make it better, and better results are achieved with a more complicated model. Effectively, the model is promoted. This continues until the model is no longer understood by everyone. It’s the Peter Principle of modeling, every model rises to a level where it is incompetent.


  2. January 8, 2013 at 10:50 am

    In finance, at least, it seems to me that “complexity” is promoted in order to winnow the field of participants with knowledge of the rules or access to technical advantages. High frequency trading is a good example.


  3. Jonathan
    January 8, 2013 at 11:31 am

    The paper you refer to as “endless discussions about how to measure risk” is really much more disturbing and important that that. It says that CDS INCREASE RISK.

    The article says that the authors give two explanations:

    “First, creditors who have hedged their default risk are less interested in helping a company stay afloat during tough times. Second, the availability of default insurance on a company attracts a larger number of lenders, increasing the complexity of reaching the kind of debt restructuring deals that can keep temporarily troubled companies out of bankruptcy.. ”

    As to the first explanation “creditors … are less interested in helping a company stay afloat” should really say “creditors may actively push a company into default because they will profit on the CDS”. They quote another paper that spells this out.

    Empty creditors are even willing to push the firm into bankruptcy if their total payoffs including CDS payments would be larger in that event.

    That is, a fund might well be in a position to profit from a default. If so, they might well try to make that happen. Anyone who doubts that hedge funds and others would not do so hasn’t been paying attention.


    • January 8, 2013 at 11:32 am

      Yes, I want to do a separate post on that issue, it’s super interesting. A different kind of unintentional consequence (we hope) of the quest to “remove risk” but actually screwing things up, complicating stuff, and skewing incentives.


  1. No trackbacks yet.
Comments are closed.
%d bloggers like this: