Overfitting
I’ve been enjoying watching Andrew Ng’s video lectures on machine learning. It requires a login to see the videos, but it’s well worth the nuisance. I’ve caught up to the current lecture (although I haven’t done the homework) and it’s been really interesting to learn about the techniques Professor Ng describes to avoid overfitting models.
In particular, he talks about iterative concepts of overfitting and how to avoid them. I will first describe the methods he uses, then I’ll try to make the case that they are insufficient, especially in the case of a weak signal. By “weak signal” I mean anything you’d come across in finance that would actually make money (technically you could define it to mean that the error has the same variance as the response); almost by definition those signals are not very strong (but maybe were in the 1980’s) or they would represent a ridiculous profit opportunity. This post can be seen as a refinement of my earlier post, “Machine Learners are spoiled for data“, which I now realize should have ended “spoiled for signal”.
First I want to define “overfitting”, because I probably mean something different than most people do when they use that term. For me, this means two things. First, that you have a model that is too complex, usually with too many parameters or the wrong kind of parameters, that has been overly trained to your data but won’t have good forecasting ability with new data. This is the standard concept of overfitting- you are modeling noise instead of signal but you don’t know it. The second concept, which is in my opinion even more dangerous, is partly a psychological one, namely that you trust your model too much. It’s not only psychological though, because it also has a quantitative result, namely that the model sucks at forecasting on new data.
How do you avoid overfitting? First, Professor Ng makes the crucial observation that you can’t possibly think that the model you are training will forecast as well on new data as on the data you have trained on. Thus you need to separate “training data” from “testing data”. So far so good.
Next, Professor Ng makes the remark that, if you then train a bunch of different models on the training data, which depend on the number of variables you use for example, then if you measure each model by looking at its performance on the testing data to decide on that parameter, you can no longer expect the resulting model (with that optimized number of parameters) to actually do so extremely well on actually new data, since you’ve now trained your model to the testing data. For that reason he ends up splitting the data into three parts, namely the training data (60%), a so-called validation data set (20%) and finally the true testing set (the last 20%).
I dig it as an idea, this idea of splitting the data into three parts, although it requires you have enough data to think that testing a model on 20% of your data will give you meaningful performance results, which is already impossible when you work in finance, where you have both weak signal and too little data.
But the real problem is that, after you’ve split your data into three parts, you can’t really feel like the third part, the “true test data”, is anything like clean data. Once you’ve started using your validation set to train your data, you may feel like you’ve donated enough to the church, so to speak, and can go out on a sin bender.
Why? Because now the methods that Professor Ng suggests, for example to see how your model is doing in terms of testing for high bias or high variance (I’ll discuss this more below), looks at how the model performs on the test set. This is just one example of a larger phenomenon: training to the test set. If you’ve looked at the results on the test set at all before fixing your model, then the test set is just another part of your training set.
It’s human nature to do it, and that’s why the test set should be taken to a storage closet and locked up, by someone else, until you’ve finished your modeling. Once you have declared yourself done, and you promise you will no longer tweak the results, you should then find the person, their key, and test your model on the test set. If it doesn’t work you give up and try something else. For real.
In terms of weak signals, this is all the more important because it’s so freaking easy to convince yourself there’s signal when there isn’t, especially if there’s cash money involved. It’s super important to have the “test data set”, otherwise known as the out-of-sample data, be kept completely clean and unviolated. In fact there should even be a stipulated statute of limitations on how often you get to go out of sample on that data for any model at all. In other words, you can’t start a new model on the same data once a month until you find something that works, because then you’re essentially training your space of models to that out-of-sample data – you are learning in your head the data and how it behaves. You can’t help it.
One method that Ng suggests is to draw so-called “learning curves” which plot the loss function of the model on the test set and the validation set as a function of the number of data points under consideration. One huge problem with this for weak signals is that the noise would absolutely overwhelm such a loss estimate, and we’d end up looking at two extremely misleading plots, or information-free plots, the only result of which would be that we’ve seen way too much of the test set for comfort.
It seems to me that the method Ng suggests is the direct result of wanting to make the craft of modeling into an algorithm. While I’m not someone who wants to keep things guild-like and closed, I just don’t think that everything is as easy as an algorithm. Sometimes you just need to get used to not knowing something. You can’t test the fuck out of your model til you optimize on every single thing in site, because you will be overfitting your model, and you will have an unrealistic level of confidence in the result. As we know from experience, this could be very bad, or it could just be a huge waste of everyone’s time.
#Occupy Wall Street meeting and news
First, I wanted to mention that there will be an Alternative Banking meeting on Sunday from 3-5pm at 1401 IAB (14th floor of the International Affairs Building), Columbia University. This is on the corner of Amsterdam and 118th. Please come!
Next I wanted to mention that I’ve been hearing from other occupied cities, namely Occupy Oakland and Occupy LA. I’ve been meeting the most amazing people this way and I’m super grateful for that. I should be talking to Cheryl from Occupy LA later today, and it sound like her posse of occupiers have really been making things happen with something called the “Responsible Banking Measure” recently passed by the LA City Council. Awesome stuff, and I’ll post more as I learn it. My favorite part was the last line of her email, “let’s do this!”
Another inspiring figure is coming from Occupy Oakland, a photographer named Miles Boisen, whose trademark phrase, “struggle and snuggle,” is very close to my heart. He recently met a group of four protesters who were so fascinating that he asked them to tell a bit more about their story. First, here’s their picture:
Next, here’s one of their stories:
Dear Myles,
It was wonderful to meet you at the time and place we did. I am happy to give you some background information. Feel free to use it (or not) any way you like.
I was born in San Francisco in 1944, the year that millions of Jews were being slaughtered by Hitlers minions in Europe. My mother, Hodee Edwards, is Jewish. My father, Harvey Richards, is of Welsh ancestry. I witnessed the 1950s in the streets of Oakland, watching my mother crying over the Rosenbergs execution, and experiencing the racism of our world through the eyes of my African American step father (George Edwards) and my sister (Lou Edwards). I dodged the FBI coming to my house to harrass my family and witnessed doors being slammed in the FBI’s faces as they sought to intimidate dissidents in Oakland. I traveled to the USSR in the summer of 1961, the year I graduated from high school, to help my father and his wife, my stepmother Alice Richards, make two films on ordinary life in the USSR. See this for a clip from that movie.
I entered UC Berkeley in the fall of 1961 and immediately joined the civil rights movement. My first demonstrations were against compulsory ROTC at Berkeley, a struggle that resulted in the cancellation of compulsory ROTC in my sophmore year. I participated in the sit-ins at Mel’s drive-in restaurants (see this for a clip of my father’s film showing those demos and a shot of me on the picket line), at the Sheraton Palace Hotel (see this which includes a shot of me being dragged out of the lobby by the SFPD) and along auto row on Van Ness in San Francisco in 1963 and 1964 against racial discrimination in hiring, then rampant all around the bay area. I spent 2 months in the San Francisco County Jail in San Bruno in 1966 for my efforts.
I resisted the draft from my graduation in 1966 when I was classified 1A to 1969 when the draft board just gave up on me and never sent me another nasty letter. See this for a slide show of events that took place on the day I was drafted (but slipped out of their grasp), Oct 18,1967 during the Stop the Draft Week events on the very same street Occupy Oakland’s general strikers walked down last Tuesday.
As you can see the events of those years are still very alive for me through the work I have done archiving my father’s film collection (see this). I have kept the memories of those movements alive because I believe in them now as I did then. So, for me, the Occupy Oakland movement is a dream come true. It finally gives voice to the cry for justice and sanity that has been silenced for so long under the hegemony of the Republican and Democratic Parties ever since the election of the crook, Richard Nixon, and continuing until the present day with the election of the most recent fraud, Obama the bomber.
I rejoice at the outpouring of outrage from our youth. They have made it possible for me to switch from the tiny minority who over the past 40 years have dissented against the American empire, to the 99% who are dissenting today. What a relief. I don’t know where this movement is headed nor how long it will last. But I do know that it has my whole hearted support and that it opens the door to ten thousand possibilities. I am in awe that I lived to witness it. Since I am retired now, my future plans are to continue to manage Estuary Press and the Harvey Richards Media Archive and to make sure that we do not forget our history, even as we are making it right now.
I guess you can’t call this brief, but I tried my best.
Thanks for asking, and hoping you don’t regret it,
Paul Richards
In praise of nerd kids
I’m a bit of an unusual mother. I don’t worry that my kids don’t have a bunch of friends, that they aren’t popular. I’d actually be more worried, to tell you the truth, if they were super popular. Actually one of my kids is verging on popular, and it’s kind of an issue.
See, the thing is, I love nerd kids. I love the way they are dreamy, and spaced out, figuring out a logical internal system that could never exist except in their minds, laughing to themselves about a joke they just made up, and rewriting songs with silly lyrics a la Weird Al Yankovic, whom they’ve never heard of.
Why? Because almost all of my favorite people in the world were total nerds, or at least outsiders, when they were kids, and I don’t think that’s a coincidence. There’s something about not fitting in, about maintaining an observer’s status, that keeps you from ever being subsumed by the vicious groupthink of seventh grade (and if you can resist that kind of peer pressure, you can resist anything).
By the way, I was a huge nerd in seventh grade, and I had exactly one friend. In fact I didn’t have any friends for a couple of months, and it was miserable. So I’m not saying that I’m hoping for an easy path for my kids. In fact I’m hoping for something kind of tough for my kids, which is probably why most other parents try hard to make sure their kids have playdates with lots of other kids and have spectacular birthday parties etc.
And it’s not that I don’t want my kids to have friends- I do, and they do, just not very many. As far as I’m concerned, two or three co-nerd friends is just about perfect. If we’ve got enough for a Dungeons and Dragons game, we’re good. I’m not ever going to make my kids feel bad for the fact that they’ve only ever played with each other, the kid who lives downstairs, and the kid who lives next door, because those kids are seriously nerdy and awesome.
One thing I hate though is the concept of the child prodigy. Unlike nerds, who live in a dream world of their own making, which I think is awesome, the child prodigy lives in a dream world of their parents’ making, which is weird and not good. I’ve known some of these kids, and they get totally addicted to a certain kind of adult attention, which of course doesn’t last past childhood. They never learn to rely on themselves for feeling accomplished.
I just don’t get it either- so what that a 10 year old can do calculus? What’s the big deal, are there calculus problems that the world needs solving that only this kid can do? No. Leave the kid alone to be a nerd for a few more years, then decide what he or she wants to do. The obsession in our culture with prodigy is just weird, it’s in the same boat with beauty pageants for 3-year-olds. So weird, bordering on creepy.
There are basically two kinds of nerds, from my experience. The first kind is the nerd who vaguely understands that there are social rules that he or she doesn’t understand but that everyone else does. This nerd thinks about that every now and then with something like surprise or maybe even alarm, but then goes back to thinking about more interesting things. My mom is a nerd like this: grew up in L.A. with the children of starlets and couldn’t wait to get to MIT. Built her own telescope in the backyard. That kind of thing.
Then there’s the second kind of nerd, which I was, the excruciatingly sensitive nerd who is fully aware of how much of an outsider he or she is – always aware of the cruelty of those in power, and always feeling for the various victims (who are usually someone else). This nerd is lucky if they survive adolescence without committing suicide, but pretty much from then on it’s good stuff (although never easy). Their favorite movie is Harold and Maude (or was in the 1980’s) and they tend to listen to music really loud whenever they can. Love those nerds.
Gaming the system
It’s not easy being a European bank right now. Everyone is telling them to boost capital, otherwise known as raise money, exactly at the time that much of their investments are losing value on the market because of the European debt crisis.
If I were the person running such a bank, I’d be looking at fewer and fewer options. Among them:
- Ask China for some of their money
- Ask investors for some of their money
- Ask a middle eastern country for some of their money
- Cut bonuses or dividends
- Change the way I measure my money
Of the above possibilities, 5) is kind of attractive in a weird way.
And guess what? That’s exactly the option that the banks are choosing. According to this article from Bloomberg, the banks have suddenly noticed they have way more assets than they previously realized:
Spanish banks aren’t alone in using the practice. Unione di Banche Italiane SCPA (UBI), Italy’s fourth-biggest bank, said it will change its risk-weighting model instead of turning to investors for the 1.5 billion euros regulators say it needs. Commerzbank AG (CBK), Germany’s second-biggest lender, said it will do the same. Lloyds Banking Group Plc (LLOY), Britain’s biggest mortgage lender, and HSBC Holdings Plc (HSBA), Europe’s largest bank, both said they cut risk-weighted assets by changing the model.
“It’s probably not the highest-quality way to move to the 9 percent ratio,” said Neil Smith, a bank analyst at West LB in Dusseldorf, Germany. “Maybe a more convincing way would be to use the same models and reduce the risk of your assets.”
Ya THINK?!
European firms, governed by Basel II rules, use their own models to decide how much capital to hold based on an assessment of how likely assets are to default and the riskiness of counterparties. The riskier the asset, the heavier weighting it is assigned and the more capital a bank is required to allocate. The weighting affects the profitability of trading and investing in those assets for the bank.
One reason I couldn’t stomach any more time at the risk company I worked at is things like this. We would spend week after week setting up risk models for our clients, accepting whatever they wanted to do, because they were the client and we were working for them. Our application was flexible enough to allow them to try out lots of different things, too, so they could game the system pretty efficiently. Moreover, this idea of the “riskiness of counterparties” is misleading- I don’t think there is an actual model out there that is in use and is useful to broadly understand and incorporate counterparty risk (setting aside the question for now of gaming such a model).
For example, the amount of risk taken on by CDS contracts in a portfolio was essentially assumed to be zero, since, as long as they hadn’t written such contracts, they were only on the hook for the quarterly payments. However, as we know from the AIG experience, the real risk for such contracts is that they won’t pay out because the writers will have gone bankrupt. This is never taken into account as far as I know- the CDS contracts are allowed for hedging and never impose risk on the overall portfolio otherwise. If the entity in question is the writer of the CDS, the risk is also viciously underestimated, but for a slightly more subtle reason, which is that defaults are generally hard to predict.
Here’s not such a crappy idea coming from Vikram Pandit (it’s in fact a pretty good idea but doesn’t go far enough). Namely, standardize the risk models among banks by forcing them to assess risk on some benchmark portfolio that nobody owns, that’s an ideal portfolio, but that thereby exposes the banks’ risk models:
Pandit is championing an idea to make it easier to compare the way banks assess risk. To accomplish this, he wants to start with a standard, hypothetical portfolio of assets agreed by regulators the world over. Each financial institution would run this benchmark collection through its risk models and spit out four numbers: loan loss reserve requirements, value-at-risk, stress test results and the tally of risk-weighted assets. The findings would be made public.
Then, Pandit wants the same financial firms to run the same measures against their own balance sheets – and to publish those results, too. That way, not only can investors and regulators compare similar risk outputs across institutions for their actual portfolios, but the numbers for the benchmark portfolio would allow them to see how aggressively different firms test for risks.
I think we should do this, because it separates two issues which banks love to conflate: the issue of exposing their risk methodology (which they claim they are okay with) and the idea of exposing their portfolios (which they avoid because they don’t want people to read into their brilliant trading strategies). I don’t see this happening, although it should- in fact we should be able to see this on a series of “hypothetical” portfolios, and the updates should be daily.
As a consumer learning about yet more bank shenanigans, I am inspired to listen to George Washington:
Truth Values
Just two quick things today.
First, I’m going to see Truth Values: One girl’s romp through M.I.T.’s male math maze this Saturday, with a couple of buddies of mine. It’s been recommended to me by a bunch of my math friends, and tickets are available here. It’s slightly scary how much I anticipate I have in common with the writer and performer Gioio De Cari. Also I think I may have taught a class in the classroom of this picture, maybe even with this haircut:
Second, I wanted to share a poem with you, written by Mary Oliver:
We will be known as a culture that feared death
and adored power, that tried to vanquish insecurity
for the few and cared little for the penury of the
many. We will be known as a culture that taught
and rewarded the amassing of things, that spoke
little if at all about the quality of life for
people (other people), for dogs, for rivers. All
the world, in our eyes, they will say, was a
commodity. And they will say that this structure
was held together politically, which it was, and
they will say also that our politics was no more
than an apparatus to accommodate the feelings of
the heart, and that the heart, in those days,
was small, and hard, and full of meanness.
Why I’m involved with #Occupy Wall Street
I get a lot of different responses when I tell people I’m heavily involved with the Alternative Banking group at #OWS. I find myself explaining, time after time, why it is I am doing this, even though I have a full-time job and three children (let’s just say that my once active knitting circle hasn’t met in a while).
I was not particularly activist before this. In fact I went to UC Berkeley for college and managed to never become politically involved, except for two anti-Gulf war protests in San Francisco which were practically required. While my best friend Becky was painting banners in protest of the U.S. involvement in El Salvador, I was studying tensor products and class field theory.
It’s true that I’ve been much more involved in food-based charities like Fair Foods since high school. One of the most attractive things about Fair Foods was that it acts as an outsider to the system, creating a network of gifts (of primarily food and lumber) which was on the one hand refreshingly generous, based on trust, and the other hand small enough to understand and affect personally.
I, like many people, figured that the political system was too big to affect. After all, it was enormous, did lots of very reasonable things and some unreasonable things; it didn’t solve every problem like hunger and crime, but the people running it were presumably doing their best with incomplete information. Even if those people didn’t know what they were doing, I didn’t know how to change the system. In short, I wasn’t an expert myself, so I deferred to the experts.
Same goes with the economic system. I assumed that the people in charge knew what they were doing when they set it up. I was so trusting, in fact, that I left my academic job and went to work at a hedge fund to “be in business”. I had essentially no moral judgement one way or the other about the financial system.
Once I got there, though, I had a front row seat to the unfolding crisis. As I wrote about, I got to see former Treasury Secretaries and Fed Chairman explain how shitty the securitized products are, that were currently being sold all around the world, and how they had no idea what to do about it except to tell everyone to get the hell out (which we can see has been harder for some than for others).
That meeting wasn’t the only clue I received that pointed to one thing: the experts had no idea what they are doing. The system was based on an overwhelming arrogance and network of vested interests. My conclusion as well as many other people’s.
On the other hand, it was a crisis. It was a fantastic opportunity to set up a better system. The optimist in me assumed that we would. I applied to work for the regulators to be part of the solution. When I didn’t get any offers I worked for a risk company instead.
After two years in risk, and no new system, I had to admit that there was no reason for optimism. The system is still being controlled by the same group of arrogant architects who argue that we can’t change it because then it would collapse. There’s always a problem with asking people on the inside to fix their problems. The politicians don’t understand enough about the financial system to know what to do, and the financial lobbyists telling them what to do are always protecting the banks.
Once I lost faith in fixing the problems, I felt pretty hopeless. I left finance, and considered working for a food-based charity in New York, but finally decided to stay a nerd since that’s what makes me happiest. I started this blog in a moment of hope that something as small as explaining how quants do modeling could be interesting to someone, that maybe the techniques used for so much destruction could also be used for construction.
Then something happened. It was the #OWS protest. I started going down to see if I could get people interested in talking about the financial system. Mostly people were only tepid, but sometimes people really wanted to know. I kept looking for a working group that would focus on this, and eventually I found one.
People talk about how the protesters are a bunch of lazy dirty hippies. There is certainly a group of people who don’t have access to baths, and there are people there who don’t work regularly (although that’s one reason they are there to protest). In fact there are some downright annoying people in the mix as well. But if you focus on those people, you are missing the point entirely. People also say they’re sympathetic, but that there’s no chance we could ever make a difference. I also politely disagree.
What this movement has done is to create a new opportunity for people to try to fix the system. I’m not saying I am 100% convinced it will work; in my practical-minded way, I think the best-case scenario is more like, we will influence the conversation.
But maybe influencing is enough, considering that very few people think things are working right now. At the policy level, we need to influence the conversation in the direction of showing what else could be done, rather than letting things stay the same for lack of a better plan. At the individual level we need to continue what has already begun: getting people interested and informed about the system and how it affects their everyday lives.
What’s the worst-case scenario? We have already seen it: it’s the past 4 years of doing nothing and letting the economy fall apart with no plan for improvement. The #OWS movement has already brought about a meaningful and hugely important conversation, and a sense of involvement by ordinary people, about what experts are doing with their lives.
This emerging conversation is perfectly illustrated in this Bloomberg article – not the article itself, which is strangely self-contradictory and even journalistically dishonest (it tells people to take their money out of Wall Street by investing with Vanguard in the stock market), but the comments on this article are some of the best comments I’ve ever seen on a business-oriented website. People are no longer letting themselves be spoon-fed bullshit.
The dirty hippies are a distraction, and the people who admit the system is broken but who obsess about them need to stop complaining about the embarrassing image and figure out how to help. People who think that we will never accomplish something without a few key experts should contact those experts and invite them to the meetings. Or tell me who they are and I’ll invite them. It’s time to try.
The Numbers are in: Round 1 to #OWS
This is a guest post by FogOfWar:
CUNA (a trade industry group for credit unions) just announced that at least 650,000 customers and USD $4.5 Billion have switched from banks to credit unions in the last five weeks. I think this is a pretty impressive showing for the lead up to “Bank Transfer Day”, and, as posted before, am a supporter of the CU transfers. A few quick points on the announcement:
Are those numbers driven by “Move Your Money” or BofA’s $5 Debit Fee?
A little of both, I suspect, and there’s no hard survey data (that I know of) splitting it out. The two points aren’t completely separate, as one of the points of “move your money” (at least in my mind) is to let people know that credit unions generally have lower fees than large commercial banks & it often makes financial sense to shift your account over regardless of your political views on “Too Big To Fail (TBTF)”.
So is 4.5 Billion a lot or a little?
It isn’t a big number in the scope of overall deposits, but it’s a really big number for transfers in just one month. For scale, the total deposits in all 11,000+ CUs nationwide are somewhere around $800Bn, and that’s roughly 10% of the total deposits in the US. So $4Bn (to make the math easy) is a 0.5% increase in deposits for CUs and somewhere around a 0.05% reduction in the deposit base of US banks in the aggregate. I suspect the transfers are concentrated in the large “final four” banks (BAC, JPM, C, WFC which, if memory serves, account for somewhere around 40% of deposits), so the reduction might be closer to 0.10% for the TBTF quartet.
Wait, those seem like really small percentages—why do you think this matters?
Well, because it’s a greater increase in deposits in one month than credit unions (in aggregate) got in the entirety of last year (and last year was a good year for CUs in which they saw their market share increase). People (rightly) don’t change their checking accounts lightly (there’s a lot of “stickiness” to having direct deposit/ATM cards/etc.—in short it’s a pain in the ass to change your financial institution), so this is a pretty impressive number of people and amount of deposits in this span of time.
Also (and this will play out over time), this could be the start of a trend. Certainly there seems to be a large uptick in discussion of “move your money”, and, as the idea percolates around there’s a lag between thought an action, so this well could be a slow build.
“A Slow Deliberative Walk Away from the TBTF Banks.”
Another important point is that, in fact, you really don’t want too many people moving their accounts in a short period of time for two reasons. First, a rush of people all removing their deposits at the same time is, in fact, a “run on the bank”. This is destructive for a host of reasons—in particular it can cause the institution on which there’s been a run to go into bankruptcy (regardless of whether that bank is otherwise solvent), and, let’s not forget, we the taxpayer are ultimately on the hook for the deposits of bankrupt commercial banks through the FDIC, which is funded by bank fees but backed by the full faith and credit of the taxpayer (and a BofA bankruptcy might put that backstop to the test). So, much as I disagree with many of the decisions of the mega banks, I don’t want the “move your money” campaign to be a catalyst for their insolvency proceedings.
Instead, what I’d really like to see is a slow steady whittling down of their deposit base, and thus their overall size, until they are no longer considered “to big to fail” and thus pose no danger to the taxpayers, as they will be free to make good or bad decisions and live or die, respectively, by them. In short, I’d like to see a “slow deliberative walk away from the TBTF banks” playing out over the course of the next 2-3 years.
The other reason is that credit union’s can only take deposits at a certain pace without running into issues with their own capital buffers and operations. This is a slightly technical issue, but with a substantive point behind it. In essence, absorbing a large growth in new deposits takes some time, not just from an operational perspective (ramping up staff, at a certain point opening new branches and ATMs), but also because more capital needs to be accumulated to provide a safety buffer for the additional deposits that have been taken on. From this perspective, the $4.5 Bn in 5 weeks is a “goldilocks” level—not to much to overheat, not too little to be a rounding error, but just right (OK, actually think it could be 2-3 times the pace without overheating, but everyone else loves to use that hackneyed “goldilocks” metaphor and I just felt peer pressure to frame it that way).
I read that it won’t have an impact on the banks—is that true?
In a word, “total fucking bullshit”. The deposit base is the skeleton of a bank—it’s what holds the whole thing together. Deposits are steady (essentially) free money. Money that can be deployed wherever the bank finds interesting at the moment: loans to customers, speculative exotic derivatives, new branches, foreign investment, whatever. Moreover, if retail deposits mean so little to banks, then why in the world do they spend so much advertising coin chasing them? Generally for profit institutions advertise for products that are profitable to them, not one’s that are irrelevant. QED.
That’s true over the long term. It is worth noting, however, that at this exact moment in time the banks are flush with cash sitting idle on deposit with the fed. http://www.cnbc.com/id/44019510/Bank_of_New_York_Puts_Charge_on_Cash_Deposits Which, by the way, makes it perfect timing for the “move your money” campaign. As I said before, I really don’t want a “run” on the banks, and the fact that banks are flush with cash currently means they’re relatively safe in the immediate moment from a loss of deposits.
But the real reason you’re still seeing Chase commercials on TV even though they’re flush with cash doing nothing at the Fed, is that Chase knows that most people rarely change their primary checking accounts. The accounts that are moving over now are (statistically) gone for good. Later, when that flush cash at the Fed is no more and the banks want the easy money of a wide retail deposit base, they’ll find it very difficult to bring those people back. Not because credit unions are really awesome, just because people really don’t like switching accounts—BofA has to spend a lot of energy to bring in a new account from another institution and doesn’t actually care if it brings it in from lower east side people’s or from Citibank (money is fungible).
Lastly, and perhaps most important, the primary checking account is the primary point of entry to our financial lives. Big banks like the free money you give them on deposits, but equally much they like the chance to have your credit card business, your mortgage and car loan business, your insurance business, your investing business and possibly your retirement and college savings business. All that ancillary business can be (and very much is) statistically quantified on a per-account average basis. All those cross-sells add up over time to big numbers.
FoW
Open Forum speeches
Last night Andrew, myself and Christian gave the Open Forum at Zucotti Park, representing the Alternative Banking working group. I wanted to share a couple of the speeches we used here. Andrew wrote the first part, and I wrote the second and third parts (the third part was adapted from FogOfWar’s breakdown of the banking system). Because we used the human microphone to speak, it had to be written almost like poetry. For now I’ll only share the ones I wrote, since I haven’t asked Andrew for permission to publish his. The event was videotaped and I’ll post a link to the YouTube when I know it. Hopefully we got a few more people to come to our meeting this afternoon.
Cathy:
I want to say something
about my background
When I was a kid
I wanted to become a mathematician
I was a nerd
I loved math because
it made everything
either true or false
it made everything
feel clean and safe
and I liked to feel that way
I worked hard
I went to college
I went to grad school
I was a post-doc
I became a professor of math
at Barnard College
but when I finally got there
I realized I wanted something else
I wanted to be in the real world
I wanted to be part of this city
I decided to work in business
the only job I knew how to get
was as a quant at a hedge fund
I didn’t know anything
about finance
I went there anyway
I learned a lot
I worked with smart people
I learned how they think
I learned how the markets work
I learned how to predict the market
I started to see the system
as an enormous junkyard
and the role we played
as the scavengers
we were the junkyard dogs
we used math and computers
we skimmed off the top
of the enormous system of money
this math is not clean
this math is not safe
there was something wrong
whose money is this?
Where does this money come from?
I couldn’t understand it
I asked other people there
whose money is this?
this is the system they said
it’s just money in the system
your question doesn’t make sense
but I thought more about it
I realized this money
it comes from somewhere
it comes from people
it is their savings
it is their mortgage payments
it is their retirements
some of them are rich
they can afford to make bets
but not all of them are rich
most of the system is made
from normal people’s money
I finally decided
I had to go
it didn’t seem right
to take that money
I went to work at a risk firm
we tried to understand risk
but after some time there
I realized something
people don’t care about risk
not the way they should
I left that place too
I left finance
but I still wanted to do something
about how the financial system works
which is why I’m here
I’m very thankful to you
that you are here too
and we know something
we know the system isn’t working
let’s figure it out
let’s talk about it
let’s understand it
and why it is failing
and let’s make it work
it needs to work for us
it needs to work for everyone
Thank you
Christian:
What is banking?
there are four parts
The first is old-school
traditional banking
the institutions in this
are Banks and Thrifts,
Credit Unions, and Payday Lenders
they take deposits
they start checking accounts
they make loans
they make mortgages
they give out credit cards
they give out debit cards.
The second category of banking
is Investment Funds
the institutions here
are pension funds
mutual funds
index funds
and hedge funds
the collect money from investors
to invest in the market
this is how your 401ks
get into the system
The third kind of banking
is investment banking
here the institutions
are the investment banks
like Goldman Sachs
they give advice to companies
like when they go IPO
or need to raise money
they also make trades
that are supposed to help
their clients
The fourth kind of banking
is Insurance
the institutions are
the insurance companies
they pool risks
they make big pools of money
from lots of people
so that when bad things happen
the pool can pay
instead of one person
we use this system
for Home insurance,
for life insurance,
for car insurance,
and for medical insurance
I am telling you this
to let you know
that this system is big
but it is not infinite
it has been set up by people
and it can be understood by people
and it can improved by people
please join us
the Alternative Banking group
we are meeting tomorrow
Thank you
#OWS meeting tomorrow
I’ve been busy writing my speech for the Open Forum today. Finally finished it just now. I’m going to do it with Andrew, who’s also involved in the Alternative Banking working group. I’m very impressed with the work he and other people are doing for this group, it’s inspiring to see how much people care. I’ve been told that the Open Forum will be videotaped and put on YouTube- stay tuned for the link to that!
One outcome of all that hard work is that we found a place to meet! We’re meeting Saturday (tomorrow) from 4 to 6pm at the Community Church of New York, located at 28 East 35th Street. Hopefully we will have skype set up for people to call in. For skype details keep an eye on the Alternative Banking webpage. That webpage also has the minutes of our last meeting as well as the invitation to this one, complete with a questionnaire that we put together to make the actual meeting more productive.
It also looks like after that we have a room at Columbia reserved for a few Sundays, which is also very awesome.
Sorry for the incredibly boring post. It’s actually really exciting but I don’t imagine that’s coming through. I actually do have plenty of things to say about other stuff, specifically how crazy I’d be if I were an unemployed Greek person right now, but I’m really lacking in time and sleep so I will have to let Floyd Norris from the New York Times say those things:
There is little reason to think that Greek citizens will be more cooperative now that it has been made clear their opinions are irrelevant to the people who run Europe.
Quantitative tax modeling?
Yes, it’s true. I’m going to talk about taxes. Don’t leave! Wait! I promise I’m going to keep it sexy. Buckle up for the most titillating tax convo of your life. Or at least the most bizarre.
Think of us as Murakami characters. I am a young woman, symbol both of purity and of unearthly sexual power, and I’ve taken your hand and led you down a well. We are crawling in underground tunnels looking for an exit, or perhaps an entrance. This is where taxes live, down here, along with talking animals and Bob Dylan recordings.
Do you know what I hate? I hate it when people say stuff like, the Cain 9-9-9 tax plan is bad for rich people. Or that it’s good for rich people. I hate both, actually, because you hear both statements and they both seem to be backed up with numbers and it’s so confusing.
But then again, this stuff is pretty confusing. Even when I think about the most ridiculously stupid questions about money I get confused. Even just the question of “what is the 1%?”, which has been coming up a lot lately, is hard to answer, for the following reasons among others:
- By income? Or by wealth? This matters because most rich people have most of their wealth in savings. They may not make any salary! Living off dividends or some such.
- Measured by individual? Or household? This matters because people with good jobs tend to marry each other.
But you know what? Just give me the answer in any of the four cases above – they are all reasonable choices. And tell me exactly how you’re doing it – which reasonable choices exactly? Better yet, write an open-source program that does this computation and give it, and the data you’re using, to me so I can tweak it.
As I write about this I realize I should confess here and now: I know nothing about taxes. However, I do know something about modeling, and I think in a certain way that makes it easier for me to imagine a tax model. And to critique the way people try to talk about taxes and tax plans.
Here’s my point. Let’s separate the measurement of a tax plan from the tax plan itself- it’s too easy to find a pseudo-quantitative reason to hate a tax plan that you just happen to disagree with politically (for example, by finding a weird theoretical example of a rich person who doesn’t benefit from a given tax plan, without admitting that on average rich people benefit hugely from that tax plan). If we already agree on a model for measurement then we could try to resist the urge to spin, even to ourselves.
Of course we’ll never agree on a model for measurement, so instead we should have many different models, each with a set of “reasonable choices”.
Characteristically for Murakami characters, we do not shirk from the manual labor and repetition of creating a million mini universes of tax scenarios, like folding so many tiny origami unicorns. We write down our thoughts in English and translate back to Japanese, or python, which gives it an overall feeling of alien text, but it has internal consistency. We can represent anyone in this country, under any tax situation. We may even throw in corporate tax structure models while we wait for our spaghetti water to boil.
Once we have the measurement machine, we feed a given tax proposal to the machine and see what it spits out. Probably a lot, depending on how many “reasonable choices” we have agreed to.
Average them! Seriously, that’s what you do in your head. Right? If you hear that so-and-so’s flat tax plan is good for rich people if you consider one year but bad when you take into account retirement issues, or some such thing, then overall, in your head, you basically conclude that it’s kind of a wash.
So by average I mean a weighted average where the weights depend on how much you actually care and believe in the given model. So someone who’s about to retire is going to weight things differently than someone who’s still changing diapers.
What could the end result of such a system be? Perhaps a graph, of how taxes in 2009 (or whatever time period) would have looked like under the putative new plan, versus what they actually looked like. A graph whose x-axis is salary and whose y-axis indicates relative change in tax burden (by percent of taxable income or something like that) of the new plan compared to what actually happened.
It’s nice to use “what actually happened” models since the current tax code is impossibly difficult, so we can duck the issue of writing a script that has the same information just by pretending that nobody cheated on their taxes in 2009. Of course we may want adjust that once we have a model for how much people actually do cheat on their taxes.
If we have decided to build a corporate tax model as well, let’s draw another graph which compares “what happened” to “what would have happened” based on the size of the company. So two graphs. With code and data so we can see what the model is doing and we can argue about it. We’re at the bottom of the well looking up and we see hazy light.
Towards a better financial system
This is a guest post by H.R., a risk management quant:
First, let’s clear the ground for new ideas by questioning the myths that have justified the current system.
Myth 1: The wisdom of the free market is the answer to everything (if only we could get rid of market frictions like regulations and taxes).
Evidence against:
- Behavioral finance has empirically demonstrated a wealth of cognitive biases that distort the market.
- An Indian perspective on the effects of the liberalization of their economy.
- When it comes to their own welfare, the 1% have abandoned the free market in favor of the predator nanny state.
Myth 2: The work of the 1% is indispensable.
Evidence against:
- Empirically, pumping resources to the “job creators” does not seem to be very effective.
- Most of the 1% are not even “job creators”.
- And the marginal value of those that are is questionable.
Myth 3: The unemployed exist because they don’t have the right skills or attitude.
Evidence against:
- Bill Mitchell argues that with the current lack of jobs, “skills training” and similar efforts may shuffle around who gets the jobs, but won’t solve the unemployment problem.
- Economist DeLong concedes that maybe Kalecki had a point about unemployment as a means of lowering wage earners bargaining power.
Myth 4: The government can never compete with the private sector.
Evidence against:
US health care is ridiculously uncompetitive compared to other countries nationalized systems.
Myth 5: We have to choose efficiency over inequality.
Evidence against: Chris Dillow makes the argument that
- better managed government can’t simultaneously deliver us maximum efficiency and equality, and
- there are good arguments for choosing equality over efficiency.
Myth 6: We are constrained by budget deficits.
Some basic ideas from Modern Monetary Theory would disagree.
Myth 7:
- Dollar hegemony,
- cheap oil,
- and infinite growth will go on forever.
Next, we need to imagine the possibilities.
There will be government giveaways. Leave it up to the 1% and they will grab the giveaways for themselves. Let’s decide for ourselves what our priorities are and imagine a system that serves us all.
We can use the tools of
We can imagine the possibilities for a better system
If we aren’t happy with are national system, we can think about how to start local action.
- by switching to alternative local currencies
- which is already happening in the Bay Area.
#OWS Alternative Banking update
I’m excited about the meeting from yesterday and I’m trying to help coordinate next steps. As before, the caliber of the people and the conversation was inspiring – people from all over the place, with so many different background and perspectives. Really exciting! At the same time, we were left with a bunch of open questions and issues; we could really use your help!
The group was quite large, on the order of 55 or 60 people, and after some deliberation we split into two groups: Carne’s group went to the other side of the room to discuss a true alternative banking system, and quite a few of us stayed on the first side of the room to discuss problems with our current system and incremental (but not minor) changes to improve it.
We discussed, (not in this order) how the financial system can be divided into four parts, according to FogOfWar:
- Traditional Banking: taking deposits, checking accounts, CDs, making loans/mortgages, credit cards, debit cards, banks, Thrifts, Credit Unions, Payday Lenders
- Investment Funds: collecting money from investors and making investment in the capital markets: 401(k), 403(b), IRAs, pension funds, mutual funds, index funds, hedge funds (also money market funds with a big asterisk)
- Investment Banking: traditionally two categories: I-banking (giving advice to companies on raising money in the capital markets, M&A, etc), and broker/dealer activities (making trades on behalf of clients and market making) including derivatives
- Insurance: pooling risks amongst large statistical pools to spread large losses into smaller, manageable premiums. Home insurance, life insurance, car insurance, etc.
We also talked about the power grid, how the capital markets and the players in the capital markets control the small businesses which leads to what we see today, with people feeling disempowered from their own money and their own business. We talked about the shadow banking system, politics and the power of lobbyists, and about how we might be able to effect change on the state level by trying to influence where pensions are being invested. We also heard from a fantastic woman who helped form the Dodd-Frank bill and is an expert on the FDIC and various other regulators and understands where their vested interests lie (this line of thought makes me want to write a post on an idea my friend has of paying SEC lawyers on commission, in reaction to the Citigroup – SEC debacle).
[We will write the minutes of the meeting soon, hopefully; the above is just my recollection. Please comment if I’ve missed something.]
It was all very stimulating, and made me want to draw a bunch of visuals to help with the (very large) educational background required to really tackle these problems. Visuals like this or this would also help me prepare for my upcoming Open Forum this Friday.
At the end, Carne invited us to form a separate group from Alternative Banking, which makes sense as we are on the one hand quite large for his office and on the other hand interested in improving the current system more than a completely alternative one. That leaves us with a bunch of things to do though:
- Formally create a new working group through #OWS
- Choose a name
- Choose a representative to go to the #OWS meetings and explain our activities
- Find a place to meet
- Find a way to communicate
#OWS Alternative Banking meeting today
I’m excited to go to the second meeting today of the Alternative Banking working group, whose web page is still a work in progress. We’re meeting at 3pm at Carne Ross’s office.
Dial-in instructions:
At 3 PM Eastern Standard Time (in the US), please dial: (530) 881-1000
When you are asked to enter the conference code, enter this: 451166
Then hit the pound key (#).
Last week we set up an ambitious agenda for ourselves. After passing the “Move your Money” campaign to Alternative Economy (because they are already on it), we decided to focus on two things:
1) What are the legislative steps we think need to be made to fix the current system? To this end the discussion has been centered on commenting on the Volcker Rule, which the public can do until January 13th.
2) Reimagining our current financial system – what would it look like where our deposits, our retirements, and small businesses weren’t being held hostage by a few powerful banks and corporations? Much of this discussion will probably be centered on how things used to work in this country and how things work or worked in other countries.
In the meantime, I’ll share some great links with you:
- Credit unions are seeing a surge in new members.
- When you hear “recapitalization of banks,” what does that mean?
- How do CDS’s work, and what is going to happen in Greece if their 50% haircut counts as a default?
- My hero judge Jed Rakoff tells the SEC to do their job.
- More background on Jed Rakoff.
- A former derivatives guys explains where Wall St. went wrong and comments on #OWS.
- Trick-or-treating tips for #OWS sympathizers
Data Science and Engineering at Columbia?
Yesterday Columbia announced a proposal to build an Institutes for Data Sciences and Engineering a few blocks north of where I live. It’s part of the Bloomberg Administration’s call for proposals to add more engineering and entrepreneurship in New York City, and he’s said the city is willing to chip in up to 100 million dollars for a good plan. Columbia’s plan calls for having five centers within the institute:
- New Media Center (journalism, advertising, social media stuff)
- Smart Cities Center (urban green infrastructure including traffic pattern stuff)
- Health Analytics Center (mining electronic health records)
- Cybersecurity Center (keeping data secure and private)
- Financial Analytics Center (mining financial data)
A few comments. Currently the data involved in media 1) and finance 5) costs real money, although I guess Bloomerg can help Columbia get a good deal on Bloomberg data. On the other hand, urban traffic data 2) and health data 3) should be pretty accessible to academic researchers in New York.
There’s a reason that 1) and 5) cost money: they make money. The security center is kind of in the middle, since you can try to make any data secure, you don’t need to particularly pay for it, but on the other hand if you can find a good security system then people will pay for it.
On the other hand, even though it’s a great idea to understand urban infrastructure and health data, it’s not particularly profitable (not to say it doesn’t save alot of money potentially, but it’s hard to monetize the concept of saving money, especially if it’s the government’s or the city’s money).
So the overall cost structure of the proposed Institute would probably work like this: incubator companies from 1) and 5) and maybe 4) fund the research going on in (themselves and) 2) and 3). This is actually a pretty good system, because we really do need some serious health analytics research on an enormous scale, and it needs to be done ethically.
Speaking of ethics, I hope they formalize and follow The Modeler’s Hippocratic Oath. In fact, if they end up building this institute, I hope they have a required ethics course for all incoming students (and maybe professors).
Hmmm… I’d better get my “data science curriculum” plan together fast.
Open Forum next Friday
I went back to Occupy Wall Street two nights ago after work. I hadn’t been there since last Friday, and all of the tents made the place awfully depressing. I was getting kind of skeeved out when I found myself next to the “red structure” and in the middle of the beginning of an “Open Forum” about Media Justice.
It was the first time I was an actual human microphone for a meeting, and the speeches were really good (they explained net neutrality and the cell phone industry). I was super impressed, and afterwards I introduced myself to the organizer. She explained it’s part of the Education and Empowerment working group.
Bottomline is, I’m giving an Open Forum about the financial system next Friday, November 4th. Very exciting! This format is exactly what I was hoping for when I tried to do the “teach-in” a couple of weeks ago. It’s also a chance to hand out my flyer.
I have to go write a speech consisting of 4-word phrases now. Kind of like poetry.
Is Big Data Evil?
Back when I was growing up, your S.A.T. score was a big deal, but I feel like I lived in a relatively unfettered world of anonymity compared to what we are creating now. Imagine if your SAT score decided your entire future.
Two days ago I wrote about Emanuel Derman’s excellent new book “Models. Behaving. Badly.” and mentioned his Modeler’s Hippocratic Oath, which I may have to restate on every post from now on:
- I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
- Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
- I will never sacrifice reality for elegance without explaining why I have done so.
- Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
- I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
I mentioned that every data scientist should sign at the bottom of this page. Since then I’ve read three disturbing articles about big data. First, this article in the New York Times, which basically says that big data is a bubble:
This is a common characteristic of technology that its champions do not like to talk about, but it is why we have so many bubbles in this industry. Technologists build or discover something great, like railroads or radio or the Internet. The change is so important, often world-changing, that it is hard to value, so people overshoot toward the infinite. When it turns out to be merely huge, there is a crash, in railroad bonds, or RCA stock, or Pets.com. Perhaps Big Data is next, on its way to changing the world.
In a way I agree, but let’s emphasize the “changing the world” part, and ignore the hype. The truth is that, beyond the hype, the depth of big data’s reach is not really understood yet by most people, especially people inside big data. I’m not talking about the technological reach, but rather the moral and philosophical reach.
Let me illustrate my point by explaining the gist of the other two articles, both from the Wall Street Journal. The second article describes a model which uses the information on peoples’ credit card purchases to direct online advertising at them:
MasterCard earlier this year proposed an idea to ad executives to link Internet users to information about actual purchase behaviors for ad targeting, according to a MasterCard document and executives at some of the world’s largest ad companies who were involved in the talks. “You are what you buy,” the MasterCard document says.
MasterCard doesn’t collect people’s names or addresses when processing credit-card transactions. That makes it tricky to directly link people’s card activity to their online profiles, ad executives said. The company’s document describes its “extensive experience” linking “anonymized purchased attributes to consumer names and addresses” with the help of third-party companies.
MasterCard has since backtracked on this plan:
The MasterCard spokeswoman also said the idea described in MasterCard’s April document has “evolved significantly” and has “changed considerably” since August. After the company’s conversations with ad agencies, MasterCard said, it found there was “no feasible way” to connect Internet users with its analysis of their purchase history. “We cannot link individual transaction data,” MasterCard said.
How loudly can you hear me say “bullshit”? Even if they decide not to do this because of bad public relations, there are always smaller third-party companies who don’t even have a PR department:
Credit-card issuers including Discover Financial Services’ Discover Card, Bank of America Corp., Capital One Financial Corp. and J.P. Morgan Chase & Co. disclose in their privacy policies that they can share personal information about people with outside companies for marketing. They said they don’t make transaction data or purchase-history information available to outside companies for digital ad targeting.
The third article talks about using credit scores, among other “scoring” systems, to track and forecast peoples’ behavior. They model all sorts of things, like the likelihood you will take your pills:
Experian PLC, the credit-report giant, recently introduced an Income Insight score, designed to estimate the income of a credit-card applicant based on the applicant’s credit history. Another Experian score attempts to gauge the odds that a consumer will file for bankruptcy.
Rival credit reporter Equifax Inc. offers an Ability to Pay Index and a Discretionary Spending Index that purports to indicate whether people have extra money burning a hole in their pocket.
Understood, this is all about money. This is, in fact, all about companies ranking you in terms of your potential profitability to them. Just to make sure we’re all clear on the goal then:
The system “has been incredibly powerful for consumers,” said Mr. Wagner.
Ummm… well, at least it’s nice to see that it’s understood there is some error in the modeling:
Eric Rosenberg, director of state-government relations for credit bureau TransUnion LLC, told Oregon state lawmakers last year that his company can’t show “any statistical correlation” between the contents of a credit report and job performance.
But wait, let’s see what the CEO of Fair Isaac Co, one of the companies creating the scores, says about his new system:
“We know what you’re going to do tomorrow”
This is not well aligned with the fourth part of the Modeler’s Hippocratic Oath (MHO). The article goes on to expose some of the questionable morality that stems from such models:
Use of credit histories also raises concerns about racial discrimination, because studies show blacks and Hispanics, on average, have lower credit scores than non-Hispanic whites. The U.S. Equal Employment Opportunity Commission filed suit last December against the Kaplan Higher Education unit of Washington Post Co., claiming it discriminated against black employees and applicants by using credit-based screens that were “not job-related.”
Let me make the argument for these models before I explain why I think they’re flawed.
First, in terms of the credit card information, you should all be glad that the ads coming to us online are so beautifully tailored to your needs and desires- it’s so convenient, almost like someone read your mind and anticipated you’d be needing more vacuum cleaner bags at just the right time! And in terms of the scoring, it’s also very convenient that people and businesses somehow know to trust you, know that you’ve been raised with good (firm) middle-class values and ethics. You don’t have to argue my way into a new credit card or a car purchase, because the model knows you’re good for it. Okay, I’m done.
The flip side of this is that, if you don’t happen to look good to the models, you are funneled into a shitty situation, where you will continue to look bad. It’s a game of chutes and ladders, played on an enormous scale.
[If there’s one thing about big data that we all need to understand, it’s the enormous scale of these models.]
Moreover, this kind of cyclical effect will actually decrease the apparent error of the models: this is because if we forecast you as being uncredit-worthy, and your life sucks from now on and you have trouble getting a job or a credit card and when you do you have to pay high fees, then you are way more likely to be a credit risk in the future.
One last word about errors: it’s always scary to see someone on the one hand admit that the forecasting abilities of a model may be weak, but on the other hand say things like “we know what you’re going to do tomorrow”. It’s a human nature thing to want something to work better than it does, and that’s why we need the IMO (especially the fifth part).
This all makes me think of the movie Blade Runner, with its oppressive sense of corporate control, where the seedy underground economy of artificial eyeballs was the last place on earth you didn’t need to show ID. There aren’t any robots to kill (yet) but I’m getting the feeling more and more that we are sorting people at birth, or soon after, to be winners or losers in this culture.
Of course, collecting information about people isn’t new. Why am I all upset about it? Here are a few reasons, which I will expand on in another post:
- There’s way more information about people nowadays than their Social Security Number; the field of consumer information gathering is huge and growing exponentially
- All of those quants who left Wall Street are now working in data science and have real skills (myself included)
- They also typically don’t have any qualms; they justify models like this by saying, hey we’re just using correlations, we’re not forcing people to behave well or badly, and anyway if I don’t make this model someone else will
- The real bubble is this: thinking these things work, and advocating their bulletproof convenience and profitability (in the name of mathematics)
- Who suffers when these models fail? Answer: not the corporations that use them, but rather the invisible people who are designated as failures.
What’s your short list of actionable complaints?
After reading this article from the New York Times about what Volcker says still needs to be done about the financial system (the title of his speech was “Three Years Later: Unfinished Business in Financial Reform”), I’m wondering if he wants to join the #OWS Alternative Banking working group. He’s got his own “short list of actionable complaints” list, not that different from mine:
- make capital requirements for banks tough and enforceable,
- make derivatives more standardized and transparent,
- ensure auditors are truly independent by rotating them periodically,
- end too big to fail,
- create and enforce reserve requirements and capital requirements for money market funds, and
- get rid of Fannie and Freddie, or at least make a plan to.
He also pointed to the weakness of the ratings agencies as one of the big reasons for the credit crisis, so I assume that “making the ratings agencies accountable” may be on the list too, at least in the top 10.
I was interviewed last night about being on the Alternative Banking working group for #OWS (I will link to the article if and when it comes out), and I mentioned this speech as well as the general fact that many of these problems named above are really non-partisan, especially “Too Big to Fail”. This column from the New York Times, written by former IMF chief economist Simon Johnson points this out as well.
That makes me encouraged and depressed at the same time. Encouraged because there really does seem to be a consensus about what’s terribly wrong with at least some of the most obvious issues, but depressed because in spite of this we haven’t solved any of them. To make this vague sense of depression precise, just take a look at what has happened to the original “Volcker Rule”: it has expanded by a factor of 100, from 3 to 300 pages, making it impossibly difficult to understand or probably to follow (unless you have fancy lawyers who do nothing else besides find loopholes). It’s reminiscent of our tax code. Speaking of which, here’s yet another “short list of actionable complaints” to fix that.
I’m enjoying how many people are now coming up with personal short lists of actionable complaints (even if it’s in response to complete stalemate of the political process). It’s a way of claiming and maintaining our freedom and agency. It isn’t as easy at is seems, because you have to sort out the important from the annoying, and the actionable from the existential. If you haven’t already, I encourage you to write your own short list, and feel free to post it here.
Emanuel Derman’s Models.Behaving.Badly.
This morning I want to talk about Emanuel Derman’s beautifully written and wise new book “Models. Behaving. Badly.”, available in some book stores now and on Amazon starting tomorrow.
It is in some sense an expanded version of this essay he wrote in January 2009 with Paul Wilmott. I particularly like the end of the essay where they present the “Modeler’s Hippocratic Oath”:
- I will remember that I didn’t make the world, and it doesn’t satisfy my equations.
- Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
- I will never sacrifice reality for elegance without explaining why I have done so.
- Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
- I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.
This was written in direct reaction to the financial crisis of 2008, clearly, but I think data scientists should all be asked to sign at the bottom of the page. As I’ve said before, financial modelers are just data scientists working in the most sophisticated (under some metrics) subfield of data science; just as they have ridiculous powers of profit using their methods, data scientists in other fields have ridiculous powers as well, sometimes in ways that affect people even more directly than money. Modelers absolutely need to be aware of and wary of these powers. This oath is an excellent step towards that.
In the book, Derman sets up a dichotomy between models and theories. For him, theories are stand-alone descriptions of how things are, whereas models are relative descriptions of how things work, by analogy. He also differentiates between the models (and theories) and the way that humans ascribe truth to them, which is to me the most profound and important message of his book. I’ll discuss below.
His examples of theories come mostly from physics: he has a really beautiful explanation of the evolution of the theory of electro-magnetism, for example, which actually explains how people can sometimes develop theories using temporary models. One idea that emerges is that, sometimes, models work out so well that they eventually become part of the theory. The obvious love that Derman has for physics (which he trained in as a young man) shines through this entire part of the book, and it’s beautiful and intimate reading.
Another example Derman gives of a theory was Spinoza’s theory of human emotion, wherein the basic objects were pleasure and pain, and everything was a derivative of those. For example, love is defined as “pleasure associated with an external object,” pity as “pain arising from another’s hurt.” My favorite: to Spinoza, cruelty is “the desire to inflict pain on someone you love.”
To Derman, Spinoza’s theory is a theory, even though it’s not mathematical, and even though it may not be even “true” in the sense that you could just as well have an alternative theory (although it may not be as beautiful). It is a theory, then, because it describes a mini universe of existence without depending on an assumed external frame of reference. It describes emotions themselves rather than comparing them to something else.
What then is a model? It is something that tried to explain (and predict) the behavior of something through analogy or proxy; its accuracy depends on external conditions. He talks about the Black-Scholes option pricing model as a good model in that, in its purest form, it is actually a model for the price of something depending on the abstract concept of “risk”. Then the fact that it can be misused is due more to the fact that people incorrectly proxy risk itself as described by some brownian motion somewhere. With a better model of risk, then, we’d be happy to use Black-Scholes.
Of course Black-Scholes, or rather its use, is not what caused the financial crisis. It was rather the Efficient Market Model and its corollary the Capital Asset Pricing Model that he considers much more dangerous (I agree). He describes these models very clearly, for the uninitiated, and talk about their ubiquitous use in the vocabulary of finance (and how money mangers describe their Sharpe ratios, which is a ratio of their (past) return and their perceived risk, as if they are meaningful statistics).
The basic mistake people make in ascribing power to their financial models, he says, is that they depend on human beings and their actions to be as predictable as physics. Electrons, it turns out, are much more predictable than people with money on the line.
He also goes into a beautiful riff on how there is, but shouldn’t be, a “Fundamental Theorem of Finance.” Not only is it not fundamental (and not even understandable), but there shouldn’t even be such a thing, because it depends on a model which isn’t true, and cannot deserve the name “theorem”, and moreover can only serve as a false sense of security. This kind of mathematical idolatry, he believes, is at the very root of the problem which led us into the financial crisis. Agreed.
Another aspect of the book I want to bring up, because I find it fascinating, is the way we model things in our lives. As Derman correctly points out, we each model our futures; he describes growing up in South Africa during Apartheid and his involvement with a youth Zionist movement, and how he felt pressure to model his life in a very precise way from the youth leaders there, to move to Israel and work as a laborer (examples among many of how sometimes people blithely model other peoples’ lives as well as their own). He then went on to talk about studying physics, moving to finance and his early desire to find a “theory of everything” in finance.
It may be reasonable to say that, until we die, we are on an endless quest for the perfect model for ourselves. In other words, it’s not only that we use models for our internal lives, but we intensely desire models as well – they give us pleasure, they alleviate our worries and stress. We have trouble letting go of our internal models, even when they don’t work (or we even ignore their failure completely; this was described in a recent New York Times article on confidence). When we model the person we love loving us back, it gives us enormous strength and hope. We model our future success: getting tenure, a promotion, or a child. We model our gods.
But our models are not always realistic, and they don’t always account for bad conditions; just look at the Eurozone for one huge example of this. Sometimes our beloved doesn’t care, and sometimes we don’t get tenure.
The most essential question for me then is: how do we react in that moment when we realize our model is bad? Otherwise put, how do we disagree with ourselves? It’s an excruciating moment that we can learn from – do we take away from a moment like that only the pain? Or do we grab hold of it as an opportunity for growth? Can I train myself to be the kind of person who learns from her broken models?
A related question, which I will take up in another post, is whether the people in finance (or mathematics, or data science) are people who react well to their models failing.
Shareholder Value
This is a guest post by Mekon:
It was late August, 1990. The barely-above-mediocre Boston Red Sox (record: 73-57) led the definitely-mediocre Toronto Blue Jays (record: 67-64) by 6½ games in the American League East. Unsure his team could hold off the hard-charging Jays with a month left in the season, Sox general manager Lou Gorman decided to shore up his bullpen. He offered the Houston Astros a young minor league prospect named Jeff Bagwell for aging-but-competent relief pitcher Larry Andersen. The Astros, long out of the race, said yes. What happened next is the stuff of legend, sort of:
- In September, the Sox went from flirting with mediocrity to wrapping it in a loving embrace (a 15-17 record down the stretch), but still managed to hold off the mighty Jays by 2 games.
- As the Sox staggered to the division title, Andersen chipped in 22 innings in 15 games with 1 save and a good ERA (1.23). If you believe in the Win Shares statistic they quote here, Mr. Aging-but-Competent contributed the equivalent of about 1 win.
- In the playoffs, the 88-74 Red Sox faced the defending World Series champion Oakland A’s (record: 103-59). The A’s won all four games, by an aggregate score of 20-4. Andersen pitched three innings and gave up two runs. He was charged with the loss in the first game when he gave up a run in the 7th inning of a 1-1 game just before the A’s broke the game open.
- The A’s, overwhelming favorites to repeat as World Series champs, lost the Series to the 91-71 Cincinnati Reds in four straight. You never know, do you, baseball fans?
- Andersen was declared a free agent after the season ended. He left the Sox, signed with the San Diego Padres, and pitched in the majors for another four years. Somewhere along the way, “aging-but-competent” became just “aging,” as he compiled an aggregate record of 8-8.
- The Astros promoted Bagwell to the major leagues in 1991. He was the 1991 rookie-of-the-year, the 1994 MVP, and the face of the franchise for 15 seasons. He retired with an exceptional lifetime OPS (On-base Plus Slugging average, today’s batting statistic of choice) of .948, as well as 449 home runs, currently 35th of all time. In his first year of Hall of Fame eligibility (2011), he got about 40% of the vote, a figure dragged down by suspicions of steroid use.
- In later years, Gorman appeared to look back on the trade with pride: “I called Bob Watson and made the trade, Bagwell for Andersen. Andersen would strengthen our bullpen and help us win the Eastern Division title, and we’d go on to face the A’s,” Gorman would write. “He was exactly what we needed to bolster the pen at a critical juncture in our run at the division title.”
- Red Sox fans and ownership agreed. At the Sox’ 1990 holiday dinner, owner and chair Jean Yawkey announced a $2.5M bonus for Gorman for “enhancing ticketholder value” by getting the team to the playoffs. Grateful fans gave him a loud ovation on Opening Day the following April. While a few cranky Boston Globe writers made fun of the Bagwell-for-Andersen trade over the next few years, the public didn’t buy it. As the Sox finished well out of the running each of the next four years, fans always found comfort in the thrilling playoff run of 1990.
OK, I made that last one up: everyone, except apparently Gorman, realized almost from the outset that trading away Bagwell for Andersen was a disaster. As the Sox stumbled through the next several years (see, I didn’t make it all up), poor Lou, who had actually had a pretty good career as a major league GM, became a laughingstock, finally losing his job in 1993. His name still shows up near the top of any “Worst Trades Ever” list that baseball writers feel obligated to make every year around the trading deadline. You could even argue that teams became more cautious about holding on to their minor league prospects after seeing how badly Gorman and the Sox screwed up.
All well and good. But what if I told you that outside sports, in the world of business and finance, the little piece of fiction I put at the end of the Bagwell-for-Andersen story actually isn’t fiction at all?
Shareholder value. Google the phrase and you’ll find almost 5 million links. Virtually all of them (or at least the first five I checked) equate it with the value of the firm or the share price. If you think that’s just a shortcut, it’s not: at the end of every year, companies whose stock grew in price that year give their CEO’s hefty bonuses for – you guessed it – boosting shareholder value. The trouble with all this is that it misses the dimension that Sox fans demanding Lou Gorman’s head understood very well: time.
Say you’re a shareholder in IBM. Maybe you bought (or received) your stock outright, or maybe you gave some money to a manager and had them buy the stock, either directly in your name or pooled with money from other people just like you (i.e., through a fund). Either way, you are, in financial parlance, an asset owner (as distinguished from an asset manager, who manages what you, the owner, hold).
But step back a bit: why do you own this asset, anyway? You’re not going to consume it, like you might a gallon of milk or a car. You don’t get any pleasure from it, like you might from a piece of art that hangs on the wall. The only reason to own a share of IBM is so you can sell it. Hopefully at a high price. And if you own IBM stock now to sell it later, you have to think about time. When will you sell it? Five minutes from now? Five years from now?
When you’ll sell is driven by why. Essentially, there can be two reasons:
- You want to spend the money. You sell your IBM shares and use the proceeds to buy a car, or a house, or a nice picture, or to send your kids to college. A variant of this is that you expect to need the money soon (your kids are going to college next year), and you don’t want to risk IBM decreasing in value, so you sell your shares and put the money in the bank (or in another low-risk asset) until you need it.
- You (or an asset manager acting on your behalf) decide to replace IBM with an asset you like better. Time is embedded implicitly here too: if you’re selling IBM to buy AAPL, you expect AAPL to do better than IBM over some time horizon. Which could be until you need the money, or it could be sooner. For example, I might plan to hold AAPL for a year (over which time I expect it to do better than IBM), and reevaluate what I hold after that.
Asset owners who sell because of (1) are called buy-and-hold investors. Asset owners who sell because of (2) are called traders. Rebalancing, where you hold assets for some period of time, then decide whether to replace them, is essentially a disciplined form of trading.
Now that we know why asset holders sell, we can talk about when. Take buy-and-hold investors first. If you’re selling assets to fund expenses, you’re usually either buying something big (a house or a car, not a gallon of milk) or you no longer have income (you’re retired). Now, truly large expenses are rare, and people usually retire late in life, so asset sales by buy-and-hold investors are spread out across time. In any given year, only a small minority of buy-and-hold investors need to sell assets to raise funds.
Traders initially seem more complicated, and probably are. But we can capture them pretty well by saying they sell when they see a new asset they think will get them better future returns. The more they like the return profile of an asset they already hold, the more they behave like buy-and-hold investors.
Now let’s ask again: what is true shareholder value? Given what we know about asset owners, what is in their best interest? A related question: who benefits when the stock price goes up? Shareholders, surely. But that’s only half the answer. The full answer is shareholders who sell.
Let’s look at buy-and-hold investors first. If the stock price goes up over a year, it benefits investors who sell that year. The remaining investors realize the gains only to the extent that those gains persist when they sell. Since they sell at different times, there’s only one way to benefit them as a group: enhance the value of the company consistently over time.
Of course, once you realize that boosting the share price over the short term doesn’t actually enhance value for most shareholders, you also see that immediate stock price gains are a terrible measure of a CEO’s performance. Red Sox fans understood this right away, and did all they could to run out of town the guy who boosted the short-term stock price (likelihood of making the 1990 playoffs) at the expense of long-term value (Jeff Bagwell’s career).
If you don’t believe me, here’s Warren Buffett making a related point:
“If you expect to be a net saver during the next 5 years, should you hope for a higher or lower stock market during that period? Many investors get this one wrong. Even though they are going to be net buyers of stocks for many years to come, they are elated when stock prices rise and depressed when they fall. This reaction makes no sense. Only those who will be sellers of equities in the near future should be happy at seeing stocks rise. Prospective purchasers should much prefer sinking prices.”
Now, this doesn’t mean you should manage a company to make the stock price drop, in part because there’s a big difference between existing shareholders and prospective ones. But the intuition is the same: investors should worry about the stock price when they buy and sell, and at no other time, and CEO’s should worry about shareholder value across time, not in the short term.
We know, though, that not all asset owners are buy-and-hold investors. Does the presence of traders – whether disciplined rebalancers, day traders, or high-frequency hedge funds – change things? Should it make management pay more attention to short-term stock price movements?
Let’s start with the basics: when you own an asset, its price matters only when you sell, and traders sell when another asset has a better return profile (or when they need to raise funds). Ignoring the latter, and assuming other assets’ profiles don’t change, we conclude that a rising stock price benefits traders if it comes with a worsening future return profile. (By the way, rapidly rising asset prices do usually mean worse expected future returns – but that’s a longer discussion.) That’s clearly no way to run a company, but it’s worth articulating two important reasons why:
- It harms the rest of their shareholders, who plan to keep their shares for longer and want a good future return profile (duh).
- In the aggregate, it even harms short-term traders. Remember, we’re taking asset owners’ points of view here, and even asset owners who are traders need to get their long-term returns from somewhere!
In a little more detail, imagine a world where all companies are managed for the (supposed) benefit of short-term traders, maximizing short-term stock price growth and (purposely or not) making longer-term growth prospects less attractive. So now a year (say) goes by, and all stock prices have gone up a bunch (i.e., there’s a bubble). Asset owners who are traders want to take their profits and invest in another asset that’s more attractive in the long-term – but what? If all companies focus on short-term profit, then all assets are worse in the long term. So, as an asset owner, you’ve made some money, but what are you going to retire on?
Put another way, there’s no way to trade out of the economy – from a global point of view, everyone’s a buy-and-hold investor. If companies manage for the benefit of short-term traders, even traders lose.
So why do we keep rewarding CEO’s for short-term stock price boosts? Every Red Sox fan knew Lou Gorman was mismanaging the team, so why can’t the best minds in business and finance see it? Or does something about the system pervert perspectives and incentives? I’ll take that on in another post.
Credit Unions in NYC flyer
Also from FogOfWar; see also this post where FoW discusses “Why Credit Unions?”:
mathbabe 
