Archive

Archive for February, 2012

How unsupervised is unsupervised learning?

I was recently at a Meetup and got into a discussion with Joey Markowitz about the difference between supervised, unsupervised, and partially (semi-) supervised learning.

For those who haven’t heard of this stuff, a bit of explanation. These are general categories of models. In every model there’s input data, and in some models there’s also a known quantity you are trying to predict, starting from the input data.

Not surprisingly, supervised learning is what finance quants do, because they always know what they’re going to predict: the money. Unsupervised means you don’t really know what you are looking for in advance. A good example of this is “clustering” algorithms, where you input the data and the number of clusters and the algorithm finds the “best” way of clustering the data into that many clusters (with respect to some norm in N-space where N is the number of attributes of the input data). As a toy example, you could have all your friends write down how much they like various kinds of foods (tofu, broccoli, garlic, ice cream, buttered toast) and after clustering you might find a bunch of people live in the “we love tofu, broccoli, and garlic” cluster and the others live over in the “we love ice cream and buttered toast” cluster.

I hadn’t heard of the phrase “partially supervised learning,” but it turns out it just means you train your model both on labeled and unlabeled data. Usually there’s a domain expert who doesn’t have time to classify all of the data, but the algorithm is augmented by their partial information. So, again a toy example, if the algorithm is classifying photographs, it may help for a human to go through some of them and classify them “porn” vs. “not porn” (because I know it when I see it).

Joey had some interesting thoughts about what’s really going on with supervised vs. unsupervised; he claims that “unsupervised” should really be called “indirectly supervised”. He followed up with this email:

I currently think about unsupervised learning as indirectly supervised learning.  The primary reason is because once you implement an unsupervised learning algorithm it eventually becomes part of a large package, and that larger package is evaluated.  Indirectly you can back out from the package evaluation the effectiveness of different implementations/seeds of the unsupervised learning algorithm.

So simply put, the unsupervised learning algorithm is only unsupervised in isolation, and indirectly supervised once part of a larger picture.  If you distill this further the evaluation metric for unsupervised algorithms are project specific and developed through error analysis whereas for supervised algorithms the metric is specific to the algorithm, irrespective to the project.

supervised learning:        input data -> learning algorithm -> problem non-specific cost metric -> output

unsupervised learning:    input data -> learning algorithm -> problem specific cost metric -> output

The main question is… once you formulate evaluation metric for an unsupervised algorithm specific to your project… can it still be called unsupervised?

This is a good question. One stupid example of this is that, if in the tofu-broccoli-ice cream example above, we had forced three clusters instead of the more natural two clusters, then after we look at the result we may say, shit this is really a two-cluster problem. That moment when we switch the number of clusters to two is, of course, supervising the so-called unsupervised process.

I think though that Joey’s remark runs deeper than that, and is perhaps an example of how we trick ourselves into thinking we’ve successfully algorithmized a process when in fact we have made an awful lot of choices.

Categories: data science

What’s going on: Greece and mortgages

There are two very confusing but important issues that you should be paying attention to in the news right now. Luckily, Naked Capitalism is covering this stuff for you (and for me).

First, it’s the mortgage settlement which was agreed on yesterday or maybe two days ago, which sucks in a lot of ways for poor homeowners but not for the banks. To see the top twelve reasons to hate the mortgage settlement, check out this post from Naked Capitalism.

Second, the Greek debt situation is not yet under control, and no matter what they do over there in Europe they can’t seem to admit it. Here’s a Naked Capitalism post from a couple of days ago, coupled with a new Bloomberg article that kind of says how awful that situation is.

I took all our money out of the money market account a few days ago because it’s not FDIC insured and because I really really don’t know what’s going to happen in Europe. Just saying.

Categories: finance, news

The future of academic publishing

I’ve been talking a lot to mathematicians in the past few days about the future of mathematics publishing (partly because I gave a talk about Math in Business out at Northwestern).

It’s an exciting time, mathematicians seem really fed up with a particularly obnoxious Dutch publisher called Elsevier (tag line: “we charge this much because we can”), and a bunch of people have been boycotting them, both for submissions (they refuse to submit papers to the journals Elsevier publishes) and for editing (they resign as editors or refuse offers). One such mathematician is my friend Jordan, for example.

Here’s a page that simply collects information about the boycott. As you can see by looking at it, there’s an absolutely exploding amount of conversation around this topic, and rightly so: the publishing system in academic math is ancient and completely outdated. For one thing, nobody I’ve talked to actually reads journals anymore, they all read preprints from arXiv, and so the only purpose publishers provide right now is a referee system, but then again the mathematicians themselves do the refereeing. So publishers are more like the organizers of refereeing than anything else.

What’s next? Some people are really excited to start something completely new (I talked about this a bit already here and here) but others just want the same referee system done without all the money going to publishers. I think it would be a great start, but who would do the organizing and get to choose the referees etc? It’s both lots of work and potentially lots of bias in an already opaque system. Maybe it’s time for some crowd-sourcing in reviewing? That’s also work to set up and could potentially be gamed (if you send all your friends online to review your newest paper for example).

We clearly need to discuss.

For example, here’s a post (hat tip Roger Witte) about using arXiv.org as a collector of papers and putting a referee system on top of it, which would be called arXiv-review.org. There’s an infant google+ discussion group about what that referee system would look like.

Update: here’s another discussion taking place.

Are there other online discussions going on? Please comment if so, I’d like to know about them. I’m looking forward to what happens next!

Categories: open source tools, rant

As predicted: watered down insider trading bill

Yesterday I posted about the insider trading bill which, in addition to making it illegal for politicians to trade on their insider knowledge, was also going to force “political intelligence firms” to register as lobbyists. Note that this is simply a form of transparency- they, people who work mostly for hedge funds and private equity, didn’t have to stop getting insider information, they’d just need to admit that they were getting it. But I guess that’s TMI from their perspective. From the Wall Street Journal article:

Rep. Eric Cantor, the No. 2 House Republican, plans to bring his version of the Stop Trading on Congressional Knowledge Act, or Stock Act, to the floor of the GOP-controlled chamber on Thursday, using a procedure that will prevent lawmakers from voting on major amendments. It is expected to pass by a wide margin.

At issue are changes Mr. Cantor made shortly before midnight Tuesday, when he unveiled his amendment to a bill that sailed through the Senate last week.

Most notably, Mr. Cantor cut a provision that would require people who mine Washington for market-moving information to disclose their activities in the same fashion as lobbyists. The provision covering what is known as the political-intelligence industry was opposed by Wall Street and its Washington lobbyists, including the Securities Industry and Financial Markets Association (SIFMA), which mounted an effort to kill it.

Just to be clear on who is writing legislation nowadays: they are called SIFMA, and they represent the players in the financial industry. You may remember them from this post, where they hired the research firm Oliver Wyman to investigate the impact of the Volcker Rule for a congressional hearing. Shockingly, that research firm thought the Volcker Rule should be watered down.

What exactly is the argument this guy Cantor is using to defend this change? I’d love to hear him come out and say, “I did it because SIFMA told me to”. How come we don’t get to see that argument made and defended? No wonder people don’t like or trust Congress. Even so I’ll give the last word to one of their members:

The House Democrat who has pushed for the legislation for the past six years—Rep. Louise Slaughter (D., N.Y.)—opposed the GOP-backed changes.

Ms. Slaughter said in a statement that the Cantor-backed version of the insider-trading bill was crafted “in secret, behind closed doors, brokering deals for special interests.” She added: “How ironic—insiders now appear to be writing a bill meant to ban insider trading.”

Categories: finance, news, rant

#OWS upcoming events

February 9, 2012 Comments off

Here ye, here ye, there will be an Occupy Town Square event this coming Saturday. Please come and help us reconstruct Zucotti Park inside a church at 86th and Amsterdam for the afternoon. Here’s the flyer:

Also, there will be a march from Liberty Plaza to the Fed and the SEC to celebrate very own Occupy the SEC’s submission of their Volcker Rule public comments, next Monday, February 13th, at 4:30pm.

Here’s the schedule:
4-430pm: Assemble at Liberty Plaza
5pm: March to the Fed (33 Liberty Street )
5:30pm: March to the SEC’s NY Office (3 World Financial Center, Suite 400)

Finally, the Alt Banking working group now has a twitter feed.

Categories: #OWS, news

This month’s Sky Mall: a sneak peek

I know I’m not the only person who loves Sky Mall magazine for those moments when you realize that you’re not allowed to use your electronic devices, that you have nothing at all physical to read, and that the plane won’t be airborne for 30 minutes due to runway congestion.

To tell you the truth it’s been a while since I’ve moseyed up to lean on it for psychological support so I was a bit hesitant- I didn’t know what to expect. Forgive my lack of faith.

Bottomline: Sky Mall has never disappointed me, which is more than I can say for most celebrated cultural icons. I want to share just a few of the highlights of this issue, and I hope you appreciate using up my precious 30 minutes of free in-air wifi (update: clear your cookies for another half hour) to do so:

  1. The Fleece Poncho With A Pillow (actual name) (see picture above). Best product description ever: The Fleece Poncho With A Pillow is an all-in-one fleece poncho-style blanket with a pillow attached.
  2. The Spongester (picture below). From the description: Made from the same steel as an industrial sink with labeled slots for your “good sponge” (utensils & dishes) and “evil sponge” (sink, counter, cat dish). Until now I (naively) didn’t realize that sponges had morals. I feel so… foolish.
  3. Touchless Sensor Seat (with video!!) (picture below): For only $159.99 you can get an automatic sensor that lifts and lowers the toilet seat for you. It may seem like this price is a bit steep but think about it some more: it sure beats a divorce attorney.

Categories: Uncategorized

More Money than God

This is a guest post from an anonymous friend. Actually is was a letter to me that I thought was hilarious and got permission to post.

———————————————————

Dear Cathy,

Earlier I mentioned that I was reading “More Money than God”, which might have been construed as an endorsement, so, in case you haven’t read it already, I thought I would save you some time by summarizing it:

Chapter 1: It wasn’t us! It was the banks! Those guys!

Chapter {2,\ldots,(N-2)}: All the hedge fund dudes you have heard of are* sages both of human nature and of economics. When they destroy foreign currencies, it’s to correct bad governments. When they attempt to short foreign currencies but fail, it’s because they (Soros) care deeply about these developing countries and are using their money to help support them. They are huge philanthropists. They increase economic stability by being contrarian. The only time they are outsmarted is when they are outsmarted by other hedge fund titans.

Chapter N-1: Take that, banks! Ha! In your FACE!!! Too bad you weren’t more like hedge funds. That would’ve never happened to a hedge fund.

Chapter N: Don’t regulate hedge funds. Regulating hedge funds would be bad for the economy and for philanthropy. There’s no need for hedge funds to be regulated. Regulate the banks or something else but for God’s sake not hedge funds. Also: no regulation!

Acknowledgments: thanks to Rubin and all my other buddies at CFR, and at Blackstone, and to Paul Tudor Jones, and all the other hedge fundies who supported me while I wrote this book for 3 years.

* They are now, but in the 60s when hedge funds started the whole “hedging” and “long-short” thing was just a distraction from organized insider trading over corned-beef sandwiches. But no one ever insider trades anymore. Except for Raj, who’s clearly not a real hedge fund guy. Who eats SIM cards? We’re not those kind of thugs.

Categories: finance, guest post

Politicians and insider trading

There’s shit going down in Washington now around the proposed ban on insider trading of politicians (which for some weird reason up til now hasn’t been illegal). According to this New York Times article, the proposed legislation would also require certain “political intelligence firms” to register as lobbyists, and that gotten them up in a huff. From the article:

“Hedge funds, private equity funds and investment advisers — many of which are not currently registered under the Lobbying Disclosure Act — might now be required either to register or to alter their business practices to avoid the need for registration,” the bulletin said. “If, for example, a hedge fund calls a Congressional committee staffer to gather information about the status of a bill that relates to the fund’s investment decisions, the fund may need to register.”

If you can judge someone by their enemies, then this bill seems kind of like my new best friend. Let’s wait to see how much it’s watered down in the next few days:

House Republicans and their floor leader, Representative Eric Cantor of Virginia, said they would amend the bill, going to the House floor this week, to strengthen it.

But Representative Louise M. Slaughter, Democrat of New York, said, “I think ‘strengthening’ here is a euphemism for ‘weakening.’”

Categories: finance, news, rant

Preggers

The below video resonates with me, but trust me when I say it’s all about the hormones, and we do get over it, at least after weaning. In any case, I apologize (hat tip Jordan Ellenberg).

While I’m here, though, I would like to say one thing that non-pregnant people do to pregnant people, which is desex them. The maternity clothes industry was part of this until recently, making all maternity dresses (and they were all dresses) look like school-girl uniforms.

It’s like, now that you’re pregnant I’m going to treat you like an innocent child who’s never had a dirty thought in her life. But, people, how do you think we got this way?

But it’s a more general phenomenon, and you kind of act like an idiot in part because people treat you like one.

Categories: rant

Opacity, noise, and overpopulation in finance

This is a guest post by Mekon:

When you come in to work nowadays, you have to read the blogs. The other day, two blogs I like to read both had pieces about Freddie Mac and whether it had inappropriately bet against people refinancing their homes. I’ll spare you the details, which live in the highly technical world of mortgage securitization, but the issue is that Freddie Mac had a large position in “inverse floaters,” which are worth more when people don’t refinance.

The first piece says this is fishy, because Freddie Mac also makes rules on who gets to refinance and who doesn’t. So they have lots of incentive to make the rules more stringent, block people from refinancing, and profit by doing so.

But the second piece says there’s nothing fishy here at all: Freddie Mac is probably holding the inverse floaters to hedge interest rate risk. That is, they might need them just to be neutral to interest rates (people prepay when interest rates go down), because the rest of their book is exposed the other way.

How do you tell who’s right?

The first thing to realize is that they’re actually disagreeing on facts. This isn’t like the usual economic disagreements, where people argue over principles (whether the Fed should worry about unemployment as well as inflation) or things you can’t prove (how bad the economy would have gotten without the stimulus). It should be easy to settle this one: take Freddie’s book and see how it goes up and down when interest rates go down/stay the same/go up and people prepay more or less.

I imagine we haven’t done this because we don’t have the book.

Some opacity in finance may be unavoidable, but sometimes it’s completely unnecessary and self-inflicted. These are government enterprises! Why don’t we make their books transparent? If we can’t do it right away, what about with some kind of time lag? We’re talking about their positions from 2010, for heaven’s sake!

The second thing – forgive me if I’m off base here, I’m a fan of both blogs – is that it doesn’t seem like either one of them has fully done their homework (to be fair, without being able to see into Freddie’s book, it’s not clear how they could have). Both sites followed up with more detail, but nothing that seems definitive – put another way, I still can’t tell who’s right.

I’d like to see people be more sure about the facts before publishing conclusions. I thought maybe this was just me, but then I ran across a paper by Andrew Lo which makes much the same point (see the last section). Andrew looks at 21 different books about the financial crisis and compares the range of conclusions they draw to Rashomon. And, like the Freddie example, he finds no agreement on the underlying facts. I hear his frustration when he urges: “By working with a common set of facts, we have a much better chance of responding more effectively and preparing more successfully for future crises.” Amen.

Finally, if you’ll indulge me, a little sociology. If you’ve been around finance for a while, I think you’ll agree with me that people being on loose ground with their arguments and a bit quick on the draw with their conclusions is more the norm than the exception. Put another way, there’s an awful lot of noise in finance. Why is this?

This blog has focused a lot on how finance today is both complicated and opaque. One thing I’d add is that finance isoverpopulated. I don’t just mean that we’d be better off if smart people thought more about curing cancer and avoiding famine and less about executing trades a millisecond faster or securitizing and sell some kind of risk that’s never been traded before. (But duh.)

What I mean is that finance today is so complicated and opaque that it requires extremely specialized skills to understand what’s going on. At the same time, the field employs way more people than could ever have those specialized skills. End result: many people working in finance don’t really understand it. Which makes noise an accepted part of the culture. Which in turn makes it even harder to understand what the hell is going on.

I don’t know how to fix this, but wouldn’t you feel a lot better about our financial system if we could (1) make it simpler, and (2) cut the number of people needed to operate it in half?

Categories: finance, guest post

Women in math

This is crossposted from Naked Capitalism.

A study recently came out which was entitled “Can stereotype threat explain the gender gap in mathematics performance and achievement?”. One of the authors created and posted a video describing the paper, which you can view here.

As a preview, there seem to be four main points of the paper and the video:

  1. The papers on stereotype threat normalize with respect to SAT scores which is bad.
  2. Evidence for stereotype threat is therefore weak.
  3. We should therefore stop putting all of our resources into combating stereotype threat.
  4. We should instead do something easy like combating stereotypes themselves.

Before we go into the details of the paper, we need a bit of context. For that reason, this post is split into three parts. The first addresses a meta-issue, namely that of the “null hypothesis” in this discussion. A frustration that I have, and that I think is shared by many of the women I know in math, is that the (often unspoken) working hypothesis is that in fact women are just not as talented, and it is somehow up to us women to prove this otherwise, presumably by convincing men that we’re geniuses.

The authors of the above paper fall prey to this disingenuous line of thought, by proclaiming stereotype threat is an insufficient explanation but not offering any alternative explanations. This sets up a kind of implied false dichotomy: if it isn’t explained by such and such, it must mean girls are dumb.

Not only does this undermine serious intellectual debate, but it often turns people off from entering the debate in the first place, because they sense the manipulative nature of the discussion. But that’s a pity, since, with the correct assumption, namely that women and men have equal talents but things are holding back women, we could probably make lots of progress on what those things are.

The second part is directly related not to the paper but to the blog post which referenced the paper, which changed the conversation from “math performance gap” to the question of “why there are no women math geniuses”. This is an interesting twist, and in my opinion warrants addressing separately.

In the third part I argue directly against the paper and its conclusions.

1. The Null Hypothesis

Needless to say, I think the onus is on the scientific community to prove that women aren’t as mathematically talented as men. In other words, I do not accept the defensive position that I need to prove we are as smart: the null hypothesis is that a series of effects, one of them stereotype threat, explains any perceived difference in talent.

In his now famous lecture at NBER in 2005, Larry Summers putatively discusses the issue of why there are fewer tenured women in science and math departments at top universities. However, if you read the transcript, you will note that, when he gets to the “different availability of aptitude at the high end” part, he does us a favor of sorts by admitting what his underlying working hypothesis is: that girls aren’t as good at math. His argument using standard deviations of test scores is ridiculous, especially if you consider 1) how differently women do versus men on the same test in different conditions, 2) how much that difference has itself changed over time, and of course 3) the question of what the tests themselves are measuring.

To test why this null hypothesis is so damaging, my friend Catherine Good suggested the following thought experiment: imagine if he’d gone up to the podium and, instead of saying that women aren’t all that good at math and it was partly explained by when he’d given boyish toys to his twin girls that they took care of them instead of constructed things, he had instead substituted gender with race. Here’s the passage:

There may also be elements, by the way, of differing, there is some, particularly in some attributes, that bear on engineering, there is reasonably strong evidence of taste differences between little girls and little boys that are not easy to attribute to socialization. I just returned from Israel, where we had the opportunity to visit a kibbutz, and to spend some time talking about the history of the kibbutz movement, and it is really very striking to hear how the movement started with an absolute commitment, of a kind one doesn’t encounter in other places, that everybody was going to do the same jobs. Sometimes the women were going to fix the tractors, and the men were going to work in the nurseries, sometimes the men were going to fix the tractors and the women were going to work in the nurseries, and just under the pressure of what everyone wanted, in a hundred different kibbutzes, each one of which evolved, it all moved in the same direction. So, I think, while I would prefer to believe otherwise, I guess my experience with my two and a half year old twin daughters who were not given dolls and who were given trucks, and found themselves saying to each other, look, daddy truck is carrying the baby truck, tells me something. And I think it’s just something that you probably have to recognize.

It begs the question, why did the women in kibbutz quit working on tractors? The way Larry tells his story, he makes it clear he thinks that it’s because the women wanted it that way (thus his story about the twins). But surely it is as plausible that: 1) Men, having a vested interest in proving their manhood (which they do and in cultures around the world leads to certain types of work being seen as “manly”) weren’t keen about day care duty and/or 2) women were hesitant to cross the lines of gender stereotype (it might lead them to be perceived as being masculine, or even worse, emasculating). And it also isn’t hard to imagine that parents ooh and ahh more when small children play with what are perceived to be gender-appropriate toys and are quietly or even vocally uncomfortable when boys play with dolls and girls play with trucks.

One last word about the null hypothesis and why I’m so devoted to this issue: when I and two other girls (and, as it happens, no boys) in the 6th grade did well enough to go into a special, advanced 7th grade algebra class, my (female) teacher brought us up to the front of the room and told the three of us “I don’t see why you would challenge yourselves like this anyway since you are girls, and you won’t be needing math when you grow up.” I was the only one of the three of us to actually choose that class, and I was the only girl in the algebra class. One of my friends was one of two women in a class of 45 students studying artificial intelligence at Yale. She was expecting praise for being one of only two students to get a program to work on a particularly tough assignment. Instead, she was accused by the professor of stealing the code from her male classmate. She left the major. Until stories like this become rare, or even uncommon, I will assume that there’s too much cultural influence to figure out the real story.

Going back to Larry Summers, his lecture did two things: 1) it breathed new life into the age-old stereotype that women aren’t as good at math as men, and 2) it attributed that difference to an underlying innate ability difference- that is, he conveyed a “fixed ability mindset” regarding math (more on mindsets below). As the leader of an educational institution he introduced the two ideas that together are like a powder keg: they can undermine women’s feelings of belonging in math, which in turn informs their mathematics achievement and intrinsic motivation to remain in math.

Now more about Catherine Good. She talked at that same conference where Larry Summers put his foot in his mouth; in fact she was the speaker after Larry at that conference, and she was talking about her paper that gives evidence that the above “powder keg” message tends to push women out of math (but Larry didn’t stick around long enough to hear her talk, unfortunately). She is also an expert on stereotype threat and helped me look at the study. More on her thoughts below, but I still want to talk about the concept of “genius.”

2. Women and the concept of genius

Let’s define, as one of the commenters does from the blog, a “genius woman in math” to be any woman who has won a Fields Medal. Since there are no women who have won Fields Medals (versus 52 men), this is a pretty tight definition. I would argue, and I might in another post, that even without the above definition, the concept of “genius” is a social construct which is rarely if ever applied to women, except perhaps after they’re dead. Please comment with counterexamples if you know of any.

So here’s what I think. There are lots of reasons that women don’t win Fields Medals. I will name a few.

  • Fields Medals are awarded to mathematicians under the age of 40, for some reason, and women mathematicians typically do good work into their retirement age, whereas men usually do their best work young (this also explains why Harvard has so much trouble hiring women- by the time they are convinced the woman is a genius, she’s 55 and has grandchildren and frankly probably sees the offer as tokenism).
  • The commenter who defined a “math genius” as a Fields Medalist said that it would be an objective measure. But Fields Medals are awarded by a bunch of guys who decide what’s important and who’s responsible for the important results. In other words it’s a political process.
  • Women don’t care as much about winning Fields Medals. This matters, because I know of men who explicitly worked on problems in order to win the Fields Medal (you know who you are). It’s a serious and bizarre case of narrow focus.
  • Why is math genius defined so narrowly? I would personally define it more broadly (a topic for another post), and there’d be plenty of women geniuses. With my definition, though, I’d guess that women who are geniuses have lots of options and they often choose something they consider more personally rewarding than an academic job.
  • Women’s intelligence may also manifest in different ways: note that most of the assholes on Wall Street are men. This kind of makes sense since women are typically not as driven by testosterone and competitiveness. This doesn’t mean they aren’t geniuses or that they couldn’t have done the work the men on Wall Street did (my experience proves that).
  • The Fields Medal distorts the mathematical process itself, by implying that there’s a single superstar who swoops in and solves the problem that all the other people were incapable of doing. In fact mathematics as a field is an enormous collaboration, a scientific project, where everyone depends on the community around them for coming up with questions, defining the “interestingness” of questions, and giving context to results. The idea that there’s one winner out of all of this, or even one metric by which we could measure such a winner, is silly. See this post from Quomodocumque.
  • Another point about genius (in any domain): research is showing that to truly express one’s genius takes thousands of hours of practice. So genius may be a latent trait but will never be expressed without many hours of hard work. This point is very often lost and is related to women in that their apparent geniusness depends to a large extent on how supportive their environment is for all that investment of time.

3. The paper against stereotype threat

I am finally ready to address (with Catherine’s help) the issues of the paper in question, which I will repeat:

  1. The papers on stereotype threat normalize with respect to SAT scores which is bad

In fact the author “discards” a bunch of stereotype threat studies on these grounds. However, it is totally standard to normalize with respect to some other metric (would you rather we didn’t normalize to anything?), and in fact it essentially penalizes the studies, since it has been shown that stereotype threat is in play even for the SATs. On the other hand, the standard for normalizing (this is called “including a covariate”) is that the groups being compared should not differ significantly in the covariate, presumably because it’s harder to argue that your are in fact correcting for that aspect. Because men and women sometimes do differ significantly in SAT scores, including them as covariates could be a technical violation of the rules of conducting a so-called ANCOVA.

Is this what the author is complaining about specifically? Did he, for example, check to see if the samples in the “discarded” studies actually differ in the covariate? It seems he’s making the assumption that they did, but it’s not clearly stated that they did. It’s certainly not a given that the men and women in these studies did differ in the covariate, and he needs to make that precise. If they did not, then there’s no valid argument against using SAT scores.

  1. Evidence for stereotype threat is therefore weak.

There is ample evidence that stereotype threat is very real. Keep in mind that the authors of this study have not shown evidence against stereotype threat, but have simply complained that they don’t like the existing studies for it. And their standard for what “replicates” the original study is overly stringent- they only wanted to include studies that found significant interactions between gender and condition. Interactions are easiest to find when you have a “crossover effect” (e.g. males are higher in condition A but lower in condition B), but often we find “span effects” in which the males and females may be equal in condition A but differ in condition B. This can also be an example of stereotype threat. For example, in a paper written by Catherine, she didn’t find a significant interaction (males and females performed equally in condition A) but when the stereotype threat was reduced, women outperformed men. To discount this and other studies as not providing evidence of stereotype threat simply because an “interaction” wasn’t found is playing games with statistics.

  1. We should therefore stop putting all of our resources into combating stereotype threat.

Nobody who studies stereotype threat claims it explains everything. It is part of a larger picture. The good news is that there are interventions for it (described below).

  1. We should instead do something easy like combating stereotypes themselves.

The idea that it’s “easy” to combat stereotypes is completely naive. There are tons of ways that stereotyping is understood to be very difficult, if not impossible, to get rid of. Some of them have to do with an evolutionary need to simplify first impressions of people (i.e. categorize) so that we can tell if they are an immediate threat to our safety. This may be the most baffling part of the whole thing, because the authors should really know better.

I want to end on a positive note, because the news is actually pretty good. There is a way to combat stereotype threat, and I’ve tried it and it works. To understand it, it helps to think about the way people think about intelligence itself. As a simplification, people either think that intelligence is fixed and rigid (you’re either born with it or you’re not) or they think that intelligence is malleable and can be learned and practiced.

It turns out that if someone believes the latter “malleable intelligence” view, then they work hard and are hopeful and stereotype threat is to a large extent alleviated. Whereas if they’re convinced of the former mindset for intelligence, the effect of stereotype threat is more pronounced. In situations where the stereotype is salient (“girls are bad at math” is salient when taking a math test), the situation itself can convey a mindset of fixed ability and all the hallmark responses that go along with that mindset then follow. To encourage a malleable view of intelligence can help combat that fixed view and thus the threat of the stereotype.

The way I used this information was as follows. I started a class in teaching proof techniques at Barnard College (there were both Barnard students and Columbia students in the class). At the beginning of every class for the first two weeks I described how mathematicians aren’t born knowing how to prove things, but rather they learn techniques, and practice them until they are proficient. Note I wasn’t directly confronting or addressing stereotypes, but rather setting up the mindset where the studies have shown stereotypes have less negative power.

The class went great, and is still going on. I will post soon about my experiences starting that class and others like it.

Raise capital gains and stop flying

There are two totally unrelated stories I want to discuss this morning, I hope you’ll forgive me.

First, take a look at this post, written by David Brin, which argues for higher capital gains tax. He points out VC’s or angel investors, in combination with entrepreneurs, are the true “job creators”, and also invest their money in a truly risky way, whereas generic rich people who only invest in established companies are taking risks but not on the same level. Yet these two classes of people are taxed at the same rate. I guess the counterarguments would be that they, the VC’s, also get more payoff (when things work out) and that they couldn’t make their investments without the fleet of passive rich people ready to invest if and when the company succeeds. Even so I think there’s a real difference.

It reminds me that, when I worked at D.E. Shaw and Lehman fell, there were lots of discussions around the water cooler about what the reaction would be by policy makers and regulators. The consensus fear was that the capital gains tax rate for hedge fund workers would be removed within weeks, if not days. Note this tax loophole allows hedge fund quants and traders to pay less taxes on their take-home pay than bankers across the street doing the same job. I don’t really know anyone who defends it, not even people who benefit from it. Please correct me if I’m wrong. Update: mostly people below the MD (managing director) level at hedge funds actually don’t get this benefit. It primarily applies to “buy and hold” people like VC’s, private equity, and long term debt firms.

Another argument I enjoy from Brin’s post is the refutation of lowering taxes in general to entice investment by rich people. As he said:

Supply Side assumes that the rich have a zillion other uses for their cash and thus have to be lured into investing it!  Now ponder that nonsense statement. Roll it around and try to imagine it making a scintilla of sense! Try actually asking a very rich person.  Once you have a few mansions and their contents and cars and boats and such, actually spending it all holds little attraction.  Rather, the next step is using the extra to become even richer. Naturally, you invest it.  Whatever the tax rates, you invest it, seeking maximum return.

This is absolutely true, and one of the funny things about (many of) the rich quants I know: they are obsessed with growing their pile, to the point of focusing more on money now that they’re rich than they ever did when they were poor physics or math graduate students. To be fair, to make the whole argument for raising taxes you’d need to consider the global response, whereby rich people essentially arb the tax systems of the various countries in search of the maximum return. Even so, I’m pretty sure the answer is not to try to compete with Caribbean island nations on how low we can tax.

Second, check out this fantastic article from the Wall Street Journal about how people respond to environmental impact issues by consuming more. In the article they describe what’s called the “Prius Fallacy: a belief that switching to an ostensibly more benign form of consumption turns consumption itself into a boon for the environment”. I love it, first of all because it’s completely snarky and second of all because it’s really true and annoying. My favorite line:

Even if you think that climate change is a left-wing crock, this ought to be a matter of gnawing concern. Global energy use is growing faster than population. It’s expected to double by midcentury, and most of the growth will be in fossil fuels. Disasters like the BP oil spill attract world-wide attention, but the main environmental, economic and geopolitical challenge with petroleum isn’t the oil that goes into the ocean; it is the oil we continue to use exactly as we intend.

By the way, I don’t claim to be particularly low-impact on the world myself: I’m flying to Amsterdam in March with my entire family, which definitely puts me on the earth’s shit list (turns out it’s all about airplane travel). For that matter I work at a company that makes it easier for consumers to buy airplane tickets. But at least I don’t pretend that buying a Prius or replacing my kitchen counters with less eco-unfriendly material makes me a good person (by the way, once you’ve got eco-unfriendly kitchen counters the damage is done. The best thing you can do for the environment at that point is never ever remodel your kitchen again. Can you handle that?!).

If I had my way, we’d know the fossil-fuel impact of every activity we engage in, and we’d be able to put ourselves on a fossil-fuel diet. Those people who carefully recycle their milk containers and buy local but also fly to East Asia every chance they get would be in for some major belt-tightening.

Categories: finance, news, rant

Data Science needs more pedagogy

Yesterday Flowing Data posted an article about the history of data science (h/t Chris Wiggins). Turns out the field and the name were around at least as early as 2001, and statistician William Cleveland was all about planning it. He broke the field down into parts thus:

  • Multidisciplinary Investigation (25%) — collaboration with subject areas
  • Models and Methods for Data (20%) — more traditional applied statistics
  • Computing with Data (15%) — hardware, software, and algorithms
  • Pedagogy (15%) — how to teach the subject
  • Tool Evaluation (5%) — keeping track of new tech
  • Theory (20%) — the math behind the data

First of all this is a great list, and super prescient for the time. In fact it’s an even better description of data science than what’s actually happening.

The post mentions that we probably don’t see that much theory, but I’ve certainly seen my share of theory when I go to Meetups and such. Most of the time the theory is launched into straight away and I’m on my phone googling terms for half of the talk.

The post also mentions we don’t see much pedagogy, and here I strongly concur. By “pedagogy” I’m not talking about just teaching other people what you did or how you came up with a model, but rather how you thought about modeling and why you made the decisions you did, what the context was for those decisions and what the other options were (that you thought of). It’s more of a philosophy of modeling.

It’s not hard to pinpoint why we don’t get much in the way of philosophy. The field is teeming with super nerds who are focused on the very cool model they wrote and the very nerdy open source package they used, combined with some weird insight they gained as a physics Ph.D. student somewhere. It’s hard enough to sort out their terminology, never mind expecting a coherent explanation with broad context, explained vocabulary, and confessed pitfalls. The good news is that some of them are super smart and they share specific ideas and sometimes even code (yum).

In other words, most data scientists (who make cool models) think and talk at the level of 0.02 feet, whereas pedagogy is something you actually need to step back to see. I’m not saying that no attempt is ever made at this, but my experiences have been pretty bad. Even a simple, thoughtful comparison of how different fields (bayesian statisticians, machine learners, or finance quants) go about doing the same thing (like cleaning data, or removing outliers, or choosing a bayesian prior strength) would be useful, and would lead to insights like, why do these field do it this way whereas those fields do it that way? Is it because of the nature of the problems they are trying to solve?

A good pedagogical foundation for data science will allow us to not go down the same dead end roads as each other, not introduce the same biases in multiple models, and will make the entire field more efficient and better at communicating. If you know of a good reference for something like this, please tell me.

The SEC needs handcuffs

February 3, 2012 Comments off

My friend Chris Wiggins sent me this link just now, about how the SEC lets big banks get away with whatever they want to in the name of investors. Aargh!

I was discussing the impotence of the SEC with someone at the SEC recently and here’s what I said. Lots of people think you need to pay people at the SEC as much as the bankers get paid in order to have an SEC with balls, but that’s not true. It’s about power, not money. If I knew that, as an SEC employee, I’d be able to walk into Citigroup, put handcuffs on Vikram Pandit, and perp walk him out of the building, that’s a job I’d take in an instant, even at government salary.

Categories: finance

Let them game the model

One of the most common reasons I hear for not letting a model be more transparent is that, if they did that, then people would game the model. I’d like to argue that that’s exactly what they should do, and it’s not a valid argument against transparency.

Take as an example the Value-added model for teachers. I don’t think there’s any excuse for this model to be opaque: it is widely used (all of New York City public middle and high schools for example), the scores are important to teachers, especially when they are up for tenure, and the community responds to the corresponding scores for the schools by taking their kids out or putting their kids into those schools. There’s lots at stake.

Why would you not want this to be transparent? Don’t we usually like to know how to evaluate our performance on the job? I’d like to know it if being 4 minutes late to work was a big deal, or if I need to stay late on Tuesdays in order to be perceived as working hard. In other words, given that it’s high stakes it’s only fair to let people know how they are being measured and, thus, how to “improve” with respect to that measurement.

Instead of calling it “gaming the model”, we should see it as improving our scores, which, if it’s a good model, should mean being better teachers (or whatever you’re testing). If you tell me that when someone games the model, they aren’t actually becoming a better teacher, then I’d say that means your model needs to improve, not the teacher. Moreover, if that’s true, then without transparency or with transparency, in either case, you’re admitting that the model doesn’t measure the right thing. At least when it’s transparent the problems are more obvious and the modelers have more motivation to make the model measure the right thing.

Another example: credit scoring. Why are these models closed? They affect everyone all the time. How is Visa or Mastercard winning if they don’t tell us what we need to do to earn a good credit card interest rate? What’s the worst thing that could happen, that we are told explicitly that we need to pay our bills on time? I don’t see it. Unless the models are using something devious, like people’s race or gender, in which case I’d understand why they’d want to hide that model. I suspect they aren’t, because that would be too obvious, but I also suspect they might be using other kinds of inputs (like zip codes) that are correlated to race and/ or gender. That’s the kind of thing that argues for transparency, not against it. When a model is as important as credit scores are, I don’t see an argument for opacity.

CDS data and open source ratings

What’s the current deal on credit default swap data? Is the Dodd-Frank bill going to force any CDS pricing to be publicly available?

A bit of background: a credit default swap is something like insurance you pay in case the underlying bond is defaulted on (but not exactly, see here), so it’s relatively easy to infer the default probability from its price, as long as you have a good estimate of the “recovery rate,” which is the amount the bond pays out even though it’s defaulted. This rate can vary widely, and people sometimes lose sight of how sensitive everything is to that assumed number.

Here’s the thing. I am super into the idea of an open source ratings model (see this post and this post on open source ratings models, as well as this post on open models in general), and I think having CDS data as input to the model might vastly improve it over just using quarterly filings and stock market data.

Right now the standard ratings models don’t use CDS data, but I think that’s because they’re just really old. I’d guess that some combination of the old ratings model and the new CDS market would be great for an open source ratings model. And it’s true that CDS coverage isn’t perfect (i.e. there are not liquid CDS markets on everything you’d want ratings for) but on the other hand, for what it does, the market is super timely and people really watch it (sovereign debt is a great example of this).

As of a year ago all of this data was essentially owned and monopolized by Markit, which is made up of a bunch of CDS brokers. So even if I had the money to pay for the data, for licensing reasons I wouldn’t be able to make the data open source, which sucks. I know that there’s been talk about making this data publicly available, but I’ve been so involved with stuff like the Volcker Rule, I just haven’t kept up with the current CDS transparency rules. I mean, if we aren’t going to remove the CDS market or regulate it, at the very least we should be using it. Please tell me if you know.

Categories: finance

Alternative Banking in FT Alphaville (#OWS)

February 1, 2012 Comments off

Alt Banking’s opinion piece about too-big-to-fail was published yesterday in FT Alphaville.

Woohoo!

Categories: #OWS, finance, news
Follow

Get every new post delivered to your Inbox.

Join 976 other followers