Archive
Stop with the man-diets already, coffee is good for you.
Every now and then my husband goes on a “man-diet”, which is a term I’ve coined meaning a restriction that has absolutely no reasonable goal except the very last event, namely breaking the diet and thus having an awesome moment of relapse.
His favorite man-diet is the coffee man-diet. He’ll suddenly decide that his two espressos in the morning and two more during the day is too much and he “needs to cut down”. He’ll make it one coffee in the morning and one in the afternoon, and he’ll stare wistfully at my second (or third) morning espresso, complain vaguely of headaches, and generally be a space cadet (by which I mean more than usual).
This will go on for about 5 days or so, until one morning where he’ll wake up in an addict’s rage and rebelliously suck down two coffees whilst raving about the magical properties of caffeine.
This happens about once a year. Every time it happens I remind him that coffee is actually good for you and there’s no reason to try to break an addiction that’s doing you good- we don’t try to go off oxygen, do we?
Well here’s more evidence of that. And in case you’re wondering, yes I do ignore all negative evidence of any one of my theories, but in this case I’m pretty sure the evidence I collect on my side is quite a bit stronger than the stuff I ignore. From the Bloomberg article:
The study found that men who drank 2 to 3 cups a day had a 14 percent lower risk of dying from heart disease, 17 percent lower risk of dying from respiratory disease, 16 percent decreased chance of dying from stroke and a 25 percent lower risk of dying from diabetes than those who drank no coffee.
…
Women who consumed 2 to 3 cups of coffee a day had a 15 percent lower chance of dying from heart disease, 21 percent lower risk of dying from respiratory disease, 7 percent decreased chance of dying from stroke and a 23 percent lower risk of dying from diabetes.
The modeling death spiral for public schools
There was recently a New York Times article about how the public schools have become super segregated by race.
I’m wondering how much of this can be explained by income rather than by race in combination with the obsession we all have with test scores. Let me explain.
If I’m living in a neighborhood with a neighborhood school and the school seems pretty good, then depending on how picky I am I might just stay living there and let my kids go there.
Now assume that suddenly there are test scores available for all the schools in the area, and it turns out my neighborhood school doesn’t do as well as a surrounding neighborhood. Then, depending on how much I think those test scores matter to my childrens’ futures, and how much resources I have, I will be tempted to move to that neighborhood for the “better schools” (read: better test scores).
Over time, people with good resources will move to the new neighborhood, which will become more expensive because there’s competition to get it, which in turn will make it easier for that town to raise local taxes to improve the school, and will also attract parents who really care about the quality of the schools, which will improve the school and presumably the test scores of that school, exacerbating the original difference of test scores.
And of course that’s just what’s happened in this country. My parents moved to Lexington Massachusetts for the schools, and they paid a premium for their house for the location and the school system. So I went to a public school but one that increasingly was attended by richer and richer kids.
Income segregated public schools are the new private schools.
In New York City, where there is more to consider than just your neighborhood, because you can get your kids into schools in other neighborhoods, and there’s a whole network of gifted and talented schools as well, it’s a much more complicated dynamic, but the underlying reasons are the same, and they again have to do with segmentation modeling: we know which schools do well on tests and we avoid poorly testing schools if we can.
The availability of the test scores is huge- if I’m thinking of moving to a new city I can just look up the SAT scores of the high schools in the area and try to find a place to live which is in one of the highest-scoring towns.
This is what I call a death spiral of modeling, and it’s the same idea I described here when insurance companies have too much information about you and deny you coverage because you need insurance so bad. And it’s very difficult to get out of a death spiral, because to do so you need to reset the whole system and re-pool resources but in this case people have already moved out of town.
Questions I am thinking about:
- Is it dumb to care so much about test scores? On the one hand I don’t want to take chances on my kids, so I will opt for the conservative route, which is to think they should be surrounded by kids who test well, because certainly in extreme cases that kind of thing is likely to be contagious behavior. But maybe we have exaggerated ideas about how contagious these things are or how important test scores really are to our kids futures. How would we test that and how would we disseminate the results? And what if we found out that everybody has been acting totally rationally?
- Which begs the other question, namely how can we get this system to work better overall for the average student that would be realistic?
- Note that in the above discussion I haven’t talked about the teachers at all, which is strange. But from my perspective, our system is all about concentrating kids who test well together, and it’s not all that clear that the teachers matter, although I’m sure they do actually. What am I missing? Is there a way of solving this death spiral problem through awesome teachers?
Tech firm mindset to avoid like the plague
There was recently an article entitled “Silicon Valley Avoids ‘B Players’ Like the Plague” which got my attention. Go ahead and read it, it’s pretty short. Here’s the heart of the story:
And not only are companies able to achieve more with less people, they’re also wary of hiring anyone but the best engineers. This is sometimes called the “bozo factor.” The late Steve Jobs often talked about the importance of hiring nothing but “A players.”
The former Apple chief executive said to an interviewer in 1998: “You’re well advised to go after the cream of the cream. That’s what we’ve done. You can then build a team that pursues the A+ players. A small team of A+ players can run circles around a giant team of B and C players.”
To avoid hiring less than A players, companies can go to extremes. At Violin Memory, managers can spend up to half of their time on screening and interviewing candidates. Reference checking alone can eat up large portions of the day. Candidates typically provide three references, but hiring managers will then tap their own networks to make contact with up to five people who have worked with the person. “Your reputation follows you,” said Vice President of Marketing Matt Barletta.
First, you know it’s going to be dripping with compassion and thoughtfulness if it’s a Steve Job’s quote. Second, I’m sure those managers who spend half their day stalking candidates think they’re super productive, when all it says to me is that the more time you spend spinning your creative genius in an environment like this the better – not a good incentive in an industry that probably needs less spin and more skepticism.
Okay, so they have some nutty ideas about hiring people. You might want to consider how they go about firing people as well. From the article:
Jay Fulcher, chief executive of online video technology start-up Ooyala, says he’s “never fired someone fast enough. By the time you know that it’s time for them to go, it’s already too late.”
Ummm… okay, but maybe the work is amazing? From the article:
“It sucks people in, and it takes away from your family life,” says Vice President of Engineering Kevin Rowett. “We have to figure out, can people tolerate that level of intensity?”
Ummm… sure, okay. But maybe this is some kind of super creative environment where you’re expected to be quirky and spontaneous, and you’re not expected to follow rules? From the article (emphasis mine):
Landing a position at Kaggle, a San Francisco-based start-up that crowdsources data analysis problems, is considered such a score that the company is able to have potential candidates move to San Francisco for one to two weeks and audition for a job.
People, people. What the fuck? Are we still wondering why there aren’t enough women engineers in Silicon Valley?
Caveats: 1) the comments on that article are scathing and worth reading, and 2) this toxic mindset is also apparent in New York.
Who wants Jamie Dimon’s job?
It’s Jamie Dimon day for me today, I’ve offered to write a first draft of a Alternative Banking piece on the JP Morgan $2,000,000,000 “hedging loss” that he announced last week and which resulted in a 12% stock price loss in the past 5 trading days. There are many sordid details to wade through to prepare, but here’s the question I’d like you to think about for a minute.
Who would want to take that job once he resigns or gets booted?
I’m thinking the world is divided into people who are realistic about how ridiculously large and unmanageable any too-big-to-fail bank actually is, and the psychopaths that think they are the guy who can tame the beast. Jamie Dimon was definitely one of the most psychopathic of the original crowd of CEO’s left over from before the credit crisis, and honestly he played his part so well it was amazing. It didn’t hurt that JP Morgan was never the worst example of any kind of underhanded and outrageous risk-taking, until now.
For example: Dimon consistently and vehemently complained about any regulation as “strangulation” for his industry, and as anti-american. He was so adamant that people (read: regulators, politicians, and the Fed) took him very very seriously and he was on the verge of fatally weakening the Volcker Rule. We’ll see how that pans out now, but I’m hopeful for something a bit less pathetic.
Is there anyone left who can take that over? Who has the required psychopathic balls?
Ideas for two thesis problems in data science
Natural Language Processing on math overflow
You know about math overflow? It’s a site where grad students in math (or anyone) go and pose questions, and other people can answer them. There are lots of uninteresting, unanswered questions (like questions that are too easy and the person should be able to look up) and there are some really popular ones and some really dumb ones. Sometimes there are interesting ones.
Here’s a thesis idea, come up with a metric for “interestingness” and try to forecast the interestingness of a question from its language. Might as well also try to forecast its popularity while you’re at it. That way, if you make a good model, some of the more interesting questions will get higher in the queue and people will have a better time at the site.
Genealogy graphs in different fields
You know about the mathematics genealogy project? It shows everyone with a Ph.D. in math and considers them to be “descended” from their advisor in a family-tree like structure. For example, I’m here, and if I got up through my ancestors in 7 steps I get to Jacobi. Actually there are lots of ways to go up since a bunch of people have more than one advisor – I’m also 7 steps away from Poisson, 8 from Lagrange and Laplace, and 9 from Euler. This is probably not because I’m so cool but because there just weren’t many mathematicians back then- probably most people descended from Euler. And because we have this cool data set we can see if that’s true!
Here’s what I think someone should do, besides visualizing this graph in an awesome way (which by itself would be really cool, has anyone done that?). They should draw the graph for other fields as well and try to see if there are graph properties that characterize mathematics as distinct from other disciplines like Physics or Law or History.
Conspiracy theorists may be right but they can’t explain why
I’m still reading Haidt’s “The Righteous Mind: Why Good People Are Divided by Politics and Religion,” very slowly, because I have approximately 15 spare minutes a week set aside for free reading.
The part I read last night had to do with how we use our brain as a press secretary for what we believe, arguing for that policy using all the persuasiveness we can muster, no matter how weak our evidence is.
Specifically, if we see evidence for our point of view, we jump on it. If we see evidence against our point of view (oh shit!), we wrinkle our foreheads and feel stressed out, causing us to search and search until we finally find evidence for our point of view again (whew!).
This all seems right to me, although I may just be excited about it because it was already my point of view.
Haidt then goes on to explain that our pleasure centers are directly stimulated when we go through this process of confirming our view, especially when it was somewhat challenged by contrary evidence, and especially if our view is pretty hardline. Finally, he explains that conspiracy theorists are addicted to that pleasure center stimulation moment like a heroin addict.
Assuming this is correct, it explains something I’ve been super puzzled by with conspiracy theorists. And I should say that, being part of OWS, you get to interact with more than your fair share of such people.
Namely, they can never explain their position. In fact I’d say that this is their characterizing feature: one is dubbed a conspiracy theorist not by the unreasonableness of one’s position but by the way one tries to communicate it to other people. If you just had a strong opinion but could explain it well and persuasively, then you’ll never be considered a conspiracy theorist, although of course you could be considered an asshole (depending on what hardline opinion you harbor).
Example: when I try to engage my conspiracy theorist friends (because I do think they are for the most part dear people), they very often get into the tangential loops where they concentrate on one of the following:
- They don’t explain this to anyone/ it’s a secret
- It’s too hard to understand
- There’s a small group of people who have all the power
In spite of trying to convince them that you are listening, you are smart, and you understand that we don’t live in a perfectly fair system, it’s really hard (but not necessarily impossible) to get them to settle down and tell you why they believe these things. I think their pleasure centers must also get stimulated when they go over these three points, because as I said they get totally distracted and it’s difficult to interrupt them.
And by the way, they are willing to try to explain their theory to you. I think one thing Haidt forgot to mention is that it must be a huge thrill to convert other people to your point of view when you are a conspiracy theorist, because often that seems to be a very serious goal.
When it comes to our current financial system especially, I’m starting to believe that many of these points are overall valid, but it’s kind of tragic how poorly my friends explain them. Maybe part of my blog can be devoted to explaining the “why” of the conspiracy once I think I’ve got a good argument.
One last thing which Haidt mentioned and that I’ve noticed too (but which I’m wary of exclaiming as his key point since, again, I already believed it). Namely, scientists are trained to look at evidence and admit when they are wrong. In the realm of mathematics this is certainly so: if you see something disproved it’s a simple waste of time to keep thinking it’s true, even if you previously fervently believed it.
But of course this only holds in the context of theorems and proofs- I’m not sure mathematicians are any better than anyone else in admitting they’re wrong outside of the context of theorems and proofs. Haidt mentioned that there’s no evidence that moral philosophers are any more moral than other philosophers, for example.
Evil finance should contribute to open source
This is a guest post from an friend who wishes to remain anonymous.
I’ve been thinking lately that a great way for “evil finance” to partially atone for its sins is to pay a lot of money to improve open source libraries like numpy/scipy/R. They could appeal to the whole “this helps cancer research” theme, which I honestly believe is true to some extent. For example, if BigData programs ran 10% faster because of various improvements, and you think there’s any value to current medical research, and the cancer researchers use the tools, then that’s great and it’s also great PR.
To some extent I think places like Google get some good publicity here (I personally think of them as contributing to open source, not sure how true it really is), and it seems like Goldman and others could and should do this as well — some sliver of their rent collection goes into making everybody’s analysis run faster and some of that leads to really important research results.
Personally I think it’s true even now; Intel has been working for years on faster chips, partially to help price crazy derivatives but it indirectly makes everybody’s iPhone a little cheaper. Contributing directly to open source seems like a direct way to improve the world and somewhat honestly claim credit, while getting huge benefit.
And it simultaneously has nice capitalist and socialist components, so who wouldn’t be happy?
There are actual problems banks care about solving faster, a lot of which uses pretty general purpose open source libraries, and then there are the maintainers of open source libraries doing this nice altruistic thing but I’m sure could do even better if only they didn’t have a “day job” or whatever.
And if the banks don’t want to give away too much information, they can just make sure to emphasise the general nature of whatever they help fund.
Privacy concerns
The theme of the day yesterday here at the IMA in Minnesota was privacy. There was a talk on privacy and information as well as a discussion later in the afternoon with the participants.
The talk, given by Vitaly Shmatikov, was pretty bad news for anyone who is still hoping to keep private info out of the hands of random internet trolls. Vitaly explained how de-identifying data is a relatively hopeless task, at least if you want to retain useful information in the data, because of the sparseness of most human data.
An example he gave was that the average Netflix user rates 200 films but can be identified on just 4 ratings, at least if you include timestamps. He also pointed out that, even though Netflix doesn’t directly expose its users’ movie preferences, you can infer quite a bit just by looking at how recommendations by Netflix (of the form “people who like House also like Friends”) evolve over time.
And never mind Facebook or other social media, which he explained people can crawl directly and infer the graph structure of; even without any labels for the nodes, which refer to people, one can infer an outrageous amount if you have a separate, labeled graph with overlap on the first.
Conclusion: when people think they’ve de-identified data they haven’t, because other people can collect lots of such data sets, and on top of that some small amount of partial private, identified information about individual users, and piece it together to get creep amounts of stuff.
An example I heard later that day is that someone has figured out how to take pictures of people and give them a large part of their social security number.
The conversation later that day focused more on how companies should protect their client data (through a de-indentifying algorithm), and how for the most part they do absolutely nothing right now. This is perhaps because the problem “isn’t solved” so people don’t see the reason to do something that’s not a perfect solution even though some basic procedures would make it a lot harder. My suspicion is that if they do nothing they are betting they’re more protected from litigation than if they do something that turns out to be too weak. Call me a skeptic but it’s always about litigation.
My contribution to both the talk and the conversation was this: why don’t we stop worrying about data getting out, since it’s kind of a done deal (no data ever disappears and our algorithms are only getting better). Why don’t we assume that all historical information about everyone is out there.
So, besides how to get into my bank account (I haven’t sorted that one out yet, maybe I’ll just have to carry all my money in physical form) I’ll assume everyone knows everything about me, including my DNA.
Then the question becomes, how can we live in a reasonable world given that? How can we, for example, prevent private insurance companies from looking up someone’s DNA or HIV status in order to deny coverage?
This is not an idle concern. As someone pointed out yesterday, 15 years ago nobody would have believed someone who described how much information about the average Facebook user is available now. We have no idea what it will look like in another 15 years.
To my Libertarian friends
First, I’d like to say thank you to the people who have been writing me very nice comments about the PBS Frontline special. It’s cool that people dug it, and it makes me really glad I did it. Thanks!
Second, I had a blast with Reno the other night doing her “Money Talks” show. You should definitely check her out soon.
Also, I’m on my way to the third day of a modeling conference at the IMA (which is part of the University of Minnesota) called User-Centered Modeling. I’ll be speaking tomorrow and I expect to be blogging quite a bit on the other talks between now and Friday.
And with that, I’d like to use the rest of my GoGo Inflight Internet service to start a conversation about the libertarian mindset.
By the way, in spite of my annoyingly opinionated personality, I actually love having friends I disagree with. It feels much more comfortable to be around people who give me friction and challenge my opinions than to be around people who all think similarly to me.
Why? Because it’s a lot easier to spot other people’s hypocrisies than it is to spot one’s own hypocrisies. So if I’m around people who agree with me, we are all very likely being totally blind to something obviously flawed in our mindset, but nobody’s there to point it out.
With that in mind, I really do want this to be a conversation about why libertarians think the way they do- so please comment if you have something to say (and feel free to tell me not to post your comment). I’ll start with what I see as an hypocrisy of the libertarian perspective.
Namely, the cry I hear over and over from the libertarian in the room (whichever one happens to be there) is that big government and welfare and socialized programs are helping people out who should be able to make shit work on their own, whereas they never asked anyone for any help.
This myth of the “pulled myself up by the bootstrap” kind of drives me nuts. It’s like they completely ignore the system in which they lived and (usually) thrived, and how advantaged they are in that system.
When people go into that riff where they talk about how they never owe anybody any money, and they put themselves through college and don’t see why they should feel bad for the students nowadays who owe a collective $1,000,000,000,000 in student loans because they managed to be successful without extra help, here’s what I ask them: do you think you could have been as successful as you are if you’d been born a female subsistence farmer in Africa?
That’s kind of an easy one (and I go from there) but what it does it contextualize the idea of what it means to not ask for help. Namely, when you have a good infrastructure set up with a good education, available health care, etc, then you don’t need to ask for help, because you can help yourself. But it doesn’t mean you’re doing it all by yourself!
So, if you were born into an honest family with a good work ethic and strong skills and intelligence, then yes it’s possible to work really hard and do well, and I’m always proud of people who work really hard and do well, but it needs to be understood that anyone who is a success in our culture is a success partly because our culture allows for such success – and then there’s the individual contribution component which is much much much smaller.
Did you ever notice in the Ayn Rand novel that there aren’t any kids in them? Or for that matter any disabled people, old people, or sick people? It’s a grownup world where you’re either brilliant and yearn to be free from the shackles of petty people trying to repress your innovation, or you’re one of those petty people.
But actually our world isn’t like that at all. We have a community of humans, and like it or not we each contribute to our culture and do our part in defining success or failure.
I always like to point out that I hate laziness, and I have no patience for laziness. It’s a distraction to talk about how lazy whiny entitled kids expect us to pay for their college and then also expect to be given a cushy job afterwards (because libertarians tend to start talking about such symbols of what is wrong with social programs).
Even if there are examples of such people, there are plenty of other examples of people who genuinely worked hard but needed to take out lots of loans and didn’t understand their terms and now are desperately looking for work but can’t find anything. If the conversation is going well I’ll even talk about how if, as a culture, we are raising a generation of entitled kids (which is an exaggeration), then maybe it’s our fault and not the kids’ fault. Because it is.
How are you a narcissist?
A few months ago I took a narcissist test on Oprah’s website (see here). I scored exactly average overall, meaning I’m not a narcissist, or rather I’m exactly in the middle for the population for the narcissistic traits, but if I looked at the actual categories I didn’t score average in each category.
In particular, I maxed out in the categories “Authority” and “Exhibitionism” but scored quite low on “Vanity,” “Exploitativeness,” and “Entitlement”.
I think about says it all for why I blog. It’s because I love you all so much and want the world to be a better place, and I know exactly how it should be and I seriously need you guys to listen to me.
Performing tomorrow night with Reno
I’ll be in the Lower East Side tomorrow night, at the Dixon Place Lounge, talking finance with Reno, who some of you may know. Here’s more info:
The meritocracy myth
Jack and Larry
Recently a Wall Street Journal article described what I’ll call a “Larry Summers” moment for women in business. Namely, Jack Welch, the former CEO of General Electric, spoke to a bunch of women about how if they work hard enough they’ll be appreciated and get ahead. From the article:
He had this advice for women who want to get ahead: Grab tough assignments to prove yourself, get line experience, and embrace serious performance reviews and the coaching inherent in them.
“Without a rigorous appraisal system, without you knowing where you stand…and how you can improve, none of these ‘help’ programs that were up there are going to be worth much to you,” he said. Mr. Welch said later that the appraisal “is the best way to attack bias” because the facts go into the document, which both parties have to sign.
Just as in the case of Larry Summer’s now-famous 2005 speech about women in science and math, a bunch of women left Welch’s talk in frustration.
There is no such thing as a meritocracy
Having been in academic mathematics and a quant in a hedge fund, I’d guess I’ve experienced what comes closest in many people’s minds as the closest to a meritocratic system. But my experience is that it’s anything but, even in these highly quantitative settings.
Instead, as it probably is everywhere, the job environment is a huge social game where it matters, a lot, what kind of priorities you demonstrate and what kind of other signals you give off or respond to. We don’t expect people to play golf and smoke cigars in academia but caring about teaching, or worse, getting a teaching award, can be the kiss of death.
I’m not saying that your personal efforts don’t matter at all, because they do, and you do need to produce stuff, and at a certain rate, but even “personal efforts” are first of all received in the context of a social order (i.e. the perceived importance of your efforts at the very least is a social invention), and second of all they’re are not really personal – one frames the questions one answers with the help of the community, so it’s important you have a good connection and social acceptance in that community (i.e. access to the experts).
Business in more generality is even less meritocratic- there’s a specific requirement that you must “play well with others,” which is absent from academics (mercifully). This means that instead of being an implicit social game, it’s been made very explicit. This is where people promote their work, take credit for others’ work, learn to say what people want to hear, etc. The performance review is a circle-jerk event for such empty-headed manipulations, which makes it particularly ironic that Welch suggested women take the criticism in an appraisal so seriously.
In my experience, it is unbelievably useful for these social games to have an alpha personality, which just kind of means you assume you’re in charge even when it’s not explicitly a situation where someone’s in charge. People respond to such personalities on a chemical level and there’s really nothing a so-called meritocratic system can do about that.
In other words, I’m not holding my breath for a truly meritocratic system. It’s just not what humans evolved for. Let’s acknowledge that and work on how to make the system responsive to good ideas anyway (whatever the system is).
Successful people want to believe that there is such a thing as meritocracy
This begs the question, why do people like Jack Welch and Larry Summers hold on so tight to the myth of meritocracy? My theory is that it serves a two-fold goal: as advertisement for new people and as a validation of the winners in the system.
People want to feel like they are entering a level playing field then the best thing you can do is advertise it as a meritocracy, because it’s human nature to think that you’re better than average. So everyone wants to enter such a field, assuming they will rise to the top.
At the same time, the `winners’ of the social game want desperately to think they did amazing stuff in order to be so successful. They hold on to the myth of meritocracy as a religious belief, and it is pure dogma by the time they reach upper management. This plays into another part of human nature where we discount luck and the infrastructure that led to our success and take it as a sign of our personal choices. Lots of people in finance in general suffer from this diseased mindset but actually anyone who is high enough up in their respective `meritocratic system’ does too.
That’s my simple explanation for why these guys can go in front of a bunch of women and be so unbelievably tone-deaf. They are true believers, because their entire egos are built on this belief, and it doesn’t matter how much counter-evidence is presented to them, even in the form of humans in the room with them.
One last thought. If I saw people leaving a room in disgust when I was giving a talk, I imagine I’d be slightly aghast- I might even pause and ask them what’s wrong. But I guess that’s because I’m not alpha enough.
Being a single mom is not a crime
I’m reading a book called “The Righteous Mind: Why Good People Are Divided by Politics and Religion” which explains how people first make moral decisions, then use their brains to argue those decisions.
It also promises to explain how you can actually change people’s minds, so I’m looking forward to that.
My goal of reading this is to understand how good, moral people can really believe some things that seem just so outrageous and illogical to me. I want to know when it makes sense to have a difficult conversation and how to approach it. I’ll tell you how that goes.
But every now and then I lose faith in the idea that those outrageous ideas come from an earnestly moral place. And one example came from my son, who is 12.
If you have a 12-year-old, or if you’ve ever been a 12-year-old, you may remember that they speak somewhat hyperbolically. So when mine told me there’s someone in Wisconsin making it illegal to be a single parent, I thought he was making it up.
But then he found this article for me. I was dumbfounded, and he said I’d have to blog about it since he was right and I was wrong. So here I am.
This just seems so so ass backwards on so so many levels, especially when you think about how it would go down if it became law. Do we get the fathers in trouble too? What if we don’t know who the father is? Do we make it illegal to have unprotected sex in the first place with someone you aren’t married to?
Beyond the crazy idea of where this would stop, I always get upset when I see vulnerable people further abused. If this guy can make a case that kids of single moms are more at risk for various things, why not take that as a cue to give them more support (rather than punishing them)?
Great news for NYC dataphiles: Microsoft Research is coming to NYC
This is a guest post by Chris Wiggins.
I was happy to see the news this morning that Microsoft is opening a new research lab in NYC, with 15 of the former members of the Yahoo R+D NYC lab as its founding members.
The Yahoo group is one of the most multidisciplinary research teams I’ve ever seen, with great research collaborations among physicists, machine learning experts, applied mathematicians, and social science, all learning about human behavior by analyzing web-scale datasets.
They have also managed to show how a research lab can make great contribution both to the local and international research communities in their field. For example, Jake Hofman at Yahoo has been teaching a great ‘Data Driven Modeling’ class at Columbia for years; John Langford has been a co-organizer of the New York Academy of Sciences’ one-day Machine Learning Symposium since it was founded (while also organizing international machine learning conferences, running a great machine learning blog, etc…)
Some particularly exciting aspects from the announcement include:
- Mathematical physicist Jennifer Chayes seems to be implying she’ll be spending at least part of her time here in NYC rather than her current home of MSR-Cambridge
- Multiple people in the story said they’re interested in ties with startups and universities, which should be good for the intellectual landscape of NYC dataphiles.
Congrats to all and to NYC!
Occupy Data!
My friend Suresh and I are thinking of starting a new working group for Occupy Wall Street, a kind of data science quant group.
One of his ideas: creating a value-added model for police in New York, just to demonstrate how dumb whole idea is. How many arrests above expected did each cop perform? [Related: you will be arrested if you bring a sharpie to the Bank of America Shareholder’s meeting in Charlotte!]
It’s going to be tough considering the fact that the crime reports are not publicly available. I guess we’d have to do it using Question, Stop, and Frisk data somehow. Maybe it could actually be useful and could highlight the most dangerous places to walk in the city.
Or I was thinking we could create a value-added model for campaign contributions, something like trying to measure how much influence you actually buy with your money (beyond the expected). It answers the age-old question, which super-PACs give you the best bang for your buck?
Please tell me if you can think of other good modeling ideas! Feel free to include non-ironic modeling ideas.
The student debt crisis
A few weeks ago I wrote about the higher education bubble that I saw at the individual level. This is the idea that, for a given student trying to decide whether or not to take on more loans to go to school, it’s essentially a no-brainer; it’s a cultural given that a college education pays off, statistically speaking, even if in a specific case it doesn’t.
As always, however, the situation at the individual level (a student going into debt) is informed by the overall system. Today I want to write a bit more about how this system got so out of whack.
I’ll actually write about a series of theories of mine, so please tell me if you think I’ve got the facts wrong. I don’t want to claim these are new ideas, but rather a storytelling version of a common understanding of how this all went down. In this case, it’s a story about money and perceived risk, no too dissimilar from the housing crisis.
Before I get into the details of the theory, let me throw in that the Ivy Leagues like Harvard and Princeton have always been super expensive, but that’s part and parcel of their brand. It’s actually intentional, they wouldn’t have it any other way, because part of being elite is being out of control expensive (although it needs to be said that their financial aid to poor kids is exemplary). In other words, I don’t think my theory is going to work on super elite colleges, but that’s fine because most people don’t attend those colleges.
And college always cost some money, although some state schools were really quite reasonable back in the day. It’s just a question of how long after college someone would have to wait or work before their student debts would be gone so they could move on with their lives and think about buying a house (more on that connection below).
Okay, with those disclaimers, let me get started. Namely, it’s all about bankruptcy laws. I know that sounds unbelievable, but it’s really true. From Justin Pope:
Until 1976, all education loans were dischargeable in bankruptcy. That year Congress began requiring borrowers to wait at least five years before they could discharge federal student loans. Since 1998, borrowers have been unable ever to discharge federal student loans, and in 2005 the then-Republican-controlled Congress made private loans almost impossible to discharge.
Why does this matter?
Because it meant that people lending to students wouldn’t need to worry about getting their money back. That sets up a perverse system where young people who are not creditworthy can take on piles of debt.
On the one hand, it’s good for students to be able to finance their education- you wouldn’t want young poor kids to not attend colleges at all for want of enough funding.
On the other hand, it meant that the tuition and fees could essentially rise without pause, since there was nothing to force them back- no supply vs. demand situation.
This is especially true because students aren’t told and do not generally “shop around” for a good deal in college, and moreover colleges are incredibly underhanded about making their tuition and fees clear (I am giving the CFPB about 3 more months to force them to present packages in a standard form before I really start complaining).
Another example: Pell Grants. These are grants given to poor kids to go to college, and they aren’t loans – the government pays them straight to the college. But the colleges have not really made it easier for kids to go to college because of this free money, but rather have raised their tuitions by the amount of the expected Pell Grant. Some colleges are better at getting Pell Grant money than others, and in particular for-profit colleges get 7 out of 10 such grants.
[Speaking of for-profit colleges, how are they allowed to exist? They are the worst of the worst and in particular have outrageous practices in terms of disclosing fees and tuitions, giving commissions to financial aid officers who then urge students to lie on their financial aid forms. Not to mention providing questionable educations.]
I hope it’s not too hard now to understand why student debt has just surpassed $1,000,000,000,000 in this country, ahead of credit cards. On the banker side of the room, these student debts are being bundled up and securitized and sold to investors just like old mortgage-backed securities (which, as you recall, couldn’t fail because the housing market always goes up) who are being told there’s very little risk since students can’t discharge student debt through bankruptcy. There’s a strong analogy with the previous housing bubble and the current education bubble: even ignoring the individual’s goal of becoming an educated citizen and qualified worker, there’s the demand side from the banking system itself which feeds on the fees of securitized products that seem riskless.
But the cost to those individual borrowers is heavy; we have a system whereby young people are being saddled with enormous and unreasonable debt in order to even qualify as a worker, and they are carrying it around like a noose. It is, for example, one thing preventing the housing market from recovering, because the generation of young people who should be buying a house right now is instead still trying to pay off student loans.
What should we do about this?
First of all, let’s focus on the culture of education and how we think about it. Is it a certification process that people should pay for? Or is it a part of what we offer our citizens as their right?
If the latter, it’s time we rethink why state schools should exist, and fund them accordingly, rather than removing more and more funding while expecting them to uphold their state-school mandates. I went to UC Berkeley in the early 1990’s and it was completely awesome, but it’s getting more and more squeezed by the state of California, which is forcing it to choose between becoming a bad school and becoming a private institution. And yes, that means taxpayer money going towards the investment of broad education rather than to banker bailouts.
Second of all, let’s rethink the bankruptcy laws. At the very least for private student debt. For public student debt it also needs to be negotiable in certain circumstances. We need to get the colleges themselves to lower their fees enough that the debt loads are tolerable.
Finally, let’s rethink how much to expect someone to pay down their debt depending on their salary. From Bloomberg:
The second crucial step is to mitigate the burdens of already distressed borrowers. The Obama administration has made progress, for instance by proposing an initiative that would let some students limit their loan repayments to 10 percent of their discretionary incomes next year and would forgive balances after two decades. Private lenders should be given incentives to offer more modifications and flexible payment options.
Yes, this means that some of those securitized student debt loans will default, just like mortgages. And that will mean that already undercapitalized banks will be even more so, because they’ve been taking risky bets yet again, and Sallie Mae will be up Shit’s Creek. But the alternative, of a continuing debt trap for young people, is an even worse alternative.
It’s May Day!
I’m leaving work early today to join the May Day Celebration with Occupy (May Day playlist here).
Bloomberg knows about it, and so does the Wall Street Journal, and if you do a search on the New York Times you can even find something there.
I’ll be participating in the Free University today, talking about “Weapons of Math Destruction” from 2-3 in Madison Square Park, and then meeting my kids there after school for a march down to Union Square with the signs we made in Alt Banking on Sunday. Woohoo!
Oh, and the PBS Frontline special on politics, banks, and Occupy that I’m in (Episodes 3 and 4) airs on PBS tonight at 10pm. Please watch it and tell me what you think, or watch it online here tomorrow, because I made a deal with myself that I’d do it as honestly as I could but never watch it myself. There are outtakes here from my interview that I consider unwatchable as well but people seem to like.
Conclusion (#OWS)
This is the final part of a four-part essay that was proposed by Cathy O’Neil, a facilitator of the Occupy Wall Street Alternative Banking Working Group, and written by Morgan Sandquist, a participant in the Working Group. The first three parts are here, here, and here. Crossposted from Naked Capitalism.
Still sitting in our breakfast nook, with the banking industry squinting grumpily back at us through the glare of the morning sun on the perfectly polished granite table top, we can sit back, rest our hands on the table, and rather than shouting what it expects to hear, playing our part in the script of codependency, we can speak, without pleading or rancor, the truths that are beyond the script.
Rather than repeating once again our expectations and the banking industry’s failure to meet them, rather than pleading with it to live up to its obligations and do what’s fair, we can speak of the mundane practical details of our life and our children’s lives after its eventual demise, of the specific process by which everything around us will be sold to pay the ruinous debts for which its insurance will prove woefully inadequate.
We can make of the inevitability something tangible, rather than a vague, abstract threat. We can catalog the likely disposition of all of the banking industry’s prized possessions and family heirlooms, the eventual owners of everything it values. We won’t engage in a debate over whether the inevitable will occur, nor will we revel in the justice of it, because we’ll all suffer.
The banking industry must take responsibility for the laws it has broken and make appropriate restitution, not because we’re vengeful, but because the connection between choices and consequences so necessary to any successful relationship must be restored. And as long as the banking industry is adamant that it can’t or won’t change, and given the suffering that will follow from that refusal, we have no choice but to regretfully plan for our own and our children’s well-being to the extent we can. Should it accuse us of betrayal, our response is that the first step toward an alternative is one the banking industry must take.
I would have liked to title this section of this essay “Recovery,” but I’m not in the position to do that. I can’t speak to outcomes, only to process. I’m neither imaginative nor prescient enough to suggest what the successful results of our efforts might look like, but I have some idea of how those efforts should be undertaken.
I could use the Twelve Steps of Alcoholics Anonymous (“We admitted we were powerless over debt–that our industry had become unmanageable,” and so on) to describe what might follow from the banking industry taking that first step, but that would be a too literal extension of the metaphor.
The truth is that history offers no examples of the sort of transformation that’s now needed. Though addiction and recovery may offer greater insight into our predicament than yet another political or economic analysis, there’s no reason to believe that the situation will stick to that metaphor as it evolves, even as we proceed with what is essentially an intervention.
Gil Scott-Heron famously said, “the revolution will not be televised.” I take this to mean that any real, fundamental change to the workings of our society, won’t be an entertainment offered by revolutionaries to the rest of us. It won’t be achieved if we sit home waiting for someone on television, or now the Internet, to present us with a number we can call or text, a petition we can sign, or a ballot we can fill out. Our opinions will effect nothing, and our agreement is neither solicited nor required.
I offer this essay not to start another in the endless string of conversations about what is to be done, but to prompt you to do it, whatever it will be, even if what you will do proves that everything I’ve written here it categorically incorrect. I also offer this as an explanation of why I’m doing what I’m doing.
Among other things, I’m working with several participants in the Alternative Banking Working Group on a Web application that will allow people in New York City (and, later, the rest of the country) to find credit unions for which they’re eligible (something that turns out to be far more complicated than you might expect). This will facilitate the return of money from banks, where it functions as an addictive substance, to community ownership, where it functions as a tool in the business of that community.
I do this on your behalf as well as my own, but not in your place. What else will be done and what will come of it will be the result of what you do. The Occupy Wall Street movement is entirely open and will become no more or less than what we make of it. Tomorrow’s May Day events will provide people with the chance to find out for themselves what that movement is, how they can become involved, and what it will become.
With events in more than 115 American cities and many more around the world, you should be able to find an event near you (if you’re in New York City, you can even hear more about the prank played on MF Global by the banks).
Take the day off and meet the people with whom you share this struggle, whether you’ve agreed to it or not. If the movement isn’t what you want it to be, it’s incumbent upon you to transform it as necessary. You can sit home waiting for a movement that checks all your boxes to somehow arise and solicit your participation, but so passive an approach is unlikely to accomplish much.
Let’s all get together on May 1st and see how much we can accomplish in this American Spring.
NYC Data Hackathon
This is a guest post by Chris Wiggins.
Last night I was a judge for the Data Viz Competition at the NYC Data Hackathon, part of the world’s first global data hackathon. Along with my fellow judges Cathy O’Neil and Jake Porway, we gave an award to the team that best found a nontrivial insight from the data provided for the competition and managed to render that insight visually.
Unlike a hackNY hackathon, where the energy is pretty high and the crowd much younger (hackNY hackathons are for full time students only; this crowd all were out of school — in fact at least one person was a professor), here everyone was really heads down. There was plenty of conversation and smiles but people were working quite hard, even 12 hours into the hackathon.
I noticed two things that were unusual about the participants, both of which I think speak well of the state of `data science’ in NYC:
* I’ve never been in a room with such a health mix of Wall Street quants and startup data scientists. Many of the teams included a mix of people from different sectors working together. The winning team was typical in this way: 1 person from Wall Street; 1 freelancer; and 1 data scientist from an established NYC startup.
* I met multiple people visiting from the Bay Area contemplating moving to NYC. In 2004-2007 many of my students from Columbia moved out to SF under the historical notion that that was `the place’ where they could work at a small company that would demand their technical mastery and give them sufficient autonomy to see their work come to light under their own direction. I was glad to meet people from the Bay Area who were sufficiently impressed with NYC’s data scene to consider moving here. Of course I told them it was exactly the right thing to do
and I looked forward to seeing them again soon once they’d become naturalized citizens of NYC.
Huge thanks to Shivon Zils and Matt Truck for hosting us in such a nice location, to Jeremy Howard for his suggestion a few weeks ago to throw the event, and to Max Shron for encouraging everyone to include a visualization prize as part of this event.
An Intervention (#OWS)
This is the third part of a four-part essay that was proposed by Cathy O’Neil, a facilitator of the Occupy Wall Street Alternative Banking Working Group, and written by Morgan Sandquist, a participant in the Working Group. The first part is here and the second part is here. Crossposted from Naked Capitalism.
What are we to do with our banking industry, sitting there at the kitchen table in its underwear, drumming on the table with one hand and scratching its increasingly coarse chin with the other in an impossibly syncopated rhythm, letting fly a dizzying stream of assurances, justifications, and accusations, and generally spoiling for a fight that can only be avoided by complete and enthusiastic agreement with a narrative that can be very difficult to follow, let alone make sense of?
Because this is our kitchen too, we have our legal and moral rights. We would be well within those rights to respond to its nonsense with far more coherent and sweeping condemnations of our own. Throwing the bum out in its underwear without so much as a cup of coffee, taking the children, and keeping our share of whatever might be left could certainly be justified.
Though the sense of release offered by those responses is tempting, they’re not likely to be of any practical use. We can’t win an argument with an irrational person, and our share of an insolvent industry is likely to be very little–certainly not enough to feed the children. We have to recognize the hard truth of our implication in the banking industry and its addiction.
This doesn’t mean that we’re responsible for the addiction and its consequences, or that we can make the choice not to continue that addiction on our own, but it does mean that the problem won’t be constructively resolved without our efforts. To the extent that denial is about obscuring the connection between decisions and results, the most effective means of undermining denial is to clarify that connection, and the process of doing that is intervention.
Whether or not its participants are thinking in these terms, Occupy Wall Street, to the extent that it can coherently be referred to as an entity, is in many respects functioning as an intervention into the banking industry’s increasingly untenable addiction to money and debt.
The movement’s core values of transparency, sustainability, and nonviolence reflect the clarity, patience, and compassion needed for an effective intervention.
This is not to say that all of the efforts directed at banks by the movement have been magnanimous or constructive. We have to remember the terrible suffering that has been inflicted upon so many and offer that clarity, patience, and compassion not just to the addict, but also to ourselves as intervenors and to everyone who has been affected by the banking industry.
On the whole, I have been deeply heartened by how this movement has evolved over the last seven months, and though intervention isn’t the easiest or most promising process, it’s one I recognize and know can work (in stark contrast to political revolution). From the beginning, the Occupiers have shown a fearless, poignant spontaneity that’s available only beyond the addiction-centered dynamic of denial, and the banking industry, its enablers, and others still within that dynamic of denial (which, to be fair, has included most of us at one point or another) have responded as would be expected.
The determination and wisdom granted to those who see more clearly is profoundly threatening to those seeking to maintain denial, though of course they wouldn’t be able to say quite why.
The initial objections from the press that Occupy Wall Street was making no demands could be seen as an enabler’s yearning for symptoms that can be isolated and addressed, without admitting to or addressing the addiction from which they arise. To keep calmly and patiently pointing to that underlying cause is simultaneously incomprehensible and maddening to those trapped within denial, and their responses have run the gamut from smug certainty that nothing could possibly be wrong to whistling past the graveyard to ill-conceived and unjustified violence. And the Occupiers’ patience, diligence, and good nature in the face of that decidedly ill will is as textbook an illustration of the process of intervention as we’re likely to see.
It would be all too easy to remain passive in the face of our increasingly delusional, erratic, and combative banking industry. Surely there must be a more palatable alternative to undermining the continued functioning of the complex and highly evolved process that is the core of our economy.
If we force it to rehabilitate, what will happen during that process? Will our economy collapse? When its rehabilitation is complete, will the banking industry be able to function as well as it once did? Or as the banking industry’s enablers would have it: Any attempts to regulate the banking industry will only harm it, making it less effective to all of our detriment; these banks are too important a part of our economy to be allowed to fail; and bankers must continue to receive bonuses for banks to remain competitive.
It’s true that the banking industry has seized upon the process that’s the basis of our economic survival, and that attempts to address the problems of the banking industry cannot be undertaken lightly. But it’s also true that the banking industry has perverted that process, and that attempts to address that won’t prevent our return to some fantasy of efficiency and plenty, though they might prevent the otherwise inevitable, tragic end of the current trajectory left unchecked.
Whatever happens while the banking industry is rehabilitated is unlikely to be worse than what will happen as it continues to indulge in its addiction unaddressed, and it’s unlikely to function any worse upon the completion of its rehabilitation than it is now. As Charles Eisenstein puts it, “any efforts we make today to ‘raise bottom’ for our collectively addicted civilization–any efforts we make to protect or reclaim social, natural, or spiritual capital–will both hasten and ameliorate the crisis.”
Once an addict has reached the point in his or her descent where an intervention is necessary, there’s no realistic possibility of a return to some pre-addiction Golden Age. The apparent paradox that an addict’s life must be destroyed to save it is, stated in those terms, false. The addict’s life only appears to be as yet undestroyed through the lens of denial, and a future life without substance abuse or consequences is an illusion.
But the more gently stated paradox that intervention will cause the addict suffering in the short term to help him or her in the long term is accurate. There are, however, deeper, more intractable paradoxes, and they are those of the psychology of addiction. The process of intervention is often crucial to an addict’s entering rehab and beginning recovery, yet only the addict can decide to enter rehab.
The addict must understand the damage he or she has done in order to stop using but mustn’t succumb to shame, which would simply cause a retreat to the substance. The addict must admit that he or she is powerless over the substance and that life has become unmanageable, but mustn’t surrender to hopelessness and despair, which would sap the considerable motivation needed in the process of recovery. An intervenor must do something, but there’s nothing that can be done. There is no single act, no grand gesture or magic bullet, that can accomplish anything meaningful or lasting. Intervention is a long, unpredictable process requiring superhuman compassion and patience of everyone involved. Prior training or practice in commitment to a process without regard to the outcome of that process is invaluable.
Yes, we can answer the banking industry’s petulant invective in kind, but that won’t fix the problem; the industry will become more defensive and reckless, and we probably wouldn’t end up feeling any better anyway. Our encyclopedic harangue would be cogent, compelling, and convince our friends in the retelling, but no matter how loud we shout it over the banking industry’s coffee cup into that sullen, bloodshot face, it will simply be brushed aside with the wave of a shaky hand and a hoarse grumble, or, worse, it will hit home, and rattled, the banking industry will glare at us and we’ll know that tonight will bring another nihilistic binge of leverage and derivatives, and maybe this time there will be no tomorrow morning. The industry will tell us that we don’t understand, that the pressure it’s under is unimaginable, that life is grim, and that even though it can’t fix that, it should be thanked for what it has accomplished, and that that’s the best it can do. What more could we want? What more could it do? And we can only sigh and shake our head, because we know the simple, honest answer would just fall on deaf ears, and even if it were understood and accepted, the broken soul sitting across the table is in no shape to do anything constructive.
The confrontations shown on television or in the movies, or that you have perhaps participated in yourself, are just part of the larger process of intervention, but they illustrate the themes that inform that larger process. Those themes can best be summarized as connection: the connection between the addict’s choices and the suffering of the addict and those who are around him or her; the connection between addiction and the addict’s choices; and the unbreakable, always available connection between the addict and the intervenors.
Where denial seeks to divide and conquer, intervention seeks to unify and transcend. Intervention doesn’t respond to denial on denial’s terms, but rather reflects reality as it is. It doesn’t engage in the petty distractions of accusations and recriminations, nor does it seek escape from the addict and his or her problems. Intervention shows the addict his or her choices as they’re made, how those choices are determined by addiction, and the consequences that follow from those choices, but it also shows the redemption that’s always available despite those choices and their consequences.
Where denial is deceptive, impulsive, and selfish, intervention is clear, patient, and compassionate. Intervention finally presents the addict with an unavoidable choice between continued deluded suffering and real, sustainable sanity. The addict may or may not respond positively to that choice, but it must continually be presented on the same terms until the addict surrenders his or her denial.
And to induce that surrender, it’s crucial that the addict be offered an alternative to his or her addiction, whether it’s formal rehab, a twelve-step program, methadone, or a recovery dog. It’s important to recognize that even before the addict became physically or emotionally dependent on the substance, that substance met an otherwise unmet need, and leaving it unmet will lead only to relapse.
Tomorrow: Conclusion









