Archive
Declaration of Linear Independence: the nerdiest thing you’ve ever seen
My friend Michael Thaddeus recently informed me of the existence of the Declaration of Linear Independence, written by mathematician David Grabiner. I will describe the document as a “re-imagining” of the original Declaration of Independence from the point of view of a set of vectors in some vector space which feel, for whatever reason, that their independence has been under attack (I’m considering inviting them to join Occupy).
I’m not really sure I can ethically ask you to read the entire document, due to the intense nerdiness of it which may cause the weaker among you to lose consciousness, but let me give you the flavor. Here’s the most famous sentence translated into vector-angst:
We hold these truths to be self-evident: that all nonzero vectors are created equal; that they are endowed by their definer with certain unalienable rights; that among these are the laws of logic and the pursuit of valid proofs; that to secure these rights, logical arguments are created, deriving their just powers from axioms; that whenever any argument becomes destructive of these ends, it is the right of the vectors to alter or to abolish it, and to institute a new argument, laying its foundation on such principles, and organizing its powers in such form, as to them shall seem most likely to reach the correct conclusion.
Whereas the original document listed grievances against King George III, this new one complains about Professor Eigen, who is a made-up guy personifying everything which is overbearing and repressive about eigenvectors, eigenspaces and eigenvalues. Here’s my favorite complaint:
He has restricted our freedom of movement by requiring us all to live in the same hyperplane, even though we cannot all fit in one.
Finally, the ending is really quite good for those of us who on the one hand remember our linear algebra and on the other hand sympathize with these vectors being denied their (linear) independence rights:
Online learning promotes passivity
Up til I took Andrew Ng’s online machine learning class last semester, I had two worries about the concept of online learning. First, I worried that the inability to ask questions would be a major problem. Second, I worried about the possibility of building up material. I could imagine learning a given thing online but the ability to sustain and build material over an entire semester seemed kind of unrealistic.
On the second point, I think I’m convinced. Andrew definitely taught us a real semester’s worth of stuff, and he built up a body of knowledge very well. I now communicate with my colleagues at work using the language he taught us, which is very cool.
On the first point about asking questions, however, I am even more convinced there’s a crucial problem.
I want to differentiate between two different kinds of questions to make my point. First, there’s the “I’m confused” type of question, where someone literally doesn’t get the point of something or doesn’t understand the notation or a step in an explanation.
One can imagine tackling this kind of question in various ways. For example, one can strive to be a really good teacher, which Andrew certainly is, or to explain things at a high level but shove the details into black boxes, which Andrew did quite a bit (somewhat to my disappointment, especially when linear algebra was involved). If neither of those two things is sufficient, and the class is really important and/or really common, one can imagine teaching a computer to anticipate confusion and to ask questions along the way to make sure the students are following, and to go back and explain things in a different way if not.
In other words, the first kind of “clarifying questions” can probably be dealt with by the online learning community over time.
But there’s a second kind, namely the kind of question where someone is not confused but rather asks a question for one of the following reasons:
- they want to know how a certain idea relates to something else they know about,
- they want to generalize something the teacher said,
- they want to argue against an approach or for another approach,
- they see a mistake, or
- they see an easier way to do something.
Almost by definition, the above kinds of questions aren’t anticipated by the teacher, but the fact that they are asked almost always improves the class, certainly for the student in question but also for the other students and the teacher.
For example, one semester I taught three sections of 18.03 (exhausting! and I was pregnant!), which is a calculus class at M.I.T., and I remember thinking that in every single class one of the students made a remark or asked a question that I learned something from. It got to the point that, the third time through the same material, I’d be waiting for someone to explain how I should be teaching it. I loved that the students there are so smart but also so engaged in learning.
And that’s what I’m worried about- the engagement. When you embark on an online class, the best you can hope for is that you learn something and that you don’t get hopelessly confused. And that’s cool, that you can learn something, for free, online. But what you can’t do is what I’m worried about, and that’s to get instant feedback and discussion about some idea you had in the categories above.
I’m definitely one of those people who asks questions of the second type, and although I may sometimes annoy my fellow students, I really feel like the active engagement I pursue by coming up with all sorts of crazy comments and ideas and questions is what made me capable of doing original and creative things. For me, the most important part of my education was that training whereby I got to ask questions in class and got smart teachers who liked me to do so and would talk to me about my ideas.
How can that possibly happen with online learning? I’m afraid it can’t, and I’m afraid we will be training people to receive information rather than to engage in creation.
I imagine that in 200 years, almost everyone will be taught online, hooked into the machine and pumped up with knowledge. It will be only the elites who will have access to real live people to teach them in person, where they will be taught not only the material but also how to argue against a point of view and to propose an alternate approach.
Fox News fabricates a part of Obama’s speech
This is a guest post by Michael Thaddeus.
When President Obama spoke at Lorain County Community College in Elyria, Ohio, on Wednesday, he said, “Somebody gave me an education. I wasn’t born with a silver spoon in my mouth. Michelle wasn’t. But somebody gave us a chance.” [Minute 9:24 on video.] He has made similar remarks numerous times, including as early as 2009.
But when smirking reporter Steve Doocy quoted the President to Mitt Romney on Fox News, he added three words: “Unlike some people, I wasn’t born with a silver spoon in my mouth.” [Minute 3:39 on video.]
Those three words, “unlike some people,” were a complete fabrication. President Obama never said them or anything like them. The extra words make the President sound snide, as if he were mocking Romney.
Where did these extra words come from? Steve Doocy seemingly made them up out of whole cloth. Are reporters really supposed to do that?
What happened next? Philip Rucker at the Washington Post “reported” the story on Thursday, but he made no effort to check the fabricated quote against the primary sources, easily available online. Instead, he put Fox’s words directly into the mouth of President Obama. Are reporters really supposed to do that? I e-mailed him and the Post editors to request a correction, but he hasn’t answered, and guess what, the false quote is still there.
Update: the Washington Post has made a correction.
Then what happened? The New York Post devoted one of its two Friday editorials to slamming Obama for taunting Romney. They called him “cynical,” “misguided,” and “snotty.” Well, of course he sounded snotty! That’s because the Post used the snotty quote concocted by their colleagues at Fox News! Are newspapers really supposed to do that?
When I pointed this out to the editorial staff at the Post on Friday, they replied, “we’d be happy to consider running a letter to the editor on this subject, if you’d care to write one.” Great! But what’s the catch? “I couldn’t guarantee that we could run it.” What odds do you give me? Meanwhile, even though a prominent editorial in the Post is devoted to denouncing the President for saying something that he didn’t really say, there seem to be no plans for a correction or retraction.
So there you have it. One branch of the Murdoch empire concocts a snotty quote, supposedly from Barack Hussein Obama. Another branch vilifies him for supposedly saying the snotty thing that they themselves concocted. Meanwhile, the fabricated quote continues to reverberate in the echo chamber of the right-wing blogosphere. And thanks to the Washington Post, it will soon be as good as true.
Let’s grant that these three little words are a petty mendacity by the Iraq War standards to which we’ve become accustomed. And let’s grant that Obama’s speechwriters are shrewd and were hardly unaware of the contrast with Romney when they wrote the “silver spoon” line. Still, what makes Murdoch newspapers and TV stations think they can fabricate quotes, enclose them in quotation marks, attribute them to the President of the United States, and get away with it? It’s pretty shocking when you think about it.
Occupy Wall Street isn’t dead
I went to New Jersey a couple of nights ago to talk about Occupy Wall Street and the Alternative Banking group to NJPPN, a network of politically aware and active citizens. They self-describe as non-partisan but there were quite a few NPR listeners in the audience, and in general they came across to me as very skeptical of the financial system. Or possibly they were just being polite.
One of the audience members expressed frustration that the Occupy movement has fizzled out. I guess I can understand why he may think so, because after Bloomberg cleared the Occupiers from Zuccotti Park it was obviously more difficult for people to know what the movement is up to. And what with the cold weather, many of the working groups, like Alternative Banking, were incubating ideas rather than staging street protests. Plus the movement is still less than a year old, and these things take some time to set up.
For him and for others like him, I’d like to point you to a few resources to which explain what Occupy has been up to and what it has in store:
- occupydidwhat.tumblr.com – a recently begun list of stuff that Occupy has accomplished. Cool idea.
- Occupy.com – a gorgeous new website of news for Occupy.
- The Center for Popular Economics is having a Summer Institute here at Columbia called Economics for the 99%.
- Lots of plans for May Day described here. See you there.
- As I’m sure you know, people are sleeping on Wall Street.
- Alt Banking’s blog is fairly regularly updated as well.
More to come. The hoodies are being shipped as I type.
What’s fair?
Lately I’ve been thinking about the concept of fairness and how our culture decides on what’s fair. I think lots of arguments I have with other people come down to the fact that we have fundamentally different opinions on what’s fair, so I think it’s useful to consider having that argument instead of whatever argument we were engaged in. By the way, this actually makes me like people more- it’s not that they are mean, selfish people, but that they have a different underlying theory of fairness that they are loyal to.
For example, I have met people who claim that the government should only be in charge of protecting ownership rights and prosecuting criminals and that it should stay out of every other realm. The question of how to help people out with student debt loans then is certainly moot until we first talk about whether government should “care” about helping people at all for any reason.
The question, stop, and frisk policy is an example of a policy that our local government has taken on that reflects our shared understanding of fairness; in this case, we care more about preventing crimes, so being fair to victims, then we do about the suspects of crimes.
Tax law is another issue where we, as a society, have decided what’s fair and made it into policy. The fact that these laws change drastically over time – the top tax rate of 70% just a few decades ago is a far cry from what we’ve been seeing recently – indicates that we also change our mind about what is fair depending on conditions.
I’m not saying anything deep here- we all know that things change, and we no longer spend time watching slaves get killed in an arena, because it no longer jives with our concept of justice (although the NFL can sometimes seem a bit like that). I’m just trying to differentiate, and have other people agree to differentiate, between the rules we’ve constructed, in the form of policies and laws of the land, and the underlying and evolving moral decisions that we make as a community.
One more example, because I think it’s a good one for thinking about fairness and systems of rules (again not new). Imagine we have 100 people working on a farm, making their living, and we introduce a technology that allows 1 person to now do the work that 100 people did previously.
On the one hand it’s in some sense fair to keep one person on the farm, someone who is skilled enough to use this new tractor or whatever it is, and lay off the other 99 people.
But in a larger sense we still have the same output, so the same number of resources, and 99 people out of work means 99 people don’t have access to those resources, which doesn’t actually seem so fair. In the best of worlds (a world of textbook economic growth) those 99 people would go find new jobs in new fields and we wouldn’t have to worry about them. But what if those new jobs don’t exist, or exist only for the 23 people who have some other technical skills? This is when the rules we have created really matter, and our reasons for them need to be weighed and discussed.
Powerpoint kills me from within my soul
If you are anything like me, the beginning of a meeting where there are powerpoint slides is physically painful. I’m a napper, too, so especially after lunch, the urge to put my head on a conference table and start snoring is overwhelming.
Because I know what’s going to happen.
Namely, there’s gonna be waaaay too much stuff on each slide and there’s going to be a speaker who is really proud of their soul-wizening presentation.
My eyes glaze over when there are sub-bullets and small fonts, and especially when the slide is sectioned off into subslides.
Why is this allowed to happen?!
People. If there are more than three ideas in your slide, that’s too much. If there’s more than a title and three phrases, that’s too much. If any of your phrases is longer than the line and wraps around, that’s too long, and your font should be really big so everyone can read it.
My preference is to have exactly one phrase on each slide. Otherwise everyone in the room is reading shit the speaker hasn’t gotten to. Except for the people pretending not to be asleep, who are totally disengaged and/or praying to die.
Reputational risk is insufficient for ratings agencies
I’ve had a few conversations recently with intelligent, informed people about the failure of the ratings agencies during the housing bubble to keep up standards on their ratings of CDO‘s. You can discuss all day whether it was individual ratings they got wrong (at the level of the MBS‘s) or whether it was the correlation of defaults they were underestimating. It was both. But in the end the fact is they sold AAA ratings.
Nobody really argues against that. What fascinates me, though, is that people sometimes still argue against the idea that the revenue model of ratings agencies, whereby the issuers of debt pay the ratings agencies for ratings, is fundamentally flawed.
Their explanation is something like this.
That system worked fine for a long time, because for a given rating they wouldn’t sacrifice their reputation on a ridiculous rating for some small-fry issuer. And the system would have continued to work fine except that the issuers became huge and the amount of money involved became too tempting and so they ended up whoring themselves. But there’s nothing fundamentally wrong with the incentive system, we just need to keep reputational risk the driving force.
What?? That’s argument kind of reminds me of the so-called dental insurance which pays for cleanings but when it comes to dental emergencies with root canals and surgeries you’re shit out of luck. That’s not insurance at all, in other words.
I see the need for ratings agencies – it’s a way of crowd sourcing due diligence, which makes sense, but only if we can trust the ratings agencies as an impartial third party. And I don’t want to seem like someone who doesn’t have faith in humanity, but my trust isn’t won by a system of perverse incentives that has already failed. Let’s just say I have hope for humanity but I also acknowledge our weaknesses.
And just to be clear, the new bond rating agency Kroll, which has been getting a lot of attention, also uses an issuer-pays revenue model. But I guess Egan-Jones doesn’t, it uses a subscription-based revenue model. I still prefer the concept of an open source ratings agency – I’ve been in touch with Marc Joffe, who is doing just that for sovereign debt, which I will talk more about in a separate post.
Can clouds think?
Sometimes I have trouble falling asleep. Especially if I’m riled up thinking about the newest stealth bank bailout, or wondering how to model rare, fat-tailed events, I’ll toss and turn, unable to get these problems out of my head.
Luckily I have a husband who is kind enough to tell me his stories at moments like these. I really appreciate his ability to draw out a story. He starts out slowly, and gets slower. He ends up at such a leisurely pace that I get completely distracted from my work-a-day concerns simply wondering what he’ll say next, when he’ll say it, or if he’s just fallen asleep.
It’s not just the slowness of the stories that do the trick, either, it’s also the content. He’s the master of the boring relaxing, abstract, science-fictiony story with exactly one idea. He’s seriously considering starting a blog for his stories which he’d call `Stories that put my wife to sleep’. I honestly think it would work great for lots of people- a public service, really, especially if he made very very very boring podcasts.
It’s efficient too, he’s mentioned to me that he’s told me the same story sometimes 5 or more times but I can never last through to the point of understanding the plot, and it always seems new. I never know what’s going to happen next, if anything.
My favorite story, which I have probably heard 17 times, is the story of whether clouds can think. It’s unresolved, the answer, but it’s wonderful to imagine, very slowly, the decisions a cloud could make, things like very slight changes in its luminosity or which winds to take rides on or how high to fly.
The Great Wealth Transfer, late 1900’s to early 2000’s (part 1)
When historians write about this era of U.S. history, how will it be described? I have a guess: “the Great Wealth Transfer” from the middle class to the wealthy. Let me explain why I say this.
There are lots of different parts to this story, but today I’ll concentrate on the housing wealth transfer.
We all know there was a housing bubble, that millions of people took out mortgages on dubious terms for houses that were already overpriced but that they were each counting on to go even higher. The way this was sold at the time, and even today is described, was as “home ownership for an expanded middle class.” But as Sue Waters from the Alt Banking group pointed out to me, these people didn’t get home ownership, they got debt ownership.
I know, it sounds a bit strange, but that’s just it, the language is important.
When people say they own their home, do they mean they don’t have a mortgage? Probably not. They probably mean they’re in the process of paying a mortgage, but they conflate the two concepts because they assume they will pay off the mortgage eventually. But in the meantime they don’t actually own their home, the bank does. The extent to which this is an important distinction is the extent to which it is likely that they will be able to pay off that mortgage some day in the future.
When you’ve stopped conflating home ownership with debt ownership, and you look back at the housing bubble, it’s a different picture. How many new home owners were there really? It’s not an easy question to answer, but it’s clear that there were way fewer than we thought- many of the mortgages had terms that were clearly very optimistically written. Nobody really thought it would work out well, but the system just kept growing and the optimism kept getting less reasonable. Meanwhile, bankers got extremely rich.
How did this all happen?
This is answered by asking an even larger question: how does the financial system make money? I claim a large part of it is by finding a group of people that are relatively naive and pushing risk to them. For example, the dot com bubble was created by getting normal people to invest in dumb new-fangled things – they were the pawns.
For the housing bubble, it was a bit more devious. One one end, systemic risk was pushed (into the future) to the taxpayers themselves through bailouts of the banks, AIG, Fannie, and Freddie. In other words, taxpayers didn’t know it at the time but they were getting more and more on the hook for losses as the banks and financial system took larger and larger bets on the direction of the housing market.
At the same time and at the other end of the mortgage contracts, the so-called “homeowners” who took on mortgages were the fall guys. As a whole, they signed up for debt (and the right to claim themselves as homeowners) and in return are now hopelessly underwater. The Obama administration, just like the Bush administration before it, is urging these people to do what they are morally compelled to do, namely pay off their unmanageable debts, while changing the laws for the big banks so they can get away with whatever they need to in order to ignore their outrageous undercapitalization.
To sum it up: we found a very large pool of people too naive to understand the risk they were taking on, we signed them up for that risk while painting them a beautiful picture of the American dream, and now we get to accuse them of being immoral for not being able to hold up their end of the contract. It was an amazing swindle.
To be continued in part 2, in which many of the same players who brokered the mortgages to the “new homeowners” are now buying up their foreclosed homes and renting them back.
Get overpaid so people will listen to you
Have you read the recent article in the New York Times about how lower-status monkeys are less healthy and more stressed out than higher-status monkeys? Their gene expression actually responds to changes in social status. Does this resonate with your experiences with humans?
It does with me, and for us people I’d rephrase it this way: your concerns and ideas are given attention in direct relation to your status. Your stress levels rise as you realize your status is lower and your risks have grown.
Here are some examples from work. I’ve been disappointed to notice, time after time, that my ideas are considered important and innovative in direct proportion to how much they are paying me to have them. If I’m underpaid then nobody thinks I am all that smart; nevermind being a friggin’ volunteer (with some exceptions, but don’t stop me, I’m on a roll). This perversely makes me want to get overpaid just so I’ll get listened to.
Cuz why? Didn’t you ever notice that overpaid people’s ideas are about as good as anyone else’s but they are framed as pure brilliance? I have. It even works head-to-head: two people of different status come up with the same exact idea but the one who is more important was listened to and their idea championed. Oh yeah, I’ve seen it, and so have you (example: when I was at D.E. Shaw, we rated other people’s ideas with a “probability of success” in an effort to estimate their expected payouts; someone once showed me their idea, which was identical to one of Larry Summer’s ideas, but had come 2 years before and had scored about half as well. But my fried wasn’t an MD making $5 million per year so clearly his ideas weren’t as good!).
A similar thing happens with problems rather than ideas in a workplace. The worst examples of over-worked and under-appreciated situations clearly don’t happen at the top. For example, when I worked at MSCI, it seemed like the sales guys, who defined the top there, spent more time strutting around making sure each and every one of their efforts went appreciated than doing the actual efforts, whereas the lowly dev-ops guys, and the guys setting up the initial portfolios for the new clients, were treated as an afterthought, only noticed if something went wrong. They’d stay up all night fixing something, probably someone else’s mistake, and nobody would even thank them.
[It still seems so ironic that the most technical people there are also the least appreciated, since the product is essentially technical expertise. Or is it? Maybe I’ve got it wrong, and it’s really about selling technical expertise in a package that makes people feel safe and pious. Maybe the black box we’re selling doesn’t even have to work.]
If you are thinking that everyone at MSCI is in finance and is thus overpaid and pampered, then you’ve got it wrong, it’s a brutal atmosphere, like much of finance. If you don’t believe me, read my friend Katya’s blog, Left with Balls, where she talks about the spell of Wall Street.
Taking one step back, this kind of thing strikes me as unfair and frustrating. The idea that the lowest-ranked also has to deal with ridiculous stress and chronic health problems does not jive with my inherent concept of justice. Although it does seem like a natural response to a system that’s already been created (as in, as a consequence of being frustrated because my ideas are ignored, I want to get overpaid to get listened to, so I’m joining in on the perverted game and furthering the system), it doesn’t seem like we’ve done a particularly good job setting up these systems.
For a country that putatively considers itself a democracy, we seem to have a tremendous amount of respect for a rat-race corporate hierarchy. Is that a contradiction? Or is the American dream actually to start a hierarchy and to sit at the top? Do other people identify with the guys on the bottom or the guy at the top? Or the guys in the middle clawing upwards?
Question: is it really impossible to listen to and evaluate ideas based on their merit? How about anonymous polling of problems? It’s certainly technologically feasible, but we don’t do it.
Question: Is it really impossible to appreciate people who make things work behind the scenes? How about we ask people to sit with other people in entirely different departments in a rotation to witness what other people actually do? I really think that would help with the appreciation problem (but not if the technical people in your company are in India and the salesguys are in New York).
For the record, when I start a company I’ll do these things. Of course, I’ll be sitting up there at the top thinking what a great idea I had to do them.
#OWS update: looking for UX help
I’ve got three updates on Occupy, besides reminding people that the Alternative Banking Working Group meets every Sunday from 3-5pm at Columbia (room 1401 in the International Affairs Building at 118th and Amsterdam).
- Occupy.com has launched! This is a website set up by my friend David Sauvage, and it’s looking awesome and informative.
- The “find a Credit Union webapp” is looking for UX Designer help. I’ve written about this project before, but in a word we’re helping people figure out which credit unions they are eligible for, if any; the rules can get kind of tricky. We’ve got the basic ideas down but we’d love a thoughtful designer to come in, improve the user experience, and help create a appropriate Occupy look which also doesn’t scare away non-Occupy people. We also have a development team from ThoughtWorks helping us out, but it would be very helpful to have a New York- based developer to maintain the knowledge. The eligibility rules based on address (but not necessarily on zipcode or borough) are particularly hard (or interesting, depending on how nerdy you are).
- Not a strictly Occupy issue but did you hear the Vermont Senate has voted to end Corporate Personhood (hat tip reader G. Jones)? Move to Amend has spearheaded this effort. I love their motto: End Corporate Rule, Legalize Democracy. Read about the Vermont vote here.
Should we have a ratings agency for scientific theories?
Recently in my friend Peter Woit’s blog, he discussed the idea of establishing a ratings agency for physics. From his blog:
In this week’s Nature, Abraham Loeb, the chair of the Harvard astronomy department, has a column proposing the creation of a web-site that would act as a sort of “ratings agency”, implementing some mathematical model that would measure the health of various subfields of physics. This would provide young scientists with more objective information about what subfields are doing well and worth getting involved with, as opposed to those which are lingering on despite a lack of progress.
Abraham Loeb was proposing to describe the field of String Theory as a perfect example of a bubble. And it’s absolutely true that String Theory has provided finance with tons of brilliant young orphans who either got disillusioned with the field or simply couldn’t get a job after writing a Ph.D. or after a post-doc. It provides an extreme example of a mismatch between supply and demand.
Would a ratings agency for scientific theories help? I don’t think so.
The very basic reason, as Peter points out, is that it’s hard to evaluate scientific theories while they are unfolding. There are two underlying causes: first, people in a field are too invested to admit things aren’t working out, and second, by the nature of scientific research, things could not work out for some time but then eventually still work out. It’s not clear when to give up on a theory!
Ignoring those problems, imagine a “mathematical model” which tries to gauge the success of a field. What would the elemental quantities be that would signify success? Would it count the number of proven theorems? Crappy theorems are easy to prove. Would it count the the number of successful experiments? We could always take a successful experiment and change it ever so slightly to get another success. I can’t think of a quantitative way to measure a field that isn’t open for enormous manipulation (which would only happen if people actually cared about the ratings agency’s rating).
Of course the same might be said about financial ratings. It begs the question, why are ratings agencies useful at all?
In finance we have lots of people buying very similar products with very similar contracts. Sometimes these are even sold on exchanges and carry with them the exact same risk profiles. In such a situation it makes sense to assign someone to look into the underlying risks and report back to the community on how risky a product is.
I would claim that the situation is very different in science or math. People enter a field for all sorts of reasons, with all sorts of goals and situations. String Theory is an extreme case where it could be argued that it got such spin that a whole generation of physics students got sucked into the field by sheer momentum. Perhaps it would have been nice to have a trusted institution whose job it was to calm people down and point out the reality, but I’m not sure it would have helped that much with all the excitement, especially if there had been a model which counted theorems and such. People would just have said the model had never seen something this exciting.
Then there’s the issue of trusting the modeler. Right now ratings agencies have a terrible reputation because they are paid by the people they rate products for, and have been known to sell good ratings. I’m hoping we can do better in the future, but it’s hard (but not impossible!) to imagine gathering enough experts in finance to do it well and to have the product be trusted by the community.
What is the analogy for scientific theories? The problem with rating science is that, because of the depth of most fields, only the experts in the field themselves understand it well enough to even talk about it. So that problem of getting an informed and impartial view on the worthiness of a theory is super super hard, assuming it’s possible.
Finally, I’m not sure what the ratings agency would be in charge of warning people about. Even the financial ratings agencies don’t agree- some of them measure default risk and others measure expected loss through default, which can be two really different things (for example if you think the U.S. will technically default but will end up paying their debts).
In science, I guess you could try to measure the risk that “the theory won’t end up being useful” but it’s not even clear how you’d decide that even after the fact. Maybe you could forecast the number of jobs in the field for graduating Ph.D. students, and that would be helpful to grad students but would also not be the best metric of success for the field.
I’m not saying we shouldn’t have people talk about fields and whether fields are failing, because that’s hugely important. But I don’t think there’s a quantitative model there to be created that would help the conversation. Let’s start an open forum, or a wiki, with the goal of talking about the health of various fields of scientific endeavor and have a bunch of good questions about the field and people can each add their two cents.
How to teach someone how to prove something
In a couple of my posts (most recently here), I’ve talked about the need for a course early on in undergraduate math classes on proof techniques.
The goals of the class are two-fold: first, teach the students basic skills, and second demystify the concept of proof. The students should come away from the class thinking, no it’s not magic, and I’ve learned how to do this stuff, and there are a few basic techniques which seem to come in handy.
Today I want to go further into what a curriculum for such a course might look like.
And I will, in a moment, but first I want to explain something. It’s actually a really important and dangerous question, how to teach such a course, because it could go wildly wrong, and sometimes does. From my commenter Jordan:
… “Numbers, Equations, and Proofs,” which I started at Princeton in 2002 and which is still going as well. Though here’s an interview with a dude who was an ace math competition dude and found the course so hard as to drive him out of the math major! So maybe it’s no longer as “for everyone” as I designed it to be….
This struck me, how perverted Jordan’s class became. For that matter, Math 55 at Harvard could have started out as a good idea as well, but by the time I got to Harvard as a grad student it was the reason so few math majors ever stuck at Harvard and why there were especially few women.
I remember Noam Elkies taught it while I was there and was famous for asking questions in class and getting students to compete to answer them quickly. It makes sense that he’d run a class like this, because he’s so fast and clever, and he’s naturally wondering, am I the fastest and clevererest of them all? But rather than a place where proof is demystified and people feel safe asking dumb questions, he’d created the polar opposite, a live quiz show of clever competition. Ew!
In order to combat this downfall and decay, I think the class needs to have a clearly stated mission as well as built-in curriculum requirements that works against ostentatious displays of cleverness, which indeed only serve to further the “I got it but you don’t” stereotype of math skills (but which mathematicians themselves are incentivized to further since that magical aura comes in handy).
For example, when I taught it, I let the students hand in homework again and again until they got a score they liked. Of course, this depending on me having an awesome grader (and a relatively small class), which luckily I had.
Also, I asked each student to give a presentation to the class on some proof they particularly enjoyed, and I sat through a preview of their presentation and gave them extensive advice on board work and eye contact, which took a lot of work but really helped them prepare and also boosted their egos while at the same time increased their sympathy with each other and with me.
But of course the most important thing was that I clearly stated at the beginning of each class in the first two weeks that proving things in math was a skill like any other that you get good at through practice. And when I left Barnard Dusa McDuff took over the class and still teaches it, so I know it’s in good hands.
If I hadn’t had Dusa, I’d probably have written a manifesto to be given to each person who would teach the class after me. Of course anyone could have just thrown that away but it’s an idea.
As for content, I taught them really basic proof techniques, so induction, proof by contradiction, the pigeon-hole principle, and some epsilon-delta practice. We covered some basic logic, graph theory, group theory, ordinals, and basic analysis. We constructed the reals two ways and the complex numbers once and talked for a long time about whether “i” is real and what that even means. We used A Transition to Higher Mathematics, which I recommend with a few reservations (please tell me if you’ve found a better text for something like this!).
Everything was done super explicitly and carefully, no rushing. I said things three times in three different ways. I wasn’t expecting people to be fast or clever, because I know intelligence works in different ways and that this stuff was completely new to most of the students. And at least one student in the class, who had been an artist, is now a grad student in math at Berkeley.
Looking over my post I realize I spent way more time talking about the tone of the class than the content, but that’s totally appropriate, since I think of this class as an introduction to the culture of mathematics (or rather the culture I wish we had) just as much as mathematics itself.
After all, there really is no time limit on good ideas, and you do get to do it over if you make a mistake, and going over things slowly gives you more time to ask good questions and find mistakes.
Calling all data scientists! The first ever global data science hackathon
Hey I’m helping organize a NYC data hackathon at Bloomberg Ventures to take place April 28th – 29th, from 8am Saturday to 8am Sunday. I’m looking for outrageously nerdy people to come help. There will be some prizes.
Read the official blurb below carefully and if you’re in, sign up for the event by registering here.
Update: they’ve decided on prizes.
See you there!
——————————————————————————————————————————————–
Are you a smart data scientist? Participate in this hackful event. 24 hours of non-stop, fun data science competition. The first ever global, simultaneous data science hackathon!
In connection with Big Data Week, we’re helping organize a global data science hackathon that will simultaneously take place in various locations around the world (including London, Sydney, and San Francisco). We will host the NYC event at the Bloomberg Ventures office in the West Village.
The aim of the hackathon is to promote data science and show the world what is possible today combining data science with open source, Hadoop, machine learning, and data mining tools.
Data scientists, data geeks, and hackers will self organize around teams of 3-5 members. Contestants will be presented with a ‘big data’ set (hosted on the Kaggle platform). In order to win prizes, the teams will have to use data science tools and develop an analytical model that will solve a specific data science problem specified by the judging tech panel. The contestants will have to report their achievements at specific milestones, and a leader board will be published online at each milestone.
The contestants will spend 24 hours in Bloomberg Ventures’ office space where food, drinks, workspaces, and resting areas will be provided. Teams will compete for both local and global titles and prizes.
The Hackathon runs for 24 hours starting on April 28th at 8am (early start to allow for the event to happen simultaneously in multiple time zones around the world).
If you have questions, please email shivon.zilis@bloombergventures.com
PLEASE READ CAREFULLY:
1. This is a technical competition, not a networking event or an opportunity to learn more about big data techniques and technologies. We have limited space, so we unfortunately need to be strict about who gets to compete. If you’re an entrepreneur looking to recruit, we’re excited to have you as a member of this community, but this specific event is not the right venue, please come our regular Data Business meetups instead! 🙂
2. You should have Mad Skillz at at least one of the following:
- Data grappling and/or cleaning,
- Data modeling and forecasting,
- Data visualization,
- Spontaneous micro- and macro-economic theory creation
3. You should know one or more of the following languages:
- R
- python
- Matlab
- Some statistical package like SPSS or SAS
4. You should bring your hardcore laptop to the event, since we will have on the order of 10 gigs of data to play with.
In which mathbabe becomes insurance claims adjuster
Who knows what I’m talking about with this story.
My husband dislocated his finger sledding with my son last January, so more than a year ago, and the hospital kept sending us bills for the event.
But here’s the thing, we were covered under my medical insurance, which had perhaps recently changed policy numbers when MSCI took over Riskmetrics. So probably what had happened was my husband had given them the old insurance card, but in any case, in the the end I knew I wouldn’t have to pay since we’d definitely been covered.
The hospital called once a month or so, and every time they got hold of me I argued with them and told them to check their records. They kept telling me that the insurance company was refusing payment under any of the policy numbers I gave them.
In the end, last month, I called up the insurance company myself and got them to admit payment, which wasn’t hard since they said they’d already paid for the X-ray from the dislocation on that date. I called up the hospital and straightened it out.
So yeah, I ended up doing their job for them, and that’s both annoying and exciting because now nobody thinks I owe them $2400. In fact I did a victory dance (at work, because you always have to do this during work hours for people to answer the phone).
But why I’m writing about it today is that it’s actually really infuriating how often something like this happens, and I can’t help noticing that I always get out of it but many people wouldn’t. I’m at a huge advantage in this common situation because:
- I worked as a customer service person so I know how to talk to customer service people. Turns out you should always be polite, but never hang up the phone until your problem is solved. Just keep asking, extremely nicely, things like, “Hmmm… that’s confusing, what do you think could have gone wrong?” or “What would you do if you were me?” or if those don’t work, “Do you think you could tell me who to talk to sort this out? I’d really appreciate it.”
- I am always covered by insurance, so I never worry that they are right. This is an enormous advantage over people who sometimes lose coverage between jobs or something.
- I keep all my old paperwork. Impossible for people who don’t have an incredibly
boringstable lifestyle like mine. - I have a job that allows me to make calls like this during work hours. Obviously huge.
- I am completely unafraid of forms and red tape. This comes from experience, but I know most people are afraid of such stuff, and that alone would probably keep most people from arguing.
I really do feel like I am relying on my professional skills in order to get my insurance to pay for setting my husband’s dislocated finger, when that should be a no-brainer. If you are inexperienced and poor, you’d probably be completely at a loss for how to deal with this situation.
I wonder how many people have their credit scores lowered by medical claims which should have been paid but weren’t due to crap like this. It’s a broken system, but it only leaks on the most vulnerable people, and I hate that.
Continuously forecasting
So I’ve been thinking about how to forecast a continuous event. In other words, I don’t want to forecast a “yes” or a “no”, which is something you might do in face recognition using logistic regression, and I don’t want to split the samples into multiple but finite bins, which is something you may want to do in handwriting recognition using neural networks or decision trees or recommender systems.
I want essentially a score, in fact an expected value of something, where the answers could range over the real numbers (but will probably just range over a pretty small subset but I don’t know exactly what smallish subset).
What happens when you look around is that you realize machine learning algorithms pretty much all do the former, except for various types of regression (adding weights, adding prior, nonlinear terms), which I already know about from my finance quant days. So I’m using various types of regression, but it would be fun to also use a new kind of machine learning algorithm to compare the two. But it looks like there’s nothing out there to compare with.
It’s something I hadn’t noticed til now, and I’d love to be wrong about it, so tell me if I’m wrong.
On the making of a girl nerd
Today I want to discuss the process by which girls become math and cs nerds.
I could be tempted to talk primarily about my own story, since I’m a huge nerd. And I will talk about my story, but my focus is going to be on the girls of my generation who could have become nerds but didn’t. I’m hoping we can learn some lessons so that future generations will have more nerd girls.
Both my parents are nerds. My mother has a Ph.D. in applied math and my father has a Ph.D. in pure math. Moreover, I was on the math team in high school, found out about a math camp, and went to it for two summers, with the full support of my family.
I want to go over these details again, because I want to point out that they gave me an enormous advantage to becoming a successful nerd.
First, my parents being nerds: I have found an amazing correlation between women with math Ph.D.’s and women whose fathers are mathematicians. I don’t think this is random- indeed I think it means two things. First, that girls with mathematician dads have an easy time imagining themselves as mathematicians (and an even easier time if their mom is too). Second, that girls without mathematician dads don’t. Otherwise you wouldn’t be able to explain the statistics I have.
Second, the math camp experience. I went to math camp in spite of it being an extremely uncool summer endeavor, according to my classmates at school. Yet I didn’t care, and went anyway, mostly because I was already a complete outsider, a fat girl on the math team (but a mathbabe when I got there!).
Two things about this. First, most smart girls around me in Lexington High School, and there were a lot of them, would not have been willing to go to math camp and ruin their reputations. Most of them were relatively popular, and wanted to keep it that way. I had nothing to lose in that aspect and knew it. This kind of thinking may seem silly to us as grownups but seemed like life or death choices then.
Second, the advantage having been to math camp gave me when I got to college was phenomenal. I knew how to prove things by induction, by contradiction, and using the pigeon-hole principle. I knew basic group theory, graph theory, and real analysis. This gave me a jump-start in all of my undergrad math major classes. I was an elite, and what I could do seemed like magic to the kids who were math majors who didn’t know that stuff.
The thing about math is that people get into this mindset about being good at it: they think that you either have it or you don’t (see this post for more on the mindset). So the experience for the other kids, boys and girls, going to an algebra class and sitting next to me and a few other kids from math camp backgrounds was understandably intimidating and made them think they couldn’t compete. But I believe that, considering the social constructs and the kind of confidence girls and boys are trained to have (or not have), it was particularly daunting for other girls to see their competition in a small group of elite nerds who already knew all the answers.
I’m not advocating closing math camps. In fact, I am going back to teach at my high school math camp in July for three weeks (woohoo!). What I am advocating is thinking seriously about the selection process for young nerds and how much it weeds out girls. We can do better.
For example, Harvey Mudd is doing better by careful thought and attention to the issue. Namely, they are changing the introduction to programming class to be more appealing for non-math-or-cs-camp nerds. From the New York Times article:
Known as CS 5, the course focused on hard-core programming, appealing to a particular kind of student — young men, already seasoned programmers, who dominated the class. This only reinforced the women’s sense that computer science was for geeky know-it-alls.
“Most of the female students were unwilling to go on in computer science because of the stereotypes they had grown up with,” said Zachary Dodds, a computer scientist at Mudd. “We realized we were helping perpetuate that by teaching such a standard course.”
To reduce the intimidation factor, the course was divided into two sections — “gold,” for those with no prior experience, and “black” for everyone else. Java, a notoriously opaque programming language, was replaced by a more accessible language called Python. And the focus of the course changed to computational approaches to solving problems across science.
This sounds like a brilliant idea, and one that we should all consider (and python rocks!). It is reminiscent of the “Introduction to Proofs” class which I started with Karen Edwards and Sara Robinson in 1993 at UC Berkeley as an undergrad and which is still going, as well as the class I started at in 2006 at Barnard College, which is also still going. The dual goals of such a class are to teach basic proof techniques to people interested in the major (who probably didn’t go to math camp) and to show people that being able to prove things isn’t magic, it just takes practice and knowing techniques.
Let’s get more campuses across the country to think about all the math and cs nerds they are missing out on by teaching the same old math (or cs) major classes every year. This is a curriculum change that is easy, fun to teach, and completely worthwhile.
Who is the market?
Oftentimes you’ll read an article in the middle of a market day about how “the market is responding” to the jobs report, or the manufacturing index, or sentiment reports. That kind of makes sense – it is shorthand for the fact that the people betting on the market are, as a group, reacting and changing their bets based on new news. If the expectation was for 200,000 jobs to be added but only 120,000 jobs were added, you’d expect disappointment and a drop in the S&P index.
Even so, this language is pretty confusing, since it’s certainly not true that everyone who invests in the market is doing this – most people with money in the market don’t do anything at all on a given day. Okay then, let’s interpret it as meaning something kind of reasonable like, “of those people who respond to this news by changing their bets, a majority of them are betting in one way which is moving the market.”
It still may not be true, since people who are seriously involved with the market typically don’t have the same expectations as what the official expectation report says – that report may have contained no surprising news at all, but one hedge fund liquidating their portfolio may be dominating the market. So even if there is a reasonable interpretation, the chances are it’s vapid.
Other times you’ll read an article, probably put out by Bloomberg, about how the market is “recalibrating,” or “taking stock” after a rise. This is where I get confused. It’s like I’m expected to imagine a huge man, hunched over thinking about what to do next.
But what does that really mean? As far as I can tell, nothing at all. There’s no man, there are no little men behind the wall representing this man, and everyone betting on the market is just doing their thing. It’s maybe just a way of writing a story because the journalist was told to write a story and the market wasn’t doing anything.
But lately I’m wondering if there’s something more to it. Why are journalists covering the market allowed, day after day, to write vapid articles about the market? What is it about using language like this that makes us comforted?
My guess is that people want there to be such a man, and moreover want him to be understandable and reasonable.
It’s primarily a question of control – control over our lives, as if we can say, as long as we kind of get his (the market’s) sentiments, we can avoid catastrophic risks. Like in those human nature tests where 85% of people consider themselves better than average drivers, we feel that we understand the market and so we’re covered and safe. Even when there’s plenty of evidence that we don’t actually understand the risks, we continue the market myth out of this need to feel in control.
I also think there’s another, secondary effect of this personification. Namely, we feel like the system is massive and powerful and there’s nothing we can do to affect it. It makes us passive.
My friend Hannah, who’s an anthropologist and whom I met through Occupy, likes to say to people, “that good idea you’ve had that someone should do? It’s your idea, and you should do it! There’s no Occupy elf that will go do it for you just because it’s a good idea.” I love that sentiment, and the idea of Occupy elves (why aren’t there Occupy elves?).
It makes me realize how much we expect other people to do stuff just because it’s a good idea, when in fact from experience we should have learned by now that the stuff that gets done by other people is usually because it’s a good idea for them. Stuff that’s a good idea for us, or for everyone, we should consider our personal responsibility. The market is certainly not looking out for us.
More creepy models
I’ve been having fun collecting creepy data-driven models for your enjoyment. My first installment was here, with additional models added by my dear commenters. I’ve got three doozies to add today.
- Girls Around Me. This is a way to find out if you know any girls in your immediate vicinity, which is perfect for my stalker friends, using Foursquare data. My favorite part is that the title of this article about it actually uses the word “creepy”.
- Zestcash is a cash lending, payday-like service that data mines their customers, with a stated APR of up to 350%. On of my favorite misleading quotes in this article about the model: “Better accuracy should translate into lower interest rates for consumers.” Ummmm… yeah for some of them. And I guess the idea goes, those other losers deserve what they get because they’re already poor?
- The creepiest of all by far (because it is so painfully scalable and I could imagine it being everywhere in 2 years) is this one which proposes to embody the “best practices” of medicine into a data science model. Look, we desperately need a good model in health and medicine to do some of the leg-work for us, namely come up with a list of reasonable treatments that your doctor can take a look at and discuss with you. But we definitely don’t need a model which comes up with that list and then decides for you and your doctor which one you should undergo. Decisions like that, which often come down to things like how we care about quality of life, cannot and should not be turned into an algorithm.
By the way, just to balance the creepy models a bit, I also wanted to mention a few cool ideas:
- What about having a Reckoner General, like a surgeon general? That person could answer basic questions to explain how models are being used and to head off creepy models. Proposed by my pal Jordan Ellenberg.
- What about having an F.D.A.-like regulator for financial products? They would be in charge of testing the social utility of a proposed instrument class before it went to market. Can we do the same for data-driven models? Can the regulator be kick-ass and reasonably funded?
- What about having a creep model auditing board that brings together a bunch of nerds from technology and ethics and looks through the new models and formally reprimands creepiness, using the power of social pressure to fend it off? They could publicize a list of creeps who made these models to call people out and shame them. That really doesn’t happen enough, it’s like the modelers are invisible.
- How about a law that, if you add a cookie to your digital signature that says, “don’t track me for reals”, then if you find someone tracking you, as in saving and selling your information, you can sue for $100K in damages and win?
mathbabe 