### Archive

Archive for the ‘rant’ Category

## Let’s experiment more

What is an experiment?

The gold standard in scientific fields is the randomized experiment. That’s when you have some “treatment” you want to impose on some population and you want to know if that treatment has positive or negative effects. In a randomized experiment, you randomly divide a population into a “treatment” group and a “control group” and give the treatment only to the first group. Sometimes you do nothing to the control group, sometimes you give them some other treatment or a placebo. Before you do the experiment, of course, you have to carefully define the population and the treatment, including how long it lasts and what you are looking out for.

Example in medicine

So for example, in medicine, you might take a bunch of people at risk of heart attacks and ask some of them – a randomized subpopulation – to take aspirin once a day. Note that doesn’t mean they all will take an aspirin every day, since plenty of people forget to do what they’re told to do, and even what they intend to do. And you might have people in the other group who happen to take aspirin every day even though they’re in the other group.

Also, part of the experiment has to be well-defined lengths and outcomes of the experiment: after, say, 10 years, you want to see how many people in each group have a) had heart attacks and b) died.

Now you’re starting to see that, in order for such an experiment to yield useful information, you’d better make sure the average age of each subpopulation is about the same, which should be true if they were truly randomized, and that there are plenty of people in each subpopulation, or else the results will be statistically useless.

One last thing. There are ethics in medicine, which make experiments like the one above fraught. Namely, if you have a really good reason to think one treatment (“take aspirin once a day”) is better than another (“nothing”), then you’re not allowed to do it. Instead you’d have to compare two treatments that are thought to be about equal. This of course means that, in general, you need even more people in the experiment, and it gets super expensive and long.

So, experiments are hard in medicine. But they don’t have to be hard outside of medicine! Why aren’t we doing more of them when we can?

Swedish work experiment

Let’s move on to the Swedes, who according to this article (h/t Suresh Naidu) are experimenting in their own government offices on whether working 6 hours a day instead of 8 hours a day is a good idea. They are using two different departments in their municipal council to act as their “treatment group” (6 hours a day for them) and their “control group” (the usual 8 hours a day for them).

And although you might think that the people in the control group would object to unethical treatment, it’s not the same thing: nobody thinks your life is at stake for working a regular number of hours.

The idea there is that people waste their last couple of hours at work and generally become inefficient, so maybe knowing you only have 6 hours of work a day will improve the overall office. Another possibility, of course, is that people will still waste their last couple of hours of work and get 4 hours instead of 6 hours of work done. That’s what the experiment hopes to measure, in addition to (hopefully!) whether people dig it and are healthier as a result.

Before I get too excited I want to mention the problems that arise with experiments that you cannot control, which is most of the time if you don’t plan ahead.

Some of you probably ran into an article from the Wall Street Journal, entitled Companies Say No to Having an HR Department. It’s about how some companies decided that HR is a huge waste of money and decided to get rid of everyone in that department, even big companies.

On the one hand, you’d think this is a perfect experiment: compare companies that have HR departments against companies that don’t. And you could do that, of course, but you wouldn’t be measuring the effect of an HR department. Instead, you’d be measuring the effect of a company culture that doesn’t value things like HR.

So, for example, I would never work in a company that doesn’t value HR, because, as a woman, I am very aware of the fact that women get sexually harassed by their bosses and have essentially nobody to complain to except HR. But if you read the article, it becomes clear that the companies that get rid of HR don’t think from the perspective of the harassed underling but instead from the perspective of the boss who needs help firing people. From the article:

When co-workers can’t stand each other or employees aren’t clicking with their managers, Mr. Segal expects them to work it out themselves. “We ask senior leaders to recognize any potential chemistry issues” early on, he said, and move people to different teams if those issues can’t be resolved quickly.

Former Klick employees applaud the creative thinking that drives its culture, but say they sometimes felt like they were on their own there. Neville Thomas, a program director at Klick until 2013, occasionally had to discipline or terminate his direct reports. Without an HR team, he said, he worried about liability.

“There’s no HR department to coach you,” he said. “When you have an HR person, you have a point of contact that’s confidential.”

Why does it matter that it’s not random?

Here’s the crucial difference between a randomized experiment and a non-randomized experiment. In a randomized experiment, you are setting up and testing a causal relationship, but in a non-randomized experiment like the HR companies versus the no-HR companies, you are simply observing cultural differences without getting at root causes.

So if I notice that, at the non-HR companies, they get sued for sexual harassment a lot – which was indeed mentioned in the article as happening at Outback Steakhouse, a non-HR company – is that because they don’t have an HR team or because they have a culture which doesn’t value HR? We can’t tell. We can only observe it.

Money in politics experiment

Here’s an awesome example of a randomized experiment to understand who gets access to policy makers. In an article entitled A new experiment shows how money buys access to Congressan experiment was conducted by two political science graduate students, David Broockman and Josh Kalla, which they described as follows:

In the study, a political group attempting to build support for a bill before Congress tried to schedule meetings between local campaign contributors and Members of Congress in 191 congressional districts. However, the organization randomly assigned whether it informed legislators’ offices that individuals who would attend the meetings were “local campaign donors” or “local constituents.”

The letters were identical except for those two words, but the results were drastically different, as shown by the following graphic:

Conducting your own experiments with e.g. Mechanical Turk

You know how you can conduct experiments? Through an Amazon service called Mechanical Turk. It’s really not expensive and you can get a bunch of people to fill out surveys, or do tasks, or some combination, and you can design careful experiments and modify them and rerun them at your whim. You decide in advance how many people you want and how much to pay them.

So for example, that’s how then-Wall Street Journal journalist Julia Angwin, in 2012, investigated the weird appearance of Obama results interspersed between other search results, but not a similar appearance of Romney results, after users indicated party affiliation.

Conclusion

We already have a good idea of how to design and conduct useful and important experiments, and we already have good tools to do them. Other, even better tools are being developed right now to improve our abilities to conduct faster and more automated experiments.

If we think about what we can learn from these tools and some creative energy into design, we should all be incredibly impatient and excited. And we should also think of this as an argumentation technique: if we are arguing about whether a certain method or policy works versus another method or policy, can we set up a transparent and reproducible experiment to test it? Let’s start making science apply to our lives.

Categories: data journalism, modeling, rant

## People who obsessively exercise are boring

I’m not saying anything you don’t know already. I’m just stating the obvious: people who obsessively exercise are super boring. They talk all the time about their times, and their workout progress, and their aching muscles, and it’s like you don’t even have to be there, you could just replace yourself with a gadget that listens, nods, and then says encouraging things like, “Way to go!” at the very end. Excruciating.

Look, don’t get me wrong. I’ve gone through bouts of obsessive exercise myself, and those bouts sometimes were pretty lengthy. And no, it didn’t ever make me skinny, just incredibly fit. I remember I trained for a sprint triathlon once, and man was I fit by the time it finally happened in the spring on 2004.

But then, when I got to the starting line, and there I was wishing I could reorder the events so the the beginning swim would 5 kilometers and the run at the end were a quarter mile – I’ve never been much of a runner – and I just looked around at myself and everyone else there, and I wondered how I’d become so incredibly boring and self-obsessed that I had paid good money and driven miles and miles just to obsessively exercise in front of other people.

What was going on with me? I became increasingly disgusted by my own boringness throughout the race. I think the worst part was how many people said “You go, girl!” when I jogged by. They were trying to encourage the fat girl, I get it, but it made it even more obvious that I was doing something that I honestly didn’t need to be getting public response to.

Look, I’m not against exercise, and I love doing it, or at least I love having done it because it makes you feel good, and I encourage everyone to be fit and happy. But I’m serious when I say I will no longer tolerate hanging out with people who obsess over it and want to talk to me about their obsession. Too frigging boring, people!

So if someone mentions that they went biking over this gorgeous spring weekend, then awesome, I’ll be happy for them. But if they want to talk about which bike they used, and what their time around Central Park was, and how they’re training for this or that event, then no. I will tell them “sorry but can we talk about something not incredibly boring now?”

Why do I mention this today? Because I finally figured out what my hostility towards the Quantified Self crowd is, and it’s this same thing. All those gadgets and doodads are essentially props to pull out and use to have that same boring conversation that I’ve already refused to give into. So please, don’t show me your sleep tracker or your step monitor and expect me to care. I don’t care.

And don’t get me wrong – again – I know some people will benefit from that kind of thing. And some people actually have illnesses or physical therapy and exercise and particularly quantified exercise might particularly help them keep track of their health! I get it!

But let’s face it, most people are not doing this for health. They are doing it for some other weird, narcissistic and anxiety-shielding coping-mechanistic self-competitive (or outright competitive) reason. And again, I’m not hating on them exactly, because I get it, and I’ve been there. But I don’t want to talk about it with them.

Categories: rant

## Let’s stop talking about HFT for a little while

It’s unusual that I find myself in the position of defending Wall Street activities, but here goes.

I just don’t think HFT is that big of a deal relative to other Wall Street evils. I have written a couple of times about HFT and I’m not a huge fan, and I don’t buy the “liquidity is good and more liquidity is better” argument: at some point enough is enough. I do think that day-to-day investors have largely benefitted from it but that people whose money is in massive funds which are regularly traded have seen their money get skimmed every month. Overall it’s a smallish negative tax on the average person, I’d expect.

Here’s why HFT deserves some of our hatred: there’s way too much human resources going into this stuff and it’s embarrassing, what with the laying of cables and blasting through mountains and such. And it’s a great sociological look into the absolutely greed-led mindset of the Wall Street trader, but honestly I think we already had that. It’s really business as usual at a microscopic scale, and nobody should really be surprised to learn that people will do anything to make money that’s technically possible and technically legal, and that they will brag about how they’re making the world a better place while they do it. Same old same old.

So I’m not saying HFT is awesome and we should encourage more of it. I’m all for thinking about how to slow down trading to once a second and make it “more fair” for more players (although that’s hard to do even as a thought experiment), or taxing transaction to make things slow down by themselves, which would be easy.

But here’s the thing, it’s not some huge awful thing we should focus on, even though Michael Lewis is a really good and engaging writer.

You wanna focus on something? Let’s talk about money laundering in HSBC and now Citi that is not under control. Let’s talk about ongoing mortgage fraud and robo-signing and the ongoing bailout/ taxpayer subsidy and people still losing their homes, and the poor still being the targets of illegal and predatory loans, and Too-Big-To-Fail getting worse, and the direct line between the bailout and the broken pension promises for civil servants and the overall price list for fraud that has been built.

Let’s talk about the people who created the underlying fraud still at work in places like Bank of America, and how few masterminds have gone to jail and how the SEC and the Obama administration has made that happen through inaction and passivity and how Congress is sitting on its hands because of the money coming in from lobbyists. Let’s talk about the increasing distance between the justice system for the poor and the justice system for the rich in this country.

Tell me what I missed.

The HFT noise is misplaced and a distraction from the ongoing real story.

Categories: finance, rant

## Navigating sexism does not mean accepting sexism

Not enough time for a full post this morning, but I’d like people to read a New York Times article ironically entitled Moving Past Gender Barriers to Negotiate a Raise (hat tip Laura Strausfeld). It has amazing and awful tidbits like the following:

“It’s totally unfair because we don’t require the same thing of men. But if women want to be successful in this domain, they need to pay attention to this.”

If you read on you realize that what they mean by “pay attention to” is “roll over and conform to stereotypes”. Super gross, and fuck that.

I feel like this is a more subtle, New York Times version of Susan Patton’s terrible advice for young women in snaring husbands. What happened to the feminists?!!

Categories: rant

## Lobbyists have another reason to dominate public commenting #OWS

Before I begin this morning’s rant, I need to mention that, as I’ve taken on a new job recently and I’m still trying to write a book, I’m expecting to not be able to blog as regularly as I have been. It pains me to say it but my posts will become more intermittent until this book is finished. I’ll miss you more than you’ll miss me!

On to today’s bullshit modeling idea, which was sent to me by both Linda Brown and Michael Crimmins. It’s a new model built in part by the former chief economist for the Commodity Futures Trading Commission (CFTC) Andrei Kirilenko, who is now a finance professor at Sloan. In case you don’t know, the CFTC is the regulator in charge of futures and swaps.

I’ll excerpt this New York Times article which describes the model:

The algorithm, he says, uncovers key word clusters to measure “regulatory sentiment” as pro-regulation, anti-regulation or neutral, on a scale from -1 to +1, with zero being neutral.

If the number assigned to a final rule is different from the proposed one and closer to the number assigned to all the public comments, then it can be inferred that the agency has taken the public’s views into account, he says.

1. I know really smart people that use similar sentiment algorithms on word clusters. I have no beef with the underlying NLP algorithm.
2. What I do have a problem with is the apparent assumption that the “the number assigned to all the public comments” makes any sense, and in particular whether it takes into account “the public’s view”.
3. It sounds like the algorithm dumps all the public comment letters into a pot and mixes it together to get an overall score. The problem with this is that the industry insiders and their lobbyists overwhelm public commenting systems.
4. For example, go take a look at the list of public letters for the Volcker Rule. It’s not unlike this graphic on the meetings of the regulators on the Volcker Rule:
5. Besides dominating the sheer number of letters, I’ll bet the length of each letter is also much longer on average for such parties with very fancy lawyers.
6. Now think about how the NLP algorithm will deal with this in a big pot: it will be dominated by the language of the pro-industry insiders.
7. Moreover, if such a model were to be directly used, say to check that public commenting letters were written in a given case, lobbyists would have even more reason to overwhelm public commenting systems.

The take-away is that this is an amazing example of a so-called objective mathematical model set up to legitimize the watering down of financial regulation by lobbyists.

Update: I’m willing to admit I might have spoken too soon. I look forward to reading the paper on this algorithm and taking a deeper look instead of relying on a newspaper.

Categories: #OWS, finance, modeling, rant

## Optimizing for Einstein and other homo-erotic theories

Jointly posted with Naked Capitalism.

At 41, I’m a grown woman. I’ve had enough weird and bad experiences as a woman in the mathematics part of “STEM,” inside and outside of academia, that my skin is relatively thick, a fact I’m proud of. Most of the time I let stuff roll off of me.

Even so, there are certain things that really get under my skin. Examples include terrible advice to young anxious women, and anything having to do with Princeton, New Jersey.

The recent appearance of the “Princeton Mom” Susan Patton (more about her below) has created a perfect storm inside me and I feel I have to comment, at the risk of giving her book more buzz. Note this post is not at all quantitative or even nerdy, except for some free market chit-chat which doesn’t really count. Instead it is much more straight-up ranting that I allow myself from time to time on mathbabe. If you want a more scientific and polite takedown, please see this Huffington Post article.

Princeton, New Jersey

There are two kinds of people in the world: people who hate Princeton, New Jersey, and people who are über successful white men (and sometimes Asian men). And I guess there’s a third kind, the people who have never visited Princeton.

I know that sounds histrionic, and I’ll make some caveats later on, but bear with me, it’s coming from personal experience.

I spent one horrific year (the academic year 1997-1998) as a visiting graduate student in the Princeton math department. Coming from the Harvard math department, I’d been socialized to think that spending all night in the library reading musty old French mathematical manuscripts was cool, and the very least one could do to impress one’s advisor.

In other words, I knew from male-dominated macho nerd culture. I girded myself for more of the same when I got to Princeton. But Princeton turned that up quite a few notches, and it wasn’t pretty. And it might have had something to do with being newly married, but that kind of makes my point stronger, not weaker, as you will see.

The first thing I noticed was that there were no other women in the math department. Well, that’s not quite true, since there were secretaries, and there was one female professor, who I never once spotted, and there was one other female graduate student, at least in theory, but it took me weeks and weeks to run into her.

But even so, I was kind used to that, being an experienced math nerd. I would normally just make do with hanging out with the social nerd boys. Unfortunately I couldn’t find any. It seemed like a department that either selected for anti-social people or efficiently turned them into anti-social people after they arrived.

As an illustration, let me tell you about the most social experience among graduate students I ever witnessed. It started out as a joyous scene: an enthusiastic young man bounded into the common room (which was almost always empty and didn’t really deserve the name “common room” at all) holding a book. He was showing off his newly bound thesis to an unusually large crowd of fellow graduate students – maybe 7 other men.

Instead of congratulating him, someone from the crowd grabbed the thesis and immediately and loudly proclaimed he’d found a typo. Everyone laughed. Long pause. The guy took his thesis and walked out of the room.

As you might imagine, I didn’t spend too much time in the math department. Instead, naïf that I was, I gave myself the task of finding friendly people I could truly connect with in the cultural wasteland that was Princeton Township.

The problem was, it felt like a village frozen in time. Of the perhaps 7 people I got to the point of trusting enough to share my desire for connection, no fewer than 3 of them suggested I join a church (that always made me wonder, what do Jewish people do in Princeton?), and the other 4 suggested I have a child in order to have company and something to do with myself. No shit. Human being as hobby.

I could go on – I could describe the pathetic attempt to attend a female graduate student mixer (“canceled for lack of attendants”) or the desperate time I sought counseling from the sole campus Mental Health Professional. Her exact words: “If it helps, I think I eventually see every female graduate student at Princeton.” Me: “Yes, it helps! I’m getting the FUCK out of here.” And I did.

I’ve been back once or twice, mostly to see the one person I became fond of in my year-long visit, and I am always amazed to see how little has changed. The last time I went, I attended a conference at the Institute for Advanced Study, and after lunch one afternoon I was in the cafeteria there, looking for coffee, when someone (a man! an oldish white man!) asked me to “find more plates, please” because there were no more clean ones. I looked down at my clothes: was I wearing a kitchen staff uniform like other people working the kitchen? Not at all, but I did suspiciously have my boobs with me. I must be kitchen staff.

Hey, I might be wrong

Other people have been to Princeton in the past 15 years, and some of them tell me it’s gotten somewhat better, and there are sightings of more than one woman at a time in the math department, and so on. I mean, the standards are super low, so “better” doesn’t necessarily mean much, but then again I don’t want to make it seem impossibly fixed. I’m glad the President of Princeton is was a woman.

On the other hand, another friend of mine had this to say about a very recent visit (less than 3 months ago):

I was a job candidate there. Put up at that Inn. Eating by myself, and there was a long table in the center of the room  - all white men, many in bow ties, I swear. They were talking loudly about curriculum changes in the humanities over time, and what a shame it was that they couldn’t teach the classics anymore, laughing about having to teach world literature, etc. And everyone serving them was black. It was disgusting.

My theory of Princeton

I have a kind of fun theory of why Princeton is like this. The short version is that the culture has optimized to producing “geniuses,” which started with Einstein. In fact, Einstein’s success story also pinpoints the moment that time froze there. It was like the lesson learned for the town was that, if they could only keep the place exactly like it was the moment Einstein entered Princeton, then maybe it would be a breeding ground for many many more geniuses to make the town proud.

So that’s what’s happened: everything that is done there is done in the hope that more Einsteins will pop up among the population. Would-be geniuses are worshipped in weird ways, and anyone who is not themselves a genius candidate has to tailor themselves to those who are.

And since by definition geniuses are not women – and nor are minority men – we know what their roles turn out to be. Women, at least white women, are seen as useful in as much as they can have man-children who may grow up to be geniuses. Everyone else is even less crucial.

Do you think I’m being too harsh? Perhaps. To be honest, there is a space for white men to be tagged as successful without being full-blown geniuses, especially if they’re undergraduates. Namely, if they are potentially super rich, preferably by working in finance. In any case it’s all about the successful male narrative. There is no room for any other narrative.

Why am I talking shit about Princeton?

Here’s the thing. I have come to appreciate Princeton, in a wry way (“If you’re suicidal,” one character says, “and you don’t actually kill yourself, you become known as ‘wry.’ ”), and only as long as I’m not actually there. It is such a perfect example of old-fashioned, fucked up shit. You can’t make that stuff up.

But you can point to it and say, I will never live like that. It’s become a convenient counterfactual for me personally.

But not everyone has my perspective. My biggest fear nowadays about Princeton is that people are not sufficiently up front about how awful it is, and because of that people are sometimes tricked into visiting or even moving there.

It is this fear that I’m writing this essay, that I might be able to warn people away from that place, and possibly other places like it, although I don’t know of any. I’m a one-person anti-PR machine, but there’s only so much I can do.

Susan Patton to the rescue

It turns out my job is getting easier, thanks to Susan Patton, self-proclaimed “Princeton Mom”.

As if to amplify my complaints about Princeton, Patton has come out with yet more advice for girls who are aspiring to be Princeton wives. Her new advice to young women is to get fake boobs and whatever other plastic surgery deemed necessary in high school so you can attract a man in college.

Let’s back up for just a moment, though. Who is this woman?

You have heard of Susan Patton. She’s the confused bitch that wrote a now-famous letter to undergraduate women telling them to stop thinking about careers and start getting engaged whilst in college.

Oh, and she also suggested in a recent Valentine’s Day column (subtitle: “Young women in college need to smarten up and start husband-hunting.”) in the Wall Street Journal (where else!?) that, if you want men to marry you, you shouldn’t fuck them too soon, because, in her words, “men won’t buy the cow if the milk is free.”

Yes, she said that. I’ve got two responses to that tidbit. First, this:

mooooooo, motherfucker, moooooooooooo!!

Next, Aunt Pythia mentioned this but it bears repeating: Patton is objectifying women by calling them cows.

She’s doing the same when she tells young women to get boob jobs in high school. That’s in fact the name of her game. She is insisting that women abandon any hope of intellectual curiosity, goals or ambitions while they are still teenagers and start in on a desperate competition to be a Princeton wife.

Why is Patton so nuts?

By her own account, Susan Patton married the wrong guy – a non-Princeton guy – and later got divorced. She’s bitter about her lack of foresight. In some sense this is just a pathetic story about one sad person.

But in another way it’s not. I’ve been reading a super interesting book called Why Love Hurts: a Sociological Explanation that explains why Susan Patton has some things right. In fact she’s kind of brilliant, but for obviously weird reasons, and her plan to deal with the issues she rightly raises is completely fucked up.

Here’s what she’s understood: there has been a revolution in mating rituals and partnering, and it has become a competition, and it has become increasingly important to be sexually attractive to win this competition. And although it’s not the only competition young women are enduring in college, it’s the one she’s fixated on.

In fact to a large extent we’ve gone from a social contract partnering society to a kind of pseudo-free market partnering society. The results of that transition include various things like how men and women see themselves, and specifically how they (women, not men) blame themselves for failed relationships, and moreover how they are incentivized (or not) to get married, or have kids, or importantly, to keep their word.

One of the most interesting points, at least as it pertains to Susan Patton, is that whereas men used to need to get married and have children to assert their masculinity, this is no longer true.

Nowadays, according to this theory, men in question increasingly assert their masculinity to each other through the sexual attractiveness of their girlfriends, and they don’t care very much whether they get married and have kids, or at least they don’t feel any urgency (which gives rise to both “the noncommittal man” and “the woman who loves too much”).

So when Patton tells women to get boob jobs, she’s essentially telling them to improve their odds in that existing free market. It’s not about sexual gratification, or even “self confidence” for the women. It’s really a homo-erotic, all-male issue: be something that other men will be jealous of.  And what is the measure of their jealousy? That other men are responding sexually to “my” woman. So this means men are focusing on signs of sexual responses in other men and deriving gratification from them.

Here’s what Patton has tragically wrong, though. Given that you’re willing to toss out your personal and intellectual growth for the sake of winning this competition, even given that, which is a sad way to approach life, it still doesn’t have a chance of working.

Because, once we’ve acknowledged and entered this free market for sexual and romantic partnership, it’s simply not going to work in this day and age to expect the men to want to get married when they’re 20 years old, and it’s also certainly not going to work to withhold sex from 20-year-old men and expect them to marry you. It’s just not where 20-year-old men are at in this system. In fact by doing those things a woman is signaling desperation, which – as is explained in this book – works against a given woman, not for them.

Patton and my theory

I’d like to square her advice with my optimized-for-geniuses theory of Princeton.

The main point of my theory is that it’s all about the men, and specifically, it’s all about the successful male narrative. Whereas before it was enough for women to subjugate their personality, personal ambitions, and long-term goals for the purpose of potential geniuses and/or rich finance guys, Patton is now calling for women to also mutilate their bodies for the cause.

As a signaling device, it indicates real hunger for the role. As some guy said:

Fake boobs say, ‘I objectify myself, therefore I have no problem with you doing the same.’

But as I mentioned above, it is a failed signaling device. It’s an indication that the cultural worship of men has gone too far in Princeton, New Jersey. I’m hopeful that the smell of desperation will be so obvious that people will have to take a closer look and scrutinize the culture.

I’d also like to start a petition to demand that the Wall Street Journal make up for the publishing Patton’s column by also printing this excellent essay on getting laid really well when you’re a divorced fat woman. We need an antidote.

Categories: rant

## What privacy advocates get wrong

There’s a wicked irony when it comes to many privacy advocates.

They are often narrowly focused on the their own individual privacy issues, but when it comes down to it they are typically super educated well-off nerds with few revolutionary thoughts. In other words, the very people obsessing over their privacy are people who are not particularly vulnerable to the predatory attacks of either the NSA or the private companies that make use of private data.

Let me put it this way. If I’m a data scientist working at a predatory credit card firm, seeking to build a segmentation model to target the most likely highly profitable customers – those that ring up balances and pay off minimums every month, sometimes paying late to accrue extra fees – then if I am profiling a user and notice an ad blocker or some other signal of privacy concerns, chances are that becomes a wealth indicator and I leave them alone. The mere presence of privacy concerns signals that this person isn’t worth pursuing with my manipulative scheme.

If you don’t believe me, take a look at a recent Slate article written by  and entitled Take My Data Please: How I learned to stop worrying and love a less private internet.

In it he describes how he used to be privacy obsessed, for no better reason than that he like to stick up a middle finger to those who would collect his data. I think that article should have been called something like, Well-educated white guy was a privacy freak until he realized he didn’t have to be because he’s a well-educated white guy.

He concludes that he really likes how well customized things are to his particular personality, and that shucks, we should all just appreciate the web and stop fretting.

But here’s the thing, the problem isn’t that companies are using his information to screw Cyrus Nemati. The problem is that the most vulnerable people – the very people that should be concerned with privacy but aren’t – are the ones getting tracked, mined, and screwed.

In other words, it’s silly for certain people to be scrupulously careful about their private data if they are the types of people who get great credit card offers and have a stable well-paid job and are generally healthy. I include myself in this group. I do not prevent myself from being tracked, because I’m not at serious risk.

And I’m not saying nothing can go wrong for those people, including me. Things can, especially if they suddenly lose their jobs or they have kids with health problems or something else happens which puts them into a special category. But generally speaking those people with enough time on their hands and education to worry about these things are not the most vulnerable people.

I hereby challenge Cyrus Nemati to seriously consider who should be concerned about their data being collected, and how we as a society are going to address their concerns. Recent legislation in California is a good start for kids, and I’m glad to see the New York Times editors asking for more.

Categories: data science, rant

## Intentionally misleading data from Scott Hodge of the Tax Foundation

Scott Hodge just came out with a column in the Wall Street Journal arguing that reducing income inequality is way too hard to consider. The title of his piece is Scott Hodge: Here’s What ‘Income Equality’ Would Look Like, and his basic argument is as follows.

First of all, the middle quintile already gets too much from the government as it stands. Second of all, we’d have to raise taxes to 74% for the top quintile to even stuff out. Clearly impossible, QED.

As to the first point, his argument, and his supporting data, is intentionally misleading, as I will explain below. As to his second point, he fails to mention that the top tax bracket has historically been much higher than 74%, even as recently as 1969, and the world didn’t end.

Hodge argues with data he took from a report from the CBO called The Distribution of Federal Spending and Taxes in 2006This report distinguishes between transfers and spending. Here’s a chart to explain what that looks, before taxes are considered and by quintile, for non-elderly households (page 5 of the report):

The stuff on the left corresponds to stuff like food stamps. The stuff in the middle is stuff like Medicaid. The stuff on the right is stuff like wars.

Here are a few things to take from the above:

1. There’s way more general spending going on than transfers.
2. Transfers are very skewed towards the lowest quintile, as would be expected.
3. If you look carefully at the right-most graph, the light green version gives you a way of visualizing of how much more money the top quintile has versus the rest.

Now let’s break this down a bit further to include taxes. This is a key chart that Hodge referred to from this report (page 6 of the report):

OK, so note that in the middle chart, for the middle quintile, people pay more in taxes than they receive in transfers. On the right chart, for the middle quintile, which includes all spending, the middle quintile is about even, depending on how you measure it.

Now let’s go to what Hodge says in his column (emphasis mine):

Looking at prerecession data for non-elderly households in 2006 in “The Distribution of Federal Spending and Taxes in 2006,” the CBO found that those in the bottom fifth, or quintile, of the income scale received $9.62 in federal spending for every$1 they paid in federal taxes of all kinds. This isn’t surprising, since people with low incomes pay little in taxes but receive a lot of transfers.

Nor is it surprising that households in the top fifth received 17 cents in federal spending for every $1 they paid in all federal taxes. High-income households hand over a disproportionate amount in taxes relative to what they get back in spending. What is surprising is that the middle quintile—the middle class—also got more back from government than they paid in taxes. These households received$1.19 in government spending for every $1 they paid in federal taxes. In the first paragraph Hodge intentionally conflates the concept of “transfers” and “spending”. He continues to do this for the next two paragraphs, and in the last sentence, it is easy to imagine a middle-quintile family paying$100 in taxes and receiving $119 in food stamps. This is of course not true at all. What’s nuts about this is that it’s mathematically equivalent to complaining that half the population is below median intelligence. Duh. Since we have a skewed distribution of incomes, and therefore a skewed distribution of tax receipts as well as transfers, then in the context of a completely balanced budget, we would expect the middle quintile – which has a below-mean average income – to pay slightly less than the government spends on them. It’s a mathematical fact as long as our federal tax system isn’t regressive, which it’s not. In other words, this guy is just framing stuff in a “middle class is lazy and selfish, what could rich people possibly be expected do about that?” kind of way. Who is this guy anyway? Turns out that Hodge is the President of the Tax Foundation, which touts itself as “nonpartisan” but which has gotten funding from Big Oil and the Koch brothers. I guess it’s fair to say he has an agenda. Categories: modeling, news, rant ## A visit to Fair Foods in winter It’s been a few days since I last posted. The reason was my trip to WPI for my talk (slides from my talk available here), and then on to Boston, where I stayed with my good friends over at Fair Foods in Dorchester, near Fields Corner. I mentioned Fair Foods before, for example in this post from two and a half years ago. I worked with the director Nancy in high school back in 1988 and 1989, when me and my sometimes guest blogger Becky would go work a couple of days a week. Here’s a picture I stole from the site with Becky in overalls: We’d drive at 6am with Nancy and her beat up old box truck to the Chelsea Produce Market and ask for donations of pallets of vegetables that were too old to be sold to supermarkets but still fresh enough to be eaten that same day. We’d also grab similarly oldish bread from an Arnold’s Bread bakery in Cambridge and then we’d distribute the food at “dollar bag” sites, raking in less than the amount of money we’d spent on gas and insurance for the truck. The program is still going, scraping by with sometime grants and contributions, many from ex-volunteers like me (feel free to send a contribution yourself – a check made out to “Fair Foods” and mailed to PO Box 220168, Dorchester, MA 02122 would be very welcome). And the hard working people there have my undying love and admiration for their incredible commitment and work ethic. To give you some idea, they live in an old drafty Victorian and heat their house with woodstoves. All I can say about this past weekend is thank god for union suits and wool socks. But here’s the thing, you get a pretty ground-level view of hardship and poverty working in a program like that, especially when you’ve done it for more than 25 years, and especially when you see increasingly long lines of people willing to wait for vegetables and bread in bitterly cold weather. Business is booming this winter. Many of the customers of Fair Foods are old friends by now, they’ve been coming weekly to various sites for many years to feed their children and their grandchildren. Many of them are immigrants with very little money, and the$2 it now costs for a big bag of vegetables and fruit is a great deal.

I guess my point is this. I worked for Fair Foods back in the crack epidemic of the late 1980′s and the early 1990′s, which was a hellish time for Dorchester and of course other parts of the country. But nowadays, when the crime rate is so much lower, we’re seeing another kind of hell. It’s a lot quieter.

It’s incredibly sad to see how much more demand there is for salvaged food now than there was 25 years ago, and how many of those old beautiful Victorian family houses are abandoned or at risk of foreclosure, and how few cars there are on the street compared to then. And most especially, how many of the kids I used to play with on the street are now in prison.

Before I left Saturday I made a delicious soup out of the vegetables that Jason and Liz had collected from the Chelsea Market, and I made banana bread that the banana guy had given them. A little boy from the neighborhood who came to talk to Nancy about his report card ate about half of that banana bread in one sitting. A small attempt to try to feed the people who work so hard to feed other people.

It makes me wonder what kind of country we’ve created where people are so hungry, we’re reducing food stamps, and Jamie Dimon is getting an extra big bonus. Where is the justice in that?

Categories: Becky Jaffe, rant

## If it’s hocus pocus then it’s not math

A few days ago there was a kerfuffle over this “numberphile” video, which was blogged about in Slate here by Phil Plait in his “Bad Astronomy” column, with a followup post here with an apology and a great quote from my friend Jordan Ellenberg.

The original video is hideous and should never have gotten attention in the first place. I say that not because the subject couldn’t have been done well – it could have, for sure – but because it was done so poorly that it ends up being destructive to the public’s most basic understanding of math and in particular positive versus negative numbers. My least favorite line from the crappy video:

I was trying to come up with an intuitive reason for this I and I just couldn’t. You have to do the mathematical hocus pocus to see it.

What??

Anything that is hocus pocus isn’t actually math. And people who don’t understand that shouldn’t be making math videos for public consumption, especially ones that have MSRI’s logo on them and get written up in Slate. Yuck!

I’m not going to just vent about the cultural context, though, I’m going to mention what the actual mathematical object of study was in this video. Namely, it’s an argument that “prove” that we have the following identity:

$1 + 2 + 3 + 4 + \dots = - \frac{1}{12}.$

Wait, how can that be? Isn’t the left hand side positive and the right hand side negative?!

This mathematical argument is familiar to me – in fact it is very much along the lines of stuff we sometimes cover at the math summer program HCSSiM I teach at sometimes (see my notes from 2012 here). But in the case of HCSSiM, we do it quite differently. Specifically, we use it as a demonstration of flawed mathematical thinking. Then we take note and make sure we’re more careful in the future.

If you watch the video, you will see the flaw almost immediately. Namely, it starts with the question of what the value is of the infinite sum

$1 -1 + 1 -1 + \dots.$

But here’s the thing, that doesn’t actually have a value. That is, it doesn’t have a value until you assign it a value, which you can do but then you might want to absolutely positively must explain how you’ve done so. Instead of that explanation, the guy in the video just acts like it’s obvious and uses that “fact,” along with a bunch of super careless moving around of terms in infinite sums, to infer the above outrageous identity.

To be clear, sometimes infinite sums do have pretty intuitive and reasonable values (even though you should be careful to acknowledge that they too are assigned rather than “true”). For example, any geometric series where each successive term gets smaller has an actual “converging sum”. The most canonical example of this is the following:

$1/2 + 1/4 + 1/8 + \dots + 1/2^k + \dots = 1.$

What’s nice about this sum is that it is naively plausible. Our intuition from elementary school is corroborated when we think about eating half a cake, then another quarter, and then half of what’s left, and so on, and it makes sense to us that, if we did that forever (or if we did that increasingly quickly) we’d end up eating the whole cake.

This concept has a name, and it’s convergence, and it jibes with our sense of what would happen “if we kept doing stuff forever (again at possibly increasing speed).” The amounts we’ve measured on the way to forever are called partial sums, and we make sure they converge to the answer. In the example above the partial sums are $1/2, 3/4, 7/8,$ and so on, and they definitely converge to 1.

There’s a mathematical way of defining convergence of series like this that the geometric series follows but that the $1-1+1-1 \dots$ series does not. Namely, you guess the answer, and to make sure you’ve got the right one, you make sure that all of the partial sums are very very close to that answer if you go far enough, for any definition of “very very close.”

So if you want it to get within 0.00001, there’s a number N so that, after the Nth partial sum, all partial sums are within 0.00001 of the answer. And so on.

Notice that if you take the partial sums of the $1-1+1-1 \dots$ series you get the sequence $1, 0, 1, 0,1,0,1, \dots,$ which doesn’t get closer and closer to anything. That’s another way of saying that there is no naively plausible value for this infinite sum.

As for the first infinite sum we came across, the $1 +2 + 3 + 4 +\dots,$ that does have a naively plausible value, which we call “infinity.” Totally cool and satisfying to your intuition that you worked so hard to achieve in high school.

But here’s the thing. Mathematicians are pretty clever, so they haven’t stopped there, and they’ve assigned a value to the infinite sum $1-1+1-1 \dots$ in spite of these pesky intuition issues, namely $\frac{1}{2}$, and in a weird mathematical universe of their construction, which is wildly useful in some contexts, that value is internally consistent with other crazy-ass things. One of those other crazy-ass things is the original identity $1 + 2 + 3 + 4 + \dots = - \frac{1}{12}.$

[Note: what would be really cool is if a mathematician made a video explaining the crazy-ass universe and why it's useful and in what contexts. This might be hard and it's not my expertise but I for one would love to watch that video.]

That doesn’t mean the identity is “true” in any intuitively plausible sense of the word. It means that mathematicians are scrappy.

Now here’s my last point, and it’s the only place I disagree somewhat (I think) with Jordan in his tweets. Namely, I really do think that the intuitive definition is qualitatively different from what I’ve termed the “crazy-ass” definition. Maybe not in a context where you’re talking to other mathematicians, and everyone is sufficiently sophisticated to know what’s going on, but definitely in the context of explaining math to the public where you can rely on number sense and (hopefully!) a strong intuition that positive numbers can’t suddenly become negative numbers.

Specifically, if you can’t make any sense of it, intuitive or otherwise, and if you have to ascribe it to “mathematical hocus pocus,” then you’re definitely doing something wrong. Please stop.

Categories: math, math education, rant

## Parents fighting back against sharing children’s data with InBloom

There is a movement afoot in New York (and other places) to allow private companies to house and mine tons of information about children and how they learn. It’s being touted as a great way to tailor online learning tools to kids, but it also raises all sorts of potential creepy modeling problems, and one very bad sign is how secretive everything is in terms of privacy issues. Specifically, it’s all being done through school systems and without consulting parents.

In New York it’s being done through InBloom, which I already mentioned here when I talked about big data and surveillance. In that post I related an EducationNewYork report which quoted an official from InBloom as saying that the company “cannot guarantee the security of the information stored … or that the information will not be intercepted when it is being transmitted.”

The issue is super important and timely, and parents have been left out of the loop, with no opt-out option, and are actively fighting back, for example with this petition from MoveOn (h/t George Peacock). And although the InBloomers claim that no data about their kids will ever be sold, that doesn’t mean it won’t be used by third parties for various mining purposes and possibly marketing – say for test prep tools. In fact that’s a major feature of InBloom’s computer and data infrastructure, the ability for third parties to plug into the data. Not cool that this is being done on the downlow.

Who’s behind this? InBloom is funded by the Bill & Melinda Gates foundation and the operating system for inBloom is being developed by the Amplify division (formerly Wireless Generation) of Rupert Murdoch’s News Corp. More about the Murdoch connection here.

Wait, who’s paying for this? Besides the Gates and Murdoch, New York has spent $50 million in federal grants to set up the partnership with InBloom. And it’s not only New York that is pushing back, according to this Salon article: InBloom essentially offers off-site digital storage for student data—names, addresses, phone numbers, attendance, test scores, health records—formatted in a way that enables third-party education applications to use it. When inBloom was launched in February, the company announced partnerships with school districts in nine states, and parents were outraged. Fears of a “national database” of student information spread. Critics said that school districts, through inBloom, were giving their children’s confidential data away to companies who sought to profit by proposing a solution to a problem that does not exist. Since then, all but three of those nine states have backed out. Finally, according to this nydailynews article, Bill de Blasio is coming out on the side of protecting children’s privacy as well. That’s a good sign, let’s hope he sticks with it. I’m not against using technology to learn, and in fact I think it’s inevitable and possibly very useful. But first we need to have a really good, public discussion about how this data is being shared, controlled, and protected, and that simply hasn’t happened. I’m glad to see parents are aware of this as a problem. Categories: data science, modeling, news, rant ## What does a really efficient market look like? The raison d’être of hedge funds is to make the markets efficient. Or at least that’s one of the raisons d’être, the others being 1) to get rich and 2) to leave early on Fridays in the summer (resp. winter) to get a jump on traffic to the Hamptons (resp. ski area, possibly in Kashmir). And although having efficient markets sounds like a great thing, it makes sense to ask what that would look like from the perspective of a non-insider. This recent Wall Street Journal article on high-tech snooping does a pretty good job setting the tone here. First, the kind of thing they’re doing: Genscape is at the vanguard of a growing industry that employs sophisticated surveillance and data-crunching technology to supply traders with nonpublic information about topics including oil supplies, electric-power production, retail traffic and crop yields. Next, who they’re doing it for: The techniques, which are perfectly legal, represent the latest advance in the longtime Wall Street practice of searching for every possible trading advantage. But the high cost of much of the new information—Genscape’s oil-supply report costs$90,000 a year—means that some forms of trading are becoming even more the province of firms with substantial resources.

Let’s put these two things together from the perspective of the public. The market is getting information from hidden cameras and sensors, and all that information is being fed to “the market” via proprietary hedge funds via channels we will never tap into. The end result is that the prices of commodities are being adjusted to real-world events more and more quickly, but these are events that are not truly known to the real world.

[Aside: I'm going to try to avoid talking about the "true price" of things like gas, because I think that's pretty much a fool's errand. In any case, let me just say that, in addition to the potentially realtime sensor information that goes into a commodity's price, we also have people trading on it because they are adjusting their exposure to some other historically correlated or anti-correlated instrument, or because they've decided to liquidate their books, or because they've decided the Fed has changed its macroeconomic policy, or because Spain needs to deal with its bank problems, or because someone wants to take money out of the market to rent their summer house in the Hamptons. In other words, I'm not ready to argue that we're getting close to the "true price" of gas here. It's just tradable information like any other.]

I am now prepared, as you hopefully are as well, to question what good this all does for people like us, who are not privy to the kind of expensive information required to make these trades. From our perspective, nothing happens, the price fluctuates, and the market is deemed efficient. Is this actually an improvement over the alternative version where something happens, and then the price adjusts? It’s an expensive arms race, taking up vast resources, where things have only become more opaque.

How vast are those resources? Having worked in finance, I know the answer is a shit-ton, if it is profitable in a short-term edgy kind of way. Just as those guys dug a hole through mountains to make the connection between New York to Chicago a few nanoseconds faster, they will go to any length to get the newest info on the market, as long as it is deemed to have a profitable edge in some time frame – i.e. the amount of time it will take a flood of competitors to do the same thing.

Just as there’s a kind of false myth that most of the web is porn, I’d like to perpetuate a new somewhat false myth that most data gathering and mining happens for the benefit of trading. And if that’s false now, let’s talk about it again in 100 years, when the market for celebrities is mature, and you can make money shorting a bad marriage.

Categories: finance, modeling, rant

## Do we really want elite youth to get more elite?

I don’t know if you guys read this recent New York Times editorial entitled Even Gifted Students Can’t Keep Up: In Math and Science, the Best Fend for Themselves.

In it, they claim there’s some kind of crisis going on in this country for smart kids (defined as good test-takers). Mostly their evidence for this is that, among other countries, our super good test takers aren’t as prevalent as in other countries. Turns out we’re in the middle of the pack in terms of super scorers. From the article:

On the 2012 Program for International Student Assessment test, the most recent, 34 of 65 countries and school systems had a higher percentage of 15-year-olds scoring at the advanced levels in mathematics than the United States did.

Why is this a problem? As far as I can see they’ve come up with two reasons.

First, it’s “bad for American competitiveness,” whatever that means. Last time I checked we were still pretty dominant in various ways in terms of technology and science, and there are still plenty of very well-educated young people trying desperately to get visas to enter or stay in this country.

As an aside: it’s a super interesting question to think about how we, as a country, are increasingly ignorant about how our technology works, because so much technical knowledge has been off-shored. But that’s not the crisis these guys are addressing.

Second, it’s bad for the smart kids in this country, because “when the brightest students are not challenged academically, they lose steam and check out.”

I’ll pause in my summary of their article to make the following point. If that’s true, if bright kids who aren’t academically challenged at school start checking out more and more, then it makes just as much sense to me to see if there’s something they can check out towards.

In other words, what else is there for bright teenagers to do besides school? I’ll speak as a former bright teenager. When I lost interest in school, I got a lot of odd jobs in town cleaning houses and raking lawns, and then I used the money to buy lots of softcover books from a local bookstore. I don’t know why I didn’t just take them out of the library, where I also worked. It just didn’t seem as cool as owning my very own Brother Karamazov.

I learned a lot with my odd jobs in high school, which included being a part-time secretary, a barista, a math tutor, and working on a truck at the New England Produce Center. In fact I learned way more about how the world worked than I would have if I’d followed the advice of this editorial, which was to take lots more AP classes and then enter college when I was 14.

My theory is that, instead of obsessing over math scores in standardized tests, we concentrate on allowing our children to enrich their lives with adventures and experiences that they come up with and that are reasonably safe. So let’s start by encouraging widespread internships for younger kids, and not just minimum wage jobs at fast food joints. And not just for super test-takers either. Enrichment happens when kids learn about stuff that’s outside their usual rhythm and when there are no adults scripting their activities and telling them what to do or how many laps to swim.

Notice my emphasis on letting kids choose stuff. What drives me nuts just as much as the idea of further separating and isolating and venerating great test-takers, which as far as I’m concerned is the opposite way you should treat future successful people, is the idea that there should be such a well-defined funnel for children at all.

Yes, kids should all go to school and learn basic things. But the idea that, just because someone’s good at tests they should be treated as if they’re already running the Fed only increases the weird worshippy aspect of how our culture treats math nerds.

Plus, it’s a bizarre time to come up with this idea, considering how many online and live resources there are for nerd kids now compared to when I was a kid. If I’m a nerd teenager now, I can find plenty of ways to share nerdy questions and learn nerdy things online if I decide not to work in a coffee shop.

Finally, let me just take one last swipe at this idea from the perspective of “it’s meritocratic therefore it’s ok”. It’s just plain untrue that test-taking actually exposes talent. It’s well established that you can get better at these tests through practice, and that richer kids practice more. So the idea that we’re going to establish a level playing field and find minority kids to elevate this way is rubbish. If we do end up focusing more on the high end of test-takers, it will be completely dominated by the usual suspects.

In other words, this is a plan to make elite youth even more elite. And I don’t know about you, but my feeling is that’s not going to help our country overall.

Categories: news, rant

## International trade agreements and big money bullying #OWS

The New York Times just put out an amazing and outrageous story, entitled Tobacco Industry Tactics Limit Poorer Nations’ Smoking Laws and written by Sabrina Tavernise.

In it she describes the bullying tactics of tobacco companies to small countries over their internal health regulations aiming to protect their citizenry from cancer. From the article:

In Africa, at least four countries — Namibia, Gabon, Togo and Uganda — have received warnings from the tobacco industry that their laws run afoul of international treaties, said Patricia Lambert, director of the international legal consortium at the Campaign for Tobacco Free Kids.

“They’re trying to intimidate everybody,” said Jonathan Liberman, director of the McCabe Center for Law and Cancer in Australia, which gives legal support to countries that have been challenged by tobacco companies. In Namibia, the tobacco industry has said that requiring large warning labels on cigarette packages violates its intellectual property rights and could fuel counterfeiting.

• This happens because, in order to protect companies from being taken over by foreign nations, and in the  name of free trade, it’s now possible for companies to sue countries directly. That’s what the tobacco industry is doing to small countries without the means to fight back.
• This is exactly the kind of thing that some people like Yves Smith have warned the TPP is going to do with financial regulation and other stuff. So imagine large companies or industries suing the United States or other nations for regulation that would “harm trade” or “violate their intellectual property rights.”
• In fact, it’s not going too far to say that the proliferation of these kinds of treaties are a serious threat to national sovereignty. Yves makes this case here.
• Think about it this way. It’s kind of a supernational Citizen’s United, in that only companies and industries that have enough lawyer power can get their way. And many companies easily have more resources than many countries. Think about Facebook and Google and their lobbying efforts on behalf of data privacy laws in Europe already. Now think how that battle might look when it’s against Namibia.
• Already, from the article, we saw that Uruguay was only able to fight back against Philip Morris because Bloomberg’s foundation helped them with the legal battle. And I’m glad Bloomberg helped, but if you count up all the money that will go to bullying and compare that to all the money available to help out the bullied, you quickly come to a sad conclusion.
• Once again, we have to remember that the TPP is being negotiated in secret, among a bunch of nations many of whom claim to be democratic.
Categories: #OWS, rant

## “People analytics” embeds old cultural problems in new mathematical models

Today I’d like to discuss recent article from the Atlantic entitled “They’re watching you at work” (hat tip Deb Gieringer).

In the article they describe what they call “people analytics,” which refers to the new suite of managerial tools meant to help find and evaluate employees of firms. The first generation of this stuff happened in the 1950′s, and relied on stuff like personality tests. It didn’t seem to work very well and people stopped using it.

But maybe this new generation of big data models can be super useful? Maybe they will give us an awesome way of throwing away people who won’t work out more efficiently and keeping those who will?

Here’s an example from the article. Royal Dutch Shell sources ideas for “business disruption” and wants to know which ideas to look into. There’s an app for that, apparently, written by a Silicon Valley start-up called Knack.

Specifically, Knack had a bunch of the ideamakers play a video game, and they presumably also were given training data on which ideas historically worked out. Knack developed a model and was able to give Royal Dutch Shell a template for which ideas to pursue in the future based on the personality of the ideamakers.

From the perspective of Royal Dutch Shell, this represents huge timesaving. But from my perspective it means that whatever process the dudes at Royal Dutch Shell developed for vetting their ideas has now been effectively set in stone, at least for as long as the algorithm is being used.

I’m not saying they won’t save time, they very well might. I’m saying that, whatever their process used to be, it’s now embedded in an algorithm. So if they gave preference to a certain kind of arrogance, maybe because the people in charge of vetting identified with that, then the algorithm has encoded it.

One consequence is that they might very well pass on really excellent ideas that happened to have come from a modest person – no discussion necessary on what kind of people are being invisible ignored in such a set-up. Another consequence is that they will believe their process is now objective because it’s living inside a mathematical model.

The article compares this to the “blind auditions” for orchestras example, where people are kept behind a curtain so that the listeners don’t give extra consideration to their friends. Famously, the consequence of blind auditions has been way more women in orchestras. But that’s an extremely misleading comparison to the above algorithmic hiring software, and here’s why.

In the blind auditions case, the people measuring the musician’s ability have committed themselves to exactly one clean definition of readiness for being a member of the orchestra, namely the sound of the person playing the instrument. And they accept or deny someone, sight unseen, based solely on that evaluation metric.

Whereas with the idea-vetting process above, the training data consisted of “previous winners” which presumable had to go through a series of meetings and convince everyone in the meeting that their idea had merit, and that they could manage the team to try it out, and all sorts of other things. Their success relied, in other words, on a community’s support of their idea and their ability to command that support.

In other words, imagine that, instead of listening to someone playing trombone behind a curtain, their evaluation metric was to compare a given musician to other musicians that had already played in a similar orchestra and, just to make it super success-based, had made first seat.

That you’d have a very different selection criterion, and a very different algorithm. It would be based on all sorts of personality issues, and community bias and buy-in issues. In particular you’d still have way more men.

The fundamental difference here is one of transparency. In the blind auditions case, everyone agrees beforehand to judge on a single transparent and appealing dimension. In the black box algorithms case, you’re not sure what you’re judging things on, but you can see when a candidate comes along that is somehow “like previous winners.”

One of the most frustrating things about this industry of hiring algorithms is how unlikely it is to actively fail. It will save time for its users, since after all computers can efficiently throw away “people who aren’t like people who have succeeded in your culture or process” once they’ve been told what that means.

The most obvious consequence of using this model, for the companies that use it, is that they’ll get more and more people just like the people they already have. And that’s surprisingly unnoticeable for people in such companies.

My conclusion is that these algorithms don’t make things objective, they makes things opaque. And they embeds our old cultural problems in new mathematical models, giving us a false badge of objectivity.

Categories: data science, modeling, rant

## People don’t get fired enough

This might surprise some of you – or not, I’m not sure. But one of the most satisfying things about leaving academia and the tenure system and going into industry is how, at least in the ideal situation, you can get fired for not doing your job.

In fact, one of the reasons I decided to leave academia is that I really thought some of my colleagues weren’t doing right by the undergraduates, and the frustrating thing was that there was essentially no way to force them to start. Tenure has great aspects and not-so-great aspects, and a total lack of leverage is not a great one. I feel for deans sometimes.

Here’s the dirty little secret of lots of industry jobs, though: lots of time people also don’t get fired when they should. And sometimes it’s super awful bullies who yell and scream and act inappropriately but also pull in amazing sales numbers. There are things like that, of course. That’s the example of how they don’t abide by the alleged social contract but they perform on the bottomline. Social contracts are hard to quantify and somewhat squishy. You see people getting away with stuff because they’re rainmakers or higher ups.

But there are also plenty of examples of people just not doing their job, and having super awful attitudes, or even just completely apathetic attitudes, and for whatever reason they don’t get fired. This demoralizes and irritates and distracts everyone around them, because they all resent the free-rider.

Plus, retaining people who should by all accounts get fired makes the veneer of the kool-aid drinking camaraderie even more flimsy and scrutinizable – what’s so great about working here if people can just slack off and not care? Why do I give two shits about this project anyway? How does this project in the larger scheme of things? Maybe that scrutiny is a good thing – I engage in it myself – but you don’t want everyone thinking that all the time.

Here’s the thing, before you think I’m super vicious and mean to want people to get fired. These people I’m talking about are generally high skilled and temporarily depressed. They’re in the wrong job. And once fired, they will find another job, which will hopefully be a better one for them. I’m not saying that nobody will ever end up jobless and homeless, but very few, and moreover there are plenty of jobless and homeless people who would be psyched to do that job really well (putting aside how difficult it is for homeless people to get seriously considered for a job).

And I’m not saying you fire people out of the blue. You definitely need to tell people they’re not performing well (or that they are) and keep them in the feedback loop on whether things are working out. But in my experience people who deserve to get fired totally know it and can’t believe their luck that they’ve not been fired yet.

To conclude, I’m going on record saying I kind of agree with Jack Welch on this issue in a way I never thought I would.

Categories: musing, rant

## I’m already fat so I may as well be smart

I seem to be in a mood this week for provocative posts about body image and appearance (maybe this is what happens when I skip an Aunt Pythia column). Apologies to people who came for math talk.

I just wanted to mention something positive about the experience of being fat all my life, but especially as a school kid. Because just to be clear, this isn’t a phase. I’ve been pudgy since I was 2 weeks old. And overall it kind of works for me, and I’ll say why.

Namely, being a fat school kid meant that I was so uncool, so outside of normal social activity with boys and the like, that I was freed up to be as smart and as nerdy as I wanted, with very little stress about how that would “look”. You’re already fat, so why not be smart too? You’re not doing anything else, nobody’s paying attention to you, and there’s nothing to gossip about, so might as well join the math team.

It’s really a testament to both the pressure to be thin and the pressure to conform intellectually, i.e. not be a nerd, when you’re a young girl: they are both intense and super unpleasant. The happy truth is, one can be cover for the other. More than that, really: being fat (or “overweight” for people who are squeamish about the word “fat”) has opened up many doors that I honestly think would have, or at least could have, remained shut had I been more socially acceptable.

Going back to dress code at work for a moment: while people claim that corporate dress codes are meant to keep our minds off of sex, that is clearly a huge lie when it comes to many categories of women’s work clothes. Who are we kidding? The mere fact that many women wear high heels to work kind of says it all. And that’s fine, but let’s freaking acknowledge it.

On the other hand, it’s pretty hard to look sexy in a plus-sized suit (although not impossible), and the idea of high heels at work is just nuts. This ends up being a weirdly good thing for me, though: people take me more seriously because I have taken myself out of the sex game altogether – or at least the traditional sex game.

By the way, I’m not saying all fat women have the same perspective on it. I’m lucky enough to have figured out pretty early on how to separate other people’s projected feelings about my body from my own feelings. I am an observer of fat hatred, in other words. That doesn’t make me entirely insulated but it does give me one critical advantage: I have a lot of time on my hands to do stuff that I might otherwise spend fretting about my body.

It also might help partly explain why some girls get on the math team and others don’t. Being fat is something you don’t have control over (the continuing and damaging myth that each person does have control over it notwithstanding) but joining the math team is something you do have control over. And if you aren’t already excluded for some other reason (being fat is one but by no means the only way this could happen of course), you might not want to start that whole thing intentionally. Just a theory.

Categories: rant, women in math

## One reason corporate culture sucks for women

Am I the only person offended by the recent wave of articles wherein “senior women” at corporate offices are going around telling “younger women” about the appropriate dress code?

For example, here’s the beginning of a WSJ piece on just that subject:

Clothes may make the man. Can they undo the woman?

When female employees at Frontier Communications Corp. show up at its headquarters in very short skirts, sweatpants or sneakers, Chief Executive Maggie Wilderotter sometimes pulls them aside for a quick, private chat on dressing for success.

“I want women to be paid attention to for what they say–and not how they look,” explains Ms. Wilderotter.

Later in the article the explain why this is ok:

Women face more pitfalls because they have more clothing choices than men. And because male bosses fear being accused of sexual harassment, it usually falls to female supervisors to confront an associate about her attire.

This is one reason I hate corporate jobs. And yes, it’s because I come from academia and because I’m essentially a hippie, but seriously, why do we need so much policing? Why can’t people just leave each other alone to express themselves? It’s also a double standard:

Rosalind Hudnell, human resources vice president of Intel Corp., occasionally intervenes when she sees young female staffers clad unprofessionally, even though Intel staffers often wear shorts and jeans.

It’s just another in a long list of things you are scrutinized on if you’re a woman. In addition to whether you are a good mother, a feminine-enough-without-being-too-feminine employee, and, as a tertiary issue, if that, whether you actually do your job well. Fuck this.

Question for you readers: what does it really mean that these “senior women” are taking it upon themselves to scrutinize and criticize young women? Am I wrong, is it actually generous? Or is it some kind of hazing thing? Or is it a media invention that doesn’t actually happen?

Categories: rant

## Alan Greenspan still doesn’t get it. #OWS

Yesterday I read Alan Greenspan’s recent article in Foreign Affairs magazine (hat tip Rhoda Schermer). It is entitled “Never Saw It Coming: Why the Financial Crisis Took Economists By Surprise,” and for those of you who want to save some time, it basically goes like this:

I’ll admit it, the macroeconomic models that we used before the crisis failed, because we assumed people financial firms behaved rationally. But now there are new models that assume predictable irrational behavior, and once we add those bells and whistles onto our existing models, we’ll be all good. Y’all can start trusting economists again.

Here’s the thing that drives me nuts about Greenspan. He is still talking about financial firms as if they are single people. He just didn’t really read Adam Smith’s Wealth of Nations, or at least didn’t understand it, because if he had, he’d have seen that Adam Smith argued against large firms in which the agendas of the individuals ran counter to the agenda of the company they worked for.

If you think about individuals inside the banks, in other words, then their individual incentives explain their behavior pretty damn well. But Greenspan ignores that and still insists on looking at the bank as a whole. Here’s a quote from the piece:

Financial firms accepted the risk that they would be unable to anticipate the onset of a crisis in time to retrench. However, they thought the risk was limited, believing that even if a crisis developed, the seemingly insatiable demand for exotic financial products would dissipate only slowly, allowing them to sell almost all their portfolios without loss. They were mistaken.

Let’s be clear. Financial firms were not “mistaken”, because legal contracts can’t think. As for the individuals working inside those firms, there was no such assumption about a slow exhale. Everyone was furiously getting their bonuses pumped up while the getting was good. People on the inside knew the market for exotic financial products would blow at some point, and that their personal risks were limited, so why not make systemic risk worse until then.

As a mathematical modeler myself, it bugs me to try to put a mathematical band-aid on an inherently failed model. We should instead build a totally new model, or even better remove the individual perverted incentives of the market using new rules (I’m using the word “rules” instead of “regulations” because people don’t hate rules as much as they hate regulations).

Wouldn’t it be nice if the agendas of the individuals inside a financial firm were more closely aligned with the financial firm? And if it was over a long period of time instead of just until the bonus day? Not impossible.

And, since I’m an occupier, I get to ask even more. Namely, wouldn’t it be even nicer if that agenda was also shared by the general public? Doable!

Mr. Greenspan, there are ways to address the mistake you economists made and continue to make, but they don’t involve fancier math models from behavioral economics. They involve really simple rule changes and, generally speaking, making finance much more boring and much less profitable.

Categories: #OWS, finance, modeling, rant

I had a great time at Harvard Wednesday giving my talk (prezi here) about modeling challenges. The audience was fantastic and truly interdisciplinary, and they pushed back and challenged me in a great way. I’m glad I went and I’m glad Tess Wise invited me.

One issue that came up is something I want to talk about today, because I hear it all the time and it’s really starting to bug me.

Namely, the fallacy that people, especially young people, are “happy to give away their private data in order to get the services they love on the internet”. The actual quote came from the IBM guy on the congressional subcommittee panel on big data, which I blogged about here (point #7), but I’ve started to hear that reasoning more and more often from people who insist on side-stepping the issue of data privacy regulation.

Here’s the thing. It’s not that people don’t click “yes” on those privacy forms. They do click yes, and I acknowledge that. The real problem is that people generally have no clue what it is they’re trading.

In other words, this idea of a omniscient market participant with perfect information making a well-informed trade, which we’ve already seen is not the case in the actual market, is doubly or triply not the case when you think about young people giving away private data for the sake of a phone app.

Just to be clear about what these market participants don’t know, I’ll make a short list:

• They probably don’t know that their data is aggregated, bought, and sold by Acxiom, which they’ve probably never heard of.
• They probably don’t know that Facebook and other social media companies sell stuff about them even if their friends don’t see it and even though it’s often “de-identified”. Think about this next time you sign up for a service like “Bang With Friends,” which works through Facebook.
• They probably don’t know how good algorithms are getting at identifying de-identified information.
• They probably don’t know how this kind of information is used by companies to profile users who ask for credit or try to get a job.

Conclusion: people are ignorant of what they’re giving away to play Candy Crush Saga[1]. And whatever it is they’re giving away, it’s something way far in the future that they’re not worried about right now. In any case it’s not a fair trade by any means, and we should stop referring to it as such.

What is it instead? I’d say it’s a trick. A trick which plays on our own impulses and short-sightedness and possibly even a kind of addiction to shiny toys in the form of candy. If you give me your future, I’ll give you a shiny toy to play with right now. People who click “yes” are not signaling that they’ve thought deeply about the consequences of giving their data away, and they are certainly not making the definitive political statement that we don’t need privacy regulation.

1. I actually don’t know the data privacy rules for Candy Crush and can’t seem to find them, for example here. Please tell me if you know what they are.

Categories: data science, modeling, rant