Archive

Author Archive

Advice for anxious sleepers

When I was young I used to suffer from depression from time to time, sometimes pretty badly. But ever since I had kids, I suffer much more from anxiety. It’s never been paralyzing but it means I have trouble falling asleep about once or twice a week because I can’t stop fretting. I’m jealous of people that can wander off into fantasy land and imagine landing on the moon or walking across a grassy plain in a magical land, but that’s not me. I basically have no imagination and spend my brain cycles trying to solve really concrete problems, and if there’s something out of my control then it bugs me and I have trouble letting go.

I’m sure I’m not the only one with this problem, and maybe I should be learning to meditate or something so I’m better at flexing my imagination muscles. Bring on the advice. In the meantime I’ve developed intense and complicated coping mechanisms. Here are a few in the form of friendly advice to all who suffer from anxiety at night:

  1. First of all, don’t worry about being worried. Chances are the next night you will be super exhausted and catch up on sleep, no harm done. Important to keep in mind!!
  2. Second, I really like to listen to the radio. Sports radio is almost always soothingly boring (although lately, what with all the wife beating talk, it has been less than helpful), and of course an actual baseball game is perfect, because nothing ever happens.
  3. But my husband can only sleep in total silence. Here’s the solution to this problem, which helps a LOT:

    Product motto: "This pillow speaker has saved a marriage or two ..."

    Product motto: “This pillow speaker has saved a marriage or two …”

  4. If that isn’t enough, then I usually go to the living room and watch boring movies on Netflix.
  5. I found the best, most boring movie EVER yesterday which I wanted to share with you. Namely, Nature’s The Private Life of Deer. That was seriously boring, and yet funny and nice too, especially when the “ghost deer photographer” was whispering to the camera about his strategies in tracking the ever-elusive albino deer in the northern woods.
  6. The video for that is available here, but I urge you to save it for when you have trouble sleeping and are trying not to think of something anxiety-provoking, it’ll be perfect.
Categories: musing

The Head First book series

I’ve been reading Head First Java this past week and I’m super impressed and want to tell you guys about it if you don’t already know.

The stuff inside is even sillier.

The stuff inside is even sillier.

I wanted to learn what the big fuss was about object-oriented programming, plus it seems like all the classes my Lede students are planning to take either require python or java, so this seemed like a nice bridge.

But the book is outstanding, with quirky cartoons and a super fun attitude, and I’m on page 213 after less than a week, and yes that’s out of more than 600 pages but what I’m saying is that it’s a thrilling read.

My one complaint is how often the book talks about motivating programmers with women in tight sweaters. And no, I don’t think they were assuming the programmers were lesbians, but I could be wrong and I hope I am. At the beginning they made the point that people remember stuff better when there is emotional attachment to things, so I’m guessing they’re getting me annoyed to help me remember details on reference types.

Here’s another Head First book which my nerd mom recommended to me some time ago, and I bought but haven’t read yet, but now I really plan to: Head First Design Patterns. Because ultimately, programming is just a tool set and you need to learn how to think about constructing stuff with those tools. Exciting!

And by the way, there is a long list of Head First books, and I head good things about the whole series. Honestly I will never write a technical book in the old-fashioned dry way again.

The problem with charter schools

Today I read this article written by Allie Gross (hat tip Suresh Naidu), a former Teach for America teacher whose former idealism has long been replaced by her experiences in the reality of education in this country. Her article is entitled The Charter School Profiteers.

It’s really important, and really well written, and just one of the articles in the online magazine Jacobin that I urge you to read and to subscribe to. In fact that article is part of a series (here’s another which focuses on charter schools in New Orleans) and it comes with a booklet called Class Action: An Activist Teacher’s Handbook. I just ordered a couple of hard copies.

I’d really like you to read the article, but as a teaser here’s one excerpt, a rant which she completely backs up with facts on the ground:

You haven’t heard of Odeo, the failed podcast company the Twitter founders initially worked on? Probably not a big deal. You haven’t heard about the failed education ventures of the person now running your district? Probably a bigger deal.

When we welcome schools that lack democratic accountability (charter school boards are appointed, not elected), when we allow public dollars to be used by those with a bottom line (such as the for-profit management companies that proliferate in Michigan), we open doors for opportunism and corruption. Even worse, it’s all justified under a banner of concern for poor public school students’ well-being.

While these issues of corruption and mismanagement existed before, we should be wary of any education reformer who claims that creating an education marketplace is the key to fixing the ills of DPS or any large city’s struggling schools. Letting parents pick from a variety of schools does not weed out corruption. And the lax laws and lack of accountability can actually exacerbate the socioeconomic ills we’re trying to root out.

The app effect

I have a theory which I’m slightly embarrassed about but whatever, that’s what blogs are for, I’m going to talk about it. And I have no data for this whatsoever, although I’d be interested to hear thoughts on how to collect some.

Namely, I think a sizable amount of social change we’ve seen in the past few decades, for better and for worse, can be ascribed to what I call “the app effect,” namely the tendency for everyone, but young men in particular to be playing games on their phones or their xbox360’s or whatever rather than interacting with each other.

Look at crime rates. I am not claiming that crime rates have fallen solely because of the app effect over other reasonable effects, like the availability of abortions, or less lead paint, or people having more air conditioning.

But, let’s face it, when I was growing up in Boston in the 1980’s, you’d just see way more people out on the streets on summer evenings because it was too freaking hot to do anything inside and people were damn bored. That’s when the trouble would start. Nowadays you just don’t see that nearly as much. What are people doing? My guess is that they’re playing a shit load of video games. Tell me if I’m wrong.

Here’s another example. People are less politically engaged. Partly it’s because Congress sucks, but partly – yes – it’s because people are playing Candy Crush! They used to maybe spend time going to work reading the paper and otherwise doing the civic duty thing but nowadays they’re just trying to pass level 187. I’ve been there so I know about it.

Also, when the train stops? In the tunnel? And it’s dark and really hot? Everyone just plays their games even harder, where you used to maybe start talking, or shouting, or freaking out. It is a pacifier for grown-ups, a nationwide babysitting service that keeps people in line.

It’s good and bad. Sometimes getting out of line serves a purpose, sometimes it’s just destructive and the wrong thing to do. My worry, as a person who wants to see political engagement, is that we have trained an entire population to take refuge in a pointless activity that doesn’t serve any real purpose except to distract us and to mollify us, not to mention collect our data for later marketing purposes.

Another way to imagine this is, if all the apps and all the video games stopped working for a few weeks, what would happen? What would people do with themselves?

Categories: musing

Aunt Pythia’s amazing advice

Well hello there, cutie, and welcome. Aunt Pythia loves you today, even more than usual!

For some reason she can’t pinpoint, but probably has to do with a general feeling of happiness and fulfillment, Aunt Pythia is even more excited than usual to be here and to toss off unreasonably smug and affectionate opinions and advice. Buckle up and get ready for the kisses and the muffins.

The kisses are harder to picture but they are even more delicious.

The kisses are harder to picture but they are even more delicious.

Everyone set? OK, fabulous, let’s get going. Oh and by the way, at the bottom of the column please please

think of something to ask Aunt Pythia at the bottom of the page!

I am almost out of questions!!!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.

——

Dear Aunt Pythia,

How should one deal with sexism and harassment at conferences?

As a white heterosexual male mathematician, I don’t experience much bias against me in my professional life, but I’ve seen (and heard of) a lot of bad stuff happening against anyone not conforming to this norm, which I think is not only bad for the people who experience this, but also bad for mathematics as a whole, for various reasons.

At a recent specialized conference, one of participants (a grad student) was very obviously sexually interested in one of the other grad students (one of only 2 female participants, my field has some serious problems in this regard), who was clearly not interested (and married).

I didn’t know these persons before the conference, and beyond me saying to the the harassing person that she was married and that he shouldn’t annoy her (which didn’t have any impact of course), I didn’t do anything. I would have liked to somehow help the harassed party feel welcome, and communicate that besides that one jerk people were interested in her mathematical ideas, but I didn’t know how to communicate this to her without making it seem inappropriate. So instead, I kept silent, which feels bad. Is there anything I could do next time I was in this type of situation, besides trying not to be a jerk?

Dr. Nonheroic Observer

Dear Dr. NO,

I gotta say, I love your question, but it’s kind of spare on details. What did the guy do? How much did it annoy the married party? It really matters, and my advice to you depends on those facts.

When I think about it, though, I don’t see why the fact that she’s married matters. Speaking as a 17-year married person (as of today!), married people like to flirt sometimes, so it’s not as if it’s intrinsically harassing for someone to express interest in a married person, or for that matter a single person.

But as soon as someone responds with a “not interested” signal, it is of course the responsibility of the interested party to tone it down.

Let me go into three scenarios here, and tell you what I think your response should be in each.

First, the guy likes her. You said it was obvious he was interested and it was also obvious she wasn’t. Depending on how that played out, it could be totally fine and not at all your responsibility to do anything. So, if he was like, hey would you like to go on a walk? and then she said, no thanks I’m going to get some work done and that was that, then whatevs. Again, not holding anything against someone for interest per se.

Now on to the second scenario, which seems more likely, since you mentioned that he annoyed her in spite of your advice to him. So that means he followed her around a lot and generally speaking glommed on her, which probably means he obstructed her normal interaction with other mathematicians at the conference. This is a big problem, because conferences are when the “mathematical socializing” happens, which very often results in collaboration and papers. The fact that men glom onto women prevents that, and might be a reason women don’t join your field.

Your responsibility, beyond telling the guy to lay off, which you did, is to first of all talk math with her explicitly, so she gets some mathematical socializing done. Also be proactive in introducing her to other people who are good math socializers.

Beyond that, I think you need to tell the guy to stop a second time. Ask the guy to think about why she came to the conference, and what she wants and doesn’t want out of the experience. In other words, make him try to think about her perspective rather than his own dick’s perspective. Who knows, it might help, he might just be super nerdy and not actually an asshat.

If that doesn’t work, and if he is in fact an asshat, I suggest you go to her and ask her if he is bothering you. Pretending not to notice isn’t helping her, and she probably has nobody to appeal to and could use an ally. If she says yes, then with her permission, go back to the guy and tell him he is officially bothering her. I guess that would actually work.

Third scenario is when even that doesn’t work, in which case I would go to the organizer of the conference and suggest that the harasser be asked to leave the conference.

I’d be super interested to hear your thoughts, and in particular what you think would happen if you had actually gone to the organizers. Of course, if you were one of the organizers yourself, I’d say you should have threatened the guy with expulsion earlier on.

Write back and tell me more details and tell me whether this advice was helpful!

Aunt Pythia

——

Dear Aunt Pythia,

Why EW? What is wrong with “He went on way too many dates too quickly”? What makes you the judge of what constitutes too many, when you yourself admit that you “have taken myself out of the sex game altogether – or at least the traditional sex game” so your opinion on traditional sex game (which is exactly what this guy is doing) is clearly biased. He is a modern empowered man who is exploring his options before settling down. What you wrote is nothing different than “slut-shaming” just reversing the gender. I hope you will exercise greater sensitivity in the future posts.

NY_NUMBERS

Dear NY_NUMBERS,

I think you must be referring to my response to Huh in this past column in reference to the alleged math genius who “hacked” OK Cupid. And I think you misunderstand me.

I am all for slutty behavior. In fact I am super sex positive. If the guy were just trying to get lots of great sex with lots of amazing women, then more power to him. I’d tell him about Tinder and I’d even direct him to critiquemydickpic for useful and amusing advice.

But actually he was having one or two dates per day looking for love. What?! That’s way too much emotional drainage. How can anyone remain emotionally receptive if they can’t even remember people’s names? I’d be much much happier for him, and I wouldn’t be judgmental, if he had been bringing home a different woman every night for mind-blowing sex. Youth!!

So, if you want to complain about my “ew”, then I think you’d need to say that, if someone can fuck anything that moves, they should also be able to love anything that moves. I’m not sure there’s a name for this but maybe “love-shaming?”.

In any case, I stand by my “ew”: I don’t think loving one or two people per day is possible. And the woman he ended up with found him, which was different and broke his cycle, kind of proving my point.

Aunt Pythia

——

Dear Aunt Pythia,

I’m a statistician with four-or-so years of work experience, but currently in the last half year or so of a applied bayesian stats PhD. I have seen the rise of R and Statistics as a hot, talked about subject. And for some reason, I am getting nervous about all the new cool kids that play around on Kaggle; that they will take ALL THE JOBS, and that there will be no space for slightly less cool, more classically trained statisticians such as myself. After all, all we’re doing is a bit of running a glm, or a cluster analysis, or some plotting. A monkey could learn that in three months. Sometimes I wish everyone would stay away and let me have all the datasets for myself.

Am I being unreasonably nervous about the future?

Have Stats Want to Analyze

Dear HSWA,

First, I wanna say, I had high hopes for your sign off until I wrote it out. Then I was like, wtf?! I even googled it but all I came back with was the Hampton Shaler Water Authority. And I am pretty sure that’s not what you meant. And keeping the “t” in didn’t help.

Second, I’ve got really good advice for you. Next time you’re in an interview, or even if you’re just on a bus somewhere with someone sitting next to you who allows you to talk, mention that Kaggle competitions are shitty bars for actual data scientists, because most of the work of the data scientist is figuring out what the actual question is, and of course how to measure success.

Those things are backed into each Kaggle competition, so hiring people who are good at Kaggle competitions is like hiring the chef who has been supplied with a menu, a bunch of recipes, and all the ingredients to run your new restaurant. Bad idea, because that’s the job of the chef if he’s actually good. In other words, it’s not actually all that impressive to be able to follow directions, you need to be creative and thoughtful.

Make sure you say that to your interviewer, and then follow it up with a story where you worked on a problem and solved it but then realized you’d answered the wrong question and so you asked the right question and then solved that one too.

I’m not nervous for you, thoughtful statisticians are in high demand. Plus you love data, so yeah you’re good.

Good luck!

Aunt Pythia

——

Dear Aunt Pythia,

I’ve been working as faculty in a new department this year and I have repeatedly had the feeling that the support staff is not treating me the way they would if I were 50 and male instead of young and female (although with the rank of professor).

It’s small things like roundly scolding me for using a coffee mug from the wrong cupboard, or hinting I should make sure the kitchen cleaning is easy for staff (I’m not messy!), or the conference support staff ceasing to help with basic support on a conference (and complaining about me to other people), or wanting me to walk some mail to another building.

I realize this is all small potatoes. But I have started to feel like by just taking it passively (e.g. smiling and nodding) I might be saving myself time and anger now but I’m helping to perpetuate the system. I rigorously avoid confrontation and I think I’m typically regarded as a very friendly and helpful team player by my peers. (How could I prove bias anyway, and would confrontation help?). But I’m not sure I can spend my whole life putting up with small potatoes along with the bigger potatoes I encounter from time to time.

Spud Farmer Considering Pesticides

Dear SFCP,

First of all, again, disappointed your sign-off didn’t spell anything. But will let it pass.

Second of all, my guess is that they are sexist. I have a prior on this because I’ve encountered so much sexism in this exact way.

Third of all, I’m also guessing they are administrative people in academia, which means they are also just barely able and/or willing to do their jobs. Again, experience, and since I am administration now in academia, I am allowed to call it. Some people are great, most people are not.

Fourth, I don’t know why you are “rigorously avoiding confrontation” here. The very first thing you should do is choose your tiny battles wisely and create small but useful confrontation. Examples:

  • Someone asks you to mail a letter. You say, “oh who usually mails letters? I will be sure to bring it to them.”
  • Someone doesn’t want to do their part in helping with basic support on conferences. You say, “Oh that’s not your job? I am so sorry. Who should I be asking for help on this?”
  • Someone scolds you for using the wrong coffee cup or some such nonsense. You say, “I am new here and I don’t know the rules but I will be sure to remember this one! I am one of those people with a strong work ethic, and it’s great to see how people around here pull together and make things happen.” You know, be aspirational.

Fifth, if it comes to it, get a faculty ally to explain which staff are bitter and why, and which of them are juts plain nuts, and which ones do everyone else’s job. Useful information. Make sure it’s an ally! Complaining about this stuff to the wrong person could give you a reputation as a complainer.

Sixth, do not let this stuff build up inside you! Make it an amusing part of your day to see how people wiggle out of their responsibilities and blame other people for their mistakes. And keep in mind that the faculty are probably the biggest and best examples of such behavior.

Love,

Auntie P

——

Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

Categories: Aunt Pythia

Nerding out: RSA on an iPython Notebook

Yesterday was a day filled with secrets and codes. In the morning, at The Platform, we had guest speaker Columbia history professor Matthew Connelly, who came and talked to us about his work with declassified documents. Two big and slightly depressing take-aways for me were the following:

  • As records have become digitized, it has gotten easy for people to get rid of archival records in large quantities. Just press delete.
  • As records have become digitized, it has become easy to trace the access of records, and in particular the leaks. Connelly explained that, to some extent, Obama’s harsh approach to leakers and whistleblowers might be explained as simply “letting the system work.” Yet another way that technology informs the way we approach human interactions.

After class we had section, in which we discussed the Computer Science classes some of the students are taking next semester (there’s a list here) and then I talked to them about prime numbers and the RSA crypto system.

I got really into it and wrote up an iPython Notebook which could be better but is pretty good, I think, and works out one example completely, encoding and decoding the message “hello”.

The underlying file is here but if you want to view it on the web just go here.

How to think like a microeconomist

Yesterday we were pleased to have Suresh Naidu guest lecture in The Platform. He came in and explained, very efficiently because he was leaving at 11am for a flight at noon at LGA (which he made!!) how to think like an economist. Or at least an applied microeconomist. Here are his notes:

Applied microeconomics is basically organized a few simple metaphors.

  1. People respond to incentives.
  2. A lot of data can be understood through the lens of supply and demand.
  3. Causality is more important than prediction.

There was actually more on the schedule, but Suresh got into really amazing examples to explain the above points and we ran out of time. At some point, when he was describing itinerant laborers in the United Arab Emirates, and looking at pay records and even visiting a itinerant labor camp, I was thinking that Suresh is possibly an undercover hardcore data journalist as well as an amazing economist.

As far as the “big data” revolution goes, we got the impression from Suresh that microeconomists have been largely unmoved by its fervor. For one, they’ve been doing huge analyses with large data sets for quite a while. But the real reason they’re unmoved, as I infer from his talk yesterday, is that big data almost always focuses on descriptions of human behavior, and sometimes predictions, and almost never causality, which is what economists care about.

A side question: why is it that economists only care about causality? Well they do, and let’s take that as a given.

So, now that we know how to think like an economist, let’s read this “Room For Debate” about overseas child labor with our new perspective. Basically the writers, or at least three out of four of them, are economists. So that means they care about “why”. Why is there so much child labor overseas? How can the US help?

The first guy says that strong unions and clear signals from American companies works, so the US should do its best to encourage the influence of labor unions.

The lady economist says that bans on child labor are generally counterproductive, so we should give people cash money so they won’t have to send their kids to work in the first place.

The last guy says that we didn’t even stop having child labor in our country until wage workers were worried about competition from children. So he wants the U.S. to essentially ignore child labor in other countries, which he claims will set the stage for other countries to have that same worry and come to the same conclusion by themselves. Time will help, as well as good money from the US companies.

So the economists don’t agree, but they all share one goal: to figure out how to tweak a tweakable variable to improve a system. And hopefully each hypothesis can be proven with randomized experiments and with data, or at least evidence can be gathered for or against.

One more thing, which I was relieved to hear Suresh say. There’s a spectrum of how much people “believe” in economics, and for that matter believe in data that seems to support a theory or experiment, and that spectrum is something that most economists run across on a daily basis. Even so, it’s not clear there’s a better way to learn things about the world than doing your best to run randomized experiments, or find close-to-randomized experiments and see how what they tell you.

Crime and punishment

When I was prepping for my Slate Money podcast last week I read this column by Matt Levine at Bloomberg on the Citigroup settlement. In it he raises the important question of how the fine amount of $7 billion was determined. Here’s the key part:

 Citi’s and the Justice Department’s approaches both leave something to be desired. Citi’s approach seems to be premised on the idea that the misconduct was securitizing mortgages: The more mortgages you did, the more you gotta pay, regardless of how they performed. The DOJ’s approach, on the other hand, seems to be premised on the idea that the misconduct was sending bad e-mails about mortgages: The more “culpable” you look, the more it should cost you, regardless of how much damage you did.

would have thought that the misconduct was knowingly securitizing bad mortgages, and that the penalties ought to scale with the aggregate badness of Citi’s mortgages. So, for instance, you’d want to measure how often Citi’s mortgages didn’t match up to its stated quality-control standards, and then compare the actual financial performance of the loans that didn’t meet the standards to the performance of the loans that did. Then you could say, well, if Citi had lived up to its promises, investors would have lost $X billion less than they actually did. And then you could fine Citi that amount, or some percentage of that amount. And you could do a similar exercise for the other big banks — JPMorgan, say, which already settled, or Bank of America, which is negotiating its settlement — and get comparable amounts that appropriately balance market share (how many bad mortgages did you sell?) and culpability (how bad were they?).

I think he nailed something here, which has eluded me in the past, namely the concept of what comprises evidence of wrongdoing and how that translates into punishment. It’s similar to what I talked about in this recent post, where I questioned what it means to provide evidence of something, especially when the data you are looking for to gather evidence has been deliberately suppressed by either the people committing wrongdoing or by other people who are somehow gaining from that wrongdoing but are not directly involved.

Basically the way I see Levine’s argument is that the Department of Justice used a lawyerly definition of evidence of wrongdoing – namely, through the existence of emails saying things like “it’s time to pray.” After determining that they were in fact culpable, they basically did some straight-up negotiation to determine the fee. That negotiation was either purely political or was based on information that has been suppressed, because as far as anyone knows the number was kind of arbitrary.

Levine was suggesting a more quantitative definition for evidence of wrongdoing, which involves estimating both “how much you know” and “how much damage you actually did” to determine the damage, and then some fixed transformation of that damage becomes the final fee. I will ignore Citi’s lawyers’ approach since their definition was entirely self-serving.

Here’s the thing, there are problems with both approaches. For example, with the lawyerly approach, you are basically just sending the message that you should never ever write some things on email, and most or at least many people know that by now. In other words, you are training people to game the system, and if they game it well enough, they won’t get in trouble. Of course, given that this was yet another fine and nobody went to jail, you could make the argument – and I did on the podcast – that nobody got in trouble anyway.

The problem with the quantitative approach, is that first of all you still need to estimate “how much you knew” which again often goes back to emails, although in this case could be estimated by how often the stated standards were breached, and second of all, when taken as a model, can be embedded into the overall trading model of securities.

In other words, if I’m a quant at a nasty place that wants to trade in toxic securities, and I know that there’s a chance I’d be caught but I know the formula for how much I’d have to pay if I got caught, then I could include this cost, in addition to an estimate of the likelihood for getting caught, in an optimization engine to determine exactly how many toxic securities I should sell.

To avoid this scenario, it makes sense to have an element of randomness in the punishments for getting caught. Every now and then the punishment should be much larger than the quantitative model might suggest, so that there is less of a chance that people can incorporate the whole shebang into their optimization procedure. So maybe what I’m saying is that arriving at a random number, like the DOJ did, is probably better even though it is less satisfying.

Another possibility to actually deter crimes would be to arbitrarily increasing the likelihood of catching people up to no good, but that has been bounded from above by the way the SEC and the DOJ actually work.

Categories: finance, modeling

Everyone has a crush on me

So I have been getting some feedback lately on how I always assume everyone has a crush  on me. People know this is my typical assumption because I say things like, “oh yeah that guy totally has a crush on me.” And when I say “feedback,” what I mean is people joyfully accusing me of lying, or maybe just outraged by my preposterous claims, usually in a baffled and friendly manner. Just a few comments about this.

First of all, I also have a crush on everyone else. Just as often as I say “that lady has a crush on me,” I am known to say, “Oh my god I have a huge crush on her.” It’s more fun that way!

Second of all, there are all sorts of great consequences of thinking someone has a crush on you. To name a few:

  1. It’s not a sexual thing at all, it’s more like a willingness to think the other person is super awesome. I have crush on all sorts of people, men and women, etc. etc.. No categories left uncrushed except meanies.
  2. When you act like someone has a crush on you, they are more likely to develop a crush on you. This is perhaps the most important point and should be first, but I wanted to get the first point out of the way. It’s what I call a positive feedback loop.
  3. It makes you feel great to be around someone if they have a crush on you, or even if you just think they do.
  4. What’s the worst thing that can happen? Answer: that you’re wrong, and they don’t have a crush on you, but then they’ll just walk away thinking that you were weirdly happy to see them, which is not so bad, and may in fact make them crush out on you. See what I mean?
  5. It’s a nice world to live in where a majority of the people you run into have a crush on you. Try it and see for yourself!
Categories: feedback loop

The platonic solids

I managed to record this week’s Slate Money podcast early so I could drive up to HCSSiM for July 17th, and the Yellow Pig Day celebration. I missed the 17 talk but made it in time for yellow pig carols and cake.

This morning my buddy Aaron decided to let me talk to the kids in the last day of his workshop. First Amber is working out the formula for the Euler Characteristic of a planar graph with the kids and after that I’ll help them count the platonic solids using stereographic projection. If we have time we’ll talk about duals (update: we had time!).

I can never remember which one is the icosahedron.

I can never remember which one is the icosahedron.

Tonight at Prime Time I’ll play a game or two of Nim with them.

Categories: math, math education

The future of work

People who celebrate the monthly jobs report getting better nowadays often forget to mention a few facts:

  • the new jobs are often temporary or part-time, with low wages
  • the old lost jobs, which we lose each month, were often full-time with higher wages

I could go on, and I have, and mention the usual complaints about the definition of the unemployment rate. But instead I’ll take a turn into a thought experiment I’ve been having lately.

Namely, what is the future of work?

It’s important to realize that in some sense we’ve been here before. When all the farming equipment got super efficient and we lost agricultural jobs by the thousands, people swarmed to the cities and we started building things with manufacturing. So if before we had “the age of the farm,” we then entered into “the age of stuff.” And I don’t know about you but I have LOTS of stuff.

Now that all the robots have been trained and are being trained to build our stuff for us, what’s next? What age are we entering?

I kind of want to complain at this point that economists are kind of useless when it comes to questions like this. I mean, aren’t they in charge of understanding the economy? Shouldn’t they have the answer here? I don’t think they have explained it if they do.

Instead, I’m pretty much left considering various science fiction plots I’ve heard about and read about over the years. And my conclusion is that we’re entering “the age of service.”

The age of service is a kind of pyramid scheme where rich people employ individuals to service them in various ways, and then those people are paid well so they can hire slightly less rich people to service them, and so on. But of course for this particular pyramid to work out, the rich have to be SUPER rich and they have to pay their servants very well indeed for the trickle down to work out. Either that or there has to be a wealth transfer some other way.

So, as with all theories of the future, we can talk about how this is already happening.

I noticed this recent Bloomberg View article about how rich people don’t have normal doctors like you and me. They just pay out of pocket for super expensive service outside the realm of insurance. This is not new but it’s expanding.

Here’s another example of the future of jobs, which I should applaud because at least someone has a  job but instead just kind of annoys me. Namely, the increasing frequency where I try to make a coffee date with someone (outside of professional meetings) and I have to arrange it with their personal assistant. I feel like, when it comes to social meetings, if you have time to be social, you have time to arrange your social calendar. But again, it’s the future of work here and I guess it’s all good.

More generally: there will be lots of jobs helping out old people and sick people. I get that, especially as the demographics tilt towards old people. But the mathematician in me can’t help but wonder, who will take care of the old people who used to be taking care of the old people? I mean, they by definition don’t have lots of extra cash floating around because they were at the bottom of the pyramid as younger workers.

Or do we have a system where people actually change jobs and levels as they age? That’s another model, where oldish people take care of truly old people and then at some point they get taken care of.

Of course, much like the Star Trek world, none of this has strong connection to the economy as it is set up now, so it’s hard to imagine a smooth transition to a reasonable system, and I’m not even claiming my ideas are reasonable.

By the way, by my definition most people who write computer programs – especially if they’re writing video games or some such – are in a service industry as well. Pretty much anyone who isn’t farming or building stuff in manufacturing is working in service. Writers, poets, singers, and teachers included. Hell, the future could be pretty awesome if we arrange things well.

Anyhoo, a whimsical post for Thursday, and if you have other ideas for the future of work and how that will work out economically, please comment.

Categories: economics, modeling

Two great articles about standardized tests

In the past 12 hours I’ve read two fascinating articles about the crazy world of standardized testing. They’re both illuminating and well-written and you should take a look.

First, my data journalist friend Meredith Broussard has an Atlantic piece called Why Poor Schools Can’t Win At Standardized Testing wherein she tracks down the money and the books in the Philadelphia public school system (spoiler: there’s not enough of either), and she makes the connection between expensive books and high test scores.

Here’s a key phrase from her article:

Pearson came under fire last year for using a passage on a standardized test that was taken verbatim from a Pearson textbook.

The second article, in the New Yorker, is written by Rachel Aviv and is entitled Wrong Answer. It’s a close look, with interviews, of the cheating scandal from Atlanta, which I have been studying recently. The article makes the point that cheating is a predictable consequence of the high-stakes “data-driven” approach.

Here’s a key phrase from the Aviv article:

After more than two thousand interviews, the investigators concluded that forty-four schools had cheated and that a “culture of fear, intimidation and retaliation has infested the district, allowing cheating—at all levels—to go unchecked for years.” They wrote that data had been “used as an abusive and cruel weapon to embarrass and punish.”

Putting the two together, it’s pretty clear that there’s an acceptable way to cheat, which is by stocking up on expensive test prep materials in the form of testing company-sponsored textbooks, and then there’s the unacceptable way to cheat, which is where teachers change the answers. Either way the standardized test scoring regime comes out looking like a penal system rather than a helpful teaching aid.

Before I leave, some recent goodish news on the standardized testing front (hat tip Eugene Stern): Chris Christie just reduced the importance of value-added modeling for teacher evaluation down to 10% in New Jersey.

The Platform starts today

Hey my class starts today, I’m totally psyched!

The syllabus is up on github here and I prepared an iPython notebook here showing how to do basic statistics in python, and culminating in an attempt to understand what a statistically significant but tiny difference means, in the context of the Facebook Emotion study. Here’s a useless screenshot which I’m including because I’m proud:

Screen Shot 2014-07-15 at 7.04.05 AM

If you want to follow along install anaconda on your machine and type “ipython notebook –pylab inline” into a terminal. Then you can just download this notebook and run it!

Most of the rest of the classes will feature an awesome guest lecturer, and I’m hoping to blog about those talks with their permission, so stay tuned.

Surveillance in NYC

There’s a CNN video news story explaining how the NYC Mayor’s Office of Data Analytics is working with private start-up Placemeter to count and categorize New Yorkers, often with the help of private citizens who install cameras in their windows. Here’s a screenshot from the Placemeter website:

From placemeter.com

From placemeter.com

You should watch the video and decide for yourself whether this is a good idea.

Personally, it disturbs me, but perhaps because of my priors on how much we can trust other people with our data, especially when it’s in private hands.

To be more precise, there is, in my opinion, a contradiction coming from the Placemeter representatives. On the one hand they try to make us feel safe by saying that, after gleaning a body count with their video tapes, they dump the data. But then they turn around and say that, in addition to counting people, they will also categorize people: gender, age, whether they are carrying a shopping bag or pushing strollers.

That’s what they are talking about anyway, but who knows what else? Race? Weight? Will they use face recognition software? Who will they sell such information to? At some point, after mining videos enough, it might not matter if they delete the footage afterwards.

Since they are a private company I don’t think such information on their data methodologies will be accessible to us via Freedom of Information Laws either. Or, let me put that another way. I hope that MODA sets up their contract so that such information is accessible via FOIL requests.

Aunt Pythia’s advice

Aunt Pythia welcomes you after one week away celebrating her middle son’s and the nation’s birthday. She’s not sure she will be able to incorporate such a topic into the Q&A so she’s jumping on the opportunity to spread the love emanating from this video (hat tip Mike Hill):

It comes from this webpage entitled Putting the Crotch Into Crochet which you really need to check out if the above video titillates, which really how could it not.

To business! Aunt Pythia is doing a speed round today, after grabbing her oldest from a JFK redeye and before making said son his favorite breakfast of banana and chocolate chip pancakes.

You ready? Strap on your seat belts, we’re still driving the luxury Winnebego!

Without further ado, let’s begin. And please, after enjoying the on-board cheese and cracker snacks, do your best to

think of something to ask Aunt Pythia at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.

——

Dear Aunt Pythia,

Seven years ago I was diagnosed with a brain tumor. It’s grow in back four times since, once during chemotherapy. Doctors consistently toss around words like “inevitable” and “incurable” when talking about my tumor and its recurrence.

But still, I have probably at least a decade, maybe more, depending on how medical science goes. And that’s a long time to spend alone.

But when I go out on dates, I feel like I’m leading women on by not disclosing my potential expiration date. When in a relationship would you recommend revealing this key fact?

Not Left Brained

Dear NLB,

First of all, I am sorry this is happening to you, it sucks.

Second of all, this is your private information, and you have no obligation to tell people private stuff before you’re ready. When you go on a date with someone, that’s merely an offer to spend an evening with someone, and most people don’t think beyond that 4 hour obligation, nor should you.

At the same time, you do have the obligation to not mislead, as everyone does. So third of all, that means that you wouldn’t want to start living with someone or otherwise get serious without them knowing your status.

I imagine this kind of thing comes up almost immediately in relationships, possible even as soon as the first date, when a woman might ask you if you want children. My suggestion is to tell her, or anyone else mentioning long term plans, that you don’t have long-term plans for anything, nor are you expecting to. That is sufficiently vague – yet also sufficiently transparent – so nobody would accuse you of being misled. Women who want kids, say, or to get married, will interpret that appropriately. It will also sort out people who hang out with you simply to enjoy your company, which I assume is what you’re going for.

Good luck!

Aunt Pythia

——

Dear Auntie P,

I’m a woman in a graduate program which is heavily female-dominated (so not math, clearly). Like most grad students, I’ve got some dear friends and some real stinkers in my cohort, with plenty in between.

I was having lunch this week with one of the newer students, “Belle”, in the program. Ostensibly this was a working lunch, but somehow Belle managed to squeeze in the fact that she was in a new and exciting relationship with another woman in the program, “Linda”.

The problem is that I’m much better friends with Linda than I am with Belle, and Linda isn’t out. To anyone (or at least anyone in the program), including me. Well, until now.

How do I handle this? Do I gently inform Belle that Linda is closeted and she needs to get her approval before outing her, even to her friends? Or do I hope that she notices on her own what she’s doing, and notices before she does something damaging? Also, when I’m around Linda, do I continue to act as if I know nothing about her sexuality? (Honestly, this isn’t that hard, since her personal life is not something she brings up much.) Also, when I’m in a social situation where both of them are present, do I act as if I don’t know they’re together (and be awkward towards Belle) or as if I do (thus putting Linda in a bind)?

Closets Inform Every Lunch I Take Out

Dear CIELITO,

Nice sign-off! I had to use a Spanish dictionary, but I’m impressed.

OK so first I’ll give you good advice and then I’ll tell you what I’d do.

The good advice is to stay out of it and pretend you are oblivious. It’s really none of your business and you don’t want to get in the middle of something potentially messy.

The thing I’d do is tell Linda what happened, so she can address it with Belle if in fact it’s not what she wants. After all, Linda is your friend and she should know what’s going on.

Tell me what happens!

Auntie P

——

Dear Aunt Pythia,

I still wonder if brute force, generate and test is a viable method for discovering good parameter settings for a system. I don’t like how long the programs take to run, but they seem to provide good information. I assume that you would have a better idea, just because you probably would be in the “neat” perspective, while I am definitely, and in long standing, a “scruffy”.

Lost in Space

Dear Lost,

I have practically no idea what you’re talking about but I like people who call themselves both lost and a scruffy. As for brute force optimization, yes go ahead but remember to have a clean data set to test your parameters on, because you’re surely overfitting.

Aunt Pythia

——

Dear Aunt Pythia,

To what extent are women obliged to “stand and fight” when working in fields that are male-dominated and where they feel slighted on a regular basis? I am tired of seeing people to go my male colleagues for information in which I have superior expertise, for example. And god forbid you should be a woman working in computational/applied mathematics since applied math is already looked down upon. Even male TA’s are disrespectful.

On the one hand, if all the women are pushed away, we have no women to serve as role models for the next generation. On the other, each of us has only one life to live. I feel that I deserve to be happy, deserve to be respected, and so on.

I am pretty fed up. I don’t want to become one of the bitter and bitchy ones, and I don’t want to give up my career goals. Any thoughts?

Woman in Computing

Dear WiC,

There is absolutely no obligation at all to stand and fight, by any woman or man, whatsoever. It’s a silly argument that one should role model for a position that’s miserable. It’s almost ludicrously bait and switch, in fact.

Having said that, there’s usually a reason that people are competitive with each other. In business it’s almost always about money (or status, but those two are highly correlated). In academics it’s all about status, and men do it to each other as well, although the fight is dirtier when it’s directed towards women.

So, I’m not sure this will help, but if you see the fighting and competition as a direct product of the system, it might help you to take it less personally. Personally, I’ve been in so many different contexts, and I exist as such a threat against other people (both men and women), that I recognize sexist pushback almost as a sport (how does sexist pushback work in journalism? Oh, that’s how).

I’m not saying it never gets to me, because it does, but not for long. Because in the end it’s an external distraction, and staying external is always a mistake, just look at the dieting industry.

My best advice is to keep your eyes on the prize: figure out what your agenda is, and go for it. And don’t be surprised that, as you get closer to the goal, people will be more threatened, not less, and they will embarrass themselves with bad behavior. Don’t get distracted, because you have to stay internally focused.

In other words, it’s not about some vague obligation to society. It’s about a very real obligation towards yourself, which you set.

Good luck!

Aunt P

——

Please submit your well-specified, fun-loving, cleverly-abbreviated question to Aunt Pythia!

 

Categories: Aunt Pythia

The Lede Program students are rocking it

Yesterday was the end of the first half of the Lede Program, and the students presented their projects, which were really impressive. I am hoping some of them will be willing to put them up on a WordPress site or something like that in order to showcase them and so I can brag about them more explicitly. Since I didn’t get anyone’s permission yet, let me just say: wow.

During the second half of the program the students will do another project (or continue their first) as homework for my class. We’re going to start planning for that on the first day, so the fact that they’ve all dipped their toes into data projects is great. For example, during presentations yesterday I heard the following a number of times: “I spent most of my time cleaning my data” or “next time I will spend more time thinking about how to drill down in my data to find an interesting story”. These are key phrases for people learning lessons with data.

Since they are journalists (I’ve learned a thing or two about journalists and their mindset in the past few months) they love projects because they love deadlines and they want something they can add to their portfolio. Recently they’ve been learning lots of geocoding stuff, and coming up they’ll be learning lots of algorithms as well. So they’ll be well equipped to do some seriously cool shit for their final project. Yeah!

In addition to the guest lectures I’m having in The Platform, I’ll also be reviewing prerequisites for the classes many of them will be taking in the Computer Science department in the fall, so for example linear algebra, calculus, and basic statistics. I just bought them all a copy of How to Lie with Statistics as well as The Cartoon Guide to Statistics, both of which I adore. I’m also making them aware of Statistics Done Wrong, which is online. I am also considering The Cartoon Guide to Calculus, which I have but I haven’t read yet.

Keep an eye out for some of their amazing projects! I’ll definitely blog about them once they’re up.

Great news: for-profit college Corinthian to close

I’ve talked before about the industry of for-profit colleges which exists largely to game the federal student loan program. They survive almost entirely on federal student loans of their students, while delivering terrible services and worthless credentials.

Well, good news: one of the worst of the bunch is closing its doors. Corinthian College, Inc (CCI) got caught lying about job placement of its graduates (in some cases, they said 100% when the truth was closer to 0%). They were also caught advertising programs they didn’t actually have.

But here’s what interests me the most, which I will excerpt from the California Office of the Attorney General:

CCI’s predatory marketing efforts specifically target vulnerable, low-income job seekers and single parents who have annual incomes near the federal poverty line. In internal company documents obtained by the Department of Justice, CCI describes its target demographic as “isolated,” “impatient,” individuals with “low self-esteem,” who have “few people in their lives who care about them” and who are “stuck” and “unable to see and plan well for future.”

I’d like to know more about how they did this. I’m guessing it was substantially online, and I’m guessing they got help from data warehousing services.

After skimming the complaint I’m afraid it doesn’t include such information, although it does say that the company advertised programs it didn’t have and then tricked potential students into filling out information about them so CCI could follow up and try to enroll them. Talk about predatory advertising!

Update: I’m getting some information by checking out their recent marketing job postings.

Categories: feedback loop, modeling

Zephyr Teachout to visit Alt Banking this Sunday

I’m excited to announce that Zephyr Teachout, a Fordham Law School professor who is running against Andrew Cuomo for Governor of New York, will be coming to speak to the Alternative Banking group next Sunday, July 13th, from 3pm-5pm in the usual place, Room 409 of the International Affairs Building at 118th and Amsterdam. More about Alt Banking on our website.

Title: Teachout-Wu vs. Cuomo-Hochul in the Democratic Primary in New York!

Description: Come hear candidate Teachout talk about her anti-corruption trust-busting campaign against Governor Cuomo.

Background: Teachout is an antitrust and media expert who served as the Director of Internet organizing for the 2004 Howard Dean Presidential Campaign. She co-founded A New Way Forward, an organization built to break up the power of big banks. Teachout was the first national director of the Sunlight Foundation. More here.

If we have time after talking to Zephyr we will discuss Stiglitz’s article,  The Myth Of America’s Golden Age.

Please make time to come hear Zephyr, and please spread the word.

Categories: #OWS

What constitutes evidence?

My most recent Slate Money podcast with Felix Salmon and Jordan Weissmann was more than usually combative. I mean, we pretty much always have disagreements, but Friday it went beyond the usual political angles.

Specifically, Felix thought I was jumping too quickly towards a dystopian future with regards to medical data. My claim was that, now that the ACA has motivated hospitals and hospital systems to keep populations healthy – a good thing in itself – we’re seeing dangerous side-effects involving the proliferation of health profiling and things like “health scores” attached to people much like we now have credit scores. I’m worried that such scores, which are created using data not covered under HIPAA, will be used against people when they try to get a job.

Felix asked me to point to evidence of such usage.

Of course, it’s hard to do that, partly because it’s just the beginning of such data collection – although the FTC’s recent report pointed to data warehouses that already puts people into categories such as “diabetes interest” – and also because it’s proprietary all the way down. In other words, web searches and the like are being legally collected and legally sold and then it’s legal to use risk scores or categories to filter job applications. What’s illegal is to use HIPAA-protected data such as disability status to remove someone from consideration for a job, but that’s not what’s happening.

Anyhoo, it’s made me think. Am I a conspiracy theorist for worrying about this? Or is Felix lacking imagination if he requires evidence to believe it? Or some combination? This is super important to me because if I can’t get Felix, or someone like Felix, to care about this issue, I’m afraid it will be ignored.

This kind of thing came up a second time on that same show, when Felix complained that the series of articles (for example this one from NY Magazine) talking about money laundering in New York real estate also lacked evidence. But that’s also tricky since the disclosure requirements on real estate are not tight. In other words, they are avoiding collecting evidence of money laundering, so it’s hard to complain there’s a lack of data. From my perspective the journalists investigating this article did a good job finding examples of laundering and showing it was easy to set up (especially in Delaware). But Felix wasn’t convinced.

It’s a general question I have, actually, and I’m glad to be involved with the Lede Program because it’s actually my job to think about this kind of thing, especially in the context of journalism. Namely, when do we require data – versus anecdotal evidence – to believe in something? And especially when the data is being intentionally obscured?

Critical Questions for Big Data by danah boyd & Kate Crawford

I’m teaching a class this summer in the Lede Program, starting in mid-July, which is called The Platform. Here’s the course description:

This course begins with the idea that computing tools are the products of human ingenuity and effort. They are never neutral and carry with them the biases of their designers and their design process. “Platform studies” is a new term used to describe investigations into these relationships between computing technologies and the creative or research products that they help to generate. How you understand how data, code, and algorithms affect creative practices can be an effective first step toward critical thinking about technology. This will not be purely theoretical, however, and specific case studies, technologies, and project work will make the ideas concrete.

Since my first class is coming soon, I’m actively thinking about what to talk about and which readings to assign. I’ve got wonderful guest lecturers coming, and for the most part the class will focus on those guest lecturers and their topics, but for the first class I want to give them an overview of a very large subject.

I’ve decided that danah boyd and Kate Crawford’s recent article, Critical Questions for Big Data, is pretty much perfect for this goal. I’ve read and written a lot about big data but even so I’m impressed by how clearly and comprehensively they have laid out their provocations. And although I’ve heard many of the ideas and examples before, some of them are new to me, and are directly related to the theme of the class, for example:

Twitter and Facebook are examples of Big Data sources that offer very poor archiving and search functions. Consequently, researchers are much more likely to focus on something in the present or immediate past – tracking reactions to an election, TV finale, or natural disaster – because of the sheer difficulty or impossibility of accessing older data.

Of course the students in the Lede are journalists, not academic researchers, which the article mostly addresses, and moreover they are not necessarily working with big data per se, but even so they are increasingly working with social media data, and moreover they are probably covering big data even if they don’t directly analyze it. So I think it’s still relevant to them. Or another way to express this is that one thing we will attempt to do in class is examine the extent to which their provocations are relevant.

Here’s another gem, directly related to the Facebook experiment I discussed yesterday:

As computational scientists have started engaging in acts of social science, there is a tendency to claim their work as the business of facts and not interpretation. A model may be mathematically sound, an experiment may seem valid, but as soon as a researcher seeks to understand what it means, the process of interpretation has begun. This is not to say that all interpretations are created equal, but rather that not all numbers are neutral.

In fact, what with this article and that case study, I’m pretty much set for my first day, after combining them with a discussion of the students’ projects and some related statistical experiments.

I also hope to invite at least one of the authors to come talk to the class, although I know they are both incredibly busy. Danah boyd, who recently came out with a book called It’s Complicated: the social lives of networked teensalso runs the Data & Society Research Institute, a NYC-based think/do tank focused on social, cultural, and ethical issues arising from data-centric technological development. I’m hoping she comes and talks about the work she’s starting up there.

Follow

Get every new post delivered to your Inbox.

Join 986 other followers