Author Archive

What are you thankful for in finance or economics?

It’s been a few days, I’ve been listening to Adele’s new album pretty much on loop while knitting and sewing curtains. So yes, it’s that nesting time of year, where we hunker down and seriously consume creamy spiked drinks.

Screen Shot 2015-11-24 at 7.12.52 AM.png

And by “we” I mean Americans, Canadians, Australians, and New Zealanders. Obviously we blame the hobbits on that last one.

Screen Shot 2015-11-24 at 7.13.05 AM.png

Well, here’s a question for you nog-quaffers: what are you thankful for from finance? I’ll extend it to the economy as well if you’d like.

The reason I’m asking is that this week, the Slate Money podcast I’m on is doing a special “thanksgiving” episode where we all talk about something we’re grateful for, and I’m having trouble coming up with something. Here’s what I’ve got so far:

  1. I’m grateful for consumer loans. After all, they help us out in rough times and allow us to invest in ourselves and our futures through mortgages and student loans. On the other hand, they also raise the price of everything through their availability. In fact I spent a couple of weeks ago on the show arguing that all college debt should be forgiven and that state colleges should be free. So I don’t think this works.
  2. I guess I’m thankful for inflation, in a sense. I mean, inflation makes it easier on debtors, since their debt is constantly dwindling in value, and it’s certainly better for an economy than deflation. But on the other hand, it can get out of hand and that’s bad, and it’s hard to control. So in the end I’m not actually all that excited by inflation.
  3. I could just be grateful for the entire financial system working at all. If you think about how much we depend on its functioning, to take out loans, to use our credit and debit cards, and to get paid monthly, it’s kind of amazing. On the other hand, if you think about the way finance deals with poor people, squeezing them for nickels and dimes, then you kind of lose respect. In fact it makes you want to be grateful for the CFPB instead, but that’s not financial enough.
  4. Finally, I’m thinking about how much I appreciate insurance. Yeah, I know there are plenty of problems with insurance (for example how cray-cray medical prices are for those without insurance, but I tend to blame a lack of reasonable transparency regulation on pricing in medicine on that, not insurance per se). But if you just think about how much insurance actually does for us, whether it’s medical or fire or car or life insurance, then you appreciate that it more or less functions as intended: to even out the bumpy risks of everyday life.

I’m still thinking about this question, and I’d love to hear your ideas!

Categories: Uncategorized

Do Charter Schools Cherrypick Students?

Yesterday I looked into quantitatively measuring the rumor I’ve been hearing for years, namely that charter schools cherrypick students – get rid of troublesome ones, keep well-behaved ones, and so on.

Here are two pieces of anecdotal evidence. There was a “Got To Go” list of students at one charter school in the Success Academy network. These were troublesome kids that the school was pushing out.

Also, I recently learned that Success Academy doesn’t accept new kids after the fourth grade. Their reasoning is that older kids wouldn’t be able to catch up with the rest of the kids, but on the other hand it also means that kids kicked out of one school will never land there. This is another form of selection.

Now that I’ve said my two examples I realize they both come from Success Academy. There really aren’t that many of them, as you can see on this map, but they are a politically potent force in the charter school movement.

Also, to be clear, I am not against charter schools as a concept. I love the idea of experimentation, and to the extent that charter schools perform experiments that can inform how public schools run, that’s interesting and worthwhile.

Anyhoo, let’s get to the analysis. I got my data from this DOE website, down at the bottom where I clicked “citywide results” and grabbed the following excel file:

With that data, I built an iPython Notebook which is on github here so you can take a look, reproduce my results with the above data (I removed the first line after turning it in to a csv file), or do more.

From talking to friends of mine who run NYC schools, I learned of two proxies for difficult students. One is ‘Percent Students with Disabilities’ and the other is ‘Percent English Language Learners’ (I also learned that charter schools’ DBN code starts with 84). Equipped with that information, I was able to build the following histograms:

Percent Students with Disabilities non-charter

Percent Students with Disabilities, non-Charter

Percent Students with Disabilities charter

Percent Students with Disabilities, Charter

Percent English Language Learners non-charter

Percent English Language Learners, non-Charter

Percent English Language Learners, Charter.png

Percent English Language Learners, Charter. Please note that the x-axis differs from above.

I also computed statistics which you can look at on the iPython notebook. Finally, I put it all together with a single scatterplot:

scatter disabilities, english learners

The blue dots to the left and all the way down on the x-axis are mostly test schools and “screened” schools, which are actually constructed to cherrypick their students.

The main conclusion of this analysis is to say that, generally speaking, charter schools don’t have as many kids with disabilities or poor language skills, and so when we compare their performance to non-charter schools, we need to somehow take this into account.

A final caveat: we can see just by looking at the above scatter plot that there are plenty of charter schools that are well inside the middle of the blue cloud. So this is not a indictment on any specific charter school, but rather a statistical statement about wanting to compare apples to apples.

Update: I’ve now added t-tests to test the hypothesis that this data comes from the same distribution. The answer is no.

Screen Shot 2015-11-19 at 11.06.13 AM

Those very small numbers are the p-values which are much smaller than 0.05. Other t-tests give similar results (but go ahead and try them yourself)

Categories: Uncategorized

Line Edits

Right now I’m eyeball deep in line edits for my book, Weapons of Math Destruction: how big data increases inequality and threatens democracy.

Or rather, I’m in the phase of minor(ish) edits from my editor (post-existential threats, anyway, and that’s a big deal!) which is before the next phase, where I’ll be dealing with issues from the actual line editor, the person who knows all about commas and what gets italicized versus quoted and so forth.

Then come the galleys, and along the way of course I help choose a cover design. After that the blurbs start (who should I ask?) and they print a bunch of copies in China and have it all shipped back here by boat. The process takes months and it’s all new to me.

Point is, my brain is completely occupied with this stuff, which is the opposite of sexy but on the other hand is exactly why the book will be better (hopefully!) than a blog – it will be actually carefully edited.

Everyone, fingers crossed, but the tentative launch date is September 5th of 2016. I know, it’s forever from now, but at least it’s an actual date. I’m already planning the party.

Categories: Uncategorized

Open Data Conference this Friday

I’m going to be on a panel Friday at a conference called Responsible Use of Open Data in Government and the Private Sector, which is being co-organized by Berkeley and NYU and is being held at NYU this Thursday evening and Friday all day.

The agenda is here. The first keynote, on Thursday night, is to be given by the Chief Analytics Officer of New York City, who I’m interested to hear from. I’m wondering what de Blasio’s administration is up to with respect to data and predictive modeling.

I’m going to miss Panel 2 because of my podcast taping, which is a shame, but I’m looking forward to Panel 1 which will discuss consequences of data sharing, unintended as well as intended: privacy concerns, discrimination concerns, and so on.

I’m on Panel 3, which with Panel 4 is devoted to the topic of private data use and collection versus the public good. The focus is on health care data and smart cities, but I will probably veer off to all kinds of ways that private companies use data to the detriment of the public, and how that should change.

Panel 5 discusses platforms for sharing data as well as the proposed governance of shared data. To be honest I’m a bit skeptical of the concept I’ve heard floated about recently that private companies will “donate” their data for the public good, but I’d love to be wrong.

Registration is free and open and available here.

Categories: Uncategorized

Debiasing techniques in science

My buddy Ernie Davis just sent me an article, published in Nature, called How scientists fool themselves – and how they can stopIt’s really pretty great – a list of ways scientists fool themselves, essentially through cognitive biases, and another list of ways they can try to get around those biases.

There’s even an accompanying graphic which summarizes the piece:


I’ve actually never heard of “blind data analysis” before, but I think it’s an interesting idea. However, it’s not clear how it would exactly work in a typical data science situation, where you perform exploratory data analysis to see what the data looks like, and you form a “story” based on that.

One thing they mentioned in the article but not in the graphic is the importance of having your research open sourced, which I think is the way to let the “devil’s advocacy” and “team of rivals” approaches actually happen in practice.

It’s all the rage nowadays to have meta analyses. I’d love for someone to somehow measure the ability of the above debiasing techniques to see which work well, and under what circumstances.

Categories: Uncategorized

Aunt Pythia’s advice

It’s a crisp autumn morning, and Aunt Pythia is deeply enjoying snuggling into her La-Z-Boy whilst wearing her cottony and fluffy hoodie, and she’s looking forward to a good long chat. She’s drinking coffee but she’s willing to make you tea if that’s your preference. In any case, make yourself comfortable, Aunt Pythia has some explaining to do.

Honestly I like both.

Honestly I like both.

Confession the first: recently Aunt Pythia has been going on somewhat of a craft binge. She’s taken to sewing lined curtains for her New York apartment, and the learning curve for someone who has never successfully done more than seam pants is steep. So far one prototype, a lopsided affair that looks much more 1970’s than she had envisioned. Stay tuned for updates, but please also make it your plan to sympathize with uneven hemming and puckers for a little while. Solidarity, people.

Also! Aunt Pythia readers, another confession/ brag. About a month ago Aunt Pythia received an email requesting her presence for an underwear modeling shoot, and she said yes (exact quote from CEO Julie at the shoot: “nobody has ever said yes that quickly. Most women have a million questions.”). It’s safe to say, dear readers, that the only question Aunt Pythia had about the underwear modeling gig was, why did it take you so long to ask me?

Two reasons this story might matter to you: first, you can check out the pictures here – please note Aunt Pythia’s hair goes with her shirt – and second, you can get an “Aunt Pythia discount” on Dear Kate underwear for women by using the discount code AuntPythia30, good through November 30th.

Now that all has been revealed! Aunt Pythia is getting on with the important stuff: your problems, ethical dilemmas, and general questions. Let’s do this, people! And after we do that, please don’t forget to:

ask Aunt Pythia any question at all at the bottom of the page!

By the way, if you don’t know what the hell Aunt Pythia is talking about, go here for past advice columns and here for an explanation of the name Pythia.


Dear Aunt Pythia,

In light of the discussion started by “Woman not at a bar”, I’ve been thinking about what women say about where it is OK to express romantic and sexual interest. I have been repeatedly told that men shouldn’t approach women they don’t know, because that shows that they only care about her looks.

Also, that a man shouldn’t approach a woman at a recreational activity (sports team, crafts, class, etc), because if she says no she might feel uncomfortable or scared and feel forced to drop out of the activity. One shouldn’t approach a woman at work, because that implies not taking her seriously as a professional. And approaching a close friend risks ruining the friendship.

All of these make sense to me individually, and if a woman says that something makes her uncomfortable I believe her, but they don’t seem to leave much room. Sometimes people tell me to wait for a woman to show clear signs of interest before making a move, but that seems to mean waiting forever (twice in the last ten years for me; one was a student in a class I was teaching, and the other one turned out not to be interested after all).

So is there any way to approach a woman that doesn’t make her feel threatened if she isn’t interested? And if not, perhaps it’s time for women to switch to making the first move as a rule?

A Single Heterosexual Adult Male Experiencing Distress


You had me until the student in your class. I don’t think it’s ever ok to “make a move” on a student in your class. I’m going to give you the benefit of the doubt and assume you meant you waited at least long enough so that she wasn’t a student anymore. That’s a requirement. But let’s put it aside. It’s over there, next to the sugar bowl.

Here’s the thing. I think you might have gotten bad advice, but I think it happened before your scrutiny of the situation set in.

Because yes, if you assume the set-up is, “I approach a woman, not knowing if she likes me at all, and I make my moves” then I agree, it’s really hard to know when that’s appropriate. In fact it might never be. But the good news is, that’s not how it actually works. Or at least I’ve never seen that approach work unless it’s at a bar or a party and everyone’s incredibly drunk and horny, and even then it often doesn’t work.

What actually works, IRL, is that you slowly orbit around someone that you’re interested in, and you pick up on positive feedback, and you test things out with the other person, and after a bit of back-and-forth, and some body language, and after she laughs at your jokes, and you laugh at hers, and after you both figure out how to spend more time together without making it seem like it’s on purpose, then you find yourselves “Interested” with a capital I. It’s a whole lot of very ambiguous, somewhat ambiguous, then not-so-ambiguous communication leading up to the first “move.”

It’s called flirting. It’s fun, and it’s the number one way you determine whether someone wants to date you. I suggest you practice doing it, because it’s basically a requirement for someone who wants to avoid the above misunderstandings.

Why do I say that? Because if you’re trying to find a girlfriend in your native environment, then yes, it’s generally speaking not appropriate unless the flirting has established it the two of you as “a possible thing.” You cannot abruptly “make a move” on someone you work with, or someone you play sports with, or someone on the streets you’ve never met, without seeming like a creep. You just can’t. And that’s because it is creepy, actually, and it’s creepy because there’s this technology called “flirting” which everyone knows about and is an expected and required lead-in to making a move. Think of flirting as a means of obtaining consent for a move.

Exceptions can be made in the following circumstances:

  1. You are being set up by mutual friends. So it’s already a date.
  2. You meet online at a dating website, so it’s already a date.

But even if the above things happen, and it’s “already a date,” I suggest you still diligently engage in the flirting phase anyway, because it’s still a great way of establishing mutual feedback, a non-creepy persona, and an atmosphere of lighthearted and sexy fun.

Wait, I hear you saying, how do I flirt so that it’s not creepy? How do I slowly but surely cross the spectrum from “friendly” to “sexy”?

So, when you encounter a woman you are attracted to, you are friendly, and you listen. You figure out what she likes, and what she likes about you. You do not think to yourself, “I am attracted to this women, when can I make a move?” but rather you think, “how do I know if she’s interested in me? what encouragement has she given me that she likes me, and what encouragement have I given her that I like her?”

Evidence of encouragement can be stuff like, in order of ambiguity (a very incomplete list!):

  1. she makes eye contact when you arrive and smiles
  2. she laughs at your jokes and asks you questions about yourself
  3. she touches your arm or hand when she talks to you
  4. she mentions she’s going somewhere and invites you to come along
  5. she sits on your lap and grinds

Flirting works kind of like a ladder, where you and the woman are both climbing it at the same time. If she is on the 4th rung, you should be on the 3rd, 4th, or 5th for you guys to stay close. If she goes one rung further than you, then you can keep up, and then even go one rung further yourself.

But by no means do you ever leave her behind on the ladder. Then she’d feel like, “Dude, I’m not keeping up with you, haven’t you noticed? Why aren’t you paying attention to where I am on this flirt ladder?” And you will come across as a creep. Creeps are people who aren’t paying attention to what the woman actually wants and are just barreling ahead based on what they want.

Does this all make sense? And, given this context, does it make sense that “making a move” on someone is almost always creepy? It’s basically like showing up at the top of the ladder. Maybe like stilts, except even less stable. And the woman is like, holy crap, you might fall right on top of my head and give me a concussion.

I hope this little story has helped. Now, go forth and flirt!


Aunt Pythia


Dear Aunt Pythia,

Re: the aunt pervert.

Another blogger I read got confused by the same sort of Google traffic and looked into it. Apparently it’s not incest, it’s that in India and Pakistan the word “aunt” is used in porn searches for older women the way people here might use “MILF”.

The blogger in question realized that his searches were coming from people who misspelled “auntie fuck” as “anti faq” and wound up on his Anti-Libertarian FAQ. No kidding. So unfortunately, a column by Aunt Pythia that mentions sex or boobs is going to get those kind of visitors…

Absurdly Understood Naughty Terminology

Dear AUNT,

I’m not complaining! But thanks, I do feel a bit better about it now that it’s less incestuous.

Auntie P


Dear Aunt Pythia,

Would you mind pointing me in the direction of resources for women in math trying to figure out how to be thoughtful about what it means to be a woman in math? (In addition to your blog, of course!)

I am a fourth year statistics and economics student at UVA, and I find myself increasingly desirous of things to read to help me articulate and name and talk about the challenges of being a lady in the math department.

As I have spent more and more time in math, there have been things that have happened that have made me upset, and I have struggled to articulate *why*. For example: I went to office hours for my probability class – maybe ~75% male, with a male professor – and there were four of my male classmates there as well. I asked a question, and immediately all five men responded with some version of “that’s a dumb/obvious question” in a please-don’t-waste-our-time-tone. I explained that they had misunderstood my question, that my question was really X not Y, and the professor kinda sorta answered X.

I left, very upset at what had just happened, but not being able to quite articular why. It would be natural to be upset if five people in general shut down my question. But what about how it had been five *men* did I find particularly offensive/upsetting?

Another example: A few weeks ago one of my (female) friends withdrew from our stochastic processes class, which had 4 women out of 22 students. For me this was a painful things to watch, but I again, I had difficulty saying why. She was a math major and withdrawing wouldn’t change that.

What was wrong? Was it that it felt so avoidable, that if things had gone so slightly differently I knew she would have stayed (and so this wasn’t even about gender at all)? Was it that I’ve watched the number of women in my math classes decrease with every successive class I’ve taken, and here I was watching this war of attrition happen before my eyes? She’s happy, so why am I upset?

Another: Until this semester I have never had either a female professor or TA in any economics, statistics, math, or CS class. I’m a fourth year, so I’ve taken a lot of classes aka had lots of opportunity to have had a female instructor in my field. I have yet to successfully explain why that’s hard – not just philosophically too bad, but hard – to my friends. Or to successfully communicate *how* that hard-ness presents itself. Sometimes I think the things that upset me maybe shouldn’t upset me, or that I’m seeing ghosts where there are none.

That math is just really hard, and it’s hard and even lonely for everyone, and that men/everyone experiences the same thing. I don’t know how to respond to that devil’s advocate in my head, or how to think well about that either. I would love your advice on what to read and where to go to learn how to coherently articulate my thoughts and frustrations in this arena. Especially because I only have room in my schedule to take either abstract algebra or a feminism class next semester, and I would really like to take the math class AND have the intellectual resources to think well about these things.


Woman Here In Math Seeking Intellectual & Constructive Assistance, Legitimately (WHIMSICAL)


I don’t know whether to be offended that you needed to spell out your sign-off for me. I supposed I deserve it, sometimes in the past I’ve missed some really good ones. Apologies to all those Aunt Pythia contributors!!

Here’s the thing. I think you’re already miles ahead of where I was at your age, because of two things. First, you’ve figured out what’s bothering you. I remember not knowing why I was upset, but simply bursting out in tears every now and then. It was bewildering.

Second, you’ve found my blog! And I’m so glad about that! One of the major goals of my blog is to be here for you.

Now, here’s the bad news. I don’t really have too much in the way of concrete advice for you. I’ll do my best though, here goes:

  1. I’d also be sad to see that woman go, and I’d also be confused as to why. I feel that way whenever I see women leave math, even though I myself left. But of course I don’t regret that I left, and I’m much happier now, so it doesn’t make sense at the individual level to feel sorry for a woman who chose to do something else.
  2. Maybe we’re both just feeling bad for math’s culture itself, that it can’t seem to get itself together to be a welcoming place for all these wonderful women. I’m sorry for you, math culture. And I’m not sure you can hear me, or what you’d say if you could answer me, but believe me you’re missing out on some majorly wonderful people by being so difficult.
  3. Having said that, the underlying math, the actual math questions and riddles and puzzles, is awesome, and when it’s just you and it, and the rest of the culture is shut out, then it can be magical.
  4. About the men: they are dumb, immature, and asinine. Including the professor. But not everyone is like that. So my advice here is: seek out men and women who are not like that, and figure out how to do math with them.
  5. I remember being in Victor Guillemin‘s MIT math grad class on differential geometry. He is so nice, and the math was super beautiful. There was this really badly off man who would come to the class, maybe he was homeless, and he’d ask bizarre and unrelated questions. Guillemin would, without fail, figure out a way to turn it into a really good question and would quite gallantly and kindly answer it, ending with something sincere like, “thanks so much for asking that!” Love that guy, and love how consistently elegant he made that potentially disastrous situation.
  6. Which is to say, we all have something to strive for. In your story above, the least we could expect from the professor is an honest answer to the question he thought you were asking, and we didn’t even get that. Lame.
  7. So, my advice to you is, trade up. Spend time with the people who are closer to Guillemin and further away from those people. And if that’s momentarily impossible, make do but keep in mind that there are better ways to run a culture, and that when you’re in charge you’ll see to it that it does get better.
  8. So when you’re a professor, you will never ridicule a question because it might end up being much deeper than you expect and after all math is really hard and sometimes we have brain farts and that’s ok too. Make your worst case scenario that you never humiliate anyone or call into question their basic dignity, and you’ll be rising the level of discourse by a mile and a half.
  9. Find women in math, at conferences and whatnot, and make friends. A little commiseration goes a long way.

Good luck!

Aunt Pythia


Dear Aunt Pythia,

Four-plus years after #OWS, I find myself within sight of a reputable liberal arts degree in both Politics and Economics, facing an uncertain job-market just over the horizon. So far, I feel like I’ve chosen well, trying to make sense of the financial system through an interdisciplinary lens. I have read a lot of the post-crisis canon, in hopes of new directions to pursue. I found your blog years ago following the Frontline feature, and was struck by mathbabe’s candor (No one understands the ‘whole financial system’) and quantitative rigor. As for my own skills, I’m good at compiling and interpreting research, written exposition, and creative analysis. I have some training in econometrics, but it’s definitely not my forte. I’ve done a couple of legislative internships, but I’m more and more certain that I need to pivot to a finance-related position or institution.

Grad school seems a remote possibility for the future, but right now I want to chart a course in the direction of economic journalism or policy analysis. I have major qualms about finance’s ability to confront long-term risk and deliver sustainable growth. Someday, I’d love to contribute to a prominent publication or think-tank, and help to craft the financial reforms we urgently need. (Stop me, please, if you see more effective ways to intervene for someone with my background). If you see fit, I am eager to hear which organizations you think are making the most progress, or what roles a newly-minted grad could hope to play therein.

So far, I’ve researched numerous SRI/ESG-based firms, government regulators (especially those agencies empowered by Dodd-Frank), industry monitors like FINRA, and even mission-oriented banks, CDFIs, B-corp lenders, and the like. I have yet to explore consulting or other professional services in as much depth. Is there a phylum that I’m neglecting here? Do you have any specific suggestions? I also wonder: if you or others in the Alt-Banking community knew as undergrads what you knew now, what organizations or roles would you have striven for?

Thanks for your consideration.

Curious About Robust Economic Empowerment & Risk Strategies


Thanks for the question. It’s a tricky one. One of the things that finance successfully does as a field is to make itself seem impenetrable for people not on the inside. At the same time, it’s an absolute requirement of a working modern economy. So there you have it, only insiders can penetrate and understand it, it has to exist and be healthy, and yet insiders are often corrupt (even when they don’t know they are).

So part of me doesn’t want you to go into field at all, because it kind of stinks. But on the other hand, the other effect that’s making things worse is that only money-grubbing jerks ever do go in. So, in the name of not wanting the field to be entirely overwhelmed by such people, I will in fact encourage you to go in, keeping your eyes open of course.

What I suggest is to get a job at a big bank and think of it as a sociological experiment, a la Karen Ho’s Liquidated. Then maybe go work for regulators or something. Honestly I know it sounds terrible – I’m suggesting you be part of the revolving door problem, but I don’t think it makes sense to be a financial regulator without actually having experience in finance.

Readers, weigh in if you have other ideas for CAREERS.

Good luck, and keep in touch!

Aunt Pythia


Readers? Aunt Pythia loves you so much. She wants to hear from you – she needs to hear from you – and then tell you what for in a most indulgent way. Will you help her do that?

Please, pleeeeease ask her a question. She will take it seriously and answer it if she can.

Click here for a form for later or just do it now:

Categories: Uncategorized

Silicon Valley and Journalism conference at Columbia’s Tow Center

Today I’m excited to attend a Tow Center Journalism Conference on the relationship between Silicon Valley and journalism. Unfortunately it’s sold out at this point, but here’s the updated schedule.

I’m particularly excited for the 10:45am panel which features Zeynep Tufecki and Kate Crawford, among others, discussing ethics arising from the relationship between Silicon Valley platforms like Facebook and Google and journalism, especially when it comes to censorship and digital rights.
Also, one of my favorite technology journalists Julie Angwin is speaking on a panel at noon, which is going to be interesting. And of course the brilliant and amazing Emily Bell, who runs the Tow Center, is moderating a couple of discussions. An excellent line up, I’m lucky to live so close.
Speaking of critical journalism, I was quoted in a recent piece on which focused on predictive policing, written by Jack Smith.
Categories: Uncategorized

Get every new post delivered to your Inbox.

Join 3,674 other followers