mathbabe

A few days ago I wrote about the idea of having a quant group working for consumers rather than against them. Today I wanted to spell out a few things I think that group could do.

The way I see it, there’s this whole revolution of data and technology and modeling going on right now, but only people with enough dough to pay for the quants are actually actively benefiting from the revolution. So people in finance, obviously, but also internet advertising companies.

The problem with this, besides the lopsidedness of it all, is that the actual models being used are for the most part predatory rather than helpful to the average person.

In other words, most models are answering question of the form:

1. how can we get you to spend money you don’t necessarily want to spend? or
2. how good a credit risk are you? or
3. how likely are you to come back and spend more money? or
4. how can we anticipate the market responding to end-of-month accounting shenanigans? I just threw this one in to give people a sense of how finance models work.

It all makes sense: these are businesses that are essentially bloodthirsty, making their money off your clicks and purchases. They are not going away.

Then there are some models that are already out there trying to make the user experience more enjoyable. They answer questions of the form:

1. If you like that, what else may you like? (Pandora, Netflix, Google)
2. If you bought that last week, maybe you’d like to buy it again this week?
3. If you bought that, maybe you’d like to buy this now?

The problem, as you can see, is that these second, helpfulish kinds of things quickly devolve into the first, predatory kinds of things. In other words, you’re being bombarded by ads and suggestions for spending more money than you actually want to spend.

What about stuff that you actually don’t want to do but that is probably not directly profitable to anyone? I’d love to see technology used to tackle some of these chores:

1. Figuring out what the best deal is on loans (credit card, student loans or mortgages), without becoming a lawyer. Here I’m not just saying they should all be clear about their terms- that’s a job for the CFPB. I mean there should be a website that asks me a few questions about what I need a loan for and points me to the best deal available.
2. Finding the best price for something. I discussed this briefly here.
3. Warnings about weird conditions and agreements, like mandatory arbitration.
4. Help finding a good doctor or a good plumber etc.
5. Help knowing when to go to the DMV or other public services building so the lines are bearable, or even better a way to do stuff on your computer or phone and avoid the trip altogether.
6. Getting your kids’ medical records to their schools and camps safely and efficiently.

Some of these things already exist (like the doctor thing) but aren’t well-known or well-publicized, so wouldn’t it be cool if this quant group also provided an app that would aggregate all these services into one place?

And for the things like #4, where you want to be warned before you buy, it’s too much work to go back to a webpage to check everything before buying. Instead, the app could follow you around the web when you’re shopping and overlay a “bullshit warning” icon on the potential purchase in situations where people have complained about unreasonable terms and conditions.

Let’s use technology in our favor. Instead of the companies collecting information about us, let’s collect information about them. Come to think of it, the CFPB should start this quant group today.

I recently read a Bloomberg article (via Naked Capitalism) about wine critics and how they have better powers of taste than we normal people do.

One of the things I love about this article is how it’s both completely dead obvious and at the same time totally outrageous when you think about it.

Obvious because when we see wine experts going on and on about what they can discern in their tiny sip, we know they either have magical powers or they’re lying, and since some of them can do this shit blindfolded we will assume they aren’t lying.

Outrageous because if you think about it, that means we follow the advice of people whose taste is provably different from ours. In other words, the word of experts is fundamentally irrelevant to us, and yet we care about it anyway.

My question is, why do we care?

Just to go over a couple of ground rules. First, yes, let’s assume that the wine experts really do have powers of discernment that are incredible and unusual, even though we have to trust an expert on that, which may seem contradictory. The truth is, this isn’t the first study that’s shown that, and I for one have hung out with these guys and they really do taste that minerally soil in the wine. I’m not even jealous.

Second, I’m not saying you care about wine experts’ opinions, but lets face it, lots of people do. And you could say it’s because of the performance that the experts give when they smell the wine and describe it, and I’ll agree that some of them can be poets, and that’s nice to see. But the truth is they also rate the wines with a number, and these numbers are printed in books, and lots of people carry these books around to wine shops and devote themselves to only buying wines with sufficiently high numbers, even though those people probably don’t themselves have the mouth smarts to tell the difference.

Now that we’ve framed the question, we can go ahead and make guesses as to why. Here are mine:

1. People are hoping that they themselves are also supertasters. This kind of seems like the most obvious one, but I can’t help think that true supertasters would not need other people’s opinions at all.
2. People think experts’ opinion of “good”, even if not completely the same as theirs, will be highly correlated, and so is better than nothing.
3. People want to be seen drinking wine that supertasters would drink, as a sort of cachet thing.

I think #3 above is pretty much the definition of snob, and I think it exists but is not the major reason people do things (but I could be wrong). I’m guessing it’s more typically #2, but even so it doesn’t explain the really expensive high-end wine market’s huge appeal, unless there are way more supertasters than I thought out there.

I think this question, of why people listen to expert advice even when it’s mostly irrelevant, is an important one, because it happens so much in our culture, and clearly not just about wine.

I for one am attracted to the idea of going one step further and ignoring expert advice. I see a natural progression: first, people are ignorant, second, they learn what experts think, and third, they ignore experts and go with their gut.

But even sexier is the idea of never listening to experts at all: skip step two. Am I the only person who thinks that’s sexy? I mean, I guess it mostly means you’re wasting your time, but it also comes out in the end with less herd mentality.

I think this desire I have of skipping the expert advice is very tied into why I despise the echo chamber of the web and how we are profiled online and how our environment is constantly updated and tailored to our profile. It’s in some sense an expert opinion on what we’d like, given our behavior, and I hate the finiteness of that concept, possibly in part because I’ve designed models like that and I know how dumb they are.

Anyway, I’d love to hear your thoughts, and if I’ve missed any reasons why people like to hear partially relevant expert opinion.

I recently posted about the new, free online course Model Thinking. I’ve watched more videos now and I am prepared to make more comments.

For the record, the lecturer, Scott Page, is by all accounts a great guy, and indeed he seems super nice on video. I’d love to have him over for dinner with my family someday (Professor Page, please come to my house for dinner when you’re in town).

In spite of liking him, though, pretty much every example he gives as pro-modeling is, for me, an anti-modeling example. Maybe I should make a complementary series of YouTube comment videos. It’s not totally true, of course- I just probably don’t notice the things we agree on. But I do notice the topics on which we disagree:

1. He talks a lot about how models make us clearer thinkers. But what he really seems to mean is that they make us stupider thinkers. His example is that, in order to decide who to vote for for president, we can model this decision as depending on two things: the likability of the person in question (presumably he assumes we want our president to be likable), and the extent to which that person is “as left” or “as right” as we are. I don’t know about you, but I actually care about specific issues and where people stand on them, and which issues I consider likely to come up for consideration in the next election cycle. Like, if I like someone for his “stick it to the banks” approach but he’s anti-abortion, then I think about whether abortion is likely to actually become illegal. And by the way, I don’t particularly care if my president is likable, I’d rather have him or her effective.
2. He bizarrely chooses “financial interconnectedness” as a way of seeing how cool models are, and he shows a graph where the nodes are the financial institutions (Goldman Sachs, JP Morgan, etc.) and the edges are labeled with an interconnectedness score, bigger meaning more interconnected. He shows that, according to this graph, back in 2008 it shows we knew to bail out AIG but that it was definitely okay to let Lehman fail. I’m wondering if he really meant that this was an example of how your model could totally fail because your “interconnectedness scoring” sucked, but he didn’t seem to be tongue in cheek.
3. He then talked about measuring the segregation of a neighborhood, either by race or by income, and he used New York and Chicago as examples. I won’t go into lots of details, but he gave a score to each block, like the census maps do with coloring, and he used those scores to develop a new score which was supposed to measure the segregation of each block. The problem I have with this segregation score is that it depends very heavily on the definition of the overall area you are considering. If you enlarge your definition of the New York City to include the suburbs, then the segregation score of New York City may (probably would) be completely different. This seems to be a really terrible characteristic of such a metric.
4. My second problem with his segregation score is that, at the end, he had overall segregation numbers for Philly and Detroit, and then showed the maps and mentioned that, looking at the maps, you wouldn’t really notice that one is more segregated than the other (Philly more than Detroit), but knowing the scores you do know that. Umm.. I’d like to rather say, if you are getting scores that are not fundamentally obvious from looking at these pictures, then maybe it’s because your score sucks. What does having a “good segregation score” mean if not that it captures something you can see through a picture?
5. One thing I liked was a demonstration of Schelling’s Segregation Model, which shows that, if you have a group of people who are not all that individually racist, you can still end up with a neighborhood which is very segregated.

I’m looking forward to watching more videos with my skeptical eye. After all, the guy is really a sweetheart, and I do really care about the idea of teaching people about modeling.

I really feel like I can’t keep up with all of the creepy models coming out and the news articles about them, so I think I’ll just start making a list. I would appreciate readers adding to my list in the comment section. I think I’ll move this to a separate page on my blog if it comes out nice.

1. I recently blogged about a model that predicts student success in for-profit institutions, which I claim is really mostly about student debt and default,
2. but here’s a model which actually goes ahead and predicts default directly, it’s a new payday-like loan model. Oh good, because the old payday models didn’t make enough money or something.
3. Of course there’s the teacher value-added model which I’ve blogged about multiple times, most recently here. And here’s a paper I’d like everyone to read before they listen to anyone argue one way or the other about the model (h/t Joshua Batson). The abstract is stunning: Recently, educational researchers and practitioners have turned to value-added models to evaluate teacher performance. Although value-added estimates depend on the assessment used to measure student achievement, the importance of outcome selection has received scant attention in the literature. Using data from a large, urban school district, I examine whether value-added estimates from three separate reading achievement tests provide similar answers about teacher performance. I find moderate-sized rank correlations, ranging from 0.15 to 0.58, between the estimates derived from different tests. Although the tests vary to some degree in content, scaling, and sample of students, these factors do not explain the differences in teacher effects. Instead, test timing and measurement error contribute substantially to the instability of value-added estimates across tests. Just in case that didn’t come through, they are saying that the results of the teacher value-added test scores are very very noisy.
4. That reminds me, credit scoring models are old but very very creepy, wouldn’t you agree? What’s in them that they want to conceal them?
5. Did you read about how Target predicts pregnancy? Extremely creepy.
6. I’m actually divided about whether it’s the creepiest though, because I think the sheer enormity of information that Facebook collects about us is the most depressing thing of all.

Before I became a modeler, I wasn’t personally offended by the idea that people could use my information. I thought, I’ve got nothing to hide, and in fact maybe it will make my life easier and more efficient for the machine to know me and my habits.

But here’s how I think now that I’m a modeler and I see how this stuff gets made and I see how it gets applied. That we are each giving up our data, and it’s so easy to do we don’t think about it, and it’s being used to funnel people into success or failure in a feedback loop. And the modelers, the people responsible for creating these things and implementing them, are always already the successes, they are educated and are given good terms on their credit cards and mortgages because they have a nifty high tech job. So the makers get to think of how much easier and more convenient their lives are now that the models see how dependable they are as consumers.

But when there are funnels, there’s always someone who gets funneled down.

Think about how it works with insurance. The idea of insurance is to pool people so that when one person gets sick, the medical costs for that person are paid from the common fund. Everyone pays a bit so it doesn’t break the bank.

But if we have really good information, we begin to see how likely people are to get sick. So we can stratify the pool. Since I almost never get sick, and when I do it’s just strep throat, I get put into a very nice pool with other people who never get sick, and we pay very very little and it works out great for us. But other people have worse luck of the DNA draw and they get put into the “pretty sick” pool and their premium gets bigger as their pool gets sicker until they are really sick and the premium is actually unaffordable. We are left with a system where the people who need insurance the most can’t be part of the system anymore. Too much information ruins the whole idea of insurance and pooled risk.

I think modern modeling is analogous. When people offer deals, they can first check to see if the people they are offering deals are guaranteed to pay back everything. In other words, the businesses (understandably) want to make very certain they are going to profit from each and every customer, and they are getting more and more able to do this. That’s great for customers with perfect credit scores, and it makes it easier for people with perfect credit scores to keep their perfect credit scores, because they are getting the best deals.

But for people with bad credit scores, they get the rottenest deals, which makes a larger and larger percentage of their takehome pay (if they even get a job considering their credit scores) go towards fees and high interest rates. This of course creates an environment in which it’s difficult to improve their credit score- so they default and their credit score gets worse instead of better.

So there you have it, a negative feedback loop and a death spiral of modeling.

I recently read this New York Times article about a company that figures out how to get the best deal when you rent a car. The company is called AutoSlash and the idea is you book with them and they keep looking for good deals, coupons, or free offers every day until you actually need the car.

Wait a minute, a data science model that actually directly improves the lives of its customers? Why can’t we have more of these? Obviously the car companies absolutely hate this idea. But what are they going to do, stop offering online shopping?

Why don’t we see this in every category of shopping? It seems to me that you could do something like this and start a meta-marketplace, where you buy something and then, depending on how long you’re willing to wait until delivery, the model looks for a better online deal, in exchange for a small commission. Then you’d have to make sure that on average the commission is paying for itself with better deals, but my guess is it would work if you allowed it a few days to search per purchase. Or if you really are a doubter, fix a minimum wait time and let the company take some (larger) percentage of the difference between the initial price and the eventual best price.

Another way of saying this, is that when you go online to buy something, depending on the scale (say it’s on the expensive side) you probably shop around for a few days or weeks. Why do that in person? Just have a computer do it for you and tell you at the end what deal it gave you. Don’t get bombarded by ads, let the computer get bombarded by ads for you.