Home > data science > Model Thinking (part 2)

Model Thinking (part 2)

February 23, 2012

I recently posted about the new, free online course Model Thinking. I’ve watched more videos now and I am prepared to make more comments.

For the record, the lecturer, Scott Page, is by all accounts a great guy, and indeed he seems super nice on video. I’d love to have him over for dinner with my family someday (Professor Page, please come to my house for dinner when you’re in town).

In spite of liking him, though, pretty much every example he gives as pro-modeling is, for me, an anti-modeling example. Maybe I should make a complementary series of YouTube comment videos. It’s not totally true, of course- I just probably don’t notice the things we agree on. But I do notice the topics on which we disagree:

  1. He talks a lot about how models make us clearer thinkers. But what he really seems to mean is that they make us stupider thinkers. His example is that, in order to decide who to vote for for president, we can model this decision as depending on two things: the likability of the person in question (presumably he assumes we want our president to be likable), and the extent to which that person is “as left” or “as right” as we are. I don’t know about you, but I actually care about specific issues and where people stand on them, and which issues I consider likely to come up for consideration in the next election cycle. Like, if I like someone for his “stick it to the banks” approach but he’s anti-abortion, then I think about whether abortion is likely to actually become illegal. And by the way, I don’t particularly care if my president is likable, I’d rather have him or her effective.
  2. He bizarrely chooses “financial interconnectedness” as a way of seeing how cool models are, and he shows a graph where the nodes are the financial institutions (Goldman Sachs, JP Morgan, etc.) and the edges are labeled with an interconnectedness score, bigger meaning more interconnected. He shows that, according to this graph, back in 2008 it shows we knew to bail out AIG but that it was definitely okay to let Lehman fail. I’m wondering if he really meant that this was an example of how your model could totally fail because your “interconnectedness scoring” sucked, but he didn’t seem to be tongue in cheek.
  3. He then talked about measuring the segregation of a neighborhood, either by race or by income, and he used New York and Chicago as examples. I won’t go into lots of details, but he gave a score to each block, like the census maps do with coloring, and he used those scores to develop a new score which was supposed to measure the segregation of each block. The problem I have with this segregation score is that it depends very heavily on the definition of the overall area you are considering. If you enlarge your definition of the New York City to include the suburbs, then the segregation score of New York City may (probably would) be completely different. This seems to be a really terrible characteristic of such a metric.
  4. My second problem with his segregation score is that, at the end, he had overall segregation numbers for Philly and Detroit, and then showed the maps and mentioned that, looking at the maps, you wouldn’t really notice that one is more segregated than the other (Philly more than Detroit), but knowing the scores you do know that. Umm.. I’d like to rather say, if you are getting scores that are not fundamentally obvious from looking at these pictures, then maybe it’s because your score sucks. What does having a “good segregation score” mean if not that it captures something you can see through a picture?
  5. One thing I liked was a demonstration of Schelling’s Segregation Model, which shows that, if you have a group of people who are not all that individually racist, you can still end up with a neighborhood which is very segregated.

I’m looking forward to watching more videos with my skeptical eye. After all, the guy is really a sweetheart, and I do really care about the idea of teaching people about modeling.

Categories: data science
  1. JSE
    February 23, 2012 at 12:53 pm

    “I actually care about specific issues and where people stand on them, and which issues I consider likely to come up for consideration in the next election cycle.” I think the point of this kind of modeling is that you perceive yourself to care about individual issues, but you actually don’t. Or rather, you care about them in a way that you think comes from your reasoning but is in fact conditioned by your party ID. Or rather, this is true of most people even if it’s not true for you. There is a reason that people who support gay rights also tend to support unions, and it’s not because there’s a coherent political philosophy that leads to both outcomes.

    Like

    • February 23, 2012 at 1:55 pm

      Agreed. In that case I’d love to see how much of the voting can be explained by these two things before going off and modeling it like that. He may have been getting to that in his mind (or in a later lecture) but it wasn’t mentioned here.

      Like

    • Dewk
      April 17, 2012 at 10:51 am

      You do realize that this example was pretty much day 1 of the course where he was giving an introduction to the course with a few examples of what modelling is? Should he have spent another 20 minutes thinking up every possible reason that someone might vote for a candidate and putting them into the model and totalling forgetting that this was the introductory lesson?

      Also, one of the cool things about modelling, is that despite thinking that every little detail matters, quite frequently it turns out that only a couple of very high level details matter for any level of significance.. So it is possible that while you believe that the little details would change your vote, in the end it really is only a couple of high level details that really matter. Probably if the example replaced likeability with political party then you might not have had as big an issue with the example.

      I’ve only watched 3 of the videos so far, and haven’t been disappointed yet because I understand that for the first 4 videos, this would be day 1 of a real college course. So this is just the course introduction material. However, he did touch upon some topics that I am anxious to hear about in the future.

      Like

  2. JSE
    February 23, 2012 at 12:57 pm

    “What does having a “good segregation score” mean if not that it captures something you can see through a picture?”

    It might mean something like: “There are different patterns of segregation which look, to the naked eye, roughly “equally segregated,” but which have different effects on other variables we care about, and our score does a good job of distinguishing between those patterns.” I mean, I haven’t watched the lecture and I have no idea whether it DOES mean that — but it doesn’t sound inherently ridiculous to me.

    I mean, the size of the spectral gap of a graph captures something you can’t see in a picture of the graph, and it really does mean something. (Not really analogous, I know.)

    Like

    • February 23, 2012 at 1:54 pm

      I’d like to hear what it does do, is all. But I agree it might do something.

      Like

    • Dewk
      April 17, 2012 at 10:57 am

      It seems to me that you should already know that math can be far better at picking out patterns than the human eye in many cases (particularly when noisy signals are involved). Try looking at a low power RF signal in the presence of noise. It just looks like random gobbly-gook to the naked eye. Apply a smoothing filter and the desired signal jumps right out at you.

      Like

  3. Mike Maltz
    February 23, 2012 at 1:14 pm

    I used to be a mathematical modeler (OR-type), for about 20 years) but soon realized that, unless I understood the data (how it was collected, reasons that some data were* missing), I fell into the GIGO trap. So for the next 20 years I devoted myself to understanding and cleaning the data I was most interested in — and then visualizing it instead of trying to model it. More fun, and makes more sense — at least, to me.

    *Yes, I know I’m using “data” as both singular and plural. Besides, like garbage, data is both a collective and collected noun!

    Like

  4. February 23, 2012 at 5:15 pm

    If you model stuff in a good GIS you can actually make spatially aware models that do make sense as pictures. Most GIS tools are horrendously expensive but Manifold System is quite reasonable at $1000 per seat (about 90% of the functionality of market leader, ESRI’s ArcGIS for about 10% of the price).

    Also, although I haven’t used it there is a program Crimestat for analysing geographic clustering which a friend recommended highly.

    The advantage of using these tools is that they allow easy access to potentially important explanatory variables like ‘near by’, ‘far away’, ‘on the same bank of the river’, ‘connected by a highway’ or ‘separated by a mountain range’. These concepts are often left out of models built using software where querying these kinds of relationships is difficult.

    http://www.manifold.net/

    http://www.icpsr.umich.edu/CrimeStat/

    Like

  1. No trackbacks yet.
Comments are closed.