## A public-facing math panel

I’m returning from two full days of talking to mathematicians and applied mathematicians at Cornell. I was really impressed with the people I met there – thoughtful, informed, and inquisitive – and with the kind reception they gave me.

I gave an “Oliver Talk” which was joint with the applied math colloquium on Thursday afternoon. The goal of my talk was to convince mathematicians that there’s a very bad movement underway whereby models are being used against people, in predatory ways, and in the name of mathematics. I turned some people off, I think, by my vehemence, but then again it’s hard not get riled up about this stuff, because it’s creepy and I actually think there’s a huge amount at stake.

One thing I did near the end of my talk was bring up (and recruit for) the idea of a panel of mathematicians which defines standards for public-facing models and vets the current crop.

The first goal of such a panel would be to define mathematical models, with a description of “best practices” when modeling people, including things like anticipating impact, gaming, and feedback loops of models, and asking for transparent and ongoing evaluation methods, as well as having minimum standards for accuracy.

The second goal of the panel would be to choose specific models that are in use and measure the extent to which they pass the standards of the above best practices rubric.

So the teacher value-added model, I’d expect, would fail in that it doesn’t have an evaluation method, at least that is made public, nor does it seem to have any accuracy standards, even though it’s widely used and is high impact.

I’ve had some pretty amazing mathematicians already volunteer to be on such a panel, which is encouraging. What’s cool is that I think mathematicians, as a group, are really quite ethical and can probably make their voices heard and trusted if they set their minds to it.

Sounds great! Would you comment on these factors? I’d like to understand them better.

LikeLike

Yeah, I’d like to see more details about “anticipating impact, gaming, and feedback loops” as well. In my experience, you know that any model with enough money riding on it will be gamed, but what could be done about it before releasing the model into the wild? My approach has always been passive: release the model, watch it, diagnose that it is being gamed, adjust, repeat. Is there a better approach / are you aware of specific examples of a successful people modeling approach?

LikeLike

I would say the key is that models should never EVER be used for evaluation or judgment. Because they WILL be gamed. And they can always be gamed.

Models are used for prediction. With an honest model, you may be able to use it to decide who to hire. You can NEVER use it to evaluate the perfomance of teachers you already have. NEVER.

LikeLike

case in point: actuaries in insurance

LikeLike