Two articles on understanding statistical error
Today I want to share two articles today which call on the public to try to understand scientific error at a deeper level than we do now.
First, an academic journal called Basic and Applied Social Psychology (BASP) has decided to ban articles using p-values. This was written up in Nature news (hat tip Nikki Leger) with an excellent discussion of the good and bad things that might result. On the one hand, p-values are thoroughly gamed and too easy to achieve with repetitive testing, resulting in a corpus that is skewed towards such testing situations. On the other hand, if you get rid of p-values you have to replace them with something to give you an idea of whether a statistical result is interesting. Of course there are plenty others out there, but they too may quickly become gamed.
Second, The Big Story has an in-depth article on evidence-based sentencing and paroling models and what can go wrong there (hat tip Auros Harman). They focus on the fact that the people filling out the questionnaires can and do lie in order to game their scores and leave jail earlier. They also mention the fact that the scores are quite unrobust to small changes in input, specifically age. Finally, they punish people for being poor or for “hanging out with the wrong crowd” or even for having parents that went to jail.