mathbabe

Great essay, Cathy. Those of us who work as scientists need to be the harshest critics of our ideas and input data. Sadly, as humans we are wired to become attached to our ideas and so it takes conscious effort to sufficiently scrutinize our pet ideas. I liked the title of the “Bad Data Handbook” – and the title of the second chapter (“Is It Just Me, or Does This Data Smell Funny?”) convinced me to buy it. Found this helpful in my quest to be chief skeptic of my own data and ideas.

LikeLike

Excellent! I am so glad you are discussing this issue as it really needs the attention. On my end of things from the healthcare side I spent 3 years writing an electronic medical records system and the important part of that time was the meeting of the minds with the doctors and the employees to build it as “their” tool and I don’t care what industry you are in today there’s nothing that replaces on hands experience you certainly have been there done that as well in an even bigger way. When you “see” the mechanics and know there’s a snow job hidden in the queries and algorithms it would almost be a crime not to talk about it.

In addition to the software development I was continuously exposed to the “human” element as these are people and not just numbers out there and no developer can sit there and create analytics and software without this, but as you have stated here it happens all the time with how marketing and news skew things. I sent your essay on to a few folks to read as well. I’m thinking too that out of the west coast here we need a “data skeptics group” as well.

I laugh at myself and tell all that writing code made me “logical” and in the same context in a humorous way ask if the world is ready to accept the fact that “logical women” do exist but we scare the daylights with folks sometimes as we vary from the old “paradigm” of what roles we are supposed to play:) Truth of the matter is that we are both that loving and nurturing being and yet can apply some logic and be a little nerdy too but we tell the truth and see through a lot of what I call the “Algo Duping” going on out there.

The reference to other report is amazing that it was written in 2006 so that tells you right there now long some of this has been buried. Good stuff. I’m with you telling folks to “be a skeptic when you need to be”…it’s getting more difficult all the time to separate the good from the junk that lives out there as we have “junk” making big money too as you so well stated. Back in 2009 I wrote a post asking if data addiction and abuse is going to be the next 12 step program on the horizon with a hint of satire but darn, we might be getting close:) Symptoms include data with lack of integrity, spasmodic algorithms combined with hand to mouse adhesion (grin).

I don’t know what the copyrights are on this but if the ability to put on SCRIBD is there or if O’Reilly will be providing that option, it would great to encourage a bigger distribution of your essay. I was not sure if it was a stand alone or part of one of their overall Strata larger publications.

LikeLike