The Black Box Society by Frank Pasquale
There’s a new book out, called The Black Box Society and written by Frank Pasquale, a lawyer focused on technology and a friend of mine. It’s published by Harvard University Press and it looks like this:
To be honest, when I first received it I was a bit worried that it would make my book, which I am utterly engaged in writing, entirely moot. After all, Frank and I had discussed his book and I’d seen earlier versions. I knew it contained information about racist secret algorithms in finance and tech, and there were also other issues in common with our two books.
Now that I’ve had a chance to read it, though, I’m not as worried. First of all, Frank’s book is aimed at a different audience, which is to say a somewhat more academic and technical audience. In particular his policy recommendations near the end of the book seem to be written for lawyers who know the current laws and need arguments to improve them.
Also, his focus is on secrecy itself as a means of power, whereas I focus on models as the object of interest.
I like a lot of what Frank says, and I think his metaphors work really well. For example, he talks about the early promise of the internet to expose information of all sorts, on powerful corporations as well as individuals. Then he talks about how reality has been a disappointment, and we’ve ended up with an internet that acts as a “one way mirror,” whereby powerful corporations can see into individual’s lives but those individuals can’t look back.
He also makes the important point that, when it comes to the NSA and other government agencies snooping around, while they might be legally prevented from gathering certain kinds of data about people, nothing prevents them from buying information and profiles from data warehouses like Acxiom, which can do the kind of collecting that they can’t. In other words, the data warehousing industry acts as a giant loophole in the set of rules protecting our civil liberties.
For another really interesting review of Frank’s book, written by a software engineer, take a look at David Auerbach’s Slate review (hat tip Jordan Ellenberg). In particular he has interesting things to say about the extent to which algorithms are intentionally evil (they’re probably not) and the extent to which engineers can fix problems (they probably can).
In any case, I recommend The Black Box Society, it’s a fascinating and important book.
mathbabe 
This is so frustrating to wake folks up on what’s going on. I go back to work I did years ago with HMOs ripping of doctors with payments with what was called the “floating patients” where patients would appear on month on their capitation payments and gone for a couple months and then reappear. This all came about as one MD had this happen to his family members who were all in fact his patients…so I made up a little MS Access data base to where they could import their spreadsheets and make a file to verify their payments. Remember this is going back a few years when spreadsheet knowledge was not like today and electronic medical records were still new on the horizon and I was still writing mine at the time. When you spend a lot of time with the data and integrate with some billing systems, you see all kinds of stuff:)
So having seen that and not been able to get straight answers on whether or not the HMOs had really bad systems or did this on purpose as money saved went as a bonus to the administrators, so there was a motive if you will for money, but could never determine which was which or if it was a bit of both. At any rate I gave it away to doctors so they could reconcile and not get shorted so having seen that first hand I’m a skeptic on data for sure. After seeing the process being an issue for so many doctors, you begin to wonder, but again the approach I took was to give them a tool so they could reconcile with what was sent to them on spreadsheets. When you see stuff like this, it never leaves you.
I did a radio show about privacy and again this stuff is going hog wild to where it appears anyone who can make a decent buck with consumer data is going for it. I had my day a few months ago where I guess I got under Acxiom’s skin too as they came out to talk to me on Twitter, which was unusual as they like to remain very quiet of course. Again I think we need data folks pursuing this as the leads here and yes you need lawyers as part of the team for sure but not running it as we have code running all over legal verbiage to be interpreted as being legal with the context that is chosen to apply. Today of course things are way beyond what I described above and again so should be the skepticism.
Anything for awareness to get people’s attention, but we also need data mechanic logic solutions versus the non tech perceptions that arise. Spending time with some doctors, and I mean a lot of time writing an EMR with how they perceived things and how the data processes actually worked tuned me in to how the other side thinks and saw things beyond my nerdy work and made me aware of that fact as I had to realize their perceptions were much different than mine. One day sat the doctors down, opened up and showed them some code and how my work went on doing such, and it was a really good thing as their perceptions were that I did something like working on a spreadsheet to make changes to the program…taught me well about perceptions and to learn how to get outside my own head sometimes as we all get buried like that at times I think, especially after pulling a few all nighters on some stupid query that won’t work right or something along that line:)
Today I tell all “the short order code kitchen burned several years ago and there was no fire sale” meaning systems are complex and the time and money it takes is not like it used to be to where adaptations were much simpler. Healthcare.Gov is probably a good example of this if we look back to what all occurred there by all means. Consumer expectations on what can be done and time limits is really not always the reality and it’s not their fault either as much of the news and other items out there just display all of this as being very simple when it’s not, thus getting consumers to pay attention on privacy is difficult too as there’s no way they can relate to what the code is doing on the back end as they can’t see it or understand a lot of this. So many of these apps I think focus on getting the back end done first to mine and collect the data, and then some kind of interface for the consumer gets added last and is of course the part that steers the marketing.
LikeLike
“one way mirror” what a great metaphor. lets face it, corporations rule the world. serious govt oversight is a facade, charade, and/or joke. kabuki theatre. also reminds me of the so-called panopticon. realized, except in cyberspace & not physical reality.
LikeLike