I’ll stop calling algorithms racist when you stop anthropomorphizing AI
I was involved in an interesting discussion the other day with other data scientists on the mistake people make when they describe a “racist algorithm”. Their point, which I largely agreed with, is that algorithms are simply mirroring back to us what we’ve fed them as training data, and in that sense they are no more racist than any other mirror. And yes, it’s a complicated mirror, but it’s still just a mirror.
This issue came up specifically because there was a recent Mic.com story about how, if you google image search “professional hairstyles for work,” you’ll get this:
but if you google image search “unprofessional hairstyles for work” you’ll instead get this:
This is problematic, but it’s also clearly not the intention of the Google engineering team, or the google image search algorithm, to be racist. It is instead a reflection of what we as a community have presented to that algorithm as “training data.” So in that sense we should blame ourselves, not the algorithm. The algorithm isn’t (intentionally) racist, because it’s not intentionally anything.
And although that’s true, it’s also dodging some other truth about how we talk about AI and algorithms in our society (and since we don’t differentiate appropriately between AI and algorithms, I’ll use them interchangeably).
Namely, we anthropomorphize AI all the time. Here’s a screenshot of what I got when I google image searched the phrase “AI”:
Out of the above images, only a couple of them do not have some reference to human brains or bodies.
In other words, we are marketing AI as if it’s human. And since we do that, we are treating it and reacting to it as quasi-humans would. That means when it seems racist, we’re going to say the AI is racist. And I think that, all things considered, it’s fair to do this, even though there’s no intention there.
Speaking of intention and blame, I am of the mind that, even though I do not suspect any Google employee of making their algorithms prone to this kind of problem, I still think they should have an internal team that’s on the look-out for this kind of thing and address it. Just as, as a parent, I am constantly on the look-out for my kids getting the wrong ideas about racism or other prejudices; I correct their mistakes. And I know I’m anthropomorphizing the google algorithms when I talk about them like children, but what can I say, I am a sucker for marketing.
mathbabe 
The most interesting point made after this was pointed out was that a bunch of the articles that showed up from the ‘unprofessional’ image search were articles commenting on racism or similar issues. So the fact that we discuss these issues is one of the main factors skewing the algorithm
LikeLike
I just want to make it clear that I think we should be discussing issues of racism in the workplace and that this is a sign of a poor algorithm or poor moderation. I’m not sure if we can hope to prevent this algorithmic issue from ever happening as we can’t anticipate every search and the source websites are always changing but we should certainly react when flaws are pointed out and attempt to improve the algorithms.
LikeLike
Exactly right. It would be weird if someone was like “Donald Trump is kind of racist” and your response was “No, he’s just reflecting the racist training data he was fed by his parents and upbringing.”
LikeLiked by 1 person
You’re point is well taken, but another layer to it, is that we all get somewhat different results when ‘Googling’ something, since Google homes in on our own past history and interests to mold its response — that is often a problem in itself (though it can also be a benefit in some circumstances). In fact, I’m even wondering how the results for a Black individual with a long Google search history, searching “professional hairstyles for work” might differ (if at all) from what you show above?
LikeLike
Indeed! When I Google the phrase with the word unprofessional this morning, I got both of the above images, I imagine because of the popularity of the Mic.com article.
LikeLike
How codewords like “professional” and “successful”—these are the first two that jump to mind, but there are surely others—are used to segregate society strikes me as worthy of analysis too. Someone commenting on the social media page (LinkedIn, if I remember right) of a financially-well-rewarded (i.e., “successful”) entrepreneur in the human potential industry (I don’t know if that’s what it’s actually called, but close enough…) described her as “the most professional person I have ever met.” My first reaction to this comment was, “what the hell does that mean?”
LikeLike
“Ask and receive…”
Literally just days after posting this, a husband-wife team did some heavy housecleaning in my condo which is going on the market tomorrow. They use pumice on ovens. But they evidently don’t agree on whether to use it dry (the husband’s preference), or with a cleaner (how the wife had actually started the job). It caused a heated argument to erupt between the two of them in my kitchen. The husband’s preference prevailed, at which time the wife turns to ME and says, “He’s my father, brother, and husband all wrapped into one.”
Perhaps “professional” is a word most easily defined by its opposite…
LikeLike
Doesn’t Image Search depend on the content of the surrounding web page, rather than training data.
That moves the racial hostility to the creators of the web pages, which is of course no surprise at all.
LikeLike
I am a little surprised there is no comment on the fact that both professional and unprofessional appear to be almost all women.
LikeLiked by 1 person
I was about to write: For men, try “hairCUTS”, but that’s wrong, still get women.
PS: Same racial disparity for professional vs. unprofessional.
LikeLike
“hairCUTS” results are interesting (at least as Google shows me). Only white women have “professional haircuts” but a very diverse group by both race and gender have “unprofessional haircuts”
LikeLike
I am curious how these algorithms get written–since Google techies are still majority white male, are we getting their biases? Do they “focus group” for algorithms? Do they read articles by “professionals” to build the proper search term bias into their algorithm?
I find that to get what I am looking for, I have to play around with different search language for at least three searches and refine from there.
LikeLike