Computers think they know who you are. Artificial intelligence algorithms can recognise objects from images, even faces. But we rarely get a peek under the hood of facial recognition algorithms. Now, with ImageNet Roulette, we can watch an AI jump to conclusions. Some of its guesses are funny, others… racist.
[referenced url=”https://www.lifehacker.com.au/2017/09/this-facial-recognition-software-knows-your-girlfriends-face-better-than-you-do/” thumb=”https://www.lifehacker.com.au/wp-content/uploads/sites/4/2017/09/Couple-410×231.jpg” title=”This Facial-Recognition AI Knows Your Girlfriend’s Face Better Than You Do” excerpt=”If someone showed you a group photo containing your boyfriend or girlfriend, you could probably spot them without much trouble. But what if the photo was from ten years ago? Or what if their face was partially obscured? What if it contained thousands of people? That’s when you might need artificial intelligence to help you out.”]
ImageNet Roulette was designed as part of an art and technology museum exhibit called Training Humans to show us the messy insides of the facial recognition algorithms that we might otherwise assume are straightforward and unbiased. It uses data from one of the large, standard databases used in AI research. Upload a photo, and the algorithm will show you what it thinks you are. My first selfie was labelled “nonsmoker.” Another was just labelled “face.” Our editor-in-chief was labelled a “psycholinguist.” Our social editor was tagged “swot, grind, nerd, wonk, dweeb.” Harmless fun, right?
But then I tried a photo of myself in darker lighting and it came back tagged “Black, Black person, blackamoor, Negro, Negroid” In fact, that seems to be the AI’s label for anyone with dark skin. It gets worse: in Twitter threads discussing the tool, people of colour are consistently getting that tag along with others like “mulatto”, “orphan” and “rape suspect.”
So far in my timeline, I’ve seen lighter-skinned participants labelled ‘adonis’ ‘baron’ ‘face’ and darker-skinned ppts labelled ‘rape suspect’ ‘mulatto’ ‘blackamoor’ ‘negroid’. Can you guess which group finds it funny? #racist #algorithms https://t.co/n6inWxPvto
— Tony Kainth (@DrTonyKainth) September 16, 2019
These categories are in the original ImageNet/WordNet database, not added by the makers of the ImageNet Roulette tool. Here’s the note from the latter:
ImageNet Roulette regularly classifies people in dubious and cruel ways. This is because the underlying training data contains those categories (and pictures of people that have been labelled with those categories). We did not make the underlying training data responsible for these classifications. We imported the categories and training images from a popular data set called ImageNet, which was created at Princeton and Stanford University and which is a standard benchmark used in image classification and object detection.
ImageNet Roulette is meant in part to demonstrate how various kinds of politics propagate through technical systems, often without the creators of those systems even being aware of them.
Where do these labels come from?
The tool is based on ImageNet, a database of images and labels that was, and still is, one of the biggest and most accessible sources of training data for image recognition algorithms. As Quartz reports, it was assembled from images collected online, and tagged mainly by Mechanical Turk workers — humans who classified images en masse for pennies.
It reveals the deep problems with classifying humans – be it race, gender, emotions or characteristics. It’s politics all the way down, and there’s no simple way to ‘debias’ it.
— Kate Crawford (@katecrawford) September 16, 2019
Because the makers of ImageNet don’t own the photos they collected, they can’t just give them out. But if you’re curious, you can look up the photos’ tags and get a list of URLs that were the original sources of the photos. For example, “person, individual, someone, somebody, mortal, soul” > “scientist” > “linguist, linguistic scientist”> “psycholinguist” leads to this list of photos, many of which seem to have come from university faculty websites.
Tried it with me and my partner, racist algorithm said he was a sociologist and I was a negro/blackamoor. So white people get titles and we get assigned a colour… https://t.co/j0RjqvHtwi
— Piccola stella (@Exhausted33) September 16, 2019
Browsing those images gives us a peek into what has happened here. The psycholinguists tend to be white folks photographed in that faculty headshot sort of way. If your photo looks like theirs, you may be tagged a psycholinguist. Likewise, other tags depend on how similar you look to training images with those tags. If you are bald, you may be tagged as a skinhead. If you have dark skin and fancy clothes, you may be tagged as as wearing African ceremonial clothing.
It reveals the deep problems with classifying humans – be it race, gender, emotions or characteristics. It’s politics all the way down, and there’s no simple way to ‘debias’ it.
— Kate Crawford (@katecrawford) September 16, 2019
Far from being an unbiased, objective algorithm, ImageNet reflects the biases in the images that its creators collected, in the society that produced those images, in the mTurk workers’ minds, in the dictionaries that provided the words for the labels. Some person or computer long ago put “blackamoor” into an online dictionary, but since then many somebodies must have seen “blackamoor” in their AI’s tags (in freaking 2019!) and didn’t say, wow, let’s remove this. Yes, algorithms can be racist and sexist, because they learned it from watching us, alright? They learned it from watching us.
Leave a Reply
You must be logged in to post a comment.