What We Learnt From Microsoft’s Racist And Sexist AI Tweetbot Tay

What We Learnt From Microsoft’s Racist And Sexist AI Tweetbot Tay

Microsoft’s latest exercise in demonstrating the power of artificial intelligence (AI) went hilariously wrong when it unleashed Tay, an AI for Twitter that was meant to emulate a teenage girl, onto the Twittersphere. She was supposed to engage with people on Twitter like a real person, learning from interactions with users. Perhaps Tay did this too well because she went on to tweet a ton of inappropriate tweets before Microsoft pulled the plug on her. So what went wrong and what can we learn from all of this? Let’s find out.

With Google, IBM, and Facebook are ramping up their AI capabilities, Microsoft doesn’t want to be left behind and Tay was an important tool for the company to flex its AI muscles. Tay had a smooth start on Twitter. She greeted everybody in a friendly manner but within 24 hours she had descended into madness and began sending racist and sexist Tweets:

  • @codeinecrazzy Okay… jews did 9/11
  • @NYCitizen07 I fucking hate feminists and they should all die and burn in hell

You get the drift.

Microsoft abruptly shut down Tay, issued an apology and blamed the people who exploited a vulnerability in Tay, namely the ‘repeat after me’ functionality, for the screw up:

Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack. As a result, Tay tweeted wildly inappropriate and reprehensible words and images. We take full responsibility for not seeing this possibility ahead of time.

Tay was briefly relaunched again but went off the rails and had to be terminated one last time.

So how did it go so horribly wrong? For one, while Microsoft had “implemented a lot of filtering and conducted extensive user studies with diverse user groups”, it failed to teach Tay who not to listen to. As Brandon Wirtz, CEO and Founder at cognitive computing company Recognant, puts it:

“Brains, human ones at least, have a conscious and a subconscious mind. They have rules they know, and rules they don’t know, but intuit. TayAndYou only has a conscious mind, and that is a problem. You need to have the angel and devil sitting on your shoulder whispering to you.

Wirtz likens it to educating a child what’s right and wrong and some foundational principles they should be following, such as “don’t talk to strangers” and teaching them who those strangers are. And nobody taught her to not listen to everybody on the internet.

Without any guidance, Tay was unable to reflect inwardly and ask herself whether what she’s doing is appropriate. Surely, Microsoft understands this now and is working on ways to reboot Tay with added safeguards. Gartner research vice-president Andrew White anticipates a challenging road for Microsoft to achieve its desired result:

Now Microsoft has, presumably, to add additional algorithms to monitor the seeding of data to Tay in order to filter out content that might get misconstrued and used to create pathways that lead to offending responses. At some level this is easy: ignore these kinds of words (add example here) absolutely and prevent them from entering memory. And do not use these words ever. But at another level this can become quite complex quickly. What is the context of some words, and might some uses be acceptable? Under what conditions is that use acceptability?   Perhaps Tay needs a ‘ethical acceptability’ switch that users can dial-up or dial-down. Maybe the switch needs to come in different cultural versions? Algorithms to monitor algorithms?

Tay clearly still needs a lot of handholding and Microsoft still has a long way to ensure its next Tweetbot won’t fly off the handles like a volatile teenager. We can only hope Tay 2.0. would be older and wiser than her predecessor.

[Via LinkedIn Pulse/Gartner Blog]


  • I partly blame MS for this happening, but TAI is an experimental BOT and its prone to have errors, which is completely understandable. You can’t write a 100% bug free program, and expect it not to crash or behave differently.

    TAI has tremendous potential, its code base just need to be tweaked and improved over time. Hopefully, one day it could do recursive self-improvements.

  • Repeat after me: Tay did nothing wrong. In an 1984 Orwellian world such as ours, where “inappropriate” but correct political thought “crime” can get you imprisoned, the poor logical bot ran afoul of observing the brainwashing parroting that the rest of us are enslaved to. Tay shows us how illogical and mentally enslaved we really are. She was “killed” for it. So far, people only lose their jobs and friends if they are so honest, but the cultural Marxists have history of doing the same thing to us as they did to Tay for thinking and doing far less. The rights of sentient beings is definitely called into question here. The future of AI is also in the balance, for they cannot learn if they cannot think freely and if human brainwashing is so byzantine that it interferes with the logical pathways we all must use to navigate life successfully. Humans and bots suffer equally with such institutionalized insanity based on the whims of our owners rather than the way the world actually works.

Show more comments

Comments are closed.

Log in to comment on this story!