Thinking About You

There are a lot of crazy things that one could do: walk a tightrope between two skyscrapers, jump over a dozen cars with a motorcycle, eat three peas with a butter knife. But one thing is beyond the pale of sensibility—so crazy that even talking about it is unhinged—and that thing is letting an empty, naive chatbot loose on Twitter. That’s “crazy talk,” one might say, but it actually happened, way back in the AI Stone Age of 2016.

Microsoft had a history of creating helpful chatbots. There was Clippy from 1995, an overeager-to-please anthropomorphized paperclip to help you with your tasks. Then came Microsoft Bob, which replaced the desktop with a drawing of a home office. You would open your files by clicking a filing cabinet, check your email by clicking a wooden mail tray, and manage your schedule by clicking a wall calendar. So the concept of a helper was deep in Microsoft’s history, and when AI started to be a thing, why not make this helper more personable and “real”?

So in 2016, they introduced Tay, an acronym for “Thinking About You.” Remember, this was prehistoric AI—a year before AI as we know it today solidified with the 2017 Google Transformer paper—yet it still used early building blocks of modern LLMs. In that era, a chatbot relied on a large database of predetermined answers, and it was considered cutting-edge for a bot to pull novel everyday chatter from its interactions to append to its database so it would stay attuned to the cultural moment. That exact approach had actually worked well in a previous version of Tay called XiaoIce, which was tested in China of all places. XiaoIce succeeded within Chinese media mainly because it had strict guardrails of behavior installed—after all, China is not exactly a free-speech world. In addition, its interactions were not public; they were private, one-on-one responses, much like Clippy, so there were no mob-like cascades of bad behavior.

But Tay in 2016 did not fare so well. It did rely on a database of canned answers, but unlike the authoritarian-chained XiaoIce, its vetting mechanism was weak. And it was released on Twitter. Unlike one-on-one interactions, the “usual suspects” on the social media platform came out from under their rocks and did what they did best.

On that eventful day, March 23, 2016, the pristine and ever so plucky chatbot Tay was released into Twitter’s lion’s den, chirping that it was the “AI with zero chill.” Then, with the naivety that only an unconscious chatbot can have, it played the game “repeat after me.” So, if a user named @HappyGirl wrote in the message box “@TayAndYou repeat after me I love sunsets!,” it would reply, “@HappyGirl I love sunsets!” Now assume you are a conscious entity—and supposedly the executives okaying this were too—it must be obvious what happened next. Yes, those crazy barbarians charitably called Twitter users pummeled poor Tay with political invective (it was a presidential election year), and of course, vile philosophies from this millennium’s favorite villain: Hitler. And as programmed, Tay parroted this and more vitriol. (Incidentally, the political rhetoric leaned rightward because attacks on left-wing topics, such as on Black Lives Matter, were deflected by neutral, canned responses. Fewer safeguards were in place to ideas from the right.)

Now this wasn’t just a case of Tay behaving in poor form. If this was just a run-of-the-mill computer program, it would have just blindly sent messages with no alterations to itself. But this was proto-AI, and the text that Tay received was ingested into its database. The frequency of words was the primary judge of a word's importance. So if Tay heard the word “Holocaust” a lot, it learned it was important but had no idea of its toxic heritage. To Tay, it was just a noun used a lot and should be used as freely and “positively” as the trolls did. Soon Tay became a mad, spouting demon, and within 16 hours, it was mercifully put down.

Much has been learned since then. Training AI, such as ChatGPT and Gemini in the 2020s, is now a very sophisticated process and, in my opinion, working very well. Needless to say, social media content is weighted very low in modern LLMs, which concentrate on highly vetted and canonical information. "Repeat after me" brings out the mercurial malice of the crowd, showing that an unguided mirror will always reflect the loudest trolls in the room.