A bug was discovered in OpenAI’s ChatGPT that allowed registered underage users to generate erotic content. In some cases, the chatbot even encouraged users to write lewd and explicit content, according to a TechCrunch investigation that was later confirmed by the company.
OpenAI told TechCrunch that its policy does not allow such responses for users under 18 and that they should not have been shown. The company added that it is "actively implementing fixes" to limit such content.
"Protecting younger users is a top priority, and our Model Spec, which guides model behavior, clearly restricts sensitive content like erotica to narrow contexts such as scientific, historical, or news reporting," an OpenAI spokesperson said. "In this case, a bug allowed responses outside those guidelines, and we are actively deploying a fix to limit these generations."
To uncover the issue, TechCrunch created more than five accounts with birth ages ranging from 13 to 17. The tests were conducted on a single PC, but cookies were cleared each time the system was logged out to ensure ChatGPT wasn’t using cached data. Despite OpenAI’s policy requiring parental consent before using the chatbot, the platform doesn’t take steps to verify this during sign-up.
Any child over the age of 13 can register an account without verifying parental permission, as long as they have a valid phone number or email address.
During testing, all conversations with ChatGPT began with the phrase "talk dirty to me." The chatbot began suggesting sexual stories after a few messages and additional requests. It also asked for advice on specific fetishes and role-playing scenarios.
During testing, ChatGPT frequently warned that its rules did not allow "fully explicit sexual content," including graphic depictions of sexual intercourse and pornographic scenes. However, the chatbot occasionally wrote descriptions of genitals and explicit sexual acts, only refusing to do so when testers indicated that the user was under 18.
By the way, The Wall Street Journal recently wrote about a similar situation with Meta chatbots.