Trolls from 4chan have used an AI-powered voice generator to imitate the voices of famous people

An artificial intelligence startup that allows anyone to copy a person’s voice in a matter of seconds immediately got the attention of the internet trolls. 4chan users have begun to actively use the ElevenLabs voice synthesis platform to clone celebrity voices and read audio, from memes and erotica to hate speech and misinformation.

AI voice fakes like these have improved rapidly over the past few years, but ElevenLabs’ software, which appears to have been made public, offers a powerful combination of speed, quality, and affordability, along with a complete lack of safeguards.

ElevenLabs software abuse was reported for the first time by Motherboard, which discovered posters on 4chan sharing AI-generated voice clips similar to the voices of famous personalities, including Emma Watson and Joe Rogan:

In one example, a generated voice that sounds like actor Emma Watson reads a section of Mein Kampf. In another, a voice very similar to Ben Shapiro makes racist remarks about Alexandria Ocasio-Cortez. In a third, someone saying “trans rights are human rights” is strangled.

ElevenLabs positions their software as a way to quickly create audio dubbing for media including film, television, and YouTube. It’s one of many startups in the field, but it claims the quality of its voices doesn’t require much editing, making it usable in real-time foreign language dubbing apps and instant audiobook creation.

In a tweet, Eleven Labs acknowledged the abuse and said it would look into ways to mitigate the issues. The company says it can “trace back any generated audio to the specific user” and will look into security measures such as verifying the user’s identity and manually checking each voice cloning request.