Academic plagiarism software provider Turnitin has demonstrated an artificial intelligence tool it claims is 98% accurate in detecting machine text. However, the first tests conducted by the Washington Post, showed that its effectiveness was insufficient.

While Turnitin was good at recognizing AI-generated text when all input was generated using ChatGPT, it did poorly with sentences that were co-authored by a human and a machine. Additionally, the tool incorrectly flagged parts of the essay written by the student as AI-generated.

New ways to identify text with artificial intelligence are becoming increasingly important as educators face a new challenge when students use chatbots like ChatGPT to write essays and complete homework.

Turnitin has been criticized for not providing transparent information to customers about how its tools work. According to the Financial Times, some leading universities of The Russell Group in Great Britain, in particular the University of Cambridge, refused to use Turnitin’s AI text recognition tool.

It is quite difficult to determine whether a passage of text was created by artificial intelligence. Even OpenAI, the company that developed ChatGPT, warned that its tool could correctly identify text as written by AI only 26% of the time. A group of computer scientists from the University of Maryland recently published a study showing that detection tools can be easily bypassed.

Turnitin’s AI discovery software is still free to use. CEO Chris Caren said, “Educators told us that being able to accurately detect AI written text is their first priority right now. They need to be able to detect AI with very high certainty to assess the authenticity of a student’s work and determine how to best engage with them.”