In response to growing concerns among educators about ChatGPT’s ability to help students cheat, OpenAI has released a tool, which can recognize text written by artificial intelligence. However, the company said, “our classifier is not fully reliable.”
“In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives)”, wrote the company that developed ChatGPT on its blog.
A classifier trained on a set of texts written by humans and artificial intelligence on the same topic is not yet reliable and should only complement other ways of determining who wrote the text. Limitations that the classifier has, according to OpenAI, include being unreliable on short texts and working much more reliably with texts from the latest artificial intelligence systems.
Recently, NYU professors told students that they could not use ChatGPT without specific permission, saying that any use of the tool would be considered plagiarism. Teachers have come up with their own ways to detect AI essay writing to prevent any cheating, such as passing their essay prompts through ChatGPT to have a benchmark of what the AI-generated text will look like.
A 22-year-old student named Edward Tian developed his own ChatGPT detector, which was launched in beta earlier this year and fully released on January 29 as GPTZeroX. In this application, you can insert text or upload one or more documents at the same time, and the program will count how much of the text was written by the artificial intelligence and highlight the sentences that were written by AI. According to Tian, the app was wrong less than 2% of the time when tested on a dataset of BBC news articles and machine-generated articles with the same tip, making GPTZeroX more reliable than OpenAI’s own detector.
On its blog, OpenAI is asking people directly affected by the speech bot, “including but not limited to teachers, administrators, parents, students, and education service providers,” to provide feedback via a Google Form survey. As ChatGPT becomes capable of writing everything from student essays to code, teachers around the world are trying to adapt the learning process to the new technology and debating how the tool can be used ethically.