AI-based search engines, such as ChatGPT or Grok 3, misinform users by linking to incorrect news sources in 60% of cases and ignoring publisher requests.
According to a study conducted by the Columbia Journalism Review and the Tow Center for Digital Journalism, serious problems were found during testing of eight popular AI tools that offer real-time information search.
To test the effectiveness of such AI systems, the researchers provided direct excerpts from real news articles and then asked each model to identify the headline, original publisher, publication date, and URL. They made 1,600 queries.
The error rate among the tools varied significantly: the Perplexity system provided incorrect information in 37% of requests, ChatGPT made mistakes in 67% of cases, and Grok 3 showed the worst result — 94% of incorrect answers.
Researchers also noted the following problems in using these AI systems:
▪️chat bots, as a rule, did not refuse to answer questions they could not answer accurately, offering incorrect or probable answers instead;
▪️paid chatbots were more likely to give incorrect answers than their free counterparts;
▪️It appears that several chatbots are bypassing the robot exception protocol settings;
▪️Generative search tools created links by citing syndicated and copied versions of articles;
▪️license agreements for news content do not guarantee the accuracy of quotes in responses.
Read also: Searching with AI — can Google lose a new battle for search?
When these AI search engines did link to sources, they often directed users to syndicated versions of content on platforms like Yahoo News rather than the original publisher’s website. This occurred even when the publishers had formal licensing agreements with AI companies. Perplexity, for example, successfully identified 10 excerpts from National Geographic’s paid content that were supposed to be private. But Gemini and Grok 3 generated a lot of non-existent URLs. In the case of Grok 3, out of 200 links checked, 154 were either incorrect or led to non-existent pages.
As a reminder, Google recently launched a new experimental feature called "AI Mode" in Search. AI models from Google, OpenAI, and Perplexity will be able to process complex queries, collect, analyze, and summarize information from multiple sources.
Read also: AI Speaks — 10 Chatbots and Voice Assistants for Communication