An artificial intelligence chatbot created by the New York City government to help small business owners was giving them strange advice and answers, including suggestions to break the law. Despite this, the city authorities refused to remove it. This was reported by the Associated Press.
The chatbot, which was launched in October in cooperation with Microsoft to help small business owners with bureaucratic questions, offered users algorithmically generated text answers to questions about navigating the city’s bureaucratic maze.
The bot contained a disclaimer that sometimes the answers could be incorrect, harmful, or biased, and a statement that its answers did not constitute legal advice. But despite the warnings, the bot’s answers went so far as to suggest that users break the law.
The error messages said that the chatbot said that an employer has every right to fire an employee who complains about sexual harassment, fails to report pregnancy, refuses to cut her dreadlocks, etc.
After these reports, the New York City authorities refused to remove the bot. Instead, Mayor Eric Adams defended the decision to leave the chatbot available for use, although he admitted that the bot’s answers were incorrect in some areas.
Adams also said that allowing users to find errors in bot responses is part of the improvement of new technologies.
“Anyone that knows technology knows this is how it’s done. Only those who are fearful sit down and say, ‘Oh, it is not working the way we want, now we have to run away from it all together.’ I don’t live that way,” said the mayor.
Despite all the shortcomings, the bot continues to work, which is a cause for concern among experts, as such a system, which produces a bunch of false and sometimes even illegal answers, is dangerous, especially if it is used by the authorities without clear boundaries and regulation.
Loading comments …