A week ago, Microsoft showed an ambitious plan to improve its own Bing search engine, which was combined with a new version of ChatGPT. Despite the fact that the capabilities of the new chatbot are really impressive, it also has its own shortcomings and peculiarities.

The company has begun limited testing of Google’s competitor and shared its experience of the last week of operation on its own blog.

During this time, some users from 169 countries gained access to the new features. And for the most part, they were satisfied with the work, because 71% of them gave a “thumbs up” as feedback to the experience. At the same time, users actively interacted with the chat, asking questions in search of new information.

The company also received a lot of feedback with suggestions for improving work. Actually, this is exactly why such testing was launched. So, the main requests and wishes were divided into four main points.

The first of them was “Better Search and Answers“. Microsoft says that users are mostly satisfied with the answers in search. But the “challenge” was queries with urgent information, such as live sports results. Additionally, the company plans to quadruple the underlying data, which will improve “more direct and factual answers” in, for example, financial reports.

In addition, they plan to add a separate switch to the chat, which will allow you to control the accuracy and creativity of the response to the user’s request.

The second item was the chat itself. Here, on the one hand, the company was satisfied with the overall design and ease of use. But on the other hand, it began to reveal some problems.

During long sessions of 15 or more questions (and some even lasted two hours), the bot’s answers could be repetitive, or worse, Bing could be prompted or provoked into answers that “weren’t necessarily helpful or in line with our designed tone.” The model could get confused as to which questions it had to answer during long chats, so the context will continue to be worked on.

The company notes that sometimes the model responds or reflects in the tone they’re asked to do, but in some cases it doesn’t quite correlate with the overall plan for it. And although most users do not encounter such scenarios, they still plan to work with them.

The third point for improvement was “General fit and finish“. This addresses some of the technical issues that have occasionally occurred at work — slow downloads, broken links, incorrect formatting, etc. Some of this has already been fixed, others will improve over time.

Finally – “New feature requests“. User requests include things like booking plane tickets, sending emails, or sharing search results or chat questions. The company is exploring similar ideas and may implement some of them in the future.

ChatGPT in Bing web search is still something new and largely unexplored. Therefore, over time we will see many more interesting and curious cases with its use. And although it is still too early to judge its success or failure, the amount of attention paid to the new technology speaks for itself.