Microsoft has introduced a major update to Windows 11 2024 (24H2), which, in addition to adding several traditional features, focuses on new capabilities of the Copilot artificial intelligence.
Copilot’s interface will be completely redesigned, with a window that will immediately suggest possible topics the user wants to learn about or actions they want to take. For example, meditate.
One of the new features of Copilot will be Click to Do, which activates artificial intelligence to help you complete tasks on the screen. For example, AI can help you remove an object from a photo or summarize a long article. This function is activated by simply holding down the Windows button and clicking the cursor on the desired item on the screen, which is a bit like Google’s Circle to Search.
In addition, Copilot will be able to communicate with the user by voice, and the virtual assistant will be able to ask questions, brainstorm with the user, or just chat. In the options, you will be able to choose one of four voice options for Copilot to use.
The Copilot Daily feature will also read out a summary of the news and weather forecast with additional information. Microsoft notes that Copilot Daily will receive information only from authorized sources such as Reuters, Axel Springer, Hearst Magazine, and the Financial Times, and plans to add more publications over time. Over time, the company promises to add additional personalization and customization options to Copilot Daily.
Copilot has been part of Microsoft’s Edge browser for some time now, but it won’t be getting a major update; users will only be able to launch it faster by typing @copilot in the address bar.
The experimental feature of Copilot Labs will allow Microsoft to test new features of Copilot that are still in development. For their part, users will be able to provide feedback and help Microsoft identify shortcomings. The first two features to be launched in Labs at the initial stage are: Copilot Vision and Think Deeper.
Copilot Vision is a feature that sees what the user sees and can talk to them about it in real time. It understands the content of web pages, both text and images, and can answer questions about their meaning.
In terms of data privacy and security, Microsoft promises that no content that Copilot Vision interacts with will be stored and used to train artificial intelligence models. And the feature itself will not work on all websites, but only with a limited list of popular pages. Apparently, Microsoft doesn’t want users to view anything illegal with Vision.
During the test period, Copilot Vision will also not work on paid and confidential websites. Microsoft notes that no special processing of the website content takes place during the function, and Copilot Vision simply reads and interprets the images and text it first sees on the page together with the user.
Think Deeper allows Copilot to think about more complex questions. The feature takes more time to respond, but allows Copilot to provide detailed answers to questions. Microsoft doesn’t say this directly, but it seems that Think Deeper is based on the OpenAI o1-preview model, which was launched only in September. It uses reinforcement learning, encouraging the model to “think” before responding by simulating a chain of thought.
Some of Copilot’s new features also apply to Windows 11 apps, including the Photos utility, which uses AI to increase the resolution of photos up to 8x the original. This feature is not unique, Adobe’s Super Resolution offers similar functionality, but its integration directly into Windows 11 will be more convenient for users.
In addition, the Paint app will feature AI-powered fill and erase options that allow users to delete or create new objects.
The update also brings long-awaited improvements to AI-powered search, allowing users to find files using natural language rather than complex search queries. This means that users will be able to find specific images by simply describing the objects in them, without having to memorize the exact file names or locations.
One of the most controversial features of Windows 11 2024, however, is Recall, the Copilot feature for finding everything that has happened on a computer. This feature was initially criticized for its insecurity, as researchers found that hackers could access the Recall screenshot database without administrator privileges.
In response, Microsoft has postponed the rollout of Recall to improve security by making it a fully optional feature that now requires Windows Hello biometric authentication and encryption of stored data.
Security remains a top concern for Copilot users, and Windows chief Pawan Davuluri acknowledged in a blog post that the Recall rollout was a lesson in handling sensitive data. He emphasized that the company has since “gone further” to instill confidence in its AI security measures, noting that AI features will require “additional security elements” beyond the traditional administrator access model.
Copilot features will not be available to everyone at once. They will first be available to Windows Insider program members and then rolled out to select devices and markets in November. The wider Windows 11 2024 update will also be rolled out in phases, and users who are eagerly awaiting it are encouraged to enable the “Get the latest updates as soon as they’re available” option in Windows Update settings.
As for improvements not related to artificial intelligence, there are very few. Windows 11 2024 will support Wi-Fi 7, HDR wallpapers on the desktop, improved battery life thanks to the Energy Saver feature, and extended Bluetooth LE support for hearing aids.
Loading comments …