Microsoft Copilot AI Update Adds Personalisation, Voice, Vision Tools

Microsoft has announced its biggest-ever update to the Copilot AI assistant in Windows 11, adding a new personalised home screen, a daily news summary, voice chat, contextual awareness, and Microsoft Edge web browser integration. The company also says conversations will be more fluid and responses will be quicker, with a more personal tone. Although these features will not be rolling out in India at present, they should be available as the company tests and refines local language support and context awareness.

Copilot is available on Windows 11 PCs as well as on Android and iOS through an app. There’s also a browser-based version, and WhatsApp integration is coming soon.

Interestingly, the company’s blog post announcing this rollout is prefaced with a letter from Mustafa Suleyman, Executive Vice President and CEO of Microsoft AI, promising “careful steps” as the company navigates how wide adoption of AI is affecting people and society. Copilot, characterised as “an AI companion for everyone”, is being positioned as helpful, supportive, and ever-present, while being mindful of users’ privacy and security.

Microsoft envisions people chatting with Copilot about their day, sharing experiences and emotions, or even venting in stressful situations. The assistant will grow to learn how a user responds to encouragement and advice. According to the company, Copilot will eventually “share the load” of mental activities such as planning schedules and remembering appointments. 

Copilot Voice presently offers four voice options. A Microsoft demo video shows an example of a response including vocal inflections, slang, pauses and filler words. Copilot Vision is being rolled out as a “Copilot Labs” feature, meaning it’s still being tested and tweaked and will only be available to Pro subscribers. It can recognise objects and text on a user’s screen, so they can ask for contextual information and have conversations based on what they are looking at or working on. 

Copilot Daily is a morning summary of news headlines, appointments, and weather updates, read out in your chosen voice. News and information sources are whitelisted by Microsoft and currently include Reuters, USA Today, and Hearst Magazines. The new Discover screen will show tips about how to use Copilot as well as prompts based on a user’s conversation history and interests. 

Microsoft promises that Copilot Vision sessions are not recorded and data is not used to train its AI ro stored in any way. On the Web, it will work only with select whitelisted websites for now, and that excludes paywalled and “sensitive” content. 

Another experimental feature, called Think Deeper, promises more thoughtful and better reasoned responses to complex questions, though these will take longer. Copilot will be able to provide detailed step-by-step instructions or compare options, based on user input.

As for Copilot+ laptops, Microsoft is reintroducing its controversial Recall feature, which is now locked behind Windows Hello face recognition. Storing snapshots of what you’re working on will now be opt-in, and sensitive personal information will be filtered out. Copilot+ laptops with Qualcomm Snapdragon processors can access this feature now, while Intel and AMD-powered machines will get it in November. 

Click To Do is a new way to bring up contextual tools such as photo editing or text summarisation. Users can hold the Windows key while clicking on anything to see these options, which Microsoft says can boost productivity. 

Windows Search can now handle requests based on content rather than just filename, date, or content type. For example, you can search for photos of an object or situation in the Windows Explorer search bar. The Photos app can use AI to upscale photos, while the humble MS Paint gets generative fill and object removal tools. These updates will roll out with the Windows 11 24H2 update.