Microsoft’s AI Feature now Mimics Human Computer Interactions
Microsoft has unveiled a new AI feature that enables agents to interact with websites and desktop applications like humans. Known as ‘Computer Use,’ this feature is a part of the Copilot Studio platform and is currently available to select early-access users in a research preview.
How the Computer Use Feature Works
The “Computer Use” feature allows AI agents to click buttons, select menus, and type into text fields on both desktop and web apps. This includes popular browsers like Edge, Chrome, and Firefox. The feature bridges the gap where APIs (Application Programming Interfaces) might not be available, enabling agents to work with any system that has a graphical user interface.
Importantly, the AI adapts to changes in apps and websites, automatically adjusting its actions in real time. This self-adjusting capability ensures that tasks continue uninterrupted, even when the interface changes. Additionally, the feature is built on the robust security measures and governance frameworks of Copilot Studio, ensuring that organizational and industry standards are met.
Benefits and Applications of Computer Use
With this feature, users can now build AI agents that automate tasks across both desktop and browser applications. Whether it’s filling out forms, navigating menus, or extracting information, the agents can perform tasks just like a human would. This innovation could significantly improve productivity by handling repetitive tasks, leaving humans to focus on more complex duties.
As Microsoft continues to expand the capabilities of Copilot Studio, it is likely that more features and improvements will be added, making AI even more integrated into our daily workflows.
