Microsoft's premium Copilot agents fail miserably in real-world work test

Microsoft is spending billions on AI infrastructure and licensing models from OpenAI, Anthropic, and others, aiming to turn Windows + Microsoft 365 into an 'agentic OS' that automates corporate drudgery. But ZDNET senior editor Ed Bott found the reality far less impressive. After paying $10 for a month of the Microsoft 365 Premium plan, he tested the exclusive Analyst agent by asking it to improve a household budget spreadsheet. The agent gave generic advice and offered to 'sketch a dashboard layout'—not build it. When Bott pushed for an actual deliverable, Copilot claimed to have created a modified workbook but provided a non-clickable sandbox path. After multiple attempts, it admitted: 'your chat interface is currently not rendering downloadable file attachments correctly.' The agent suggested using ChatGPT as a workaround.

Bott's broader experience with Copilot agents revealed a pattern of hallucinations, misinformation, and wasted time. The agents flash occasional competence but consistently fail at real tasks. Troubleshooting with Copilot itself was futile—it couldn't fix even its own delivery problems. While developers find value in tools like Claude Code and GitHub Copilot, Microsoft's business-focused agents appear half-baked. The promised 'agentic OS' that writes memos, builds presentations, and automates meetings remains a distant goal. For now, corporate users paying extra for Copilot Premium are getting confidence without capability.