Researchers Uncover New Phishing Risk Hidden Inside Microsoft Copilot

Permiso security researchers have uncovered a significant new phishing vulnerability within Microsoft Copilot, revealing how cross-prompt injection attacks (XPIA) can manipulate AI-generated summaries. The research demonstrates that attacker-controlled text embedded in emails can influence how Copilot processes and summarizes content across Microsoft 365 services like Outlook and Teams. When users request email summaries, hidden instructions within the email body can steer the AI to append deceptive security alerts, malicious prompts, or phishing-style messages directly into the trusted Copilot interface. This represents a fundamental shift in attack methodology, where adversaries now target the AI assistant's credibility rather than relying solely on traditional email spoofing techniques.

Testing across three Copilot interfaces revealed varying levels of vulnerability, with the Teams Copilot interface showing the highest likelihood of reproducing attacker-supplied content. In some cases, Outlook's built-in summarize feature detected suspicious instructions and refused to generate summaries, indicating protective mechanisms exist but aren't consistently applied. The most concerning finding wasn't that Copilot followed attacker instructions, but how much more convincing malicious content becomes when delivered through the assistant's authoritative voice. Users have developed skepticism toward suspicious emails over decades, but that distrust doesn't transfer to AI-generated summaries appearing within familiar productivity tools.

The research highlights a critical security boundary issue as AI systems increasingly process untrusted external content. Organizations deploying Copilot and similar assistants must recognize that these tools create new attack surfaces where traditional email security measures may be insufficient. As AI becomes deeply integrated into workplace workflows, security teams need to develop specific protections against prompt injection attacks and educate users about this emerging threat vector. The findings suggest that AI assistant security requires fundamentally different approaches than traditional email security, focusing on how models interpret and process potentially malicious instructions embedded within otherwise normal-looking content.