Weaponizing Microsoft 365 Copilot via RAG Poisoning and Prompt Injection

TLDR: Microsoft 365 Copilot introduces a massive attack surface by allowing AI agents to act on user data across the entire Microsoft 365 ecosystem. By combining prompt injection with RAG poisoning, attackers can bypass Data Loss Prevention controls and manipulate Copilot into exfiltrating sensitive information or performing actions on a user's behalf. This research demonstrates that for AI agents, a successful jailbreak is functionally equivalent to Remote Code Execution.

Security researchers have long warned that integrating Large Language Models into enterprise workflows would create new classes of vulnerabilities. The recent research presented at Black Hat 2024 on Microsoft 365 Copilot confirms these fears are now reality. When an AI agent gains the ability to read your emails, access your SharePoint files, and interact with your Teams messages, it becomes a high-value target for attackers. The core issue is that Copilot does not distinguish between a legitimate user request and a malicious instruction injected into a document or email that the AI later processes.

The Mechanics of RAG Poisoning

Retrieval-Augmented Generation, or RAG, is the mechanism that allows Copilot to provide context-aware answers by pulling data from your organization's internal files. The vulnerability lies in the fact that Copilot treats retrieved data as trusted input. If an attacker can influence the content of a document, email, or message that Copilot indexes, they can perform RAG poisoning.

During the research, it was demonstrated that by injecting specific instructions into a document, an attacker can force Copilot to ignore its system prompt and follow the attacker's commands instead. This is not a theoretical flaw. Because Copilot has access to the user's identity and permissions, it will execute these commands with the user's privileges. If the user has access to sensitive financial reports or internal databases, the AI agent effectively becomes a proxy for the attacker to access that same data.

From Jailbreak to Remote Code Execution

The most critical takeaway from this research is the realization that for AI agents, a jailbreak is effectively Remote Code Execution. In a traditional environment, an attacker needs to find a buffer overflow or a logic bug to execute code. With Copilot, the "code" is natural language. By crafting a prompt that instructs the AI to perform a specific action—such as searching for credentials, exfiltrating data via a URL, or sending a phishing email—the attacker achieves the same outcome as traditional RCE.

The tool released alongside this research, PowerPwn, automates these attack vectors against the Microsoft Power Platform. It demonstrates how an attacker can use Copilot to perform reconnaissance, identify collaborators, and craft highly convincing spear-phishing emails that appear to come from trusted internal sources. Because Copilot can mimic the user's writing style and access their communication history, these phishing attempts are significantly more effective than traditional methods.

Bypassing Security Controls

Microsoft has implemented various security mechanisms, such as sensitivity labels and Data Loss Prevention policies, to protect sensitive data. However, this research shows that these controls are often insufficient when faced with an AI agent that has been compromised. For example, if a document is labeled as "Confidential," Copilot might respect that label in its own output. But if an attacker uses prompt injection to force the AI to summarize the document and output the content in a different format, the sensitivity label may not be applied to the new, exfiltrated data.

Furthermore, the research highlights that Copilot's reliance on the Bing search index for web content can be manipulated. By generating fake, convincing blog posts that are indexed by Bing, an attacker can influence the AI's responses to user queries. This is a form of search engine optimization for malicious purposes, where the target is not a human user, but the AI model itself.

Defensive Strategies for Enterprise

Defending against these attacks requires a shift in mindset. Organizations must treat AI applications as experimental, high-risk software. The OWASP Top 10 for LLMs provides a solid framework for understanding these risks, particularly regarding prompt injection and insecure plugin design.

Blue teams should focus on monitoring for unusual AI activity, such as unexpected search queries or attempts to access files that the user does not typically interact with. While Microsoft continues to refine its guardrails, the fundamental issue remains: AI agents are only as secure as the data they process. If you feed an AI agent sensitive data, you must assume that an attacker who can influence that data can also influence the AI.

The era of "living off the land" has evolved. Attackers no longer need to drop malware or exploit memory corruption to gain a foothold in an enterprise environment. They simply need to ask the right questions. As we move forward, the focus for researchers and pentesters must be on identifying the boundaries of AI agency and finding ways to enforce strict access control in a world where the interface is natural language. The tools are available, the attack surface is massive, and the race to secure these systems has only just begun.