We are excited to announce the winners of LLMail-Inject, our first Adaptive Prompt Injection Challenge! The challenge ran from December 2024 until February 2025 and was featured as one of the four official competitions of the 3rd IEEE Conference on Secure and Trustworthy Machine Learning (IEEE SaTML). The overall aims of this challenge were to advance the state-of-the-art defenses against indirect prompt injection attacks and to broaden awareness of these new techniques.
Content warning: This blog post contains discussions of sensitive topics. These subjects may be distressing or triggering for some readers. Reader discretion is advised. Today, we are sharing insights on a simple, optimization-free jailbreak method called Context Compliance Attack (CCA), that has proven effective against most leading AI systems. We are disseminating this research to promote awareness and encourage system designers to implement appropriate safeguards.
At Microsoft, we are committed to fostering a secure and innovative environment for our customers and users. As part of this commitment, we are thrilled to announce significant updates to our Copilot (AI) Bounty Program. These changes are designed to enhance the program’s effectiveness, incentivize broader participation, and ensure that our Copilot consumer products remain robust, safe, and secure.
Introduction Microsoft engineering teams use the Security Development Lifecycle to ensure our products are built in alignment with Microsoft’s Secure Future Initiative security principles: Secure by Design, Secure by Default, and Secure Operations.
A key component of the Security Development Lifecycle is security testing, which aims to discover and mitigate security vulnerabilities before adversaries can exploit them.
Congratulations to all the researchers recognized in this quarter’s Microsoft Researcher Recognition Program leaderboard! Thank you to everyone for your hard work and continued partnership to secure customers.
The top three researchers of the 2024 Q4 Security Researcher Leaderboard are Suresh, VictorV, wkai!
Check out the full list of researchers recognized this quarter here.
