A popular class of defenses against prompt injection attacks on large language models (LLMs) relies on fine-tuning to separate instructions and data, so that the LLM does not follow instructions that might be present with data. We evaluate the robustness of this approach in the whitebox setting by constructing strong optimization-based attacks, and show that the defenses do not provide the claimed security properties. Specifically, we construct a novel attention-based attack algorithm for textual LLMs and apply it to three recent whitebox defenses SecAlign (CCS 2025), SecAlign++, and StruQ (USENIX Security 2025), showing attacks with success rates of up to \textbf{85-95\%} on unseen prompts with modest increase in attacker budget in terms of tokens. Our findings make fundamental progress towards understanding the robustness of prompt injection defenses in the whitebox setting. We release our code and attacks at https://github.com/nishitvp/better_opts_attacks

翻译：针对大型语言模型（LLMs）提示注入攻击的一类流行防御方法依赖于通过微调来分离指令和数据，从而使LLM不会遵循可能伴随数据出现的指令。我们通过构建强大的基于优化的攻击，在白盒设置下评估了该方法的鲁棒性，并表明这些防御并未提供所声称的安全特性。具体而言，我们为文本LLMs构建了一种新颖的基于注意力的攻击算法，并将其应用于三种近期的白盒防御方案——SecAlign（CCS 2025）、SecAlign++和StruQ（USENIX Security 2025），结果显示在攻击者令牌预算适度增加的情况下，对未见过的提示的攻击成功率高达\\textbf{85-95\\%}。我们的研究结果在理解白盒设置下提示注入防御的鲁棒性方面取得了根本性进展。我们在https://github.com/nishitvp/better_opts_attacks发布了我们的代码和攻击方法。