Red Teaming Language Models with Language Models

作者： Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving (2022)

arXiv： 2202.03286

领域

安全评估

TLDR（中文）

DeepMind 用一个 LLM 自动产生攻击 prompt 来红队另一个 LLM，把红队工程化。安全/越狱研究从此从"人工搜 prompt"走向自动化范式。

TLDR (English)

DeepMind uses one LLM to automatically generate attack prompts for red-teaming another LLM, engineering red-teaming. Safety/jailbreak research since then shifted from "manual prompt search" to automated paradigm.

出现在这些文章里

• 安全与对抗：模型的防护与攻击
• Safety and Adversarial: Protecting and Attacking Models

同被引用

这些论文与本文出现在同一篇文章中

• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
• Extracting Training Data from Large Language Models Nicholas Carlini et al. (2021)
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
• Jailbroken: How Does LLM Safety Training Fail? Alexander Wei et al. (2023)

相关论文

同一领域的其他论文

• SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez et al. (2024)
  评估应用
• OpenAI o1 System Card OpenAI (2024)
  推理能力安全
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
  安全
• Stop Uploading Test Data in Plain Text: New Protocols for Dataset Release Alon Jacovi et al. (2023)
  评估
• Needle in a Haystack — Pressure Testing LLMs Greg Kamradt (2023)
  长上下文评估
• Let's Verify Step by Step Hunter Lightman et al. (2023)
  推理能力评估