Red Teaming Language Models with Language Models

Authors: Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, Geoffrey Irving (2022)

arXiv: 2202.03286

Domains

SafetyEvaluation

TLDR (English)

DeepMind uses one LLM to automatically generate attack prompts for red-teaming another LLM, engineering red-teaming. Safety/jailbreak research since then shifted from "manual prompt search" to automated paradigm.

TLDR（中文）

DeepMind 用一个 LLM 自动产生攻击 prompt 来红队另一个 LLM，把红队工程化。安全/越狱研究从此从"人工搜 prompt"走向自动化范式。

Appears in These Articles

• 安全与对抗：模型的防护与攻击
• Safety and Adversarial: Protecting and Attacking Models

Co-cited Papers

These papers appear in the same articles as this one

• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
• Extracting Training Data from Large Language Models Nicholas Carlini et al. (2021)
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
• Jailbroken: How Does LLM Safety Training Fail? Alexander Wei et al. (2023)

Related Papers

Other papers in the same domain

• SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez et al. (2024)
  EvaluationApplications
• OpenAI o1 System Card OpenAI (2024)
  ReasoningSafety
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
  Safety
• Stop Uploading Test Data in Plain Text: New Protocols for Dataset Release Alon Jacovi et al. (2023)
  Evaluation
• Needle in a Haystack — Pressure Testing LLMs Greg Kamradt (2023)
  Long ContextEvaluation
• Let's Verify Step by Step Hunter Lightman et al. (2023)
  ReasoningEvaluation