Jailbroken: How Does LLM Safety Training Fail?

作者： Alexander Wei, Nika Haghtalab, Jacob Steinhardt (2023)

arXiv： 2307.02483

领域

安全

TLDR（中文）

系统化分类越狱手段（在分布外、目标冲突），并解释为什么 RLHF 难以根治。是越狱研究"taxonomy"参考资料。

TLDR (English)

Systematically classifies jailbreak methods (out-of-distribution, goal conflict) and explains why RLHF struggles to eradicate them. Reference material for jailbreak research "taxonomy".

出现在这些文章里

• 安全与对抗：模型的防护与攻击
• Safety and Adversarial: Protecting and Attacking Models

同被引用

这些论文与本文出现在同一篇文章中

• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
• Extracting Training Data from Large Language Models Nicholas Carlini et al. (2021)
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
• Red Teaming Language Models with Language Models Ethan Perez et al. (2022)

相关论文

同一领域的其他论文

• OpenAI o1 System Card OpenAI (2024)
  推理能力安全
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
  安全
• Universal and Transferable Adversarial Attacks on Aligned Language Models Andy Zou et al. (2023)
  安全
• Constitutional AI: Harmlessness from AI Feedback Yuntao Bai et al. (2022)
  对齐安全
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
  对齐安全
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli et al. (2022)
  安全评估