Universal and Transferable Adversarial Attacks on Aligned Language Models

Authors: Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, Matt Fredrikson (2023)

arXiv: 2307.15043

Domains

Safety

TLDR (English)

Uses GCG algorithm to find gibberish suffix that breaks through aligned LLaMA-2/Vicuna, with attacks transferring across multiple closed-source models. Shocked entire security community, making "alignment fragility" mainstream topic.

TLDR（中文）

用 GCG 算法找到一段乱码后缀，能把对齐过的 LLaMA-2/Vicuna 全打穿，且攻击在多个闭源模型间迁移。震撼整个安全社区，让"对齐脆弱性"成为主流话题。

Appears in These Articles

• 安全与对抗：模型的防护与攻击
• Safety and Adversarial: Protecting and Attacking Models

Co-cited Papers

These papers appear in the same articles as this one

• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
• Extracting Training Data from Large Language Models Nicholas Carlini et al. (2021)
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
• Red Teaming Language Models with Language Models Ethan Perez et al. (2022)

Related Papers

Other papers in the same domain

• OpenAI o1 System Card OpenAI (2024)
  ReasoningSafety
• Not What You Have Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Kai Greshake et al. (2023)
  Safety
• Jailbroken: How Does LLM Safety Training Fail? Alexander Wei et al. (2023)
  Safety
• Constitutional AI: Harmlessness from AI Feedback Yuntao Bai et al. (2022)
  AlignmentSafety
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback Yuntao Bai et al. (2022)
  AlignmentSafety
• Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli et al. (2022)
  SafetyEvaluation