AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

作者： Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han (2023)

arXiv： 2306.00978

领域

推理

TLDR（中文）

发现"少数关键权重对应大激活"，按重要性做 per-channel scaling。在 4-bit 上比 GPTQ 更鲁棒、推理更快，是当下 INT4 部署的主流方案之一。

TLDR (English)

Discovers "few critical weights correspond to large activations", applies per-channel scaling by importance. More robust and faster than GPTQ at 4-bit, one of mainstream INT4 deployment solutions today.

出现在这些文章里

• KV Cache 与量化：让大模型跑得更快
• KV Cache and Quantization: Making Large Models Faster

同被引用

这些论文与本文出现在同一篇文章中

• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Tim Dettmers et al. (2022)
• QLoRA: Efficient Finetuning of Quantized LLMs Tim Dettmers et al. (2023)
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar et al. (2022)
• Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon et al. (2023)

相关论文

同一领域的其他论文

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  架构推理
• FlashAttention-3: Fast and Accurate Attention with Asympotic IO Complexity Jay Shah et al. (2024)
  推理
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Charlie Snell et al. (2024)
  推理能力推理
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  推理架构
• Accelerating Large Language Model Decoding with Speculative Sampling Charlie Chen et al. (2023)
  推理
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning Tri Dao (2023)
  推理