QLoRA: Efficient Finetuning of Quantized LLMs

Authors: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer (2023)

arXiv: 2305.14314

Domains

AlignmentInference

TLDR (English)

4-bit NF4 + LoRA + paged optimizer enables SFT of 65B on single 48GB GPU. Open-source community fine-tuning of LLaMA-2/3, Qwen uses this approach almost 100%.

TLDR（中文）

4-bit NF4 + LoRA + paged optimizer，让 65B 在单张 48GB 显卡上 SFT。开源社区微调 LLaMA-2/3、Qwen 几乎 100% 用这套方案。

Appears in These Articles

• KV Cache 与量化：让大模型跑得更快
• KV Cache and Quantization: Making Large Models Faster

Co-cited Papers

These papers appear in the same articles as this one

• LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale Tim Dettmers et al. (2022)
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers Elias Frantar et al. (2022)
• Efficient Memory Management for Large Language Model Serving with PagedAttention Woosuk Kwon et al. (2023)
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration Ji Lin et al. (2023)

Related Papers

Other papers in the same domain

• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI (2025)
  ReasoningAlignment
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  ArchitectureInference
• KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh et al. (2024)
  Alignment
• FlashAttention-3: Fast and Accurate Attention with Asympotic IO Complexity Jay Shah et al. (2024)
  Inference
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Charlie Snell et al. (2024)
  ReasoningInference
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  InferenceArchitecture