Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

Authors: Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar (2024)

arXiv: 2408.03314

Domains

ReasoningInference

TLDR (English)

Systematically presents scaling laws for "spending more compute at inference time": with fixed budget, adding inference-time search to small models often more cost-effective than training larger models. Theoretical foundation for o1/R1 era.

TLDR（中文）

系统性给出"推理时多花 compute"的 scaling law：在固定预算下，对小模型加推理时搜索往往比训练更大模型更划算。是 o1/R1 时代理论支撑。

Appears in These Articles

• Sampling 与 Decoding：从概率到文字
• Sampling and Decoding: From Probabilities to Text

Co-cited Papers

These papers appear in the same articles as this one

• Language Models are Few-Shot Learners Tom Brown et al. (2020)
• Accelerating Large Language Model Decoding with Speculative Sampling Charlie Chen et al. (2023)
• Fast Inference from Transformers via Speculative Decoding Yaniv Leviathan et al. (2023)
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason Wei et al. (2022)

Related Papers

Other papers in the same domain

• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI (2025)
  ReasoningAlignment
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model DeepSeek-AI (2024)
  ArchitectureInference
• OpenAI o1 System Card OpenAI (2024)
  ReasoningSafety
• FlashAttention-3: Fast and Accurate Attention with Asympotic IO Complexity Jay Shah et al. (2024)
  Inference
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints Joshua Ainslie et al. (2023)
  InferenceArchitecture
• Accelerating Large Language Model Decoding with Speculative Sampling Charlie Chen et al. (2023)
  Inference