On-Policy Self-Distillation for Reasoning Models

作者： Self-Distilled Reasoner Team (2026)

arXiv： 2601.18734

领域

对齐推理能力

TLDR（中文）

2026 开年后训练热点"自蒸馏"的代表工作：模型在自身策略分布上生成数据并蒸馏回自身，不再依赖外部强教师模型，配合可验证奖励实现自我改进闭环，显著降低后训练对蒸馏管线的依赖。

TLDR (English)

A representative work of the early-2026 "self-distillation" trend in post-training: the model generates data on its own policy distribution and distills it back into itself, removing the need for an external strong teacher and closing a self-improvement loop with verifiable rewards, reducing post-training dependence on distillation pipelines.

出现在这些文章里

• 微调与对齐：让模型听指令、守规矩
• Fine-Tuning and Alignment: Making Models Follow Instructions

同被引用

这些论文与本文出现在同一篇文章中

• Agent World Model: Scaling Agentic RL with Synthetic Verifiable Environments Agent World Model Team (2026)
• Constitutional AI: Harmlessness from AI Feedback Yuntao Bai et al. (2022)
• Deep Reinforcement Learning from Human Preferences Paul Christiano et al. (2017)
• The Evaluation Differential: How Evaluation Awareness Distorts LLM Benchmarks The Evaluation Differential Team (2026)

相关论文

同一领域的其他论文

• Agent World Model: Scaling Agentic RL with Synthetic Verifiable Environments Agent World Model Team (2026)
  应用推理能力
• Adaptive Parallel Reasoning (UC Berkeley Technical Report EECS-2026-213) UC Berkeley (2026)
  推理能力推理
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning DeepSeek-AI (2025)
  推理能力对齐
• KTO: Model Alignment as Prospect Theoretic Optimization Kawin Ethayarajh et al. (2024)
  对齐
• OpenAI o1 System Card OpenAI (2024)
  推理能力安全
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters Charlie Snell et al. (2024)
  推理能力推理