Kimi K2: Open Agentic Intelligence
arXiv: 2507.20534
Domains
TLDR (English)
The Kimi K2 technical report: a 1T-total / 32B-active MoE pretrained on 15.5T tokens. On the optimization side it pushes Muon to trillion-parameter scale with MuonClip (QK-clip: anticipatorily clip attention logits once they exceed a threshold of 100) together with a WSD learning-rate schedule, achieving zero loss spikes and zero training rollbacks; weights and detailed infra notes are open-sourced.
TLDR（中文）
Kimi K2 技术报告：1T 总参 / 32B 激活的 MoE，15.5T tokens 预训练。优化系统上把 Muon 推到万亿规模并给出 MuonClip（QK-clip：当注意力 logits 前瞻性地超过阈值 100 时提前裁剪），配合 WSD 学习率调度，全程零 loss spike、零训练回滚；报告同时开源权重与详细 infra 细节。
Appears in These Articles
Co-cited Papers
These papers appear in the same articles as this one
Related Papers
Other papers in the same domain