Distributed Training · 2025
Muon is Scalable for LLM Training
Shows the matrix-orthogonalization Muon optimizer scales to large MoE training with weight decay and update-scale fixes, reaching roughly 2x the compute efficiency of AdamW.
Editorial record
Plain-language summary
Shows the matrix-orthogonalization Muon optimizer scales to large MoE training with weight decay and update-scale fixes, reaching roughly 2x the compute efficiency of AdamW.
Knowledge graph
Relationships
Descendants
Conceptual ancestorEvidence: Strongly supported
Megatron-LM: Training Multi-Billion Parameter LMs Using Model Parallelism
Builds on the distributed training lineage in the archive
freshness sweep 2026
Source record
Provenance
- Record ID
- P-615
- Record created
- 2026-07-13
- Last reviewed
- 2026-07-14
- Record version
- 2
- https://arxiv.org/abs/2502.16982
- arXiv:2502.16982
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.