iMuon extends norm-constrained optimizers to Riemannian matrix manifolds
The paper's authors present intrinsic Muon, a framework giving closed-form Muon-style updates on fixed-rank, SPD, Stiefel and Grassmann manifolds, with convergence rates and experiments on LoRA fine-tuning.
In plain words
Muon is an optimizer used for training large models, but it was designed for ordinary parameter matrices and does not transfer directly to parameters that must live on curved geometric spaces, such as low-rank or orthogonal matrices. The authors propose iMuon, which adapts Muon's update rule to those spaces while preserving their symmetry. They report closed-form updates and convergence guarantees, plus experiments on LoRA fine-tuning of large language models.
What was announced
The authors note that Muon and related norm-constrained optimizers compute updates through a linear maximization oracle over a matrix-norm ball, and that restricting this oracle to a manifold's tangent space breaks quotient symmetries and couples the tangent constraint with an ambient norm bound, blocking closed-form solutions. Their fix relies on the observation that every Riemannian metric canonically lifts a unitarily invariant Euclidean norm to an intrinsic norm on each tangent space, yielding an intrinsic oracle that preserves symmetry and, under a structural condition, reduces to a standard Euclidean one.
Building on this, the paper introduces iMuon with closed-form updates on the fixed-rank, SPD, Stiefel and Grassmann manifolds for unitarily invariant norms including spectral, Frobenius and nuclear norms. The authors establish deterministic and stochastic convergence rates whose constants depend only on manifold dimension; on the fixed-rank manifold the constant depends only on rank, which they say removes factor-conditioning dependence and the runtime rescaling needed by prior work. Experiments cover LoRA fine-tuning of LLMs, image classification and subspace learning.
What it does not show
- The evidence is the paper's abstract only; experimental results, baselines and comparisons are not shown in the supplied passages, and no independent evaluation is available.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.