The authors challenge the geometric narrative behind the Muon optimizer by showing that precise geometric structure isn't what drives performance. They introduce Freon (based on Schatten quasi-norms) and Kaon (which replaces singular values with random noise), both matching Muon's performance despite lacking coherent geometry. Their analysis reveals that optimizer performance is controlled by two local quantities — alignment and descent potential — rather than global geometric structure, suggesting Muon succeeds by guaranteeing step-size optimality, not by tracking ideal geometry.
Main takeaways:
- Freon interpolates between SGD and Muon using Schatten quasi-norms; best GPT-2 parameters lie in the quasi-norm regime, which can't be represented by any unitarily invariant LMO.
- Kaon replaces singular values with random noise yet matches Muon's performance, proving precise geometry isn't necessary.
- Optimizer performance is controlled by two local quantities: alignment and descent potential, not global geometric structure.
- Muon succeeds by guaranteeing step-size optimality around these local quantities, not by tracking ideal global geometry.
- Both Freon and Kaon retain classical convergence guarantees despite their unusual constructions.