Skip to content
Sagan

Paper

Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success

Unreadunread

AI summary

arXiv:2606. 14560v1 Announce Type: cross Abstract: Non-Euclidean optimisation methods with matrix-valued updates, such as Muon and Scion, have recently shown strong empirical performance for training Transformer models, yet their theoretical advantages over Euclidean methods remain poorly understood.