Skip to content
Sagan

Paper

How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization

Unreadunread

AI summary

arXiv:2605. 14200v1 Announce Type: cross Abstract: Recent frontier large language models predominantly rely on Mixture-of-Experts (MoE) architectures.