Skip to content
Sagan

Paper

Weibull Weight-Scale Parameter Evolution under AdamW Training Dynamics

Unreadunread

AI summary

arXiv:2606. 19367v1 Announce Type: cross Abstract: Building on a two-parameter Weibull framework for diagnosing transformer weight distributions, we study why the Weibull weight-scale parameter $\lambda$ grows, overshoots, and then relaxes during AdamW training.