Skip to content
Sagan

Paper

Gefen: Optimized Stochastic Optimizer

Unreadunread

AI summary

arXiv:2606. 13894v1 Announce Type: new Abstract: AdamW is a default optimizer for modern deep learning, but its first and second moment states add roughly two parameter-sized buffers to training memory.