Skip to content
Sagan

Paper

A Polyak-Ruppert Central Limit Theorem for SA-Adam with Momentum and Non-Convergent Adaptive Preconditioning

Unreadunread

AI summary

arXiv:2606. 17364v1 Announce Type: cross Abstract: Adaptive optimizers combining preconditioning, momentum, and weight decay (Adam and AdamW) are, under Polyak-Ruppert averaging, candidate engines for one-pass inference.