Skip to content
Sagan

Paper

Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers

Unreadunread

AI summary

arXiv:2605. 20756v1 Announce Type: cross Abstract: Preconditioned optimizers are central to language model training, but their stochastic update rules are usually treated as direct approximations to population preconditioned descent.