The authors derive a closed-form upper bound for how large a learning-rate step can be while still guaranteeing that an update "contracts" the model's beliefs in KL-divergence terms. Instead of treating learning rate as a hyperparameter you tune empirically, they model updates as projected steps on the probability simplex and compute the maximum step size that keeps the update well-behaved in the natural information geometry.
Main takeaways:
- Provides a formula (not a tuning heuristic) for the largest safe learning-rate step
- Models updates as projected forward steps on the probability simplex
- Step is "admissible" if it's contractive in KL/Bregman geometry
- Positions learning-rate choice as a geometric calculation rather than pure trial-and-error