Skip to content
Sagan

Paper

The Hidden Power of Scaling Factor in LoRA Optimization

Unreadunread

AI summary

arXiv:2606. 12883v1 Announce Type: new Abstract: In Low-Rank Adaptation (LoRA), the scaling factor $\alpha$ is often treated as a mere complement to the learning rate, yet its role in optimization remains poorly understood.