Skip to content
Sagan

Paper

Negligible in Size, Significant in Effect: On Scale Vectors in Large Language Models

Unreadunread

AI summary

arXiv:2605. 26895v1 Announce Type: cross Abstract: Normalization layers in modern large language models (LLMs) consist of a deterministic normalization operation and a learnable scale vector.