Skip to content
Sagan

Paper

Attention is Just Another Name for Coupling?: A Fast-Slow ODE Perspective on Hierarchical Pretraining

Unreadunread

AI summary

arXiv:2606. 16730v1 Announce Type: new Abstract: Causal self-attention is a coupling mechanism: each token's hidden state is updated by a learned mixture of preceding tokens at the same timescale.