Skip to content
Sagan

Paper

A Mean-Field Analysis of Multi-Head Self-Attention under Cross-Entropy Training

Unreadunread

AI summary

arXiv:2606. 10469v1 Announce Type: cross Abstract: This paper develops a mean-field theory for a simplified single-layer causal multi-head self-attention model trained by cross-entropy minimization.