Skip to content
Sagan

Paper

Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps

Unreadunread

AI summary

arXiv:2605. 16928v1 Announce Type: new Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attention.