Skip to content
Sagan

Paper

LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

Unreadunread

AI summary

arXiv:2606. 04302v1 Announce Type: new Abstract: Key-value (KV) caching accelerates inference of large language models (LLMs) by reusing past computations for generated tokens.