Skip to content
Sagan

Paper

CachePrune: Privacy-Aware and Fine-Grained KV Cache Sharing for Efficient LLM Inference

Unreadunread

AI summary

arXiv:2605. 23640v1 Announce Type: new Abstract: Large Language Models (LLMs) rely on Key-Value (KV) caching to accelerate inference, and many serving systems further share the KV cache across users' requests to reduce redundant computation.