The authors present GRIEF, a greybox fuzzer for LLM inference engines (vLLM, SGLang) that treats timed multi-request traces as first-class inputs. GRIEF targets serving-layer failures (KV-cache isolation, cross-request interference, scheduling bugs) that emerge only under realistic concurrent workloads and are missed by model or API tests. Using lightweight oracles and controlled replay with log-probability checks, GRIEF discovers 15 vulnerabilities across vLLM and SGLang, including 2 CVEs, spanning KV-cache isolation failures, cross-request performance interference, and crash/liveness bugs.
Main takeaways:
- Modern LLM serving systems (vLLM, SGLang) use shared state (KV cache, prefix sharing, batching, multi-tenant scheduling) that creates concurrency bugs missed by standard model or API tests.
- GRIEF treats timed multi-request traces as inputs, fuzzing the serving layer rather than individual prompts or model behavior.
- Lightweight oracles detect crashes, hangs, performance pathologies (e.g., noisy-neighbor denial of service), and silent output corruption.
- Controlled replay with log-probability checks confirms reproducible serving-layer failures (e.g., cross-request contamination where one request's KV cache leaks into another's output).
- Early campaigns on vLLM and SGLang discovered 15 vulnerabilities (10 confirmed by developers, 2 CVEs), including KV-cache isolation failures, cross-request performance interference, and delayed crashes.