The authors tackle the "LLM Wiki" pattern—compiling domain knowledge into a persistent artifact served via KV cache for sub-second, zero-retrieval-failure access. The core problem is the "compilation gap": blindly summarizing documents into a wiki catastrophically drops facts (53–60% failure rate), while serving the full raw context works well but doesn't scale due to attention dilution. They propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided refinement that evaluates compiled wikis against diagnostic questions, identifies dropped facts, and forces their preservation in the next compilation round. One or two iterations recover 80% of lost quality and reduce catastrophic failures by 55%.
Main takeaways:
- Full-context KV cache inference beats RAG on curated knowledge (4.38 vs. 4.08 out of 5, 7× faster) but degrades at scale due to attention dilution.
- Blind compilation into a wiki fails catastrophically: drops quality from ~3.5 to ~2.2 and causes 53–60% failure rates.
- WiCER iteratively diagnoses dropped facts and forces the next compilation to preserve them, recovering 80% of lost quality in 1–2 rounds.
- Targeted diagnosis (identifying specific missing facts) drives the gains (+0.95); generic "pinning" strategies only help marginally (+0.16).
- All code and benchmarks are released across 17 RepLiQA domains (6,800 questions).