Skip to content
Sagan

Paper

WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems

Unreadunread

AI summary

The authors tackle the "LLM Wiki" pattern—compiling domain knowledge into a persistent artifact served via KV cache for sub-second, zero-retrieval-failure access. The core problem is the "compilation gap": blindly summarizing documents into a wiki catastrophically drops facts (53–60% failure rate), while serving the full raw context works well but doesn't scale due to attention dilution. They propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided refinement that evaluates compiled wikis against diagnostic questions, identifies dropped facts, and forces their preservation in the next compilation round. One or two iterations recover 80% of lost quality and reduce catastrophic failures by 55%.

Main takeaways:

  • Full-context KV cache inference beats RAG on curated knowledge (4.38 vs. 4.08 out of 5, 7× faster) but degrades at scale due to attention dilution.
  • Blind compilation into a wiki fails catastrophically: drops quality from ~3.5 to ~2.2 and causes 53–60% failure rates.
  • WiCER iteratively diagnoses dropped facts and forces the next compilation to preserve them, recovering 80% of lost quality in 1–2 rounds.
  • Targeted diagnosis (identifying specific missing facts) drives the gains (+0.95); generic "pinning" strategies only help marginally (+0.16).
  • All code and benchmarks are released across 17 RepLiQA domains (6,800 questions).