WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systems
Juan M. Huerta Zinnia Tech Solutions 600 Steamboat Road Greenwich, CT 06830, USA
Abstract
The LLM Wiki pattern—to compile and provide domain knowledge into a persistent artifact and serve it to LLMs via KV cache inference—promises context access at sub-second latency with zero retrieval failure. Realizing this requires solving the compilation gap: LLM-compilation distilling raw documents into a wiki without catastrophically discarding critical facts. We characterize this gap across 17 RepLiQA domains (6,800 questions): we observe that full-context KV cache inference outperforms RAG on curated knowledge (4.38 vs. 4.08/5, $7.3\times$ faster TTFT) but degrades below RAG at scale due to attention dilution, and blind compilation fails entirely (2.14–2.32 vs. 3.46, 53–60% catastrophic failure rate). To address the compilation gap, we propose WiCER (Wiki-memory Compile, Evaluate, Refine), an iterative algorithm inspired by counterexample-guided abstraction refinement (CEGAR) that closes this gap. WiCER evaluates compiled wikis against diagnostic probes, identifies dropped facts, and forces their preservation in subsequent compilations. One to two iterations recover 80% of lost quality (mean 3.24 vs. 3.47 for raw full-context across the 15 topics with baselines), reducing catas
中文速览
把大量领域文档"编译"成一份精简的知识库（wiki），再通过KV缓存让大模型一次性加载并反复复用，从而实现毫秒级响应、零检索失败——这就是"LLM Wiki"方案的核心承诺，但实践中存在一个致命的编译缺口：让大模型自动压缩原始文档时，它会过度删减、丢失关键事实，导致回答质量断崖式下跌（得分从3.47跌至2.14–2.32，超过半数问题彻底答错）。为此，作者提出WiCER算法——借鉴软件验证领域的反例引导抽象精化（CEGAR）思路，在每轮编译后用诊断探针测试wiki、定位被丢弃的事实，再强制下一轮编译保留这些事实，如此迭代一到两次即可找回约80%的丢失质量，灾难性失败率相对降低55%。消融实验进一步证明，真正起作用的是"精准定位缺失事实"（+0.95分），而非笼统地"钉住"所有内容（+0.16分）。这项工作为低延迟、高可靠的领域知识问答系统提供了一套可落地的编译-评估-精化完整方案，并在17个领域、6800道题上开源了全部代码与基准测试。
原文 arXiv:2605.07068;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2605.07068v1