Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources
Xingxuan Li1,2 , Ruochen Zhao , Yew Ken Chia1, , Bosheng Ding1, , Shafiq Joty2,4 Soujanya Poria3, Lidong Bing1,5 1DAMO Academy, Alibaba Group, Singapore, 2Nanyang Technological University, 3Singapore University of Technology and Design, 4Salesforce Research, 5Hupan Lab, 310023, Hangzhou, China {xingxuan.li, yewken.chia, bosheng.ding, {ruochen002, Equal contribution. Xingxuan Li, Yew Ken Chia, and Bosheng Ding are under the Joint Ph.D. Program between DAMO Academy and their corresponding universities. Ruochen Zhao is under the AISG Ph.D. Fellowship Programme.
Abstract
We present chain-of-knowledge (CoK) , a novel framework that augments large language models (LLMs) by dynamically incorporating grounding information from heterogeneous sources. It results in more factual rationales and reduced hallucination in generation. Specifically, CoK consists of three stages: reasoning preparation, dynamic knowledge adapting, and answer consolidation. Given a knowledge-intensive question, CoK first prepares several preliminary rationales and answers while identifying the relevant knowledge domains. If there is no majority consensus among the answers from samples, CoK corrects the rationales step by step by adapting knowledge from the identified domains. These corrected rationales can plausibly serve as a better foundation for the final answer consolidation. Unlike prior studies that primarily use unstructured data, CoK also leverages structured knowledge sources such as Wikidata and tables that provide more reliable factual information. To access both unstructured and structured knowledge sources in the dynamic knowledge adapting stage, we propose an adaptive query generator that allows the generation of queries for various types of query languages, includin
中文速览
大型语言模型（LLMs）容易"一本正经地说错话"（幻觉问题），尤其在回答需要真实知识的复杂问题时表现明显。研究者提出了一个叫做"知识链"（Chain-of-Knowledge, CoK）的新框架，让模型在推理过程中动态地从多种异构知识来源——既包括Wikipedia这类非结构化文本，也包括Wikidata知识图谱、数据库表格等结构化数据——中按需检索事实，并设计了一个"自适应查询生成器"（AQG）来自动生成SPARQL、SQL或自然语言等不同格式的查询语句。最关键的是，CoK采用逐步纠错的方式：先修正第一条推理句，再用修正后的结果去生成并纠正下一条，从而避免错误像滚雪球一样层层传播。在涵盖事实、医学、物理、生物等多个领域的知识密集型任务上，CoK平均比标准的思维链（CoT）基线提升了4.3%，为减少LLMs幻觉、提升回答可靠性提供了一条实用可行的路径。
原文 arXiv:2305.13269;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.13269v4