ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models
Haoran Luo1, Haihong E1 , Zichen Tang1, Shiyao Peng1, Yikai Guo3, Wentai Zhang1, Chenghao Ma1, Guanting Dong2, Meina Song1, Wei Lin4, Yifan Zhu1, Luu Anh Tuan5 1School of Computer Science, Beijing University of Posts and Telecommunications, China 2School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China 3Beijing Institute of Computer Technology and Application 4Inspur Group Co., Ltd., China 5College of Computing and Data Science, Nanyang Technological University, Singapore {luohaoran, ehaihong, Corresponding author.
Abstract
Knowledge Base Question Answering (KBQA) aims to answer natural language questions over large-scale knowledge bases (KBs), which can be summarized into two crucial steps: knowledge retrieval and semantic parsing. However, three core challenges remain: inefficient knowledge retrieval, mistakes of retrieval adversely impacting semantic parsing, and the complexity of previous KBQA methods. To tackle these challenges, we introduce ChatKBQA, a novel and simple generate-then-retrieve KBQA framework, which proposes first generating the logical form with fine-tuned LLMs, then retrieving and replacing entities and relations with an unsupervised retrieval method, to improve both generation and retrieval more directly. Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and CWQ. This work can also be regarded as a new paradigm for combining LLMs with knowledge graphs (KGs) for interpretable and knowledge-required question answering. Our code is publicly available111https://github.com/LHRLAB/ChatKBQA.
中文速览
知识库问答（KBQA）的难点在于：传统方法先检索再解析，检索结果一旦出错就会拖累后续的语义理解，整个流程还异常繁琐。ChatKBQA 反其道而行之，提出"先生成、再检索"的新范式——先用经过指令微调的开源大语言模型（LLM）直接把自然语言问题转换成逻辑表达式，再用无监督的短语级语义检索去知识库里找到并替换其中的实体和关系，最终生成可执行的 SPARQL 查询来得出答案。在 WebQSP 和 CWQ 两个标准数据集上，ChatKBQA 均刷新了最优性能，同时大幅简化了以往多步骤的复杂流程。这项工作不仅为 KBQA 提供了一条更简洁、可复现的技术路径，也为大语言模型与知识图谱的深度结合提供了一种可解释的新思路。
原文 arXiv:2310.08975;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2310.08975v3