WebCPM: Interactive Web Search for Chinese Long-form Question Answering
Yujia Qin1, Zihan Cai1, Dian Jin1, Lan Yan1, Shihao Liang3, Kunlun Zhu3, Yankai Lin2∗, Xu Han1, Ning Ding1, Huadong Wang1, Ruobing Xie4, Fanchao Qi1, Zhiyuan Liu1 , Maosong Sun1∗, Jie Zhou4 1NLP Group, DCST, IAI, BNRIST, Tsinghua University, Beijing 2Gaoling School of Artificial Intelligence, Renmin University of China, Beijing 3ModelBest Inc. 4Pattern Recognition Center, WeChat AI, Tencent Inc. Corresponding author.
Abstract
Long-form question answering (LFQA) aims at answering complex, open-ended questions with detailed, paragraph-length responses. The de facto paradigm of LFQA necessitates two procedures: information retrieval, which searches for relevant supporting facts, and information synthesis, which integrates these facts into a coherent answer. In this paper, we introduce WebCPM, the first Chinese LFQA dataset. One unique feature of WebCPM is that its information retrieval is based on interactive web search, which engages with a search engine in real time. Following WebGPT (Nakano et al., 2021), we develop a web search interface. We recruit annotators to search for relevant information using our interface and then answer questions. Meanwhile, the web search behaviors of our annotators would be recorded. In total, we collect $5,500$ high-quality question-answer pairs, together with $15,372$ supporting facts and $125,954$ web search actions. We fine-tune pre-trained language models to imitate human behaviors for web search and to generate answers based on the collected facts. Our LFQA pipeline, built on these fine-tuned models, generates answers that are no worse than human-written ones in $32.5
中文速览
针对"如何让AI系统像人一样通过搜索引擎逐步收集信息来回答复杂开放性问题"这一挑战，研究团队构建了首个面向中文的交互式网络搜索长文本问答（Long-form QA）数据集WebCPM，包含5500对高质量问答、15万余条网页搜索行为记录，并配套开放了完整的搜索界面与代码。他们训练预训练语言模型分别模仿人类的搜索行为（动作预测、查询生成、事实抽取）和答案综合能力，形成一套完整的问答流水线。实验表明，该流水线生成的答案在自有测试集上有32.5%的情况不逊于人工撰写，在DuReader数据集上更有47.5%的情况优于人工标注答案，且交互式搜索明显优于传统的一次性检索方式。这项工作不仅填补了中文开放域长文本问答的数据空白，还为行为克隆、工具学习等研究方向提供了可复现的公开基准平台。
原文 arXiv:2305.06849;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.06849v2