HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Junyi Li1,3,4, Xiaoxue Cheng, Wayne Xin Zhao1,4 , Jian-Yun Nie3 and Ji-Rong Wen1,2,4 1Gaoling School of Artificial Intelligence, Renmin University of China 2School of Information, Renmin University of China 3DIRO, Université de Montréal 4Beijing Key Laboratory of Big Data Management and Analysis Methods Equal contributions Corresponding author
Abstract
Large language models (LLMs), such as ChatGPT, are prone to generate hallucinations, i.e., content that conflicts with the source or cannot be verified by the factual knowledge. To understand what types of content and to which extent LLMs are apt to hallucinate, we introduce the Hallucination Evaluation benchmark for Large Language Models (HaluEval), a large collection of generated and human-annotated hallucinated samples for evaluating the performance of LLMs in recognizing hallucination. To generate these samples automatically, we propose a two-stage framework, i.e., sampling-then-filtering. Besides, we hire some human labelers to annotate the hallucinations in ChatGPT responses. The empirical results suggest that ChatGPT is likely to generate hallucinated content related to specific topics by fabricating unverifiable information (i.e., about $19.5\%$ responses). Moreover, existing LLMs face great challenges in recognizing the hallucinations in texts. However, our experiments also prove that providing external knowledge or adding reasoning steps can help LLMs recognize hallucinations. Our benchmark can be accessed at https://github.com/RUCAIBox/HaluEval.
中文速览
大型语言模型（LLM）如 ChatGPT 会生成"幻觉"（hallucination）——即编造无法被事实核实或与来源相矛盾的内容，但学界对这一问题的范围和规律缺乏系统认识。为此，研究者构建了 HaluEval 基准数据集，包含 3 万条自动生成的任务专项幻觉样本（覆盖问答、对话、摘要三类任务）和 5000 条由人工标注的 ChatGPT 真实回复，自动生成部分采用"先采样、后过滤"的两阶段流程来保证样本的多样性和难度。实验发现，ChatGPT 约有 19.5% 的回复含有幻觉，且现有 LLM 普遍难以识别文本中的幻觉——即便是生成这些幻觉样本的 ChatGPT 自身，在问答任务上的识别准确率也仅有约 62.6%；不过，向模型提供外部知识或引导其进行链式推理可以显著改善识别能力。这项工作填补了 LLM 幻觉评测的空白，为后续研究提供了标准化基准，有助于推动更可靠、更安全的 LLM 落地应用。
原文 arXiv:2305.11747;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.11747v3