Evaluation and Analysis of Hallucination in Large Vision-Language Models
Junyang Wang♣, , Yiyang Zhou♠,∗, Guohai Xu△, Pengcheng Shi♠, Chenlin Zhao◇, Haiyang Xu△, Qinghao Ye△, Ming Yan△, Ji Zhang△, Jihua Zhu♠, Jitao Sang♣,, Haoyu Tang♡,† ♣ School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China ♠ School of Software Engineering, Xi’an Jiaotong University, Xi’an, China ♡ School of Software, Shandong University, Jinan, China ◇ MAIS, Institute of Automation, Chinese Academy of Sciences(CASIA), Beijing, China △ DAMO Academy, Alibaba Group {guohai.xgh, Equal contribution Corresponding author Work done during internship at DAMO Academy, Alibaba Group.
Abstract
Large Vision-Language Models (LVLMs) have recently achieved remarkable success. However, LVLMs are still plagued by the hallucination problem, which limits the practicality in many scenarios. Hallucination refers to the information of LVLMs’ responses that does not exist in the visual input, which poses potential risks of substantial consequences. There has been limited work studying hallucination evaluation in LVLMs. In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework. HaELM achieves an approximate 95% performance comparable to ChatGPT and has additional advantages including low cost, reproducibility, privacy preservation and local deployment. Leveraging the HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we analyze the factors contributing to hallucination in LVLMs and offer helpful suggestions to mitigate the hallucination problem. Our data and code are available at https://github.com/junyangwang0410/HaELM.
中文速览
大型视觉语言模型（Large Vision-Language Models, LVLMs）在回答问题时常常会"幻觉"——也就是在回复里编造出图片中根本不存在的内容，而现有的评测手段（如基于目标检测的POPE）其实只是在利用模型对"是/否"问题天然倾向于回答"是"的偏差，并不能真实反映模型在日常使用场景下的幻觉程度。为此，研究者提出了HaELM——一个基于大语言模型（LLM）的幻觉评测框架：先分析LVLMs真实的幻觉模式，再用ChatGPT批量生成与之匹配的幻觉/非幻觉训练样本，最后用LoRA方法在LLaMA上微调出一个专用评测模型。实验表明，HaELM的评测准确率可达ChatGPT的约95%，同时具备成本低、可复现、可本地部署、保护数据隐私等优势。这项工作不仅填补了LVLMs幻觉评测的空白，还通过系统分析揭示了幻觉的成因并给出了缓解建议，为后续改进多模态模型的可靠性提供了实用工具和参考依据。
原文 arXiv:2308.15126;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2308.15126v3