Is ChatGPT a Good Causal Reasoner? A Comprehensive Evaluation.
Jinglong Gao Xiao Ding Bing Qin Ting Liu Research Center for Social Computing and Information Retrieval Harbin Institute of Technology, China Corresponding Author
Abstract
Causal reasoning ability is crucial for numerous NLP applications. Despite the impressive emerging ability of ChatGPT in various NLP tasks, it is unclear how well ChatGPT performs in causal reasoning. In this paper, we conduct the first comprehensive evaluation of the ChatGPT’s causal reasoning capabilities. Experiments show that ChatGPT is not a good causal reasoner, but a good causal explainer. Besides, ChatGPT has a serious hallucination on causal reasoning, possibly due to the reporting biases between causal and non-causal relationships in natural language, as well as ChatGPT’s upgrading processes, such as RLHF. The In-Context Learning (ICL) and Chain-of-Thought (CoT) techniques can further exacerbate such causal hallucination. Additionally, the causal reasoning ability of ChatGPT is sensitive to the words used to express the causal concept in prompts, and close-ended prompts perform better than open-ended prompts. For events in sentences, ChatGPT excels at capturing explicit causality rather than implicit causality, and performs better in sentences with lower event density and smaller lexical distance between events. The code is available on https://github.com/ArrogantL/ChatGP
中文速览
ChatGPT在各类NLP任务上表现亮眼，但它的因果推理（causal reasoning）能力究竟几何，此前从未有人系统评测过。研究者通过四类任务——句中事件因果识别、因果发现（多选与二分类两种格式）、因果解释生成——在五个主流数据集上对四个版本的ChatGPT进行了全面测试，发现ChatGPT并不是一个合格的因果推理器，却是一个出色的因果解释生成器：它在判断因果关系是否存在时表现糟糕，精确率极低，严重倾向于把任何事件对都标记为"有因果关系"，呈现出严重的因果幻觉（causal hallucination）。这一问题的根源在于自然语言语料中因果关系表述远多于非因果关系表述的报告偏差，以及RLHF等对齐训练过程的副作用，而In-Context Learning和Chain-of-Thought提示策略非但没能改善，反而进一步加剧了这种幻觉。这项研究警示我们：不能因为ChatGPT在多选题上成绩不错就误判其具备真正的因果推理能力，对大语言模型的评测必须设计更严格、更全面的任务形式。
原文 arXiv:2305.07375;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.07375v4