Large Language Models and Causal Inference in Collaboration: A Survey
Xiaoyu Liu1,*, Paiheng Xu1,*, Junda Wu2, Jiaxin Yuan1, Yifan Yang1, Yuhang Zhou1, Fuxiao Liu1, Tianrui Guan 1, Haoliang Wang3, Tong Yu3, Julian McAuley2, Wei Ai1, Furong Huang1 1University of Maryland, College Park, 2University of California San Diego, 3Adobe Research Correspondence: * denotes equal contribution
Abstract
Causal inference has demonstrated significant potential to enhance Natural Language Processing (NLP) models in areas such as predictive accuracy, fairness, robustness, and explainability by capturing causal relationships among variables. The rise of generative Large Language Models (LLMs) has greatly impacted various language processing tasks. This survey focuses on research that evaluates or improves LLMs from a causal view in the following areas: reasoning capacity, fairness and safety issues, explainability, and handling multimodality. Meanwhile, LLMs can assist in causal inference tasks, such as causal relationship discovery and causal effect estimation, by leveraging their generation ability and knowledge learned during pre-training. This review explores the interplay between causal inference frameworks and LLMs from both perspectives, emphasizing their collective potential to further the development of more advanced and robust artificial intelligence systems.
中文速览
大型语言模型（LLMs）在推理、公平性等方面仍存在明显短板，而因果推断（causal inference）长期面临专家知识依赖和反事实数据稀缺等瓶颈，这篇综述正是要系统梳理二者如何相互赋能。一方面，因果推断的框架和方法被用来诊断并改善LLMs的因果推理能力、减少幻觉、提升公平性与安全性、增强可解释性，以及解决多模态场景下的对齐难题；另一方面，LLMs凭借预训练积累的海量知识和强大的文本生成能力，反过来辅助因果图发现（causal discovery）和处理效应估计（treatment effect estimation），为传统因果方法提供先验知识或高质量反事实样本。综述覆盖了推理评估基准、去偏方法、可解释性框架、多模态因果建模，以及LLMs辅助因果发现与反事实生成等多个研究方向，汇总了该领域的最新进展与开放挑战。这项工作的价值在于：它首次将"用因果推断改进LLMs"和"用LLMs推进因果推断"这两条平行线索整合进同一框架，为构建更可靠、更可解释的人工智能系统指明了协同发展的路径。
原文 arXiv:2403.09606;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2403.09606v3