Zero-shot Causal Graph Extrapolation from Text via LLMs
Alessandro Antonucci, Gregorio Piqué, and Marco Zaffalon Corresponding author:
Abstract
We evaluate the ability of large language models (LLMs) to infer causal relations from natural language. Compared to traditional natural language processing and deep learning techniques, LLMs show competitive performance in a benchmark of pairwise relations without needing (explicit) training samples. This motivates us to extend our approach to extrapolating causal graphs through iterated pairwise queries. We perform a preliminary analysis on a benchmark of biomedical abstracts with ground-truth causal graphs validated by experts. The results are promising and support the adoption of LLMs for such a crucial step in causal inference, especially in medical domains, where the amount of scientific text to analyse might be huge, and the causal statements are often implicit.
中文速览
大语言模型（LLM）在医学文献等科学文本中自动提取因果关系图（causal graph）的能力尚未得到充分验证，而人工标注或随机实验的成本又极高。研究者以GPT-4 Turbo为核心工具，通过精心设计的提示词（prompt engineering），让模型在无需标注训练数据的情况下，对文本中实体间的因果方向进行逐对判断，再迭代扩展为完整的因果关系图。在SemEval因果方向识别基准上，GPT的F1分数约为99%，超过使用8000条训练样本的竞赛冠军方法；在20篇生物医学摘要的因果图提取实验中，召回率高达97%，但精确率约74%，主要误差来源于模型难以区分直接与间接因果关系。这项研究表明，大语言模型在从海量科学文献中自动构建因果知识图谱方面具有切实可行的潜力，对医学等需要大规模因果推断的领域尤具价值，但如何减少误报仍是下一步需要解决的关键问题。
原文 arXiv:2312.14670;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2312.14670v1