Maven-Ere: A Unified Large-scale Dataset for Event Coreference, Temporal, Causal, and Subevent Relation Extraction
Xiaozhi Wang1 , Yulin Chen2∗, Ning Ding1, Hao Peng1, Zimu Wang5, Yankai Lin6,7 , Xu Han1, Lei Hou1, Juanzi Li1,3 , Zhiyuan Liu1, Peng Li4†, Jie Zhou8 1Department of Computer Science and Technology, BNRist; 2Shenzhen International Graduate School; 3THU-Siemens Ltd., China Joint Research Center for Industrial Intelligence and IoT; 4Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China 5Xi’an Jiaotong-Liverpool University, Suzhou, China 6Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China 7Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China 8Pattern Recognition Center, WeChat AI, Tencent Inc, China indicates equal contribution. Partly done while Y.Lin and P.Li were at Tencent. Corresponding author: J.Li.
Abstract
The diverse relationships among real-world events, including coreference, temporal, causal, and subevent relations, are fundamental to understanding natural languages. However, two drawbacks of existing datasets limit event relation extraction (ERE) tasks: (1) Small scale. Due to the annotation complexity, the data scale of existing datasets is limited, which cannot well train and evaluate data-hungry models. (2) Absence of unified annotation. Different types of event relations naturally interact with each other, but existing datasets only cover limited relation types at once, which prevents models from taking full advantage of relation interactions. To address these issues, we construct a unified large-scale human-annotated ERE dataset Maven-Ere with improved annotation schemes. It contains $103,193$ event coreference chains, $1,216,217$ temporal relations, $57,992$ causal relations, and $15,841$ subevent relations, which is larger than existing datasets of all the ERE tasks by at least an order of magnitude. Experiments show that ERE on Maven-Ere is quite challenging, and considering relation interactions with joint learning can improve performances. The dataset and source codes
中文速览
真实世界中的事件之间存在共指、时序、因果、子事件等多种关系，准确抽取这些关系（事件关系抽取，ERE）是自然语言理解的核心难题，但现有数据集普遍存在规模太小、且各类关系分散在不同数据集中无法统一利用的缺陷。为此，研究者基于大规模事件检测数据集MAVEN，通过精心设计的众包标注流程，构建了首个统一、大规模的人工标注事件关系数据集MAVEN-ERE，涵盖逾10万条共指链、121万条时序关系、近6万条因果关系和约1.6万条子事件关系，比现有同类数据集至少大一个数量级。实验表明，该数据集上的ERE任务颇具挑战性，现有模型性能仍有较大提升空间，而将多种关系联合训练可以利用关系间的相互约束来提升各子任务的表现。这一数据集的发布有望打破数据瓶颈，推动事件关系抽取及时序推理、因果推理等下游应用的系统性进步。
原文 arXiv:2211.07342;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2211.07342v1