A Discrete Hard EM Approach for Weakly Supervised Question Answering
Sewon Min1, Danqi Chen2,3, Hannaneh Hajishirzi1,4, Luke Zettlemoyer1,3 1University of Washington, Seattle, WA 2Princeton University, Princeton, NJ 3Facebook AI Research, Seattle, WA 4Allen Institute for Artificial Intelligence, Seattle, WA
Abstract
Many question answering (QA) tasks only provide weak supervision for how the answer should be computed. For example, TriviaQA answers are entities that can be mentioned multiple times in supporting documents, while DROP answers can be computed by deriving many different equations from numbers in the reference text. In this paper, we show it is possible to convert such tasks into discrete latent variable learning problems with a precomputed, task-specific set of possible solutions (e.g. different mentions or equations) that contains one correct option. We then develop a hard EM learning scheme that computes gradients relative to the most likely solution at each update. Despite its simplicity, we show that this approach significantly outperforms previous methods on six QA tasks, including absolute gains of 2–10%, and achieves the state-of-the-art on five of them. Using hard updates instead of maximizing marginal likelihood is key to these results as it encourages the model to find the one correct answer, which we show through detailed qualitative analysis.111Our code is publicly available at https://github.com/shmsw25/qa-hard-em.
中文速览
弱监督问答任务中，模型往往只知道最终答案是什么，却不知道该从文章里选哪个片段、用哪个方程式或哪条SQL语句来推导出它，这种"一题多解"的歧义让模型训练很棘手。研究者将这类任务统一建模为离散隐变量学习问题：先为每道题预先枚举出一小批可能的推导方案（候选解集合），再用"硬EM（hard EM）"策略训练——每次只选当前模型认为最可能的那个解来更新参数，而不是像最大边际似然（MML）那样对所有候选解求和优化。在TriviaQA、DROP、WikiSQL等六个问答数据集上，这一方法相比此前用启发式规则或MML训练的方法取得了2–10个百分点的绝对提升，并在五个数据集上刷新了当时的最优成绩。这项工作的意义在于，它提供了一个简单通用的训练框架，无需修改模型结构，仅凭更好的学习目标就能大幅提升弱监督问答的性能。
原文 arXiv:1909.04849;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1909.04849v1