Unsupervised Question Answering by Cloze Translation
Patrick Lewis Facebook AI Research University College London、Ludovic Denoyer Facebook AI Research、Sebastian Riedel Facebook AI Research University College London
Abstract
Obtaining training data for Question Answering (QA) is time-consuming and resource-intensive, and existing QA datasets are only available for limited domains and languages. In this work, we explore to what extent high quality training data is actually required for Extractive QA, and investigate the possibility of unsupervised Extractive QA. We approach this problem by first learning to generate context, question and answer triples in an unsupervised manner, which we then use to synthesize Extractive QA training data automatically. To generate such triples, we first sample random context paragraphs from a large corpus of documents and then random noun phrases or named entity mentions from these paragraphs as answers. Next we convert answers in context to “fill-in-the-blank” cloze questions and finally translate them into natural questions. We propose and compare various unsupervised ways to perform cloze-to-natural question translation, including training an unsupervised NMT model using non-aligned corpora of natural questions and cloze questions as well as a rule-based approach. We find that modern QA models can learn to answer human questions surprisingly well using only synthetic
中文速览
为抽取式问答（Extractive QA）系统收集标注数据费时费力，且现有数据集覆盖的语言和领域十分有限，因此研究者们尝试探索一种完全不依赖人工标注的无监督路径。这项工作的核心思路是：先从维基百科中随机抽取段落，再从中选出名词短语或命名实体作为"答案"，把答案所在句子变成填空式的完形问题（cloze question），最后通过无监督神经机器翻译（NMT）模型将这些生硬的完形问题"翻译"成自然语言问题，从而自动生成大量问答训练数据来训练问答模型。在经典基准数据集 SQuAD v1 上，该方法在完全不使用任何 SQuAD 训练数据的情况下达到了 56.4 F1（答案为命名实体时高达 64.5 F1），超越了早期有监督模型，还在仅用 32 条标注样本的少样本场景下进一步提升至 59.3 F1。这项工作证明了通过纯合成数据就能让现代问答模型习得基本问答能力，为低资源语言和新领域的问答系统建设提供了一条切实可行的无监督路线。
原文 arXiv:1906.04980;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1906.04980v2