ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue Systems (ClariQ)
Mohammad Aliannejadi University of Amsterdam, Amsterdam, The Netherlands, Julia Kiseleva Microsoft Research AI, Seattle, USA, Aleksandr Chuklin Google Research, Zürich, Switzerland, Jeff Dalton University of Glasgow, Glasgow, UK, Mikhail Burtsev MIPT, Moscow, Russia,
Abstract
This document presents a detailed description of the challenge on clarifying questions for dialogue systems (ClariQ) [pronounce as Claire-ee-que]. The challenge is organized as part of the Conversational AI challenge series (ConvAI3)111http://convai.io/ at Search-oriented Conversational AI (SCAI) EMNLP workshop in 2020.222https://scai.info. The main aim of the conversational systems is to return an appropriate answer in response to the user requests. However, some user requests might be ambiguous. In IR settings such a situation is handled mainly thought the diversification of search result page [Radlinski and Dumais, 2006]. It is however much more challenging in dialogue settings. Hence, we aim to study the following situation for dialogue settings:
中文速览
对话系统在面对模糊问题时，往往不知道该直接回答还是先追问用户，ClariQ挑战赛正是为了系统研究"何时追问"与"如何追问"这两个核心难题而设立的。研究者众包构建了一个新数据集，包含用户的模糊请求、候选澄清问题及用户回答，并设计了两阶段评测：第一阶段让系统在静态数据集上完成问题必要性打分和最优澄清问题检索，第二阶段则引入真实用户进行多轮对话交互评估。评测结果表明，该框架能够有效衡量系统在相关性和自然度两个维度上的澄清表现。这项工作为开放域对话系统中的主动追问能力提供了标准化的数据资源和评测基准，对推动对话式信息检索研究具有重要参考价值。
原文 arXiv:2009.11352;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2009.11352v1