The Second Conversational Intelligence Challenge (ConvAI2)
Emily Dinan Facebook AI Research Varvara Logacheva Moscow Institute of Physics and Technology Valentin Malykh Moscow Institute of Physics and Technology Alexander Miller Facebook AI Research Kurt Shuster Facebook AI Research Jack Urbanek Facebook AI Research Douwe Kiela Facebook AI Research Arthur Szlam Facebook AI Research Iulian Serban University of Montreal Ryan Lowe McGill University Facebook AI Research Shrimai Prabhumoye Carnegie Mellon University Alan W Black Carnegie Mellon University Alexander Rudnicky Carnegie Mellon University Jason Williams Microsoft Research Joelle Pineau Facebook AI Research McGill University Mikhail Burtsev Moscow Institute of Physics and Technology Jason Weston Facebook AI Research
Abstract
We describe the setting and results of the ConvAI2 NeurIPS competition that aims to further the state-of-the-art in open-domain chatbots. Some key takeaways from the competition are: (i) pretrained Transformer variants are currently the best performing models on this task, (ii) but to improve performance on multi-turn conversations with humans, future systems must go beyond single word metrics like perplexity to measure the performance across sequences of utterances (conversations) – in terms of repetition, consistency and balance of dialogue acts (e.g. how many questions asked vs. answered).
原文 arXiv:1902.00098;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1902.00098v1