Accelerating Real-Time Question Answering via Question Generation
Yuwei Fang1 Shuohang Wang1 Zhe Gan1 Siqi Sun1 Jingjing Liu2 Chenguang Zhu1 1Microsoft Cognitive Services Research Group, 2Tsinghua University {yuwfan, shuowa, zhe.gan, siqi.sun,
Abstract
Although deep neural networks have achieved tremendous success for question answering (QA), they are still suffering from heavy computational and energy cost for real product deployment. Further, existing QA systems are bottlenecked by the encoding time of real-time questions with neural networks, thus suffering from detectable latency in deployment for large-volume traffic. To reduce the computational cost and accelerate real-time question answering (RTQA) for practical usage, we propose to remove all the neural networks from online QA systems, and present Ocean-Q (an Ocean of Questions), which introduces a new question generation (QG) model to generate a large pool of QA pairs offline, then in real time matches an input question with the candidate QA pool to predict the answer without question encoding. Ocean-Q can be readily deployed in existing distributed database systems or search engine for large-scale query usage, and much greener with no additional cost for maintaining large neural networks. Experiments on SQuAD(-open) and HotpotQA benchmarks demonstrate that Ocean-Q is able to accelerate the fastest state-of-the-art RTQA system by 4X times, with only a 3+% accuracy drop.
中文速览
问答系统（QA）速度慢、耗能高，根源在于每来一个新问题都要用神经网络实时编码——Ocean-Q 的思路是把这个瓶颈彻底绕开：事先用问题生成（Question Generation）模型离线生成海量候选问答对，等真实问题到来时直接用 n-gram 匹配从候选池里找答案，全程不再运行任何神经网络。为了让候选问答对既多又准，系统还引入了数据增强、多任务学习和多样化束搜索来提升问题生成质量，并用一个两级 TF-IDF 排序器把实时检索范围压缩到可控规模。在 SQuAD、SQuAD-open 和 HotpotQA 基准上的实验表明，Ocean-Q 能把当时最快的实时问答系统再提速 4 倍，准确率仅下降约 3 个百分点，同时因为彻底去掉了在线神经网络推理，系统的能耗和运维成本也大幅降低，对绿色 AI 落地具有实际价值。
原文 arXiv:2009.05167;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2009.05167v2