Learning Semantic Textual Similarity from Conversations
Yinfei Yanga𝑎a, Steve Yuanc𝑐c, Daniel Cera𝑎a, Sheng-yi Konga𝑎a, Noah Constanta𝑎a, Petr Pilarc𝑐c, Heming Gea𝑎a, Yun-Hsuan Sunga𝑎a, Brian Stropea𝑎a, Ray Kurzweila𝑎a \ANDa𝑎aGoogle Research Mountain View, CA, USA、b𝑏bGoogle Cambridge, MA, USA、c𝑐cGoogle Zurich, Switzerland
Abstract
We present a novel approach to learn representations for sentence-level semantic similarity using conversational data. Our method trains an unsupervised model to predict conversational input-response pairs. The resulting sentence embeddings perform well on the semantic textual similarity (STS) benchmark and SemEval 2017’s Community Question Answering (CQA) question similarity subtask. Performance is further improved by introducing multitask training combining the conversational input-response prediction task and a natural language inference task. Extensive experiments show the proposed model achieves the best performance among all neural models on the STS benchmark and is competitive with the state-of-the-art feature engineered and mixed systems in both tasks.
中文速览
用对话数据来学句子语义相似度，这个思路来自一个朴素的观察：语义相近的句子（比如"你多大了？"和"你几岁？"）往往能引发相似的回复，而字面相近但语义不同的句子（比如"你好吗？"和"你多大了？"）回复却大相径庭。基于此，研究者利用 Reddit 上约6亿条问答对，训练了一个无监督的输入-回复预测模型（input-response prediction），让模型在大量对话中自动习得句子的语义表示（sentence embedding）；进一步地，他们将该任务与自然语言推理（NLI）任务联合训练，构成多任务学习框架。实验表明，这套方法在语义文本相似度（STS）基准测试上超越了所有神经网络模型，在社区问答（CQA）问题相似度子任务上也与最佳系统不相上下。这项工作的意义在于，它证明了海量无监督对话数据是学习通用语义表示的有效来源，为低成本、高质量的句子向量训练提供了新思路。
原文 arXiv:1804.07754;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1804.07754v1