Unsupervised Online Multitask Learning of Behavioral Sentence Embeddings
Shao-Yen Tseng and Brian Baucom and Panayiotis Georgiou
Abstract
Unsupervised learning has been an attractive method for easily deriving meaningful data representations from vast amounts of unlabeled data. These representations, or embeddings, often yield superior results in many tasks, whether used directly or as features in subsequent training stages. However, the quality of the embeddings is highly dependent on the assumed knowledge in the unlabeled data and how the system extracts information without supervision. Domain portability is also very limited in unsupervised learning, often requiring re-training on other in-domain corpora to achieve robustness. In this work we present a multitask paradigm for unsupervised contextual learning of behavioral interactions which addresses unsupervised domain adaption. We introduce an online multitask objective into unsupervised learning and show that sentence embeddings generated through this process increases performance of affective tasks.
中文速览
用无监督方式从海量未标注文本中学出高质量的句子表示（sentence embedding）一直面临两大难题：学出来的表示容易包含冗余信息、且对目标领域的泛化能力很弱。本文提出一种"在线多任务"无监督学习框架，在用序列到序列（seq2seq）模型做上下文预测的同时，额外加入一个情感极性分类任务——分类标签不需要人工标注，而是在训练过程中实时用情感词表自动生成，从而在几乎不增加标注成本的前提下，把情感领域知识"嵌入"到表示学习中。在人际互动行为识别（如心理咨询会话评分）这一下游任务上，用 k 均值聚类、k 近邻及 LSTM 回归等多种简单方法评估后，所提框架学到的句子表示均优于普通无监督基线，证明在线多任务引导能让无监督句子表示在特定行为分析任务上更具判别力，且不需要大规模标注数据或复杂模型。
原文 arXiv:1807.06792;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1807.06792v2