BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis
Hu Xu1, Bing Liu1, Lei Shu1 Philip S. Yu1,2 1Department of Computer Science, University of Illinois at Chicago, Chicago, IL, USA 2Institute for Data Science, Tsinghua University, Beijing, China {hxu48, liub, lshu3,
Abstract
Question-answering plays an important role in e-commerce as it allows potential customers to actively seek crucial information about products or services to help their purchase decision making. Inspired by the recent success of machine reading comprehension (MRC) on formal documents, this paper explores the potential of turning customer reviews into a large source of knowledge that can be exploited to answer user questions. We call this problem Review Reading Comprehension (RRC). To the best of our knowledge, no existing work has been done on RRC. In this work, we first build an RRC dataset called ReviewRC based on a popular benchmark for aspect-based sentiment analysis. Since ReviewRC has limited training examples for RRC (and also for aspect-based sentiment analysis), we then explore a novel post-training approach on the popular language model BERT to enhance the performance of fine-tuning of BERT for RRC. To show the generality of the approach, the proposed post-training is also applied to some other review-based tasks such as aspect extraction and aspect sentiment classification in aspect-based sentiment analysis. Experimental results demonstrate that the proposed post-training
中文速览
电商问答场景中，用户常常需要从海量产品评论里快速获取答案，但现有方法要么让用户自己读完整评论，要么依赖社区众包而回复迟缓。为此，研究者提出了一个新任务——评论阅读理解（Review Reading Comprehension，RRC），即让模型直接从评论文本中抽取出精准的答案片段来回应用户提问，并为此构建了首个标注数据集 ReviewRC。由于评论领域数据量有限，且预训练语言模型 BERT 主要在维基百科等正式文本上训练，缺乏对口语化评论和细粒度观点的理解，研究者提出了一种联合后训练（post-training）方法，同时利用无监督的领域评论语料和有监督的通用阅读理解数据对 BERT 进行适配，再在下游任务上微调。实验表明，该方法不仅显著提升了 RRC 性能，还对方面抽取、方面情感分类等情感分析任务同样有效，为资源有限的电商智能问答提供了一套实用且通用的解决思路。
原文 arXiv:1904.02232;中英对照 + 大白话阅读 https://aha.fim.ai/paper/1904.02232v2