How would Stance Detection Techniques Evolve after the Launch of ChatGPT?
Bowen Zhang1, Daijun Ding1, Liwen Jing2 , Genan Dai1, Nan Yin3 1College of Big Data and Internet, Shenzhen Technology University, Shenzhen, China 2Faculty of Information and Intelligence, Shenzhen X-Institute, Shenzhen, China 3Mohamed bin Zayed University of Artificial Intelligence Corresponding authors:
Abstract
Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least cha
中文速览
社交媒体上的立场检测（stance detection）任务长期面临两大痛点：标注数据不足，以及深度学习模型"黑箱"、无法解释预测原因。研究者直接用零样本提示（zero-shot prompting）的方式向ChatGPT提问，让它判断推文对特定目标的态度（支持/反对/中立），无需任何训练数据。实验在SemEval-2016和P-Stance两个标准数据集上表明，ChatGPT不仅超越了所有零样本基线模型，在平均F1上比最强竞争者提升约16.6%，甚至在多数任务上优于用80%数据微调过的有监督模型。更难得的是，ChatGPT还能用自然语言解释自己的判断依据，对那些它无法给出明确分类的样本也能说明原因，这为构建可解释的立场检测AI打开了全新可能，有望推动整个领域的研究范式转变。
原文 arXiv:2212.14548;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2212.14548v4