PTR: Prompt Tuning with Rules for Text Classification
Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu , Maosong Sun State Key Lab on Intelligent Technology and Systems, Institute for Artificial Intelligence, Department of Computer Science and Technology, Tsinghua University, Beijing, China Corresponding author:
Abstract
Fine-tuned pre-trained language models (PLMs) have achieved awesome performance on almost all NLP tasks. By using additional prompts to fine-tune PLMs, we can further stimulate the rich knowledge distributed in PLMs to better serve downstream tasks. Prompt tuning has achieved promising results on some few-class classification tasks such as sentiment classification and natural language inference. However, manually designing lots of language prompts is cumbersome and fallible. For those auto-generated prompts, it is also expensive and time-consuming to verify their effectiveness in non-few-shot scenarios. Hence, it is still challenging for prompt tuning to address many-class classification tasks. To this end, we propose prompt tuning with rules (PTR) for many-class text classification and apply logic rules to construct prompts with several sub-prompts. In this way, PTR is able to encode prior knowledge of each class into prompt tuning. We conduct experiments on relation classification, a typical and complicated many-class classification task, and the results show that PTR can significantly and consistently outperform existing state-of-the-art baselines. This indicates that PTR is a p
中文速览
预训练语言模型（PLM）的提示微调（prompt tuning）在情感分类等少类别任务上效果不错，但面对关系分类这类动辄几十个类别的任务时，手工为每个类设计合适的提示词既费力又难以奏效，自动生成的提示又代价高昂且性能打折。为此，研究者提出了"带规则的提示微调"方法（PTR，Prompt Tuning with Rules），核心思路是用逻辑规则将复杂的多类别分类拆解为若干简单的子提示，再组合成针对具体类别的完整提示，从而把人类先验知识编码进微调过程。在关系分类的四个主流基准数据集（TACRED、TACREV、ReTACRED、SemEval 2010 Task 8）上，PTR 均显著超越现有最优方法。这项工作表明，将逻辑规则与预训练语言模型结合，是解决复杂多类别文本分类任务的一条高效且可解释的新路径。
原文 arXiv:2105.11259;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2105.11259v3