Instruction Tuned Models are Quick Learners
Himanshu Gupta1♢ Saurabh Arjun Sawant1♢ Swaroop Mishra1♣ Mutsumi Nakamura1 Arindam Mitra2 Santosh Mashetty1 Chitta Baral1 1Arizona State University 2Microsoft Research {hgupta35, ssawan13, srmishr1, mutsumi,
Abstract
Instruction tuning of language models has demonstrated the ability to enhance model generalization to unseen tasks via in-context learning using a few examples. However, typical supervised learning still requires a plethora of downstream training data for finetuning. Often in real-world situations, there is a scarcity of data available for finetuning, falling somewhere between few shot inference and fully supervised finetuning. In this work, we demonstrate the sample efficiency of instruction tuned models over various tasks by estimating the minimal downstream training data required by them to perform transfer learning and match the performance of state-of-the-art (SOTA) supervised models. We conduct experiments on 119 tasks from Super Natural Instructions (SuperNI) in both the single task learning (STL) and multi task learning (MTL) settings. Our findings reveal that, in the STL setting, instruction tuned models equipped with 25% of the downstream train data surpass the SOTA performance on the downstream tasks. In the MTL setting, an instruction tuned model trained on only 6% of downstream training data achieve SOTA, while using 100% of the training data results in a 3.69% points
中文速览
让指令微调模型（instruction-tuned model）在真实场景中落地，往往面临一个尴尬：零样本/少样本推理性能不足，但全量有监督微调又需要大量标注数据。这项研究聚焦于"只用少量下游数据，指令微调模型能否追上有监督SOTA"这一问题，在SuperNI的119个任务上系统评估了Tk-Instruct 3B的样本效率。实验结果表明，在单任务学习（STL）场景下，仅用25%的下游训练数据就能超越有监督SOTA；在多任务学习（MTL）场景下，只需6%的数据即可与SOTA持平，用全量数据更能将SOTA提升3.69个百分点（ROUGE-L达74.68）。此外，研究还发现，在预微调阶段引入指令一致可带来约4%的性能提升，但问题改写和标题生成类任务反而会因指令微调而受损。这项工作首次系统量化了指令微调模型的样本效率上限，为数据稀缺的真实应用场景提供了重要的实践指导。
原文 arXiv:2306.05539;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.05539v1