Fine-tuned Language Models are Continual Learners
Thomas Scialom1 Tuhin Chakrabarty2∗ Smaranda Muresan 2 1Meta AI 2Department of Computer Science, Columbia University Both Authors Contributed Equally
Abstract
Recent work on large language models relies on the intuition that most natural language processing tasks can be described via natural language instructions and that models trained on these instructions show strong zero-shot performance on several standard datasets. However, these models even though impressive still perform poorly on a wide range of tasks outside of their respective training and evaluation sets. To address this limitation, we argue that a model should be able to keep extending its knowledge and abilities, without forgetting previous skills. In spite of the limited success of Continual Learning we show that Fine-tuned Language Models can be continual learners. We empirically investigate the reason for this success and conclude that Continual Learning emerges from self-supervision pre-training. Our resulting model Continual-T0 (CT0) is able to learn 8 new diverse language generation tasks, while still maintaining good performance on previous tasks, spanning in total 70 datasets. Finally, we show that CT0 is able to combine instructions in ways it was never trained for, demonstrating some level of instruction compositionality.111Our code is publicly available at https:
中文速览
指令调优（instruction tuning）后的大模型在面对训练集之外的新任务时往往表现糟糕，而重新微调又会让模型"忘掉"之前学会的技能——即灾难性遗忘（catastrophic forgetting）问题。研究者以公开可用的30亿参数模型T0为基础，引入带有记忆回放（rehearsal）的持续学习（Continual Learning）机制：每次学习新任务时，只把之前数据的极小比例（1%）混入训练，充当"记忆缓冲区"。实验结果显示，由此得到的Continual-T0（CT0）能够依次习得写俳句、对话情感回复、文本简化、新闻标题生成等8项全新语言生成任务，同时几乎完整保留了在原有70个数据集上的性能，甚至对从未参与回放的零样本评估集也维持了良好表现。进一步分析揭示，这种持续学习能力源于预训练阶段本身，而非多任务微调或模型规模所赋予，这一发现为构建能不断自我扩展的实用语言模型提供了重要的理论依据和实践路径。
原文 arXiv:2205.12393;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2205.12393v4