OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Srinivasan Iyer111Equal contribution; alphabetical order. , Xi Victoria Lin∗, Ramakanth Pasunuru∗, Todor Mihaylov, Dániel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, Xian Li, Brian O’Horo, Gabriel Pereyra, Jeff Wang, Christopher Dewan, Asli Celikyilmaz, Luke Zettlemoyer, Ves Stoyanov \AND \addr Meta AI Work done while at Meta AI.
Abstract
Recent work has shown that fine-tuning large pre-trained language models on a collection of tasks described via instructions, a.k.a. instruction-tuning, improves their zero and few-shot generalization to unseen tasks. However, there is a limited understanding of the performance trade-offs of different decisions made during the instruction-tuning process. These decisions include the scale and diversity of the instruction-tuning benchmark, different task sampling strategies, fine-tuning with and without demonstrations, training using specialized datasets for reasoning and dialogue, and finally, the fine-tuning objectives themselves. In this paper, we characterize the effect of instruction-tuning decisions on downstream task performance when scaling both model and benchmark sizes. To this end, we create OPT-IML Bench: a large benchmark for Instruction Meta-Learning (IML) of 2000 NLP tasks consolidated into task categories from 8 existing benchmarks, and prepare an evaluation framework to measure three types of model generalizations: to tasks from fully held-out categories, to held-out tasks from seen categories, and to held-out instances from seen tasks. Through the lens of this frame
中文速览
大规模语言模型在经过"指令微调"（instruction tuning）后能大幅提升零样本和少样本泛化能力，但究竟哪些微调决策真正起作用——比如任务规模、采样策略、示范样本的加入方式、推理链数据的使用——学界至今缺乏系统性认识。为此，研究者整合了8个现有基准，构建了包含近2000个NLP任务的大型评测框架OPT-IML Bench，并设计了三个层级的泛化评估：对全新任务类别的泛化、对已见类别中未见任务的泛化、以及对已见任务中未见样本的泛化。借助这一框架，他们系统分析了各种微调决策对性能的影响，并据此训练出OPT-IML（30B和175B两个规模），在PromptSource、FLAN、Super-NaturalInstructions、UnifiedSKG四个评测集上均显著超越原始OPT基座模型，同时与各基准上专门微调的模型保持高度竞争力。该工作不仅开源了OPT-IML模型和评测框架，还为未来大规模指令微调的最佳实践提供了可复现的实验依据。
原文 arXiv:2212.12017;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2212.12017v3