How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources
Yizhong Wang ♣♠ Hamish Ivison††♣ Pradeep Dasigi♣ Jack Hessel♣ Tushar Khot♣ Khyathi Raghavi Chandu♣ David Wadden♣ Kelsey MacMillan♣ Noah A. Smith♣♠ Iz Beltagy♣ Hannaneh Hajishirzi♣♠ ♣Allen Institute for AI ♠University of Washington Equal contribution.
Abstract
In this work we explore recent advances in instruction-tuning language models on a range of open instruction-following datasets. Despite recent claims that open models can be on par with state-of-the-art proprietary models, these claims are often accompanied by limited evaluation, making it difficult to compare models across the board and determine the utility of various resources. We provide a large set of instruction-tuned models from 6.7B to 65B parameters in size, trained on 12 instruction datasets ranging from manually curated (e.g., OpenAssistant) to synthetic and distilled (e.g., Alpaca) and systematically evaluate them on their factual knowledge, reasoning, multilinguality, coding, safety, and open-ended instruction following abilities through a collection of automatic, model-based, and human-based metrics. We further introduce Tülu , our best performing instruction-tuned model suite finetuned on a combination of high-quality open resources.
中文速览
开源社区一直声称某些指令微调（instruction-tuning）模型的表现已能媲美 ChatGPT，但这些说法背后的评测往往残缺不全、缺乏可比性。为此，研究团队系统地对 12 个公开指令数据集（从人工标注的 OpenAssistant 到模型蒸馏的 Alpaca）分别训练了参数量从 6.7B 到 65B 不等的 LLaMa 系列模型，并从事实知识、推理、多语言、代码生成、安全性和开放式指令遵循六个维度，同时采用自动指标、GPT-4 偏好打分和人工评估进行全面对比。结果发现，不同数据集各有所长，没有哪个单一数据集能在所有维度上称霸；将多个高质量数据集混合训练得到的 Tülu 模型综合表现最佳，但即便是 65B 的 Tülu，平均得分也仅达到 ChatGPT 的 87%、GPT-4 的 73%，差距依然显著。研究还揭示了一个关键隐患：基于模型偏好的评估与输出文本的长度和词汇多样性高度相关，容易掩盖模型真实能力的差距，因此系统性的多维度评测不可或缺。团队已开源全部模型权重、训练代码和评测框架，为后续研究提供基础。
原文 arXiv:2306.04751;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2306.04751v2