QLoRA: Efficient Finetuning of Quantized LLMs
Tim Dettmers、Artidoro Pagnoni∗、Ari Holtzman、Luke Zettlemoyer University of Washington Equal contribution.
Abstract
We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates gradients through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters (LoRA). Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark, reaching 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU. QLoRA introduces a number of innovations to save memory without sacrificing performance: (a) 4-bit NormalFloat (NF4), a new data type that is information theoretically optimal for normally distributed weights (b) Double Quantization to reduce the average memory footprint by quantizing the quantization constants, and (c) Paged Optimizers to manage memory spikes. We use QLoRA to finetune more than 1,000 models, providing a detailed analysis of instruction following and chatbot performance across 8 instruction datasets, multiple model types (LLaMA, T5), and model scales that would be infeasible to run with regular finetuning (e.g. 33B and 65B pa
中文速览
大型语言模型的微调一直面临显存墙难题——用常规16位精度微调650亿参数的LLaMA模型需要超过780GB显存，普通研究者根本负担不起。QLoRA提出了三项组合创新：把预训练模型压缩到4位精度存储（采用专为正态分布权重设计的NF4新数据类型）、对量化常数再做一轮二次量化以进一步节省内存、以及用分页优化器（Paged Optimizers）平抑训练时的显存峰值；在此基础上，梯度只流经冻结的4位权重，去更新少量可训练的低秩适配器（LoRA）参数。最终，微调650亿参数模型所需显存从780GB以上压缩到不足48GB，在单张48GB专业GPU上24小时即可完成，训练出的Guanaco聊天模型在Vicuna基准上达到ChatGPT性能的99.3%。这项工作的重要意义在于，它让原本只有大型机构才能负担的顶级模型微调变得触手可及，同时通过对超过1000个模型的系统评测，揭示了数据质量远比数据规模重要这一关键规律，并公开了全部代码、CUDA内核与模型权重，大幅降低了开源社区进行大模型定制的门槛。
原文 arXiv:2305.14314;中英对照 + 大白话阅读 https://aha.fim.ai/paper/2305.14314v1