QLoRA: Efficient Finetuning of Quantized LLMs
About
We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates gradients through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters~(LoRA). Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark, reaching 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU. QLoRA introduces a number of innovations to save memory without sacrificing performance: (a) 4-bit NormalFloat (NF4), a new data type that is information theoretically optimal for normally distributed weights (b) double quantization to reduce the average memory footprint by quantizing the quantization constants, and (c) paged optimziers to manage memory spikes. We use QLoRA to finetune more than 1,000 models, providing a detailed analysis of instruction following and chatbot performance across 8 instruction datasets, multiple model types (LLaMA, T5), and model scales that would be infeasible to run with regular finetuning (e.g. 33B and 65B parameter models). Our results show that QLoRA finetuning on a small high-quality dataset leads to state-of-the-art results, even when using smaller models than the previous SoTA. We provide a detailed analysis of chatbot performance based on both human and GPT-4 evaluations showing that GPT-4 evaluations are a cheap and reasonable alternative to human evaluation. Furthermore, we find that current chatbot benchmarks are not trustworthy to accurately evaluate the performance levels of chatbots. A lemon-picked analysis demonstrates where Guanaco fails compared to ChatGPT. We release all of our models and code, including CUDA kernels for 4-bit training.
Key insights
What problem does QLoRA solve and how?
QLoRA cuts finetuning memory by freezing a 4-bit quantized base model and backpropagating gradients into Low Rank Adapters. It introduces 4-bit NormalFloat, Double Quantization, and Paged Optimizers to preserve 16-bit performance.
How does QLoRA differ from prior work?
Unlike prior quantization methods that work only for inference, QLoRA enables training through 4-bit weights. NF4 outperforms FP4 and Int4 on zero-shot accuracy across LLaMA models up to 65B, and QLoRA matches 16-bit LoRA on MMLU.
What are the practical uses of these findings?
QLoRA enables finetuning a 65B model on a single 48GB GPU and a 33B model on a 24GB consumer GPU in under 12 hours, making high-quality chatbot training accessible to researchers with limited hardware.
What limitations did the authors identify?
The authors did not verify QLoRA matches full 16-bit finetuning at 33B and 65B scales, evaluated on only three benchmarks, and did not test other bit precisions or adapter methods. Responsible AI evaluation of Guanaco was limited.