Locate the performance–precision trade-off for QLoRA tuning

Determine where the performance–precision trade-off lies for QLoRA finetuning, which backpropagates through 4-bit quantized pretrained language model weights into Low-Rank Adapters, by identifying the precision levels at which QLoRA ceases to match full 16-bit finetuning performance.

Background

The paper shows that QLoRA with 4-bit NormalFloat (NF4) quantization and double quantization can match 16-bit full finetuning and 16-bit LoRA performance across several models and tasks. This result suggests that substantial precision reduction is possible without sacrificing performance.

However, the authors explicitly raise an unresolved question about the broader trade-off between precision and performance for QLoRA, leaving open exactly how far precision can be reduced before performance deteriorates, and at which precision levels the method stops matching 16-bit baselines.