Mechanism behind LoRA’s advantage over full fine-tuning in AE-Adapt-V

Ascertain whether the superior performance of LoRA-based end-to-end fine-tuning over full fine-tuning during AE-Adapt-V is attributable to LoRA’s improved preservation of the knowledge encoded in the pre-trained diffusion transformer backbone.

Background

In end-to-end adaptation following embedding space alignment, the authors compared LoRA tuning against full fine-tuning and observed that LoRA achieved higher VBench scores and better visual quality with fewer trainable parameters.

They explicitly conjecture that LoRA’s low-rank adaptation better preserves the pre-trained model’s knowledge, which could explain its empirical advantages. Validating this mechanism would clarify why parameter-efficient fine-tuning can outperform full fine-tuning in this setting and inform future adaptation designs.