Model merging with SVD to tie the Knots (2410.19735v1)

Abstract: Recent model merging methods demonstrate that the parameters of fully-finetuned models specializing in distinct tasks can be combined into one model capable of solving all tasks without retraining. Yet, this success does not transfer well when merging LoRA finetuned models. We study this phenomenon and observe that the weights of LoRA finetuned models showcase a lower degree of alignment compared to their fully-finetuned counterparts. We hypothesize that improving this alignment is key to obtaining better LoRA model merges, and propose KnOTS to address this problem. KnOTS uses the SVD to jointly transform the weights of different LoRA models into an aligned space, where existing merging methods can be applied. In addition, we introduce a new benchmark that explicitly evaluates whether merged models are general models. Notably, KnOTS consistently improves LoRA merging by up to 4.3% across several vision and language benchmarks, including our new setting. We release our code at: https://github.com/gstoica27/KnOTS.

• The paper presents KnOTS, a novel technique leveraging SVD to transform LoRA weight updates for improved model merging.
• It enhances merging accuracy by aligning models in a shared representation space, achieving up to 4.3% improvement on key benchmarks.
• The joint-task evaluation demonstrates KnOTS' scalability and robustness across diverse datasets and architectures.

Model Merging with SVD: Enhancing LoRA Alignment

Introduction

The concept of model merging has gained traction as a technique to consolidate the abilities of multiple task-specific models into a single multitask model. While effective for fully-finetuned models, these methods often falter when applied to Low-Rank Adaptation (LoRA) finetuned models due to poor parameter alignment. The paper introduces "KnOTS" (Knowledge Orientation Through SVD), leveraging Singular Value Decomposition (SVD) to align the weights of LoRA models, facilitating improved model merging.

Key Contributions

1. LoRA Alignment with SVD: The authors identified that LoRA finetuned models exhibit lower alignment compared to fully-finetuned models, as evidenced by diminished pairwise centered kernel alignment (CKA) scores. KnOTS uses SVD to transform task-specific updates into a shared representation space, enhancing the alignment of these models.
2. Improved Merging Performance: KnOTS augments existing merging algorithms by aligning models in a common space, yielding up to 4.3% improvement in accuracy across vision and language benchmarks.
3. Joint-Task Evaluation: A novel benchmark that assesses a merged model's generality by evaluating on the union of inputs and labels across multiple datasets, advancing the understanding of merging efficacy in more comprehensive settings.

Methodology

KnOTS operationalizes its model merging improvements through a layer-wise application of SVD on LoRA weight updates. This transforms the updates into a more aligned space by decomposing them into shared $U\Sigma$ terms and task-specific $V$ terms. KnOTS then applies existing merging methods such as Task Arithmetic (TA) and TIES to these aligned parameters, creating a coherent merged model.

Experimental Results

1. Vision and Language Tasks: KnOTS consistently outperformed traditional merging methods across various datasets, achieving higher normalized accuracy. For instance, merging eight models finetuned with LoRA on distinct image classification tasks using KnOTS-TIES surpassed baseline accuracy by 4.3%.
2. Scalability with Model Size: On larger models like ViT-L/14 and LLaMA3-8B, KnOTS maintained or enhanced effectiveness, suggesting robustness and scalability across different model architectures and sizes.
3. Positive Impact on Joint-Task Performance: In the newly introduced joint-task setting, KnOTS demonstrated superior performance in Hits@$k$ metrics, showcasing its capability in forming general models capable of working beyond task-specific constraints.

Implications and Future Work

KnOTS signifies a methodological advancement in efficiently merging LoRA finetuned models without the need for additional finetuning. This contribution holds promise not only for improving model merging practices but also for encouraging new developments in creating more generalizable AI systems. Future research could explore extending KnOTS to other parameter-efficient finetuning methods and further refine the joint-task evaluation framework for broader applicability.

In conclusion, the KnOTS approach provides a substantial enhancement to the model merging landscape by effectively addressing the alignment issues inherent in LoRA models. Its adaptability across tasks and architectures underscores its potential impact on the development of more integrated AI systems.