$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning (2405.17258v1)

Abstract: Low-rank adapters (LoRA) and their variants are popular parameter-efficient fine-tuning (PEFT) techniques that closely match full model fine-tune performance while requiring only a small number of additional parameters. These additional LoRA parameters are specific to the base model being adapted. When the base model needs to be deprecated and replaced with a new one, all the associated LoRA modules need to be re-trained. Such re-training requires access to the data used to train the LoRA for the original base model. This is especially problematic for commercial cloud applications where the LoRA modules and the base models are hosted by service providers who may not be allowed to host proprietary client task data. To address this challenge, we propose $\textit{Trans-LoRA}$ -- a novel method for lossless, nearly data-free transfer of LoRAs across base models. Our approach relies on synthetic data to transfer LoRA modules. Using LLMs, we design a synthetic data generator to approximate the data-generating process of the $\textit{observed}$ task data subset. Training on the resulting synthetic dataset transfers LoRA modules to new models. We show the effectiveness of our approach using both LLama and Gemma model families. Our approach achieves lossless (mostly improved) LoRA transfer between models within and across different base model families, and even between different PEFT methods, on a wide variety of tasks.

• The paper introduces Trans-LoRA, a novel method enabling transferable parameter efficient finetuning without accessing original task data.
• It leverages synthetic data generation, discriminator filtering, and knowledge distillation to adapt LoRA models between different base models.
• Experimental results show up to 10% performance gains, offering a scalable solution for confidential, cloud-based AI services.

An Overview of Trans-LoRA: Towards Data-Free Transferable Parameter Efficient Finetuning

The paper "Trans-LoRA: Towards Data-Free Transferable Parameter Efficient Finetuning" addresses a significant challenge within the field of Parameter Efficient Finetuning (PEFT)—namely, the dependence of Low-Rank Adapters (LoRA) and other PEFT methods on base models that are often subject to deprecation and replacement. The proposed Trans-LoRA method offers a novel solution by allowing LoRA models to be transferred between different base models without accessing original task-specific data, leveraging synthetic data generation and a discriminative filtering process.

Introduction

The advancements in LLMs have multiplied the parameters into billions, necessitating fine-tuning for downstream tasks to achieve enhanced specialization. Conventional fine-tuning is resource-intensive, especially for large-scale deployment, prompting the need for PEFT techniques like LoRA. LoRA adapts models by training a minimal number of additional parameters atop a frozen pre-trained model. However, when the underlying base model is deprecated, all associated LoRA models must be retrained, which is impractical in many cloud-based applications due to client confidentiality constraints.

Trans-LoRA: Approach and Methodology

Trans-LoRA introduces a mechanism for transferring LoRA models to new base models using synthetic data. The key steps in the Trans-LoRA approach include:

1. Synthetic Data Generation: Using a LLM to simulate the data-generating process of the original task. This involves creating synthetic prompt-completion pairs that approximate the original task-specific data.
2. Discriminator Training: A discriminator model is trained on a mix of synthetic and real data, filtering the generated synthetic data to closely resemble the original task distribution. This step ensures the quality and relevance of the synthetic data used in transfer.
3. Knowledge Distillation: Transferring the capabilities of the source LoRA to the target LoRA through knowledge distillation on the filtered synthetic data.

The paper details a dual-model framework, utilizing a discriminator to filter non-representative synthetic data and ensure high fidelity to the original task distribution, thereby improving the effectiveness of knowledge distillation.

Experimental Validation

The efficacy of Trans-LoRA was validated through extensive experiments involving multiple LLM families (Llama2 and Gemma) and a variety of tasks drawn from datasets such as BBH, MMLU, GSM8K, and MBPP. The results consistently demonstrated that Trans-LoRA not only achieves lossless transfer but also enhances performance beyond that of either the source LoRA or the target base model.

Key numerical results include:

• Performance improvements of up to 10% in some tasks.
• Robust performance in transferring within and across different model families and PEFT variants.

Implications and Future Directions

Practical Implications: The ability of Trans-LoRA to perform nearly data-free transfers has significant implications for cloud-based AI services, where client data confidentiality is paramount. This method allows for centralized and automated model transfers without demanding retraining from clients, simplifying logistics and enhancing scalability.

Theoretical Implications: The work opens avenues for exploring synthetic data utility beyond general model training, particularly in PEFT contexts. It demonstrates that carefully curated synthetic data, combined with discriminator models, can approximate the required training distributions effectively for knowledge transfer tasks.

Future Directions: Potential future research could focus on minimizing the computational overhead required for data synthesis and discriminator training. Additionally, exploring direct PEFT transfer mechanisms without synthetic data generation could further simplify the approach. There is also scope for extending Trans-LoRA to other modalities and domains, enhancing its applicability.

Conclusion and Limitations

Trans-LoRA offers an innovative solution to the challenge of model dependency in PEFT approaches, leveraging synthetic data to enable nearly data-free model transfer. While it demonstrates substantial performance gains and practical viability, the requirement for synthetic data generation indicates room for optimization. The work holds promise for advancing scalable, confidential, and efficient model serving in AI applications.

The paper contributes to the state-of-the-art in PEFT by addressing a critical gap and providing a robust, theoretically sound framework for model transfer, which can significantly influence future research and application in the AI domain.