Towards a Unified View of Parameter-Efficient Transfer Learning (2110.04366v3)

Abstract: Fine-tuning large pre-trained LLMs on downstream tasks has become the de-facto learning paradigm in NLP. However, conventional approaches fine-tune all the parameters of the pre-trained model, which becomes prohibitive as the model size and the number of tasks grow. Recent work has proposed a variety of parameter-efficient transfer learning methods that only fine-tune a small number of (extra) parameters to attain strong performance. While effective, the critical ingredients for success and the connections among the various methods are poorly understood. In this paper, we break down the design of state-of-the-art parameter-efficient transfer learning methods and present a unified framework that establishes connections between them. Specifically, we re-frame them as modifications to specific hidden states in pre-trained models, and define a set of design dimensions along which different methods vary, such as the function to compute the modification and the position to apply the modification. Through comprehensive empirical studies across machine translation, text summarization, language understanding, and text classification benchmarks, we utilize the unified view to identify important design choices in previous methods. Furthermore, our unified framework enables the transfer of design elements across different approaches, and as a result we are able to instantiate new parameter-efficient fine-tuning methods that tune less parameters than previous methods while being more effective, achieving comparable results to fine-tuning all parameters on all four tasks.

Towards a Unified View of Parameter-Efficient Transfer Learning

The fine-tuning of large pre-trained LLMs (PLMs) is integral to achieving state-of-the-art performance in numerous NLP tasks. However, the conventional method of fine-tuning all parameters of a PLM for each downstream task becomes computationally prohibitive as both the model size and the number of tasks increase. This paper takes an important step forward by providing a comprehensive examination and a unified framework for parameter-efficient transfer learning methods.

Summary of the Paper

Motivation and Context

The prevalent fine-tuning approach results in distinct models for each task, necessitating the storage and maintenance of multiple large models. An alternative strategy is parameter-efficient transfer learning, which modifies only a small subset of parameters while keeping the majority of the PLM's parameters frozen. This results in a more manageable number of parameters that need to be updated and stored, thus saving computational resources and making quick adaptation to new tasks feasible.

Parameter-Efficient Methods

Several recent methods have aimed to address the inefficiencies of full fine-tuning:

• Adapter Tuning: Introduces small neural modules known as adapters into each layer of the network, fine-tuning only these adapters.
• Prefix Tuning: Prepends a set of tunable prefix tokens to the input or hidden layers, similar to the concepts behind prompt-tuning.
• LoRA: Utilizes trainable low-rank matrices for approximating parameter updates.

Each method restricts the number of parameters being fine-tuned to a small fraction of the total, with reported performance close to full fine-tuning across various tasks.

Unified Framework

The paper proposes a unified framework that reinterprets these methods as various ways to modify specific hidden states within the PLM:

• Modified Representation: It specifies whether the attention outputs, feed-forward network (FFN) outputs, or other components are modified.
• Functional Form: The specific function used to compute the modifications.
• Insertion Form: Whether the modifications are introduced in parallel or sequentially.
• Composition Function: How these modifications are integrated with the existing hidden states.

By comparing these methods along these dimensions, the framework simplifies understanding their design choices and effectiveness.

Detailed Analysis

Empirical Studies

The empirical studies span several benchmarks:

• Text Summarization (XSum)
• Machine Translation (WMT 2016 en-ro)
• Language Understanding (MNLI)
• Text Classification (SST2)

The paper reveals that although existing parameter-efficient methods achieve strong performance in low-resource and simpler tasks, they often lag behind on higher-resource and more complex tasks.

Design Dimensions

Insertion Form: Parallel adapters generally outperform their sequential counterparts. For example, parallel adapters at the FFN modification manifest superior results compared to other parallel or sequential placements.

Modified Representation: The modified FFN representations tend to be more effective than attention-based modifications, as they leverage task-specific textual patterns more efficiently.

Composition Function: The scaling composition function from LoRA enhances the performance of the basic additive composition used in adapter tuning.

Hybrid Approach

Combining beneficial elements from various methods leads to the "Mix-And-Match (MAM) Adapter," which uses prefix tuning with a small bottleneck dimension for attention modifications and scaled parallel adapter for FFN modifications. Notably, MAM Adapter achieves comparable performance to full fine-tuning by only updating 6.7% of the parameters in PLMs.

Implications and Future Directions

Practical Implications

The unified view and the analysis suggest several practical advantages:

• Adaptability: The framework enables researchers and practitioners to tailor highly efficient models for specific applications.
• Memory Efficiency: Reducing the storage and computational footprints, beneficial for deployment in resource-constrained environments.
• Robustness: More robust across diverse tasks, paving avenues for practical applications involving frequent model updates.

Theoretical Implications

Although this work extensively details the empirical performance of various methods, theoretically understanding why particular dimensions are more effective could be a valuable direction. Additionally, exploring scenarios where parameter-efficient methods could mitigate or exacerbate issues such as catastrophic forgetting and model biases remains an open area of investigation.

Conclusion

This paper's framework provides an important step in advancing parameter-efficient transfer learning, offering both empirical results and a theoretical basis for design choices within parameter-efficient methods. This unified view not only sheds light on the underlying mechanics of these methods but also facilitates the development of more effective and resource-efficient NLP models. Future work can build on this framework to optimize further and experiment with parameter-efficient methods, ultimately advancing the field of AI and NLP.