Differentially Private Fine-tuning of Language Models (2110.06500v2)

Abstract: We give simpler, sparser, and faster algorithms for differentially private fine-tuning of large-scale pre-trained LLMs, which achieve the state-of-the-art privacy versus utility tradeoffs on many standard NLP tasks. We propose a meta-framework for this problem, inspired by the recent success of highly parameter-efficient methods for fine-tuning. Our experiments show that differentially private adaptations of these approaches outperform previous private algorithms in three important dimensions: utility, privacy, and the computational and memory cost of private training. On many commonly studied datasets, the utility of private models approaches that of non-private models. For example, on the MNLI dataset we achieve an accuracy of $87.8\%$ using RoBERTa-Large and $83.5\%$ using RoBERTa-Base with a privacy budget of $\epsilon = 6.7$. In comparison, absent privacy constraints, RoBERTa-Large achieves an accuracy of $90.2\%$. Our findings are similar for natural language generation tasks. Privately fine-tuning with DART, GPT-2-Small, GPT-2-Medium, GPT-2-Large, and GPT-2-XL achieve BLEU scores of 38.5, 42.0, 43.1, and 43.8 respectively (privacy budget of $\epsilon = 6.8,\delta=$ 1e-5) whereas the non-private baseline is $48.1$. All our experiments suggest that larger models are better suited for private fine-tuning: while they are well known to achieve superior accuracy non-privately, we find that they also better maintain their accuracy when privacy is introduced.

Analyzing Differentially Private Fine-tuning of LLMs

The paper "Differentially Private Fine-tuning of LLMs" presents innovations to enhance differentially private (DP) fine-tuning of large-scale LLMs. The authors seek to address the challenge where fine-tuning large models under DP constraints traditionally leads to significant utility losses due to noise added for privacy guarantees. This work contributes by proposing a framework that attains competitive utility with state-of-the-art non-private models while respecting privacy requirements.

Core Innovations

The primary innovation in this paper lies in leveraging parameter-efficient fine-tuning techniques to mitigate the detrimental effects of DP noise. The authors adapt cutting-edge non-private parameter-efficient tuning methods to the DP setting. These methods exploit the intrinsic property that only a small fraction of parameters need updating to maintain model accuracy while mitigating computational and noise implications associated with larger parameter spaces.

1. Meta-framework: The authors propose a meta-framework that introduces new trainable parameters to the pre-trained models and limits modification to only a small fraction. This approach ensures significantly reduced computational overhead and noise addition, as seen by focusing training on a low-rank subspace or additional lightweight modules.
2. Parameter-efficient Methods: Key methods include LoRA (Low-Rank Adaptation), Adapter-based fine-tuning, and a further improvement via Compacter (Compact Adapters). These techniques provide a low-footprint, modular approach to model adaptation suitable for various downstream tasks while maintaining a small parameter set for fine-tuning.
3. Resultant Insights: Larger models, such as RoBERTa-Large and GPT-2-XL, are demonstrated to retain more utility under DP constraints, implying that the power of large models persists even in the presence of differential privacy. Remarkably, the authors find that the accuracy of private models often closely approaches that of non-privately fine-tuned counterparts.

Experimental Validation

The paper validates its hypothesis through a series of experiments using RoBERTa and GPT-2 models across common NLP tasks, such as those from the GLUE benchmark and tasks involving natural language generation like E2E and DART datasets. The results indicative of the paper's claims include:

• Utility vs. Privacy Trade-off: The proposed methods undergo rigorous evaluation and show improved accuracy versus privacy trade-offs compared to traditional DP methods. For instance, on the MNLI task with RoBERTa-Large, a slight utility loss was observed when achieving close to non-private model accuracy at stringent privacy settings.
• Efficiency: The experiments underscore the reduced memory and computational costs when employing these parameter-efficient models compared to full-fine-tuning approaches.
• Quantitative Success: Specifically, models like GPT-2-XL privately fine-tuned maintain high utility scores with BLEU and other performance metrics close to non-private counterparts, proving the functional scalability of the proposed method across different model sizes.

Implications and Future Directions

The findings suggest a paradigm shift in approaching DP model tuning by prioritizing parameter-efficient methodologies that deftly balance privacy, utility, and resource efficiency. The implications of such a shift include:

• Scalability and Deployment: The reduced footprint ensures that a single pre-trained model can be deployed efficiently across various tasks by attaching small, task-specific tuning parameters. This modulates broader usage potential without extensive storage or computational burdens.
• Practical Utility: By demonstrating competitive utility on standard datasets, the methods could expedite the integration of robust, privacy-preserving LLMs in real-world applications, particularly where data sensitivity is paramount.
• Theoretical Underpinnings: The paper raises important theoretical questions about the intrinsic dimensionality and adaptability of models in DP contexts—offering rich ground for future exploration on why certain parameters afford better utility stabilizing effects under noise.

In conclusion, this paper decisively advances the field of private machine learning by proving the efficacy of parameter-efficient tuning frameworks for ensuring privacy without substantial trade-offs in model accuracy. As differential privacy remains crucial in translating machine learning advancements into privacy-sensitive domains, the work could serve as a benchmark and inspiration for future research paths optimizing for scalable and utility-preserving privacy solutions in AI.