Deep Leakage from Gradients (1906.08935v2)

Abstract: Exchanging gradients is a widely used method in modern multi-node machine learning system (e.g., distributed training, collaborative learning). For a long time, people believed that gradients are safe to share: i.e., the training data will not be leaked by gradient exchange. However, we show that it is possible to obtain the private training data from the publicly shared gradients. We name this leakage as Deep Leakage from Gradient and empirically validate the effectiveness on both computer vision and natural language processing tasks. Experimental results show that our attack is much stronger than previous approaches: the recovery is pixel-wise accurate for images and token-wise matching for texts. We want to raise people's awareness to rethink the gradient's safety. Finally, we discuss several possible strategies to prevent such deep leakage. The most effective defense method is gradient pruning.

• The paper introduces an optimization algorithm that reconstructs original training data from shared gradients in distributed learning.
• It demonstrates successful data recovery on image and language tasks, emphasizing gradient inversion's effectiveness even with minimal auxiliary information.
• The study highlights significant privacy risks and recommends defense strategies such as gradient perturbation and compression to mitigate leakage.

Deep Leakage from Gradients

The paper "Deep Leakage from Gradients" by Ligeng Zhu, Zhijian Liu, and Song Han introduces the concept of deep leakage from gradients (DLG), which represents a significant security concern in distributed training and collaborative learning environments. The authors demonstrate that it is feasible to reconstruct private training data solely from the gradients shared among nodes in such systems.

Introduction

In distributed machine learning, gradients are typically exchanged between nodes to collaboratively train models. The assumption that sharing gradients does not compromise data privacy has underpinned the design of many distributed systems, including those employed in privacy-sensitive applications like medical data analysis. The paper challenges this assumption by showing that precise training inputs and labels can be recovered from the shared gradients with an optimization algorithm.

Methodology

The DLG algorithm presented by the authors leverages an optimization approach that minimizes the difference between shared gradients and gradients calculated on randomly initialized "dummy" data. The process iteratively adjusts these dummy inputs and labels to match the real gradients, effectively reconstructing the original training data.

Key steps of the method include:

1. Random initialization of dummy inputs and labels.
2. Computation of gradients using the dummy data.
3. Optimization of dummy data to minimize the gradient distance to the shared gradients.

The iterative optimization process facilitates the recovery of data in both single input and batched scenarios, though the latter requires more iterations for convergence.

Experimental Results

The paper showcases the efficacy of DLG on several tasks:

1. Vision Tasks: On datasets like MNIST, CIFAR-100, SVHN, and LFW, the reconstructed images from gradients closely match the original images. The authors also highlight the superiority of DLG over previous methods that typically require additional information such as generative models or class labels.
2. Language Tasks: Using a BERT model for masked LLM tasks, DLG effectively reconstructs sentences from gradients. With iterative refinement, the revealed text closely aligns with the original sentences.

Implications and Defense Strategies

The implications of this research are profound. The ability to extract exact training data from gradients signifies a severe vulnerability in the gradient sharing schemes used in federated learning and other distributed training paradigms.

The authors propose several defense strategies:

1. Gradient Perturbation: Introducing Gaussian or Laplacian noise to the gradients can distort the gradient data enough to prevent successful recovery. However, this approach requires a careful balance to avoid degrading model accuracy.
2. Gradient Compression and Pruning: By reducing the precision or pruning insignificant gradients, it is possible to obscure critical information. The paper finds that pruning more than 20% of the gradients successfully defends against DLG.
3. Increased Batch Size and Resolution: Although not as generalizable, increasing the batch size or input resolution complicates the gradient inversion process, thus offering some level of protection.
4. Cryptographic Techniques: Secure aggregation protocols and encryption schemes can ensure that shared gradients do not reveal sensitive training information, but these methods might introduce compatibility and performance trade-offs.

Future Directions

Future research will need to explore the balance between maintaining model accuracy and ensuring privacy. Enhancements to existing privacy-preserving methods, such as more robust noise-adding mechanisms that minimally impact accuracy or advanced cryptographic techniques suitable for a wide range of models, could offer pathways forward. Additionally, further investigation into optimizing DLG for various model architectures and training settings can deepen our understanding of privacy risks in distributed learning.

Conclusion

The paper "Deep Leakage from Gradients" presents a compelling case for revisiting and potentially overhauling current practices in gradient sharing within distributed machine learning systems. By exposing the vulnerability of these systems to data leakage through gradient matching, the authors underscore the necessity for integrating secure and privacy-preserving mechanisms. This work contributes critical insights to the fields of machine learning and data privacy, prompting further exploration of secure distributed training techniques.