Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning (1702.07464v3)

Abstract: Deep Learning has recently become hugely popular in machine learning, providing significant improvements in classification accuracy in the presence of highly-structured and large databases. Researchers have also considered privacy implications of deep learning. Models are typically trained in a centralized manner with all the data being processed by the same training algorithm. If the data is a collection of users' private data, including habits, personal pictures, geographical positions, interests, and more, the centralized server will have access to sensitive information that could potentially be mishandled. To tackle this problem, collaborative deep learning models have recently been proposed where parties locally train their deep learning structures and only share a subset of the parameters in the attempt to keep their respective training sets private. Parameters can also be obfuscated via differential privacy (DP) to make information extraction even more challenging, as proposed by Shokri and Shmatikov at CCS'15. Unfortunately, we show that any privacy-preserving collaborative deep learning is susceptible to a powerful attack that we devise in this paper. In particular, we show that a distributed, federated, or decentralized deep learning approach is fundamentally broken and does not protect the training sets of honest participants. The attack we developed exploits the real-time nature of the learning process that allows the adversary to train a Generative Adversarial Network (GAN) that generates prototypical samples of the targeted training set that was meant to be private (the samples generated by the GAN are intended to come from the same distribution as the training data). Interestingly, we show that record-level DP applied to the shared parameters of the model, as suggested in previous work, is ineffective (i.e., record-level DP is not designed to address our attack).

• The paper demonstrates that adversaries can leverage GANs to generate high-fidelity reconstructions of private training data from model updates.
• It details how embedding an artificial class in collaborative learning enables iterative parameter exploitation to progressively leak sensitive information.
• The research highlights that even differential privacy measures may fail to fully protect data unless model accuracy is significantly compromised.

Overview of "Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning"

The paper "Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning," authored by Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz, explores the privacy vulnerabilities inherent in collaborative deep learning frameworks. Specifically, it targets the premise that federated or decentralized learning methodologies provide enhanced privacy protections compared to centralized approaches.

Introduction and Problem Definition

Collaborative deep learning schemes have been proposed to mitigate the privacy risks associated with centralized models, where all data is aggregated and processed at a single location. In these decentralized approaches, participants train models locally and share only a subset of model parameters, potentially obfuscated via differential privacy (DP). However, this paper presents a critical analysis showing that these collaborative settings are vulnerable to a novel form of attack that exploits Generative Adversarial Networks (GANs) to breach participants' privacy.

Attack Mechanism

The core of the attack leverages GANs' ability to generate data from the same distribution as the training data without direct access to it. While GANs are traditionally used for generating realistic data samples, this paper details how an adversary within the collaborative learning framework can use GANs to infer and reconstruct sensitive information originally intended to remain private.

The attack operates as follows:

1. GAN Configuration: The adversary utilizes a GAN to generate prototypical samples that mimic the private data of target participants.
2. Parameter Exploitation: By participating in the collaborative training loop, the adversary receives iterative updates of model parameters.
3. Gradual Inference: Utilizing the GAN-discriminator dynamic, the adversary refines the generated samples based on the feedback from the shared model parameters.
4. Release of Information: The adversary can generate realistic samples that are effectively indistinguishable from the original private training data of the collaborators.

Experimental Setup

The paper conducts exhaustive experiments on two prominent datasets: MNIST and AT&T (Olivetti) Faces Dataset. The experiments involve a controlled simulation of the collaborative learning environment with multiple participants, among which one is an adversarial insider. The adversary injects an artificial class into the training setup, compelling the victim to distinguish between authentic and generated data, thereby leaking more information about the genuine data.

Key findings include:

• The GAN attack achieves high-fidelity reconstructions of the private training data even when differential privacy mechanisms are employed.
• The attack's effectiveness diminishes only when the learning algorithm's accuracy is significantly compromised due to excessively tight differential privacy bounds.

Differential Privacy and Limitations

The paper further critiques the robustness of differential privacy as applied in collaborative deep learning scenarios. While differential privacy aims to obfuscate sensitive information by adding noise, the paper demonstrates that the level of noise needed to theoretically guarantee privacy often undermines the model's ability to learn effectively. Consequently, the adversary can still extract meaningful patterns as long as the collaborative model retains reasonable accuracy.

Implications and Future Directions

The implications of this research are substantial, highlighting that collaborative deep learning frameworks, despite their security promises, may introduce new avenues for privacy violations that do not exist in centralized systems. Specifically:

• Trust Dynamics: Collaborative learning assumes trust among participants, but this assumption is critically undermined by active adversaries.
• Model Sharing Risks: The act of sharing parametric updates in a collaborative framework may lead to inadvertent leakage of sensitive data to malicious insiders.

Future research directions suggested by the authors include:

• Enhanced Privacy Mechanisms: Investigating more robust differential privacy implementations or alternative privacy-preserving methods, possibly at a device or user level, which can handle active adversary scenarios.
• Cryptographic Protections: Exploring more computationally demanding cryptographic primitives such as secure multiparty computation (MPC) or homomorphic encryption that can provide stronger privacy guarantees, albeit with higher computational costs.
• Adversarial Training Defenses: Developing defense mechanisms against GAN-based attacks specifically tailored for collaborative learning environments.

Conclusion

"Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning" provides a comprehensive analysis of the vulnerability of collaborative deep learning frameworks to GAN-based attacks. The insights challenge the assumption that decentralized learning paradigms inherently offer better privacy protections compared to their centralized counterparts. This paper serves as a crucial call to action for the AI and security research communities to reconsider and reinforce privacy strategies in collaborative settings.