A Survey of Privacy Attacks in Machine Learning (2007.07646v3)

Abstract: As machine learning becomes more widely used, the need to study its implications in security and privacy becomes more urgent. Although the body of work in privacy has been steadily growing over the past few years, research on the privacy aspects of machine learning has received less focus than the security aspects. Our contribution in this research is an analysis of more than 40 papers related to privacy attacks against machine learning that have been published during the past seven years. We propose an attack taxonomy, together with a threat model that allows the categorization of different attacks based on the adversarial knowledge, and the assets under attack. An initial exploration of the causes of privacy leaks is presented, as well as a detailed analysis of the different attacks. Finally, we present an overview of the most commonly proposed defenses and a discussion of the open problems and future directions identified during our analysis.

• The paper presents a detailed taxonomy that categorizes privacy attacks into membership inference, reconstruction, property inference, and model extraction.
• It analyzes attack methods such as shadow model training and GAN-based reconstruction, linking vulnerabilities to factors like overfitting and model complexity.
• The paper evaluates defenses including differential privacy and model regularization, underscoring the trade-off between privacy protection and model utility.

Privacy Attacks in Machine Learning: A Comprehensive Survey

Machine learning models, widely deployed across various domains, are increasingly scrutinized for privacy vulnerabilities. As these models leverage expansive datasets, often containing sensitive information, privacy attacks have emerged as a significant concern for researchers and practitioners alike. The paper "A Survey of Privacy Attacks in Machine Learning" presents a thorough examination of over 40 studies focused on privacy attacks against machine learning systems, categorizing these attacks and discussing their implications.

Attack Taxonomy and Threat Model

The research delineates a taxonomy of privacy attacks specifically targeting machine learning. It identifies four main categories: membership inference, reconstruction, property inference, and model extraction attacks. Each category captures a distinct approach by adversaries to extract sensitive information from models:

1. Membership Inference Attacks: These attacks aim to determine whether a particular data instance was part of the model's training dataset. Black-box attacks in this category mainly employ shadow model training, which involves training auxiliary models to mimic the behavior of the target model and infer membership.
2. Reconstruction Attacks: These attacks attempt to reconstruct data samples or their attributes using access to the model's internal parameters or outputs. This includes model inversion attacks, where an adversary aims to deduce input data from outputs.
3. Property Inference Attacks: These attacks focus on extracting dataset properties that are not directly encoded, such as biases or aggregated statistics, which can offer insights into the dataset used for training.
4. Model Extraction Attacks: This category involves reconstructing a model's parameters or its entire functionality. The goal can vary from task accuracy (replicating the model's performance) to fidelity (replicating decision boundaries).

The survey also emphasizes a threat model encompassing the actors and assets involved. It identifies data owners, model owners, model consumers, and adversaries, detailing how each interacts with machine learning systems, thereby influencing privacy risks.

Causes and Effectiveness of Privacy Leaks

The paper explores factors contributing to privacy leaks in machine learning models:

• Overfitting: Poor generalization is frequently linked to membership inference vulnerabilities, as models tend to leak information when they memorize specific training instances.
• Model Complexity: Complex models, particularly neural networks with numerous parameters, can be more susceptible to privacy attacks.
• Adversarial Training: Techniques designed to enhance model robustness against adversarial attacks may inadvertently increase susceptibility to privacy breaches due to heightened sensitivity to inputs.

Attack Implementations

The research comprehensively covers the methodologies employed across different privacy attacks against centralized and distributed learning settings. Techniques such as shadow training, generative modeling with GANs, and adaptive querying strategies are highlighted as prevalent methods for launching effective attacks.

Defenses and Countermeasures

The survey discusses various defensive strategies employed to mitigate privacy attacks:

• Differential Privacy (DP): DP involves adding controlled noise to the outputs or gradients during training, providing formal privacy guarantees. However, the trade-off between privacy and utility remains a critical challenge.
• Model Regularization: Techniques like dropout and model stacking aim to reduce overfitting, thereby lowering the likelihood of membership inference attacks.
• Output Manipulation: Restricting access to prediction vectors or modifying outputs to limit information leakage serves as a practical defense against certain attacks.

Conclusion and Future Directions

"A Survey of Privacy Attacks in Machine Learning" is an exhaustive resource that not only categorizes existing privacy attacks but also identifies the pressing challenges in defending against them. The research highlights the need for further theoretical exploration to understand why these attacks are effective and how they scale in real-world scenarios. Additionally, the survey calls for expanding research focus beyond supervised learning, recognizing the potential vulnerabilities in other paradigms and algorithms. Ultimately, this paper serves as both an informative guide for current privacy concerns and a catalyst for future research in safeguarding machine learning systems.