Stealing Machine Learning Models via Prediction APIs (1609.02943v2)

Abstract: Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some allow users to train models on potentially sensitive data and charge others for access on a pay-per-query basis. The tension between model confidentiality and public access motivates our investigation of model extraction attacks. In such attacks, an adversary with black-box access, but no prior knowledge of an ML model's parameters or training data, aims to duplicate the functionality of (i.e., "steal") the model. Unlike in classical learning theory settings, ML-as-a-service offerings may accept partial feature vectors as inputs and include confidence values with predictions. Given these practices, we show simple, efficient attacks that extract target ML models with near-perfect fidelity for popular model classes including logistic regression, neural networks, and decision trees. We demonstrate these attacks against the online services of BigML and Amazon Machine Learning. We further show that the natural countermeasure of omitting confidence values from model outputs still admits potentially harmful model extraction attacks. Our results highlight the need for careful ML model deployment and new model extraction countermeasures.

• The paper introduces model extraction attacks on MLaaS by replicating models solely using black-box prediction APIs.
• Researchers apply equation-solving and path-finding techniques, achieving near-perfect accuracy with minimal queries.
• Findings highlight significant vulnerabilities, including monetization risks and training data privacy violations.

Stealing Machine Learning Models via Prediction APIs

The paper "Stealing Machine Learning Models via Prediction APIs" by Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart, investigates model extraction attacks in the context of Machine Learning-as-a-Service (MLaaS). The authors provide a comprehensive paper on how adversaries can duplicate ML models purely through black-box access to prediction APIs.

Overview

The paper focuses on the security implications of ML models that are considered confidential due to their sensitive training data, commercial value, or use in security applications. With the increasing trend of deploying ML models as part of publicly accessible services (e.g., Amazon ML, Google ML, and BigML), the tension between maintaining model confidentiality and providing public access has been a growing concern. The paper evaluates model extraction attacks, where adversaries can replicate the functionality of an ML model without knowledge of its parameters or the training data.

Attack Scenarios

Two primary attack scenarios are considered:

1. Cross-user model extraction: Attackers extract models trained by other users to evade query charges by the service provider.
2. Training-data privacy violations: Successfully extracted models can leak information about sensitive training data, aiding in attacks like model inversion.

Methodologies and Results

Equation-Solving Attacks

The authors introduce equation-solving attacks for extraction, showing their efficiency in various ML models, including logistic regression, neural networks, and decision trees:

• Binary Logistic Regression: The attack requires $d+1$ samples to recover the model parameters via classic equation-solving techniques.
• Multiclass Logistic Regression and Multilayer Perceptrons: These models form a nonlinear equation system which can be solved via logistic-loss minimization. The authors demonstrate extraction against both softmax and one-vs-rest logistic models, extending these attacks to deep neural networks.

For example, when applied to a softmax regression model trained on the Adult dataset with a target of 'Race,' the attack achieved 99.98% accuracy with computationally efficient techniques.

Decision Tree Path-Finding Attacks

The authors propose new path-finding attacks that exploit the rich outputs (confidence scores) provided by APIs. These attacks are demonstrated against decision trees on platforms like BigML. The algorithms involve:

1. Direct queries: Finding the exact path of a query in the decision tree by exploiting confidence values as pseudo-identifiers.
2. Top-down approach: Extracting the tree layer by layer using incomplete queries to identify decision splits effectively.

In experiments with eight public decision trees from BigML, the authors show an accuracy of over 99% in most cases.

Practical Attacks on ML Services

The attacks are experimentally validated against two MLaaS providers: BigML and Amazon:

• BigML: The extraction attacks on decision trees set up for monetization demonstrated over 99% accuracy using up to 4,013 queries.
• Amazon ML: Efficient extraction of logistic models with feature extraction (one-hot encoding and binning) was performed using a reverse-engineering approach, achieving near-perfect model duplication with minimal queries.

Implications and Future Directions

The authors discuss the implications of their findings, both practical and theoretical:

• Monetization risk: Model extraction undermines the business model of charging for predictions.
• Training data leakage: Extracted models can facilitate privacy violations.

Possible countermeasures examined include:

1. API Minimization: Providing only class labels rather than confidence scores significantly reduces the attack surface but does not eliminate the threat.
2. Rounding Confidences: Reducing the precision of confidence scores hampers the efficiency of equation-solving attacks but does not prevent them entirely.
3. Differential Privacy: While promising, applying differential privacy effectively to prevent model extraction remains an open challenge.
4. Ensemble Methods: Aggregating multiple models might offer resilience against extraction attacks.

The paper emphasizes the need for careful model deployment and new countermeasures to safeguard against extraction threats. Future research directions may involve more sophisticated defensive strategies and a deeper understanding of the trade-offs between model utility and security.