High Accuracy and High Fidelity Extraction of Neural Networks (1909.01838v2)

Abstract: In a model extraction attack, an adversary steals a copy of a remotely deployed machine learning model, given oracle prediction access. We taxonomize model extraction attacks around two objectives: accuracy, i.e., performing well on the underlying learning task, and fidelity, i.e., matching the predictions of the remote victim classifier on any input. To extract a high-accuracy model, we develop a learning-based attack exploiting the victim to supervise the training of an extracted model. Through analytical and empirical arguments, we then explain the inherent limitations that prevent any learning-based strategy from extracting a truly high-fidelity model---i.e., extracting a functionally-equivalent model whose predictions are identical to those of the victim model on all possible inputs. Addressing these limitations, we expand on prior work to develop the first practical functionally-equivalent extraction attack for direct extraction (i.e., without training) of a model's weights. We perform experiments both on academic datasets and a state-of-the-art image classifier trained with 1 billion proprietary images. In addition to broadening the scope of model extraction research, our work demonstrates the practicality of model extraction attacks against production-grade systems.

• The paper introduces a taxonomy that clarifies the trade-off between accuracy-oriented and fidelity-oriented extraction attacks.
• It demonstrates learning-based and functionally equivalent extraction techniques that boost query efficiency and replicate model behavior.
• The study highlights vulnerabilities in black-box models and prompts future research on robust defensive strategies.

High Accuracy and High Fidelity Extraction of Neural Networks

The paper thoroughly explores the notion of model extraction attacks, an increasingly pertinent topic in the context of machine learning models deployed in industry, where proprietary models may be susceptible to adversaries seeking to clone the models via oracle access. The authors delineate the landscape of model extraction attacks, primarily categorizing them based on two key objectives: accuracy and fidelity. Accuracy-oriented extraction aims to replicate the performance of the original model on its intended task, whereas fidelity-oriented extraction is primarily concerned with mirroring the victim model's behavior across possibly all inputs, including replicating any errors the original model might make.

Key Contributions

1. Taxonomy of Extraction Attacks: This work creates a systematic taxonomy for model extraction attacks with a particular emphasis on the interplay between accuracy and fidelity. The taxonomy underscores the inherent conflict between achieving high accuracy—whereby an extracted model is optimal in performing the original task—and high fidelity, which involves closely matching the victim model's responses, even if imperfect.
2. Learning-Based Extraction Techniques: The paper evaluates various learning-based extraction methods, where the adversarial model is trained using labels generated by querying the target model. These techniques, tested on datasets including a state-of-the-art convolutional neural network trained on one billion proprietary images, show substantial improvements in query efficiency and performance relative to existing baselines. Notably, attacks leveraging only a fraction of the available data were shown to exceed the performance of models trained directly on publicly available datasets. This highlights the practicality of theft-motivated attacks with access to just a portion of public data.
3. Functionally Equivalent Extraction: The exploration extends to pioneering a practical functionally equivalent extraction attack, where the goal is to directly extract the weights of the target model, particularly in two-layer ReLU networks. This approach leverages precise gradient differences to facilitate the delineation of network weights, requiring only input-output access to the model—a method superior in scalability and efficiency compared to prior strategies based on high-precision computations or additional model access.
4. Hybrid Strategies: Additionally, the paper assesses hybrid strategies that amalgamate direct extraction and learning-based techniques. These strategies aim to harness the advantages of both, employing learning strategies to rectify errors inherent in direct extraction methods, thereby achieving nearly perfect fidelity even in larger models.

Implications and Future Directions

The implications of these findings are substantive, both practically and theoretically. Practically, the research underscores the vulnerabilities of deployed machine learning models to both theft and reconnaissance. Models, especially those restricted by black-box access, are demonstrably susceptible to these extraction methods, indicating the need for better security protocols in model deployment such as query rate limits or noise addition to model outputs to mitigate efficiency of such attacks.

Theoretically, the work also contributes to understanding the landscape of adversarial capabilities and limitations in model extraction. Given the trade-offs and operational challenges illustrated in achieving high fidelity against complex models, there are avenues for future work exploring more robust defensive mechanisms as well as investigating the scalability of functionally equivalent extraction to deeper and more intricate architectures.

Overall, this paper marks a significant step towards a comprehensive understanding of model extraction attack strategies and their potential defenses, serving as a critical resource for researchers focused on secure AI model deployment and resilience against adversarial attacks.