Model Stealing Through the Lens of Model Multiplicity

Abstract: Model stealing attacks, where adversaries create high-fidelity surrogate models, are a significant threat to the intellectual property of machine learning services. Conventional wisdom suggests these surrogates could provide adversaries with economic leverage comparable to the original service providers. This paper challenges this assumption by evaluating model stealing attacks beyond mere fidelity to the target model. Because query-based extraction provides only partial supervision of the target's input-output behavior, the surrogate is not uniquely identified: many near-optimal surrogates can achieve comparable fidelity while differing in deployment-relevant properties. Instead of performing a classic learning-based model stealing attack, we compute the Rashomon Set (i.e., the set of almost-equally-accurate models) of surrogate models, and evaluate its diversity using multiplicity metrics (ambiguity, discrepancy, and Rashomon Capacity) and group fairness metrics. Across tabular, medical imaging, and NLP tasks, our experiments on real-world datasets reveal that despite exhibiting similar fidelity to the target model, surrogate models can display significant variances in other critical performance metrics. These findings cast doubt on the presumed equivalence between high-fidelity surrogates and the target model in practical deployment scenarios.

• The paper reexamines model theft by showing that high fidelity does not ensure functional equivalence, as surrogates form a diverse Rashomon set.
• It introduces dropout-based sampling to approximate multiplicity, revealing significant variability in predictions and fairness outcomes.
• The study demonstrates that reliance on fidelity alone can obscure critical behavioral differences, questioning standard risk assessments in MLaaS.

Model Stealing Through the Lens of Model Multiplicity

Introduction and Motivation

The paper "Model Stealing Through the Lens of Model Multiplicity" (2606.15493) presents a reevaluation of the prevailing narrative around model stealing attacks, which typically equate high-fidelity surrogates to faithful operational replicas of the victim model. The core argument is that fidelity—defined as label-wise agreement with the target—is an insufficient metric to assert behavioral equivalence between a stolen model and its source. Rather, query-based model extraction yields only partial supervision, resulting in a Rashomon set: a diverse family of near-equivalent surrogates displaying multiplicity in deployment-relevant properties, including sample-level predictions and fairness metrics.

Figure 1: Overview of extraction-induced multiplicity, illustrating the adversary’s pipeline from query to surrogate and the subsequent evaluation of the induced Rashomon set via fidelity, multiplicity, and fairness criteria.

This work departs from the conventional single-surrogate analysis by empirically characterizing the multiplicity within the set of models neighbors to the surrogate extracted under common attack pipelines. The authors introduce dropout-based sampling for efficient Rashomon set approximation and explore the resulting diversity in predictions and fairness profiles across tabular, medical imaging, and NLP tasks.

Theoretical Framework: Rashomon Sets and Multiplicity Metrics

The Rashomon set $S_{\epsilon}(h_0)$ is formally defined as the subset of a hypothesis space $H$ containing all models whose empirical risk is within $\epsilon$ of the reference model $h_0$. Multiplicity arises when the extraction process, due to the limited visibility into the target’s function, leaves the surrogate only partially identified, resulting in a set of hypotheses with similar observed fidelity but differing in downstream predictions and fairness outcomes.

Three core multiplicity metrics are operationalized:

• Ambiguity: The average instance-level disagreement within the Rashomon set relative to a reference.
• Discrepancy: The maximum possible rate of disagreement between the reference and any Rashomon set member.
• Rashomon Capacity: The dispersion of soft predictive scores, indicative of local variability in confidence estimates among near-optimal models.

Additionally, fairness is measured using statistical parity, predictive equality, equal opportunity, and equalized odds across major demographic partitions.

Experimental Design

Extraction Attacks

Two widely used black-box extraction attacks are compared: Knockoff Nets [Orekondy_2019_CVPR], leveraging adaptive query selection, and standard prediction-API stealing [tramer2016stealing], which uses auxiliary data to train posteriors matching the target model.

Rashomon Set Approximation

To address computational intractability, the Rashomon set is approximated via dropout sampling at inference time, drawing thousands of near-equivalent surrogates for each extracted model. This is performed across a dropout grid to manipulate the induced diversity.

Datasets and Architectures

• Tabular: U.S. Census tasks (ACSIncome, ACSMobility, ACSEmploy) with one-hidden-layer neural networks. Race is the sensitive attribute for fairness quantification.
• Medical Imaging: MedMNIST datasets using ResNet-50 architectures.
• NLP: Financial Sentiment Task (FST) on BERT classifiers.

Main Results

Fidelity is Unstable as an Equivalence Proxy

High-fidelity surrogates (agreement $\geq0.91$ on tabular and $\geq0.96$ on PneumoniaMNIST, FST) coexist with substantial multiplicity as measured by ambiguity and Rashomon Capacity.

Figure 2: Fidelity results indicate that dropout-based Rashomon sets maintain high aggregate agreement across attacks and datasets.

Figure 3: Ambiguity and discrepancy metrics reveal substantial instance-level disagreement inside high-fidelity Rashomon sets, even at low dropout.

Tabular Domain

• Ambiguity within Rashomon sets reaches up to $1.0$ (i.e., all individuals see conflicting predictions among surrogates) for ACSMobility.
• Even in the low dropout regime (fidelity $>0.9$), ambiguity exceeds $0.4$ in most cases and up to $0.74$ for ACSIncome.
• Maximum discrepancy for surrogate-versus-reference reaches $H$0 but is typically lower for ACSMobility.

Medical Imaging

Figure 4: Surrogates in breast cancer and pneumonia detection tasks exhibit stable aggregate fidelity under dropout-based sampling.

Figure 5: Multiplicity metrics for medical imaging datasets indicate that ambiguity regularly exceeds $H$1, despite high fidelity.

• PneumoniaMNIST and OCTMNIST display persistent ambiguity ($H$2–$H$3 at low dropout), disproving the hypothesis that fidelity guarantees functional equivalence in high-dimensional domains.
• Rashomon Capacity likewise increases with dropout, reflecting instability in the predicted softmax distributions.

NLP

Figure 6: Dropout-induced Rashomon sets for the NLP task show high ambiguity and Rashomon Capacity growth even at high fidelity.

• BERT-based FST surrogates preserved fidelity $H$4, yet ambiguity surpassed $H$5 and reached $H$6 in the regime of larger dropout.

Fairness Variability Among High-Fidelity Surrogates

Figure 7: Group fairness metrics for surrogates highlight significant variance across the Rashomon set in tabular tasks, even at fixed fidelity.

• For ACSIncome and ACSEmploy, fairness metrics (e.g., equal opportunity, statistical parity differences) shift substantially across sampled surrogates, implying that a model thief cannot guarantee preservation of the victim’s fairness traits even when overall fidelity is matched.

Impact and Contradictory Claims

A central, explicit claim supported by comprehensive empirical evidence is that high-fidelity extraction attacks do not yield functionally unique surrogates; instead, a large space of behaviorally diverse models is consistent with limited query supervision. Aggregate performance metrics systematically obscure this diversity, affecting both individual-level and group-level deployment outcomes.

Practical and Theoretical Implications

This work undermines prevailing assumptions in security/privacy risk assessment for MLaaS, casting doubt on fidelity as a sufficient criterion for model equivalence. For service providers, the implication is that intellectual property risks from model theft are more nuanced than previously assumed: attackers can reproduce statistical performance but not necessarily critical operational or legal characteristics, such as subgroup calibration or bounded individual recourse.

From a theoretical perspective, the results further generalize Rashomon-based phenomena, demonstrating that extraction-induced under-specification emerges in neural, convolutional, and transformer models alike. This highlights the need for measuring risk and policy effects in downstream deployment not by single-model analysis, but by quantifying the variability inherent to the surrogate's local hypothesis class.

Future Directions

• Defense Design: Mitigations for model stealing should account for the fact that query obfuscation or watermarking may not only reduce fidelity but could amplify Rashomon set diversity, complicating detection and recourse.
• Attack Evaluation: Risk audits for extraction attacks must include multiplicity and fairness metrics, not only fidelity.
• Multiplicity Beyond Classification: Extensions to regression, structured prediction, and explanation-level multiplicity (e.g., SHAP, LIME) are relevant.
• Heterogeneous Architectures: Preliminary results are for surrogate-victim pairs with architectural matching; multiplicity is likely to increase with more realistic, mismatched architectural settings.

Conclusion

The authors provide a rigorous challenge to the equivalence of high-fidelity model stealing and behavioral replication, demonstrating that extraction is fundamentally an identification problem under multiplicity. Effective risk assessment and defense strategies in MLaaS must transition from fidelity-centric views toward explicit exploration of extraction-induced Rashomon sets, employing multiplicity and fairness metrics to characterize the operational non-uniqueness of stolen models. These findings anchor model stealing in the wider theoretical context of multiplicity and underspecification, opening avenues for more robust evaluation and control of intellectual property risks in deployed learning systems.