MER-Inspector: Model Extraction Risk Framework
- MER-Inspector is a framework for assessing model extraction risk that combines an NTK-based theoretical metric (MRC) with victim model accuracy (VMA).
- It employs an attack-agnostic approach by leveraging worst-case threat modeling, kernel-based analysis, and a lightweight comparator to rank models.
- The framework demonstrates robust performance across various architectures and datasets, emphasizing practical insights for both pre-attack risk assessment and operational monitoring.
Model Extraction Risk Inspector (MER-Inspector) denotes a family of mechanisms for assessing or monitoring model extraction risk in machine learning services. In its primary and most formalized sense, MER-Inspector is an attack-agnostic framework for comparing how vulnerable trained models are to model extraction attacks (MEAs) by combining an NTK-based theoretical metric, Model Recovery Complexity (MRC), with an empirical metric, victim model accuracy (VMA), without committing to any specific extraction algorithm (Zhang et al., 23 Sep 2025). The term has also been used in adjacent operational settings, including benign-calibrated traffic-window detection for LLM APIs and extraction-warning monitors for decision-tree MLaaS deployments, which emphasizes that the name spans both ex ante risk assessment and online attack monitoring rather than a single implementation (Liu et al., 4 Jun 2026, Kesarwani et al., 2017).
1. Conceptual basis and problem setting
MER-Inspector, in the 2025 formulation, is designed for the problem of model function leakage: an adversary reproduces a victim model’s decision function through black-box access. The underlying concern differs from data privacy leakage such as membership inference or reconstruction. MEAs copy the victim by querying it with labels or posteriors and training a surrogate to match either ground truth, measured as attack accuracy, or the victim’s outputs, measured as fidelity (Zhang et al., 23 Sep 2025).
The central design choice is attack agnosticism. Attack-empirical risk estimates are costly, specific to the attack in vogue, and quickly outdated; moreover, model owners do not control the attacker’s data, query budget, or strategy. MER-Inspector therefore adopts a worst-case threat model defined as the union of capabilities across MEAs and seeks risk metrics that depend on the victim’s model and data rather than on a particular attack implementation (Zhang et al., 23 Sep 2025).
The formal setting is a -class classification task with training data
A deep model outputs logits, with prediction
The victim parameters are , the surrogate parameters are , and the initialization is (Zhang et al., 23 Sep 2025).
A common misconception is that MER-Inspector is itself an MEA or a benchmark suite. In the 2025 formulation it is neither: it is a framework for comparing extraction risk between trained models without executing an attack, although attack-empirical fidelity is used as ground truth to train and evaluate its comparator model (Zhang et al., 23 Sep 2025).
2. NTK-based theoretical formulation
The theoretical foundation of MER-Inspector is Neural Tangent Kernel (NTK) theory. For wide networks, the model output is linearized around initialization through the first-order Taylor expansion
where . The corresponding output change is
The NTK is defined as
0
Over training samples, the empirical NTK matrix 1 has blocks 2. In the infinite-width limit and under mild non-parallel input assumptions, 3 is positive definite and remains stationary during training (Zhang et al., 23 Sep 2025).
Under the worst-case threat model, linearized model extraction becomes a regularized kernel classification problem in the RKHS induced by 4:
5
By the representer theorem,
6
For squared loss, the unique minimizer has the closed form
7
and
8
where 9 concatenates 0 across training points, and 1 collects kernels 2 (Zhang et al., 23 Sep 2025).
The same NTK regime yields a projection form for parameter dynamics:
3
where
4
is a projection onto the span of the Jacobians at the chosen inputs. As the attacker’s sample set expands, 5 approaches identity on the subspace spanned by training gradients, so 6 tends toward 7 (Zhang et al., 23 Sep 2025).
This formulation is significant because it links MEA performance to kernel geometry and training dynamics rather than to any particular attack heuristic. A plausible implication is that the framework is meant to remain informative even when the attack surface changes, provided the NTK approximation continues to capture the relevant function-space structure.
3. Fidelity bounds, generalization, and Model Recovery Complexity
The theoretical analysis centers on fidelity, defined as the probability that victim and surrogate predictions agree:
8
Using the fidelity margin
9
the fidelity gap is
0
MER-Inspector derives a bound on this disagreement probability in terms of three components: empirical margin loss on training samples, a complexity term involving 1 and the NTK-induced quadratic form 2, and a sample-complexity term depending on 3 and 4 (Zhang et al., 23 Sep 2025).
The generalization error of the surrogate is bounded by the fidelity gap and the victim’s own error:
5
where
6
This gives an explicit route from victim properties and NTK geometry to an upper bound on surrogate generalization error (Zhang et al., 23 Sep 2025).
MER-Inspector’s principal theoretical metric is Model Recovery Complexity:
7
This quantity is the RKHS norm of the optimal kernel solution when empirical loss is zero, so it measures the “size” of the victim’s output change in the NTK-induced feature space. Using the projection dynamics, it can also be rewritten as
8
The interpretation given in the paper is operational: a small MRC implies that the surrogate can closely recover the victim’s weight change, increasing extraction risk, whereas a large MRC indicates harder recovery and thus lower risk. No additional normalization is required because the quadratic form is already an intrinsic NTK-RKHS norm (Zhang et al., 23 Sep 2025).
This is the most distinctive feature of the framework. Prior attack benchmarking typically reports fidelity or agreement after running a concrete attack; MRC instead attempts to quantify extractability from the victim model’s geometry before an attack is instantiated.
4. Integration with victim model accuracy and the comparison framework
MER-Inspector does not rely on MRC alone. The paper reports that victim model accuracy has a strong positive correlation with extraction risk, measured by fidelity. Across models and datasets, the Pearson correlation coefficient between VMA and attack fidelity reaches 9, and Kendall rank correlation is 0. At the same time, VMA saturates when models have similar accuracies, for example when accuracies cluster near 1, which limits its discriminative power for fine-grained comparisons within an architecture family (Zhang et al., 23 Sep 2025).
The framework therefore integrates a theoretical metric and an empirical metric. For each model, MER-Inspector computes a two-dimensional risk vector
2
To increase separability, a pair 3 is augmented with the difference 4, yielding the feature vector
5
A lightweight fully connected comparator is then trained on labeled pairs indicating which model has higher measured attack fidelity. The comparator has three hidden layers of sizes 6, 7, and 8, uses ReLU activations, and has a sigmoid output. Its output is a binary decision indicating whether 9 (Zhang et al., 23 Sep 2025).
The comparator is trained using attack fidelity ground truth from an MEA validation protocol in which the attacker uses the same architecture and hyperparameters as the victim, queries all training samples with full posteriors, trains the surrogate for 0 epochs, and records maximum fidelity on the test set. This empirical attack is used only to label pairs for training and evaluation, not as the deployed risk measure (Zhang et al., 23 Sep 2025).
The reported experimental scope includes five datasets—CIFAR-10, CIFAR-100, FashionMNIST, STL-10, and CelebA—and sixteen victim architectures from four groups: ResNet20, ResNet32, ResNet44; WRN22-2, WRN22-4, WRN22-8, WRN28-2, WRN34-2, WRN40-2; DenseNet121, DenseNet169, DenseNet201; and LeViT-128, LeViT-192, LeViT-256, LeViT-384 (Zhang et al., 23 Sep 2025).
The most cited performance summary is comparative accuracy (CAcc). In intra-group comparisons, MRC alone achieves 1, VMA alone 2, and combined VMA+MRC 3. In inter-group comparisons, VMA alone achieves 4, MRC alone 5, and combined VMA+MRC 6. Across all pairs, VMA+MRC reaches 7 CAcc with feature augmentation, outperforming VMA alone at 8 and MRC alone at 9; without feature augmentation, combined performance drops to 0 (Zhang et al., 23 Sep 2025).
These results motivate the framework’s basic division of labor: MRC is strongest within architecture groups, whereas VMA is strongest across more heterogeneous architecture families. This suggests that neither metric is sufficient in isolation across the full model-selection problem.
5. Practical computation, hyperparameters, and limitations
Finite-width implementation requires several engineering choices. Because the empirical NTK is not perfectly stationary at initialization in finite-width networks, the paper recommends computing 1 after training for stability. Since 2 may be nearly singular, its eigenvalues are floored at 3 before inversion, with 4 used as the default. Exact NTK computation scales as 5 and is infeasible at full scale, so MER-Inspector samples a small set of 6 inputs to construct 7 (Zhang et al., 23 Sep 2025).
Sample selection is based on prediction-margin extremes. For each sample,
8
The method selects 9 samples with largest margins as simple points and 0 with smallest margins as hard points, using the difficulty ratio 1. The approximate MRC computation then proceeds in five steps: select 2 samples by difficulty ratio 3; compute gradients and assemble the NTK; floor eigenvalues below 4 and reconstruct 5; compute the victim’s output probability change vector on the same set; and return
6
The default hyperparameters are 7, 8, and 9 (Zhang et al., 23 Sep 2025).
Sensitivity studies clarify the operational regime. Larger 0 improves MRC-fidelity correlation; for the LeViT group, PCC improves from approximately 1 at 2 to approximately 3 at 4. A balanced difficulty ratio 5 is reported as robust across groups. Hard samples carry more weight for less generalizable models such as ResNet and LeViT, whereas simple samples matter more for better-generalizing models such as WideResNet. For the NTK eigenvalue floor, 6 gives the best trade-off by ensuring most 7 are positive definite, approximately 8, while preserving fine-grained NTK information; smaller 9 or larger 0 harms performance (Zhang et al., 23 Sep 2025).
The cost advantage over explicit attack execution is substantial. Computing MRC with 1 samples for a model with approximately 2 million parameters takes approximately 3 seconds, whereas executing even a simple MEA with a surrogate trained for 4 epochs takes approximately 5 seconds (Zhang et al., 23 Sep 2025).
The framework nonetheless has explicit assumptions and failure modes. The NTK analysis assumes wide, over-parameterized networks and identical initialization; finite-width models, especially transformers such as LeViT, can deviate from NTK idealizations, weakening absolute correlations across architecture families. MRC depends on 6 and sample selection, and too few samples can under-represent the Jacobian span. VMA saturates when accuracies are similar. The fidelity and generalization bounds depend on RKHS margin analysis with margin 7, confidence 8, and the data-distribution quantity 9, so bound tightness is not uniform across settings (Zhang et al., 23 Sep 2025).
The paper also reports robustness under several practical threat variations. For an “Only Label” attacker, overall CAcc with VMA+MRC is 00; for “CIFAR100 01 CIFAR10,” overall CAcc is 02; under budgeted queries with uncertainty-based or K-center active querying, VMA+MRC achieves 03 to 04 CAcc across inter-group pairs (Zhang et al., 23 Sep 2025).
6. Related formulations, prior work, and broader uses of the name
The 2025 MER-Inspector is situated against attack-execution-based assessment. Prior risk assessments are described as relying on MEAs such as Knockoff Nets, ActiveThief, and MAZE, or on holistic attack benchmarking such as ML-DOCTOR and SecurityNet; these approaches are characterized as expensive, attack-specific, and not future-proof (Zhang et al., 23 Sep 2025). The motivation is reinforced by query-efficient attacks such as Marich, which uses public data and label-only APIs, and experimentally extracts models that achieve approximately 05–06 of the target’s accuracy using approximately 07–08 queries, while producing predictive distributions approximately 09–10 closer to the target than prior active-sampling attacks (Karmakar et al., 2023).
The name “MER-Inspector” is also used for operational detection in LLM APIs. In that setting, the objective is not architecture comparison but benign-calibrated traffic-window distribution testing. Incoming windows of queries are embedded with a fixed encoder, and a detector flags the window when its distribution differs from historical benign traffic. The instantiated method uses Maximum Mean Discrepancy with benign-only threshold calibration, Gaussian RBF kernels, multi-kernel averaging over scales 11, reference averaging with 12, and a benign null distribution built from 13 benign-only comparisons. With window size 14, the reported averages are 15 benign FPR, 16 pure-attacker TPR, 17 average TPR over attacker fractions, and 18 balanced accuracy (Liu et al., 4 Jun 2026).
A still earlier monitoring line appears in the MLaaS literature on decision trees. “Model Extraction Warning in MLaaS Paradigm” defines extraction status using either information gain or coverage of the input feature space. The information-gain monitor computes
19
while the coverage-based monitor maintains per-leaf per-feature min-max summaries and computes a weighted overall extraction status from leaf hypervolumes and class priors. The paper studies both single-user and colluding adversaries on BigML decision-tree APIs and reports monotone increases in extraction status for colluding groups, including an example on GSS Survey with approximately 20 at 21, approximately 22 at 23, and approximately 24 at 25 under Tramer attack with 26 queries per user (Kesarwani et al., 2017).
| Formulation of MER-Inspector | Core mechanism | Reported scope |
|---|---|---|
| Attack-agnostic risk comparison | NTK theory, MRC, VMA, pairwise comparator | 16 architectures, 5 datasets (Zhang et al., 23 Sep 2025) |
| LLM API traffic monitoring | Benign-calibrated MMD on query windows | 14 attacker-normal pairs, mixed multi-user traffic (Liu et al., 4 Jun 2026) |
| MLaaS extraction warning | Information gain and query-summary coverage | Decision-tree models on BigML (Kesarwani et al., 2017) |
Taken together, these uses indicate that “MER-Inspector” has become a broader label for systems that quantify extraction risk either structurally, through model-and-data geometry before an attack, or operationally, through observed query streams during an attack. The primary technical distinction is that the 2025 framework compares the relative extractability of models without running an MEA, whereas the LLM and MLaaS variants are monitors for suspicious extraction behavior in deployed services (Zhang et al., 23 Sep 2025, Liu et al., 4 Jun 2026, Kesarwani et al., 2017).