An Embarrassingly Simple Detector for Model Extraction Attacks in Large Language Model API Traffic

Abstract: LLMs are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. However, individual extraction queries often resemble benign requests, and existing evaluations often focus on single-query anomaly scoring or pure benign-versus-attacker user settings. We formulate model extraction monitoring as benign-calibrated traffic-window distribution testing and show that an embarrassingly simple detector is effective: embed incoming queries into a semantic space and test whether their aggregate distribution deviates from historical benign traffic. We instantiate the detector with maximum mean discrepancy (MMD), using only benign-vs-benign comparisons to set the decision threshold. We evaluate on fourteen attacker-normal query pairs from four extraction scenarios and compare with adapted PRADA, SEAT, CAP, DATE, and marginal Mahalanobis baselines. Across three random seeds, MMD achieves 0.3% benign FPR, 100.0% pure-attacker TPR, 90.5% average TPR over attacker fractions, and 95.1% balanced accuracy. These results show that benign-calibrated distribution testing is a strong empirical baseline for model extraction detection in both user-level and mixed multi-user LLM API traffic. Code is released at: https://github.com/LabRAI/mmd-LLM-mea-detection.

• The paper introduces a benign-calibrated maximum mean discrepancy test over query embeddings that detects distribution shifts in aggregate API traffic without attack labels or account-level assumptions.
• The detector achieves 0.3% benign false-positive rate, 100% detection on pure attacker traffic, 90.5% average detection across attacker fractions, and 95.1% balanced accuracy across 14 attack-normal pairs.
• The results show that larger traffic windows improve diluted-attack detection, while low-contamination and adaptive attacks remain challenging; original per-query detector protocols also transfer poorly to LLM API traffic.

Motivation and problem formulation

The paper addresses the detection of model extraction attacks against LLMs deployed behind hosted APIs. Its starting point is a mismatch between how extraction detectors are typically evaluated and how attacks actually manifest in production traffic. Individual extraction queries are drawn from natural text sources—Wikipedia-derived sentences, SQuAD-style prompts, medical knowledge questions—and therefore look benign in isolation. Existing detectors such as PRADA, SEAT, CAP, and DATE are evaluated at the user or account level, where a benign account issues only legitimate queries and an attacker account runs a complete extraction workflow, or they score queries independently as text anomalies. Neither setting captures the deployment scenario the authors target: aggregate API traffic windows that mix many users, with attacker queries constituting only a small fraction of each window.

The paper formulates model extraction monitoring as benign-calibrated traffic-window distribution testing. The defender holds only a historical benign query set $B$ sampled from $P_b$; at test time, an incoming batch $T$ is modeled as a contaminated distribution $(1-\rho)P_b + \rho P_a$, where $\rho$ is the attacker fraction, ranging from $\rho=1$ (pure attacker traffic) to small values (diluted or distributed extraction). The goal is to maximize detection power subject to a false positive constraint $\Pr[h(T;B)=1 \mid T\sim P_b] \le \alpha$. This formulation deliberately avoids assumptions about the attacker's generation process, labeled attack data, or account-level stream structure.

Method

The detector is intentionally minimal. Queries are embedded with a fixed sentence encoder (BAAI/bge-small-en-v1.5 by default), and each incoming traffic window is compared against benign reference batches using maximum mean discrepancy (MMD), an unbiased kernel two-sample statistic computed with a multi-kernel RBF kernel whose bandwidths are set via the median heuristic scaled by $\{0.5, 1, 2, 4\}$. To reduce variance from any single reference sample, the final window score averages MMD over 20 sampled benign reference batches.

Threshold calibration uses only benign-vs-benign comparisons: 1,000 pairs of benign batches yield a null distribution $\mathcal{S}_0$, and the threshold is its 95th percentile. No attack labels, no task-specific encoders, and no self-supervised anomaly models are required. The output is a batch-level flag rather than per-query labels, matching the operational reality that extraction is visible only through repeated querying.

Evaluation setup

The evaluation covers fourteen attacker-normal query pairs spanning four extraction families: Query-Efficient-Med, Model-Leeching, MeaeQ, and BERT-based API extraction (2606.05725). Attacker queries derive from medical domain exploration, SQuAD-style templates, or WikiText-103; normal queries come from WildChat, SQuAD, GLUE, BoolQ, AG News, Hate Speech, SST-2, and IMDB. Traffic windows contain 1,500 queries; results average three random seeds. Five baselines—PRADA, SEAT, CAP, DATE, and marginal Mahalanobis distance—are adapted to the same embedding space and benign-calibration protocol so that all methods share identical representation, batch construction, and thresholding conventions.

Main results

The headline numbers are strong. Across the fourteen pairs, MMD achieves 0.3% benign FPR, 100.0% TPR on pure attacker traffic, 90.5% average TPR across attacker fractions, and the highest balanced accuracy at 95.1%:

Method	Benign FPR	5% TPR	10% TPR	25% TPR	Avg. TPR	Balanced Acc.
MMD	0.3	59.0	93.7	100.0	90.5	95.1
Mahalanobis	14.2	70.5	83.2	91.9	88.9	87.4
DATE	12.5	39.7	56.7	86.1	75.4	81.4
SEAT	13.6	31.7	64.5	93.2	77.5	82.0
CAP	16.8	26.4	34.8	61.9	60.9	72.0
PRADA	7.3	18.9	25.4	58.2	49.3	71.0

The trade-off structure among baselines is informative. Marginal Mahalanobis achieves the best 5%-fraction TPR (70.5%) but pays for it with 14.2% benign FPR—an unacceptable rate for monitoring systems where alarm volume drives analyst burden. DATE and SEAT detect nearly all pure-attacker windows but degrade sharply under dilution. PRADA has moderate FPR but the weakest detection overall. No adapted baseline simultaneously matches MMD's near-zero false positives, high TPR, and best balanced accuracy. The practical implication is that aggregate distributional deviation, calibrated on benign data alone, dominates per-query anomaly scoring for this problem.

Difficulty is not uniform across attack families: template-based settings such as Model-Leeching and Query-Efficient-Med are easy, while WikiText-derived BERT-API queries paired with natural task inputs are hardest at low attacker fractions because attacker and benign embeddings overlap semantically. On BERT-API-SQuAD-Wiki, for example, MMD's 5% TPR drops to 4.0%, though 10% TPR remains 60.7%.

Original protocols do not transfer

A notable negative result concerns RQ4: when baseline methods are run under their original protocols without adaptation, they fail substantially. PRADA's original nearest-neighbor normality check raises alarms on 51.6% of benign streams while detecting only 49.0% of attacker streams—a near-inversion of its reported image-domain performance. SEAT's similar-pair protocol yields 14.3% benign FPR and only 15.0% attacker TPR. DATE drops to 49.5 AUROC (below chance) and Mahalanobis to 70.1 AUROC on individual extraction queries, far below their original anomaly/OOD benchmarks. The authors conclude that model extraction queries are neither generic text anomalies nor ordinary OOD utterances, which justifies their unified adaptation protocol. This finding also implies that comparisons of extraction detectors across papers using different protocols may be misleading.

Sensitivity analysis

The main configuration choices are shown not to be arbitrary. Larger traffic windows dramatically improve low-fraction detection: at 5% attacker fraction, TPR rises from 1.2% with 100-query windows to 74.4% with 1,500-query windows, while FPR stays between 0.0% and 1.2%. The 95th-percentile threshold provides a strong operating point; relaxing to the 90th percentile buys 83.6% TPR at 5% but costs 1.6% FPR. Detection at 25% contamination and above is essentially insensitive to these choices.

Two further findings strengthen the design. First, two-sided decision rules matter for adapted baselines: allowing deviations from benign calibration in either direction improves average TPR by 22.8–44.3 points for PRADA, SEAT, CAP, and DATE, indicating that text-query extraction traffic can shift the benign distribution in either direction. Second, the detector is robust to encoder choice: five sentence encoders all maintain low FPR and roughly 90% average TPR, with e5-base-v2 improving balanced accuracy to 96.7%. Runtime is competitive—the full configuration takes 395.7 seconds end-to-end on one pair, comparable to CAP, and a lightweight configuration (5 reference repeats, 250 null samples) reduces this by 11.4% with negligible accuracy loss.

Limitations and open questions

The paper concedes several boundaries explicitly. Low-contamination mixed traffic remains the principal weakness: at 5% attacker fraction, especially when extraction queries are semantically close to benign task inputs (e.g., WikiText-derived BERT-API queries paired with GLUE or SQuAD data), detection degrades considerably. More fundamentally, the mixed-traffic evaluation models cover-query insertion and slow distributed extraction, but does not evaluate adaptive attackers who explicitly optimize queries to match the benign embedding distribution—paraphrasing, style transfer, or semantic-preserving rewriting could plausibly shrink the distributional gap that MMD measures. Whether the detector survives such adversarial optimization is left open.

The evaluation scope is also bounded: the detector uses only query text in an offline window setting, ignoring timestamps, account metadata, response content, and cross-window accumulation, all of which real deployments could exploit. Coverage spans four extraction scenarios and fourteen pairs but not all domains, languages, multi-turn interactions, tool-use queries, or production traffic patterns. A specific open question is whether temporal and account-aware extensions can push reliable detection below the 10% attacker-fraction regime where current performance becomes uneven.

Conclusion

This paper demonstrates that a benign-calibrated MMD distribution test over semantic query embeddings is a strong empirical baseline for detecting model extraction in LLM API traffic, achieving 0.3% benign FPR, 100% pure-attacker TPR, and 95.1% balanced accuracy across fourteen attacker-normal pairs, including mixed multi-user windows down to 10% contamination. Equally valuable are its negative findings: original protocols of established extraction detectors do not transfer to semantic text-query traffic, and per-query anomaly scoring misses weak aggregate shifts entirely. The work's honest limitations—vulnerability at very low attacker fractions and untested robustness against embedding-aware adaptive attackers—define the concrete questions that subsequent research on LLM extraction monitoring must answer.