Anchoring the Unknown: Open-Set Model Attribution via Proxy-Anchor Learning

Abstract: The proliferation of text-to-speech (TTS) systems capable of generating realistic synthetic speech poses growing challenges for audio forensics. While binary deepfake detection has received considerable attention, source tracing (i.e., identifying which TTS system produced a given audio sample) remains underexplored, particularly in open-set scenarios where unknown systems may be encountered. We propose a metric learning framework based on the Proxy-Anchor loss function that operates on Wav2Vec2-BERT embeddings to learn a discriminative embedding space for TTS source attribution and out-of-distribution (OOD) detection of unseen systems. We evaluate it on the MLAAD v9 dataset spanning 140 TTS systems across 51 languages, and introduce an architecture merging strategy that groups TTS system versions into unified classes, reducing inter-class confusion. Our system achieves 99.76% accuracy on 110 in-distribution classes and a False Positive Rate (FPR@95) as low as 2.04% for OOD detection. Also, for a fair comparison against the current state of the art, we further evaluate it on the MLAAD v5 official dataset splits, improving the OOD accuracy by almost doubling it. These results demonstrate that Proxy-Anchor metric learning, combined with architecture-aware class design and post-hoc OOD scoring, provides an effective framework for forensic TTS source tracing in both closed-set and open-set settings.

• The paper introduces a TTS attribution system combining frozen Wav2Vec2-BERT embeddings, Proxy-Anchor metric learning, architecture-aware class merging, and calibrated out-of-distribution scoring.
• Merging related TTS versions improved closed-set accuracy from 98.23% to 99.76% and reduced OOD FPR@95 from 9.59% to 2.04% on MLAAD v9.
• On MLAAD v5, the method achieved 98.57% closed-set accuracy and 89.20% OOD accuracy, substantially outperforming prior reported OOD accuracy of 44.82%.

This paper addresses open-set source tracing of text-to-speech (TTS) systems: given a synthetic utterance, determine which generative system produced it, while also flagging samples from systems never seen during training. The proposed approach couples frozen Wav2Vec2-BERT embeddings with a Proxy-Anchor metric learning head and post-hoc out-of-distribution (OOD) scoring, and is evaluated on MLAAD v9 (140 TTS systems, 51 languages) and the MLAAD v5 official splits used by prior work (2606.10758). Reported results include 99.76% closed-set attribution accuracy over 110 merged architecture classes, an OOD FPR@95 of 2.04%, and, on MLAAD v5, an OOD accuracy of 89.20% that nearly doubles the 44.82% of Kulkarni et al. (2606.10758).

Problem formulation and motivation

Source tracing is framed as a $K$-way classification problem over TTS systems, augmented with a rejection stage for unseen generators. The authors argue that binary deepfake detection is insufficient for forensic analysis, and that existing open-set evaluations typically hold out only a small number of unseen systems. A further practical complication motivates one of the paper's central design choices: many TTS systems differ only in model size, version, or training data (e.g., Llasa-1B vs. Llasa-3B), producing high inter-class confusion when treated as fully independent classes.

Method

The system uses the facebook/w2v-bert-2.0 self-supervised model, extracting frame-level features from the fourth of 24 Conformer layers — following prior evidence that lower SSL layers are more discriminative for deepfake tasks — and aggregates them via temporal mean pooling into a 1024-dimensional utterance embedding. A trainable linear projection head maps these to a 1024-dimensional space with L2 normalization onto the unit hypersphere. Each class is represented by a learnable proxy, and the model is trained with the Proxy-Anchor loss (Kim et al., 2020), with margin $\delta = 0.1$ and scale $\alpha = 32$, using AdamW, batch size 256, and 100 epochs. Attribution at inference assigns the class of the proxy with maximum cosine similarity.

OOD detection is performed with three post-hoc scoring functions over the proxy similarities: softmax energy (a log-sum-exp aggregation of softmax-normalized similarities), Shannon entropy of the softmax-normalized similarities, and maximum proxy distance (cosine distance to the nearest proxy). The decision threshold $\tau$ is calibrated so that the true positive rate is 95% on a held-out OOD calibration set of 10 unseen systems.

Architecture-level class merging

The paper introduces a manually constructed merging strategy that groups TTS system versions sharing the same underlying architecture, reducing 140 systems to 130 classes and the ID training set from 120 to 110 classes. The effect is consistent and substantial: closed-set accuracy improves from 98.23% to 99.76% for Proxy-Anchor (+1.53 points), and OOD FPR@95 improves from 9.59% to 2.04% — a factor-of-4.7 reduction. The mechanism is that proxies for merged architectures cluster more tightly, increasing the separation between ID and OOD samples in similarity space. This is a strong, practically relevant claim: the choice of class granularity, not just the learning objective, drives much of the open-set performance.

Closed-set and OOD results on MLAAD v9

On the closed-set task, Proxy-Anchor (98.23% / 99.76% for Experiments 1 and 2) performs nearly identically to logistic regression on the same frozen embeddings (98.16% / 99.59%) and clearly above $k$-NN with $k=21$ (92.58% / 95.15%). The authors state plainly that this near-parity implies the frozen Wav2Vec2-BERT features are already highly linearly separable; the benefit of metric learning appears almost exclusively in the OOD stage. This is an honest and important concession: the Proxy-Anchor loss is not improving ID discrimination so much as shaping the embedding geometry for uncertainty estimation.

OOD results bear this out. With 120 individual classes, entropy scoring is best (FPR@95 of 9.59%, AUROC 97.98%); with 110 merged classes, maximum proxy distance is best (FPR@95 of 2.04%, AUROC 99.35%), with softmax energy close behind (3.15%). In cross-method comparison using each system's best OOD scorer, the FPR@95 advantage of Proxy-Anchor over logistic regression is 4.1 points in Experiment 1 and 14.5 points in Experiment 2, while the $k$-NN baseline is essentially non-discriminative for OOD detection (FPR@95 of 54.63% and 66.90%), indicating that raw embedding distances without a learned projection are inadequate for open-set rejection. A t-SNE visualization of raw versus projected embeddings supports the quantitative claim that the Proxy-Anchor space separates ID from OOD samples.

Comparison with the state of the art on MLAAD v5

To enable direct comparison, the system is re-evaluated on the MLAAD v5 official splits. The headline numbers are strong: closed-set accuracy of 98.57% versus 95.61% (Kulkarni et al.) and 95.80% (Klein et al.); OOD accuracy of 89.20% versus 44.82% (Kulkarni et al., with Klein et al. not reporting this metric); and FPR@95 of 3.36% versus 8.30% (Klein et al.), a 60% relative reduction. Notably, the paper claims theirs is the only method evaluated that performs strongly on both closed-set attribution and open-set rejection simultaneously, framing the two objectives as complementary rather than competing within this framework.

Limitations and open questions

Several caveats are stated or implicit. The architecture merging is performed manually based on model versions, which does not scale automatically and depends on accurate external knowledge of system lineage; a principled or learned merging criterion is left open. The near-identical closed-set performance of Proxy-Anchor and logistic regression suggests the metric learning contribution is confined to OOD geometry, and the paper does not ablate whether simpler contrastive losses (e.g., N-pair, supervised contrastive) would achieve the same OOD gains. The OOD threshold is calibrated per deployment on a held-out set of unseen systems, and the paper does not address how calibration transfers to genuinely novel architectures that differ substantially from both training and calibration systems. Robustness to post-processing, codec artifacts, reverberation, and adversarial perturbation — conditions studied in related work — is explicitly deferred to future work, as is few-shot adaptation to newly emerging TTS systems. All evaluation is on synthetic speech from MLAAD; no bona-fide (real) speech rejection is evaluated, so the system assumes a prior binary detection stage.

Conclusion

The paper presents a compact and effective recipe for open-set TTS source tracing: frozen lower-layer Wav2Vec2-BERT features, a Proxy-Anchor projection head, architecture-aware class merging, and calibrated post-hoc OOD scoring. Its principal empirical contributions are the near-doubling of OOD accuracy on MLAAD v5 relative to prior work, the 2.04% FPR@95 on MLAAD v9 with merged classes, and the demonstration that closed-set attribution and open-set rejection can be jointly optimized in a single embedding space. The main open questions concern automated class-granularity selection, calibration transfer to truly novel architectures, and robustness under realistic channel and adversarial conditions.