Open-Set Source Tracing as Compositional Factors via Structured Prototypes

Abstract: Recent research expands beyond binary anti-spoofing with the emergence of Source Tracing, the task of identifying the specific generative origins of synthetic speech. However, current research often equates a "source" with its generative architecture. We propose redefining a source as a compositional tuple of Architecture, Training Data, and other training factors affecting the generated speech. We propose a framework using Structured Orthonormal Prototypes to minimize class overlap and intra-class variance. Our Subspace Partitioning strategy splits the embedding into architecture and data subspaces, while a residual subspace captures stochastic variability, enabling "compositional generalization" for novel factor combinations. This approach improves performance for partially seen sources and maintains robustness in fully open-set scenarios. MLAAD evaluations for Few-Shot open-set Identification show our approach significantly outperforms angular-margin baselines.

• The paper introduces a compositional framework for tracing synthetic speech sources by decomposing generative identity into architecture, training data, and residual factors.
• It presents a prototype-based metric learning approach using orthonormal and factorized subspaces to enhance few-shot and open-set attribution performance.
• Experimental results demonstrate improved generalization with reduced F1-macro gaps and provide forensic transparency through interpretable subspace alignment.

Open-Set Source Tracing as Compositional Factors via Structured Prototypes

Introduction and Motivation

Source tracing in synthetic speech forensics has matured beyond the binary anti-spoofing paradigm, which is becoming insufficient given the diversity of neural speech synthesizers. Prevailing approaches reduce the notion of "source" to its generative architecture, which is inadequate for practical forensic attribution, especially as generative modeling becomes increasingly parameterized (e.g., different training corpora, fine-tuning, or optimization regimes). This paper introduces a compositional framework for source tracing, explicitly decomposing the generative source identity into architecture ($\mathcal{A}$), training data ($\mathcal{D}$), and a residual component ($\mathcal{H}$) capturing training configuration and unobserved factors. It then proposes structured prototype-based metric learning and subspace partitioning strategies to improve compositional generalization, particularly in open-set and few-shot attribution scenarios.

Figure 1: Variability in synthetic speech attribution, illustrating how sources share architecture, data, and residual configurations, with attribution targeting inter-source differences while compensating for intra-source variability.

Compositional Redefinition of Source Identity

The formalization of the synthetic speech source as a tuple $$\mathcal{S} = (\mathcal{A}, \mathcal{D}, \mathcal{H})$$ addresses a central limitation of previous methods: the entanglement of architectural and data-driven traits in metric learning systems. Standard Deep Metric Learning via contrastive or angular margin losses, as well as monolithic end-to-end classification, compress this variation into a single axis, precluding compositional reasoning about novel combinations. This work reframes attribution as a factorial task: architectural and data-driven factors are steered into orthogonal subspaces, while a residual manifold absorbs unmodeled stochasticity (such as random seeds, optimization details, or unlabelled inference-time variability).

Prototype-Based Structured Metric Learning Framework

Three strategies are developed to structure the embedding space produced by a ResNet18 over log-mel spectrograms:

1. Angular-Margin Baseline (ArcFace): The traditional approach maximizes the geodesic margin across seen sources but imposes no explicit geometric structure linking sources that share architecture or data lineage.
2. Orthonormal Prototypes: Embedding targets are fixed, maximally separated, and orthonormal in the metric space, theoretically improving inter-class discrimination and open-set generalization by avoiding overlap in representation across sources.
3. Subspace Partitioning (Factorized Prototypes and Residual Modeling): The most advanced strategy factorizes the embedding into $\mathcal{Z}_{\mathcal{A}}$ (architecture), $\mathcal{Z}_{\mathcal{D}}$ (data), and $\mathcal{Z}_{\mathcal{R}}$ (residual). Explicit prototype vectors for architecture and dataset are concatenated to build source prototypes; the residual manifold is regularized to maintain bounded energy without vector collapse or dominance.

This approach allows construction of source prototypes in compositional fashion, supporting generalization to previously unobserved combinations of architectures and datasets without requiring exhaustive retraining as the landscape of generative models expands.

Figure 2: Overview of the proposed source tracing framework, contrasting monolithic ArcFace supervision (top) with subspace-partitioned factorization (bottom), where embeddings are split into architecture, data, and residual subspaces aligned with respective prototypes.

Experimental Setting

Experiments are conducted on the Multi-Language Audio Anti-Spoofing Dataset (MLAAD) with a carefully controlled partition ensuring multiple open-set evaluation scenarios:

• Closed-set (IID): Seen architecture, seen data.
• Compositional (OOD): Unseen combination of seen architecture and seen data.
• Partial Open (OOD): Either architecture or dataset unseen.
• Fully Open (OOD): Both architecture and dataset are unseen at train time.

Few-shot protocols are employed to simulate realistic forensic investigation scenarios, in which K support utterances per source are provided for dynamic prototype construction.

Results: Few-Shot Attribution and Generalization

Closed-set performance of all methods is consistently high ($>$94% F1-macro). The critical evaluation is open-set, especially on unseen sources and combinations. Structured prototypes yield clear improvements in few-shot OOD conditions:

• Orthonormal Prototypes increase All-vs-All F1-macro by up to 2.6 pp compared to ArcFace at $K=5$, but the main gain is reduced generalization gap (from 13.70 pp to 12.07 pp).
• Factorized Subspaces (Strategy 2.A and 2.B) outperform both baselines, reducing the generalization gap further (down to 8.75–9.19 pp for $\lambda=0$, surpassing 80% F1-macro in all OOD conditions at $\mathcal{D}$0).
• Residual Modeling is particularly effective for challenging generalization cases where inter-factor interactions or unmodeled variability dominate.

Importantly, excessively strong MSE regularization harms generalization by over-constraining the latent space, while a modest $\mathcal{D}$1 (0.05–0.1) provides the best balance. Forensic analysis shows that architectural fingerprints generalize better than data-driven ones; this is observed in asymmetric results between unseen architecture vs. unseen dataset conditions.

Figure 3: F1-macro scores for few-shot source attribution at $\mathcal{D}$2 supports, decomposed across generalization scenarios and methods. The best configuration per strategy is selected via validation.

Theoretical and Practical Implications

Factorial source modeling via structured prototypes supports robust compositional generalization, enabling scaling as the number and diversity of generative models increases. The explicit alignment of architecture and data subspaces provides interpretability: analysts can assess whether a given attribution was driven by architectural cues, data artifacts, or residual properties, which is directly relevant for forensic transparency and legal proceedings. The approach also demonstrates that deep metric learning systems need explicit geometric priors to avoid overfitting to monolithic label indices, especially for open-set attribution.

Furthermore, the method's prototype factorization—requiring only $\mathcal{D}$3 and $\mathcal{D}$4 basis vectors—scales efficiently compared to orthonormal approaches, particularly as catalogs of generative models and datasets grow.

Future Directions

A direction for extension is explicit modeling of inference-time variables ($\mathcal{D}$5), such as speaker identity or prompt, which are currently subsumed into the residual manifold. Explicit compensation for these via further subspace expansions or disentanglement techniques (e.g., variational autoencoders, multi-domain translation) could further improve performance and interpretability. Another avenue is the application to other media, such as LLMs [suzuki2025naturalfingerprintslargelanguage] or image generators [vonderfecht2022fingerprints], where compositional source modeling may similarly improve attribution and forensic analysis.

Conclusion

This paper introduces a prototype-based, compositional factorization framework for synthetic speech source tracing that decomposes source identity into architecture, training data, and residual subspaces. Structured prototypes, especially when coupled with subspace partitioning, produce significant improvements in open-set, few-shot attribution, minimizing the generalization gap and enabling tractable scaling as generative AI diversifies. The factorial approach not only improves accuracy but also enhances interpretability and extensibility, positioning it as a robust foundation for next-generation forensic attribution systems.