Split and Conquer Partial Deepfake Speech

Abstract: Partial deepfake speech detection requires identifying manipulated regions that may occur within short temporal portions of an otherwise bona fide utterance, making the task particularly challenging for conventional utterance-level classifiers. We propose a split-and-conquer framework that decomposes the problem into two stages: boundary detection and segment-level classification. A dedicated boundary detector first identifies temporal transition points, allowing the audio signal to be divided into segments that are expected to contain acoustically consistent content. Each resulting segment is then evaluated independently to determine whether it corresponds to bona fide or fake speech. This formulation simplifies the learning objective by explicitly separating temporal localization from authenticity assessment, allowing each component to focus on a well-defined task. To further improve robustness, we introduce a reflection-based multi-length training strategy that converts variable-duration segments into several fixed input lengths, producing diverse feature-space representations. Each stage is trained using multiple configurations with different feature extractors and augmentation strategies, and their complementary predictions are fused to obtain improved final models. Experiments on the PartialSpoof benchmark demonstrate state-of-the-art performance across multiple temporal resolutions as well as at the utterance level, with substantial improvements in the accurate detection and localization of spoofed regions. In addition, the proposed method achieves state-of-the-art performance on the Half-Truth dataset, further confirming the robustness and generalization capability of the framework.

• The paper introduces a two-stage pipeline that separates boundary detection from segment classification, enhancing detection accuracy and interpretability.
• Methodology leverages adaptive histogram-based thresholding, multi-duration training, and fusion of feature representations to mitigate label ambiguity.
• Experimental results show significant improvements in F1 scores and reduced error rates across PartialSpoof and cross-corpus benchmarks.

Split and Conquer Partial Deepfake Speech

Introduction

The growing sophistication of neural TTS, VC, and diffusion-based methods enables the generation of natural-sounding deepfake audio that challenges the integrity of voice-driven systems. While utterance-level spoof detection systems have matured, the practical adversarial model has shifted toward partial manipulations—wherein only select regions, such as keywords or authorization statements, are replaced or synthetically inpainted. Partial deepfakes present unique challenges: transitions between bona fide and manipulated segments are ambiguous, artifacts can be subtle, and global cues dilute local anomalies. Existing countermeasures—either full-utterance or frame-based—are constrained by difficulties in contextual aggregation, label ambiguity near boundaries, and optimization instability.

"Split and Conquer Partial Deepfake Speech" (2604.02913) addresses these limitations via a two-stage pipeline: boundary detection to localize candidate transition points, and segment-level classification for fine-grained authenticity assessment. This modularization enables each model to focus on a precise, well-defined objective, decoupling the issues of temporal localization and spoof detection, thus reducing cross-objective interference and increasing interpretability.

Figure 1: Overview of the Split-and-Conquer pipeline, linking input audio through boundary detection, signal segmentation, independent segment-level classification, and final frame-level detection.

Methodology

Pipeline Decomposition

The central contribution is a pipeline separating boundary localization from spoof classification, operationalized as follows:

1. Boundary Detection: A dedicated model predicts transition likelihoods at 20 ms resolution, focusing exclusively on detecting bona fide–spoof boundaries rather than frame-by-frame authenticity. The task is framed as a change detection problem, encouraging the model to learn robust transition cues with minimal ambiguity.
2. Audio Splitting: Detected transition points segment the utterance into putatively spoof-uniform regions, isolating contiguous stretches with presumed consistent acoustic class.
3. Segment-level Classification: An independent classifier operates on each segment, assigning an authenticity score. This classifier is trained on ground-truth-derived, class-uniform segments, which isolates decision units, reduces label noise, and enables the use of compact models focused strictly on local context.

This structure allows performance and robustness analysis at each stage, and clarifies the impact of boundary errors on downstream decisions.

Feature Extraction and Data Augmentation

Both stages share a high-level architecture: feature extraction front-ends (wav2vec 2.0 XLSR53/128 and log-magnitude spectrograms), followed by ResNet34 backbones. MaskedSpec (frequency-domain masking) and MaskedFeature (feature-space Gaussian masking) augmentations are applied to increase generalization, following positive results in anti-spoofing literature. For segment-level modeling, reflection-based multi-length training converts variable-duration segments into multiple fixed input lengths (1s, 2s, 4s) through time-symmetric padding, providing temporal context diversity and implicit feature-space augmentation.

Boundary Thresholding

Due to the non-stationarity of score distributions across utterances, fixed global thresholds fail to generalize. Instead, a histogram-based, utterance-dependent thresholding is adopted: the least-populated histogram bin determines the adaptive splitting threshold, offering robust separation of boundary and non-boundary candidates despite intra-dataset variability. This adjustment stabilizes the translation of transition scores to discrete boundary events, crucial for consistent segmentation.

Experimental Results

Boundary Detection and Segment Classification

Boundary detectors exhibit consistent performance across feature sets, with MaskedSpec augmentation systematically enhancing generalization (PartialSpoof EVAL F-EER as low as 3.47% for XLSR128+MS). Score-level fusion of multiple detectors further improves boundary localization (down to 2.49% F-EER), demonstrating high complementarity across models. Segment-level classifiers achieve even lower S-EER under ground-truth segmentation, with fusion across input durations and representations yielding 2.78% S-EER on PartialSpoof and 0.01% on HAD.

Full Pipeline Evaluation

End-to-end, with predicted boundaries, the best single system achieves a 7.65% frame-level EER on PartialSpoof (XLSR128+MS+4s). Multi-system fusion reduces total EER to 6.55%, further elevating performance. The pipeline’s robustness is validated by high F1 scores (94.58%) and precision/recall, and its advantages persist across settings—from fine-grained frame-level to coarse-grained segmental evaluation.

Figure 2: DET curves evidencing distinct trade-offs for varied model configurations and the dominant, more robust fusion performance across operating points.

Per-utterance error distributions confirm the framework’s efficacy: 54.5% of PartialSpoof samples are classified perfectly (EER=0%), and the mean utterance EER (5.47%) is lower than the aggregate frame-level EER, underscoring balanced performance free from domination by outlier utterances.

Figure 3: Histogram of per-utterance EER for the PartialSpoof evaluation set, capturing overall system reliability and tails of challenging cases.

Cross-Corpus and Temporal Localization

The method generalizes robustly to HAD (EVAL F-EER 0.01%), indicating low sensitivity to dataset shift or language differences and validating the cross-corpus design of boundary and segment models. Visual analysis of log-magnitude spectrograms highlights substantial spectral, prosodic, and linguistic differences between datasets, corroborating the method’s adaptability.

Figure 4: Log-magnitude spectrogram samples from PartialSpoof (English), HAD (Mandarin), and LPS (English), illustrating cross-corpus acoustic diversity.

Moreover, the framework’s temporal localization capability (as measured by AP, AR, and mean AP at various IoU thresholds) exceeds that of previous methods (mAP: 71.27% vs. 55.22% for CFPRF), particularly at strict criteria, indicating precise boundary alignment and efficient forged segment recovery.

Comparative Analysis and Implications

Compared to recent frame-based (MRM), joint boundary–labeling, and proposal refinement methods (IFBDN, CFPRF), the split-and-conquer approach consistently achieves lower EERs, higher F1, and better localization metrics under various resolutions. The explicit decoupling of localization and classification, as well as exploitation of input length diversity and fusion, produces systems that are not only more accurate but also more robust to errors in individual submodules.

This design introduces important implications for partial audio forgery detection:

• Decoupled objective optimization leads to more stable, interpretable training and clearer model diagnostics.
• Segment-level models can leverage temporal statistics from self-supervised pretraining, even for short signals, using data augmentation to artificially extend context and reduce overfitting.
• Score fusion across heterogeneous models flattens individual weaknesses, yielding a system that is performant across a range of operating points and temporal granularities.
• The method’s resilience to dataset shift and boundary ambiguity suggests suitability for practical use in security-sensitive environments.

Limitations and Future Directions

Key limitations persist:

• Boundary errors, especially near ambiguous transitions, can propagate and impact classification, indicating a potential avenue for further mitigating cross-stage error accumulation.
• The assumption of piecewise-uniform spoof segments may not encompass all forms of manipulations, notably gradual or highly localized forgeries.
• Fully joint learning or end-to-end adaptation remains unexplored—hybrid approaches may harness benefits of both modularization and shared representation learning.

Future work could integrate domain-adaptive objectives, investigate finer-grained or non-uniform segmentation frameworks, and explore more flexible, proposal-free temporal localization architectures.

Conclusion

The split-and-conquer framework sets a new operational paradigm for partial deepfake speech detection by explicitly separating boundary detection from authenticity classification. Empirical evidence validates its superiority over prior art in both detection and temporal localization on multiple challenging benchmarks. Its complementary modeling strategies—feature and augmentation diversity, multi-length segment classification, and robust score-level fusion—collectively enhance performance and generalization. These findings not only have implications for practical anti-spoofing deployments but also inform the continued evolution of interpretable, modular architectures for high-resolution, localizable forgery detection in speech.

---

Reference: "Split and Conquer Partial Deepfake Speech" (2604.02913)