Improving Generalization for AI-Synthesized Voice Detection

Abstract: AI-synthesized voice technology has the potential to create realistic human voices for beneficial applications, but it can also be misused for malicious purposes. While existing AI-synthesized voice detection models excel in intra-domain evaluation, they face challenges in generalizing across different domains, potentially becoming obsolete as new voice generators emerge. Current solutions use diverse data and advanced machine learning techniques (e.g., domain-invariant representation, self-supervised learning), but are limited by predefined vocoders and sensitivity to factors like background noise and speaker identity. In this work, we introduce an innovative disentanglement framework aimed at extracting domain-agnostic artifact features related to vocoders. Utilizing these features, we enhance model learning in a flat loss landscape, enabling escape from suboptimal solutions and improving generalization. Extensive experiments on benchmarks show our approach outperforms state-of-the-art methods, achieving up to 5.12% improvement in the equal error rate metric in intra-domain and 7.59% in cross-domain evaluations.

• The paper presents a disentanglement module that extracts domain-agnostic artifact features to improve cross-domain AI-synthesized voice detection.
• It employs adaptive mutual information loss and sharpness-aware minimization to smooth the loss landscape and enhance detection robustness.
• Experimental results show up to 5.12% and 7.59% improvements in Equal Error Rate for intra-domain and cross-domain scenarios, respectively.

Improving Generalization for AI-Synthesized Voice Detection

The task of detecting AI-synthesized voices has gained paramount importance due to the dual nature of its applications. While this technology is leveraged for beneficial uses, such as voice assistants and audiobooks, it also poses threats through misuse, giving rise to phenomena like audio deepfakes. The current methodologies have shown competence in handling intra-domain scenarios but face significant challenges when addressing cross-domain data, primarily because they are often bound by predefined vocoders and are sensitive to extraneous elements such as background noise and speaker identity.

This paper presents an innovative disentanglement framework aimed at improving the generalization of AI-synthesized voice detection. The framework is designed to extract domain-agnostic artifact features associated with vocoders, which enhances the model's learning process in a flattened loss landscape. This configuration allows the model to eschew suboptimal solutions, thus advancing its generalization capabilities.

Key Contributions

The authors have outlined several contributions through their novel approach:

1. Disentanglement Framework: The core of the proposed solution is a disentanglement learning module that captures both domain-specific and domain-agnostic artifact features. This is achieved through a multi-task strategy with contrastive learning applied to separate these features.
2. Adaptive Mutual Information Loss: Emphasis is placed on aligning domain-agnostic features with content features using mutual information loss, ensuring these features are universally applicable across various domains.
3. Flattened Loss Landscape: A sharpness-aware minimization process is employed to smooth out the loss landscape, enhancing model generalization and robustness against the inherent variability of voice synthesis technologies.

Experimental Evaluation

The effectiveness of this framework is demonstrated through comprehensive experiments conducted on prominent audio deepfake detection datasets, including LibriSeVoc, WaveFake, ASVspoof2019, and FakeAVCeleb. The results indicate that the proposed framework outperforms existing methods, showing up to 5.12% improvement in the Equal Error Rate (EER) for intra-domain scenarios and 7.59% for cross-domain evaluations.

A detailed analysis reveals that the framework not only excels in distinguishing synthetic voices within the same vocoder type but also significantly enhances detection capabilities for voices emerging from previously unseen vocoders. Moreover, ablation studies on the proposed components like mutual information loss and sharpness-aware minimization further validate the efficacy of these modules in contributing to overall model improvement.

Implications and Future Directions

The proposed framework's ability to generalize across domains has profound implications in enhancing the security and reliability of systems dependent on voice authentication and AI-synthesized voice usage. The capability to detect voices synthesized by unseen vocoder architectures is particularly crucial, given the rapidly evolving landscape of deepfake generation technologies.

Looking forward, the framework sets a foundation for future explorations in AI-synthesized voice detection that could consider multi-modal data involving both audio and video. Extending these methodologies beyond vocoder artifacts to capture latent patterns across divergent synthesis techniques remains an open frontier, posing challenges yet offering opportunities for further innovation in artificial intelligence research.

This work stands as a noteworthy stride in addressing the complexities of synthetic voice detection, contributing toward a more robust framework capable of adapting to the dynamic challenges of AI-generated media.