- The paper introduces a novel psychoacoustic adversarial framework that manipulates explanation maps without altering predictions.
- It employs dynamic masking thresholds and cosine similarity to measure divergence between original and perturbed attributions.
- Empirical results reveal that explanation fragility varies with signal topology and architectural design across audio deepfake detectors.
Psychoacoustically-Constrained Manipulation of Explanations in Audio Deepfake Detectors
Summary and Context
This work addresses a critical gap in explainability robustness for audio deepfake detectors by empirically demonstrating the vulnerability of post-hoc attribution methodsโspecifically, Grad-CAM and Layer-wise Relevance Propagation (LRP)โto adversarial manipulations that leave model predictions unchanged. Prior research on explanation fragility has predominantly focused on computer vision, employing Lp norm constraints with limited relevance to perceptual integrity in audio. This paper pioneers a psychoacoustic adversarial framework, showing that explanation maps for audio classifiers can be systematically decoupled from their underlying predictions via imperceptible perturbations.
Methodological Contributions
The principal methodological innovation is an adversarial optimization framework that incorporates a dynamic psychoacoustic masking threshold into the loss. The attack objective simultaneously:
- Maximizes the divergence between the original and perturbed explanation maps (via cosine similarity on attributions).
- Penalizes perturbations that are perceptible according to domain-specific psychoacoustic metrics (e.g., masking threshold).
- Strictly preserves the predicted class by including a hinge loss on prediction invariance and bounding waveform amplitude.
The study systematically evaluates this approach across multiple state-of-the-art architectures:
- The convolutional VGGish (local spectrogram focus)
- The transformer-based AST (global attention-based features)
- SpecTTTra, designed for long-range dependencies and music deepfake detection.
Additionally, the authors introduce the Audio Fragility Score (AFSstable), a quantitative metric capturing the magnitude of explanation map displacement conditioned on preserved prediction and perceptual transparency.
Empirical Results
Experiments on the SONICS benchmark dataset confirm several significant claims:
- Explanations are highly vulnerable: Across all architectures, attribution maps can be distorted to a substantial degree without affecting predictive output or introducing perceptible artifacts when constrained by the psychoacoustic masking model.
- Strong perceptual quality preservation: The psychoacoustic attack method maintains high scores across PEAQ, VISQOL, CDPAM, PESQ, and STOI metrics, outperforming unconstrained PGD approaches, which degrade the audio beyond the threshold of perceptibility.
- Architecture-dependent robustness: SpecTTTra (long-range attention-based) demonstrates the greatest resistance to attribution manipulation (mean rank 7.83), while AST is the most fragile (mean rank 3.00). This highlights the architectural dependence of explanation robustnessโparticularly the susceptibility of self-attention mechanisms to adversarial steering.
- Signal topology effects: "Busy" audio samplesโhigh spectral bandwidth, broadband energy, high ZCRโare more amenable to explanation manipulation, as their dense textures offer greater psychoacoustic masking budgets. By contrast, sparse, high dynamic-range signals limit adversarial optimization flexibility.
Principal Component Analysis further reveals that the psychoacoustic attack induces smooth, directional shifts in explanation space for transformer models, while inducing more isotropic dispersion in convolutional architectures.
Implications and Theoretical Impact
The findings underscore a fundamental limitation of contemporary post-hoc XAI in the audio domain. Unlike vision, where Lp metrics and visual artifacts provide intuitive guidance, audio requires perceptually aligned constraints for meaningful robustness auditing. The demonstrated decoupling of attribution maps from predictions under strict psychoacoustic constraints challenges the reliability of visual explanations as trustworthy interpretability tools for critical applications, including audio deepfake forensics.
From a theoretical standpoint, this work extends the adversarial explainability threat model by operationalizing attacks bounded by human perception. The results indicate that XAI methods not mathematically tethered to model decision boundaries are insufficient for defense against adversarial deception, especially in modalities with complex perceptual structure.
Practical Relevance and Future Directions
Practically, the psychoacoustic masking-based attack models pose a significant security concern for trustworthy ML pipelines in speech and audio forensics, where legal and operational decisions may be informed by explanation maps. The paper advocates for a paradigm shift towards explanation mechanisms with provable alignment to classifier decision boundaries or leveraging inherently interpretable models.
Future research directions include:
- Designing XAI techniques robust to perceptually-aligned adversarial manipulation, potentially through joint training of classifier and explanation generator with explicit robustness constraints.
- Developing explanation auditing protocols that integrate psychoacoustic models at the system level.
- Exploring theoretical lower bounds on explanation-manipulation trade-offs across modalities and tasks.
Conclusion
This paper provides compelling evidence of the fragility of post-hoc explanations in audio deepfake detector models by introducing a perceptually-constrained adversarial paradigm. Psychoacoustic masking enables imperceptible manipulation of attribution maps, decoupling them from predictive accuracy and revealing new attack surfaces in audio ML explainability. The architectural and data-dependent analysis highlights the urgent need for interpretability frameworks that ensure explanation integrity in adversarial settings.
