---
title: 'Audio-XAI: Explainable Audio AI'
url: https://www.emergentmind.com/topics/audio-xai
type: topic
---

# Audio-XAI: Explainable Audio AI

Audio-XAI denotes the application of explainable artificial intelligence to audio models, with the aim of identifying which parts of an acoustic signal, intermediate representation, or conditioning input contribute to a model’s output. In the literature represented here, Audio-XAI spans spoken-digit and speaker analysis, audio event classification, automatic speech recognition, anomalous sound detection, audio deepfake detection, cough analysis, neural acoustic embeddings, and text-to-audio generation. The field includes both post-hoc attribution methods and model-intrinsic approaches, and it addresses explanation targets at multiple granularities: waveform samples, time-frequency bins, phoneme-aligned segments, token-to-frame alignments, latent channels, and text-token influences on generated audio [1807.03418][2304.14019][2305.18011][2406.05199][2501.13887][2502.00459][2601.19017][2605.10153][2606.14466][2606.14647][2508.14949].

## 1. Scope and development of the field

An early benchmark for Audio-XAI was introduced by "AudioMNIST: Exploring Explainable Artificial Intelligence for Audio Analysis on a Simple Benchmark" [1807.03418]. That work coupled an open dataset of 30,000 recordings of spoken English digits with Layer-wise Relevance Propagation (LRP), systematic input manipulations, and audible heatmaps. It established a template in which explanations are not merely visual overlays but testable hypotheses about model feature selection.

Subsequent work broadened both the application range and the methodological repertoire. In audio event classification, LRP and DFT-LRP were used to compare waveform and spectrogram models and to expose representation-dependent decision strategies [2304.14019]. In ASR, model-agnostic perturbation methods were adapted to phoneme recognition with explicit ground truth from TIMIT, and later transformer-specific token-to-frame explanations were proposed for Whisper and speech-augmented decoder-only models [2305.18011][2606.14647]. Audio deepfake detection became a major focus, with Grinberg et al. introducing a waveform-level transformer relevancy method and large-scale analyses of speech, non-speech, phonetic content, and voice onsets or offsets [2501.13887]. Parallel developments addressed industrial anomalous sound detection through band-level perturbation faithfulness tests [2601.19017], clinically oriented cough analysis through occlusion-weighted spectral features [2508.14949], and text-to-audio generation through token-level factual and counterfactual masking [2502.00459].

A distinct line of work has attempted to make the representations themselves explainable. XANE learns acoustic embeddings whose dimensions are tied to 14 acoustic parameters, including reverberation, noise, overlap, and CODEC properties [2406.05199]. APEX instead keeps a pretrained classifier frozen and inserts an invertible disentanglement layer to obtain audio-centric prototype explanations while preserving the original logits exactly [2605.10153]. This suggests that Audio-XAI is not limited to retrospective saliency estimation; it also includes representation design and architecture-level constraints intended to make explanations structurally meaningful.

## 2. Signal representations and explanation targets

Audio-XAI operates on a wider range of representations than image XAI. At the input level, models may consume raw waveforms, magnitude spectrograms, log-mel spectrograms, or latent acoustic embeddings. For waveform models, explanations can be assigned directly to samples or remapped to time-frequency space. In the representation-comparison study, DFT-LRP inserted a virtual STFT and inverse DFT identity block immediately after the waveform input so that relevance could be visualized in a mel-spectrogram-shaped map without changing the forward computation [2304.14019]. In AudioMNIST, waveform explanations were also converted into audible heatmaps by masking the signal with positive relevance, $\tilde x(t)=\mathrm{ReLU}(R(t))\odot x(t)$ [1807.03418].

For spectrogram models, the explanation target is typically a time-frequency relevance map $R(t,f)$ or an occlusion map over patches. This is the setting used in anomalous sound detection, cough analysis, Grad-CAM-style deepfake explanation, and many prototype methods [2601.19017][2508.14949][2606.14466][2605.10153]. However, the audio domain also requires more specialized explanation targets. In phoneme recognition, LIME-TS explains the prediction of a particular phoneme instance by ranking fixed-duration time blocks within a sliding window around the phoneme’s ground-truth time [2305.18011]. In transformer-based ASR, LEAF-X assigns each decoded token $y_i$ a normalized attribution vector $s_i\in\Delta^{T-1}$ over acoustic frames, using cross-attention structure, entropy weighting, rollout, and optional causal ablations [2606.14647].

Generative models introduce another target: the contribution of text tokens to generated audio. AudioGenX wraps a frozen text-to-audio model with an Explainer that outputs a soft mask $M_{U,z_t}\in[0,1]^{L\times 1}$ for text-token embeddings at each audio generation step, then aggregates these masks into token-level importance scores across the output sequence [2502.00459]. By contrast, XANE makes the explanation target the latent embedding itself: a 128-dimensional acoustic embedding is explicitly trained to predict 11 regression variables and 3 classification variables, making the embedding explainable in terms of measurable acoustic parameters rather than post-hoc saliency alone [2406.05199].

## 3. Methodological families

A large part of Audio-XAI consists of adapted vision methods. LRP, Grad-CAM, Integrated Gradients, SmoothGrad, SHAP variants, and LIME appear repeatedly across tasks [1807.03418][2601.19017][2501.13887][2305.18011]. Their adaptation is rarely trivial, because audio signals have strong temporal structure, phase-sensitive waveform structure, and acoustically meaningful decompositions that do not align cleanly with image superpixels or visual attention maps.

Perturbation-based methods form one major family. LIME constructs perturbed samples $x'=x\odot z$ and fits a locally weighted surrogate $g(z)=w_0+\sum_{j=1}^d w_j z_j$ by minimizing a locality-weighted square loss plus a sparsity penalty [2305.18011]. The time-partitioned LIME-TS variant replaces manual segments with uniform temporal blocks and a sliding-window perturbation regime centered on the phoneme of interest [2305.18011]. Occlusion methods instead zero out local spectrogram patches or frequency bands and observe output changes; in cough analysis the occlusion map is defined as $M(f,t)=P(y|S)-P(y|S_{\mathrm{occl}}(f,t))$, whereas in anomalous sound detection frequency-band removal produces a functional ground truth for band importance through $\Delta \mathrm{Pred}_b=|\mathrm{Pred}_{\mathrm{orig}}-\mathrm{Pred}_b|$ [2508.14949][2601.19017].

Gradient- and relevance-based methods form a second family. Audio deepfake detection provides a particularly explicit example. Because Wav2Vec2-AASIST has no dedicated classification token, Grinberg et al. adapt transformer relevancy into Gradient Average Transformer Relevancy (GATR), with layer update
$$
R_{\text{upd}}=R_{\text{old}}+\bar A_iR_{\text{old}},\qquad
\bar A_i=\mathbb{E}_h[(\nabla A_i\odot A_i)^+],
$$
followed by a gradient-weighted averaging across token rows and interpolation back to waveform length [2501.13887]. This is an audio-specific solution to a transformer architecture that does not fit standard vision-transformer relevance extraction. LEAF-X is similarly architecture-aware: it computes head entropy
$$
H_i^{(\ell,h)}=-\sum_{t=1}^T a_{i,t}^{(\ell,h)}\log(a_{i,t}^{(\ell,h)}+\epsilon)
$$
and converts it into a confidence weight $w_{\ell,h}(i)=\left(1-H_i^{(\ell,h)}/\log T\right)^{1/\tau}$ to privilege low-entropy, high-impact heads during token-to-frame attribution [2606.14647].

Prototype and disentanglement methods constitute a third family. APEX inserts an invertible matrix $U=\exp(A)$ between a frozen feature map and global average pooling, reparameterizing the classification head as $W_{\rm new}=W_{\mathrm{cls}}U^{-1}$ so that the logits remain identical. It then maximizes channel purity and extracts four types of prototypes: square-based, time-based, frequency-based, and time-frequency hybrid [2605.10153]. AudioGenX is conceptually related in that it uses a dedicated Explainer module rather than raw gradients, but its objective is causal text-token selection through factual and counterfactual losses at each audio token [2502.00459]. XANE, finally, is closer to interpretable representation learning than to attribution: its explainability derives from multitask prediction of physically meaningful acoustic parameters [2406.05199].

## 4. Evaluation protocols, faithfulness criteria, and human interpretability

A central methodological issue in Audio-XAI is that visual plausibility is insufficient. Several works therefore define task-specific quantitative evaluation protocols. In phoneme recognition, explanation quality is measured against TIMIT alignments by asking whether the true phoneme segment appears in the top-$k$ ranked segments, yielding $P(1)$, $P(3)$, and $P(5)$ [2305.18011]. In machine anomalous sound detection, faithfulness is measured by the Spearman correlation between mean band-level relevance $\bar R_b$ and frequency-band sensitivity $\Delta \mathrm{Pred}_b$ obtained by systematic band removal [2601.19017]. In ASR, LEAF-X uses D-AOPC, insertion, temporal localization, sparsity, stability, and infidelity, combining perturbation sensitivity with forced-alignment grounding and robustness to mild label-preserving perturbations [2606.14647].

In audio deepfake detection, evaluation is especially elaborate. Grinberg et al. define Average Drop, Average Increase, Average Gain, and Input Fidelity on peak-normalized relevance-masked waveforms, then supplement these with positive and negative perturbation tests based on progressively noise-masking the top or bottom $n\%$ of time frames and measuring the area under the resulting EER curve [2501.13887]. They also introduce a partial-spoof localization protocol with Relative Contribution Quantification (RCQ), Relevance Mass Accuracy (RMA), and Relevance Rank Accuracy (RRA) on utterances containing contiguous spoof segments [2501.13887]. For text-to-audio generation, AudioGenX defines factual and counterfactual fidelity drops, KL divergences under a frozen audio classifier, and a mask-size measure to capture simplicity [2502.00459].

Human interpretability is evaluated more rarely, but AudioMNIST provides a notable example. Its user study compared waveform-only, waveform-plus-LRP, raw audio, and masked-audio explanations using informedness and markedness. Audible explanations outperformed visual explanations for both correct and incorrect model outputs, and for incorrectly classified samples the reported informedness and markedness were approximately $0.12$ and $0.10$ for audible explanations versus approximately $0.03$ and $0.01$ for visual explanations [1807.03418]. A plausible implication is that audio-specific presentation formats may matter as much as the attribution algorithm itself when explanations are intended for human audit.

## 5. Empirical findings across tasks

The empirical record shows that explanation results are highly task- and representation-dependent. In AudioMNIST, LRP on a spectrogram model for speaker-sex classification concentrated on low frequencies, leading to the hypothesis that the model used differences in fundamental frequency $F_0$; when male spectrograms were frequency-scaled by $1.5\times$ and female spectrograms by $0.66\times$, accuracy on the manipulated test set dropped to $20\%\pm 12.6\%$ [1807.03418]. For the waveform model, zeroing only $1\%$ of samples selected by LRP relevance reduced digit-task accuracy from approximately $92.5\%$ to approximately $77\%$, larger than amplitude-based or random deletion [1807.03418].

In audio event classification, relevance maps exposed major differences between raw-waveform and log-mel models trained on UrbanSound8k. The 1D-CNN showed within-class cosine similarity $S_C=0.207\pm0.057$ and between-class similarity $S_C=0.076\pm0.017$, whereas YAMNet showed within-class similarity $0.593\pm0.099$ and between-class similarity $0.584\pm0.062$ [2304.14019]. The waveform model was also less sensitive to high-pass, low-pass, and pitch-shift perturbations than YAMNet [2304.14019]. This suggests that the choice of input representation can alter not only accuracy but also the semantic distinctiveness and robustness of the learned decision strategy.

ASR studies make two complementary points. On TIMIT phoneme recognition, LIME-TS placed the ground-truth phoneme segment in its top three audio segments $96\%$ of the time and reached $P(5)=0.97$ [2305.18011]. On large transformer ASR models, LEAF-X reported D-AOPC $=0.45$ versus a best baseline of $0.51$, SPR $=0.70$ versus $0.68$, STAB $=0.78$ versus $0.72$, and INF $=0.45$ versus $0.50$, with results summarized as roughly $32\%$ improved faithfulness, $35$-$39\%$ stronger locality or sparsity, and the most stable attributions [2606.14647].

Audio deepfake detection reveals both performance differences among explainers and instability of small-sample narratives. On ASV19, GATR achieved fidelity $0.998$ and Average Drop $0.35\%$, and on PartialSpoof it reached RMA $=0.44$ and RRA $=0.51$, outperforming the compared XAI methods [2501.13887]. Large-scale RCQ analysis further found that non-speech regions were most influential for bona-fide classification, that low-energy speech frames often carried higher importance except on ITW spoof utterances where high-energy frames dominated, and that unstressed vowels had the highest normalized RCQ on ASV19 spoof files while consonants were least important on ITW [2501.13887]. The authors explicitly note that conclusions drawn from a handful of hand-picked samples may not generalize across datasets or conditions [2501.13887].

In anomalous sound detection, low-frequency bands dominated the anomaly model’s decisions: removing Band 1 $(0$-$1.6\,\mathrm{kHz})$ caused the largest performance drop, whereas removing Band 5 $(6.4$-$8.0\,\mathrm{kHz})$ slightly improved Dev AUC to $\mathbf{76.8\%}$ [2601.19017]. Faithfulness evaluation showed Occlusion with $\bar\rho_s=0.884$, higher than Integrated Gradients $(0.530)$, Grad-CAM $(0.536)$, and SmoothGrad $(0.400)$ [2601.19017]. In cough analysis, occlusion maps highlighted mid-frequency bands of approximately $0.5$-$2\,\mathrm{kHz}$ around the central portion of the cough epoch, and XAI-weighted spectral features revealed significant group differences for COPD comparisons that were absent in raw spectrogram features [2508.14949]. In text-to-audio generation, AudioGenX concentrated importance on sound-related nouns, down-weighted absent optional concepts in some generations, and revealed very low scores for negation tokens in prompts such as “without thunder” and “without no thunder,” exposing a data-bias in the TAG model [2502.00459].

## 6. Fragility, interpretable-by-design alternatives, and open problems

A recurring concern is whether post-hoc audio explanations are stable under innocuous input changes. "The Perceived Fragility of Explanations in Audio Models" formalizes this concern by optimizing an inaudible perturbation $\delta$ that minimizes explanation cosine similarity while preserving the original prediction under a hinge-loss constraint [2606.14466]. The reported attacks retained high perceptual quality, preserved the final “real” or “fake” label with $100\%$ preservation under the hinge loss, reduced explanation cosine similarity from approximately $0.9$ to approximately $0.1$, and yielded median $\mathrm{AFSstable}>0.85$ across VGGish, AST, and SpecTTTra [2606.14466]. The paper also recommends explanation adversarial training, multiple complementary XAI methods, perceptual consistency constraints, decision-tethered explanations, and deployment-time monitoring of fragility [2606.14466].

One response to such fragility is to make explanations more tightly coupled to model structure. LEAF-X is model-intrinsic for transformer ASR and uses entropy-guided attention weighting plus optional causal layer ablations [2606.14647]. APEX preserves logits exactly through an invertible reparameterization and offers prototype explanations from four audio-centric perspectives without fine-tuning the original backbone [2605.10153]. XANE goes further toward explainability-by-design by forcing a compact embedding to reproduce 14 acoustic parameters and reporting a mean F1 score of $95.2\%$ on three clustering tasks, alongside parameter estimation and a CPU real-time factor of $0.07$ [2406.05199]. These approaches do not eliminate the need for evaluation, but they reduce the conceptual gap between internal computation and explanation.

Several limitations recur across the literature. APEX requires a pooling-to-linear classifier head and does not yet handle transformer backbones or self-supervised representations directly [2605.10153]. LEAF-X experiments are confined to English-only Whisper and Canary models [2606.14647]. AudioGenX currently does not address diffusion-based text-to-audio models without explicit discrete tokens [2502.00459]. AudioMNIST notes that spectrogram explanations were not made audible because waveform reconstruction would require phase reconstruction [1807.03418]. The cumulative evidence suggests that Audio-XAI remains heterogeneous: explanation faithfulness depends on task, architecture, and representation, and explanation usability depends on both quantitative validation and audio-appropriate presentation.

Source: https://www.emergentmind.com/topics/audio-xai