- The paper proposes Modified RISE-eval to improve quantitative evaluation of attention maps in speaker recognition models.
- It compares GradCAM and LayerCAM across network layers, highlighting their respective strengths in coarse and fine feature localization.
- Results indicate that decisive accuracy drops from deletion masking correlate with higher attribution fidelity, guiding model tuning.
Explainable AI in Speaker Recognition: Attention Map Visualisation and Evaluation
Introduction
This paper rigorously investigates the analysis and evaluation of attention mechanisms in neural networks for speaker recognition, focusing on improving the transparency and interpretability of AI systems. The authors systematically review and revise the evaluation procedures for attention map visualisation methods, primarily targeting CAM-based approaches such as GradCAM and LayerCAM. The context is provided by a ResNet34-based CNN trained on VoxCeleb datasets, with attention analysis focused on the identification and discrimination of speaker identity from mel spectrogram utterances. The primary contribution is the proposal and validation of Modified RISE-eval, an improved quantitative algorithm for assessing attention map quality in neural network-based speaker recognition.
Attention Mechanisms and Map Visualisation Strategies
The study adopts an expansive definition of network attention, considering any computational mechanism in a neural network that selectively processes relevant input information during decision-making. Attention maps are visualised as two-dimensional matrices that highlight influential input regions based on activations and class-guided gradients extracted at various network layers. The authors contrast perturbation-based methods (e.g., LIME, SHAP, RISE) and internal representation-based methods (e.g., GradCAM, LayerCAM, LRP), emphasizing the latter's ability to directly leverage activation gradients for fine-grained explanatory heatmaps.
GradCAM computes weighted sums of channel activations via globally pooled gradients, producing coarse maps that prioritise high-level semantic regions. LayerCAM, in contrast, multiplies spatially resolved gradients by activation values, yielding finer-grained maps capable of capturing more precise input features. These methods are benchmarked at different depths within the ResNet34 architecture to examine layer-wise variation in attention selectivity.
Evaluation Algorithms: RISE-eval and Its Limitations
RISE-eval is reviewed as the canonical input-masking-based quantitative metric for attention map quality, whereby regions highlighted by an attention map are progressively masked in the model input, and the magnitude of change in classification performance is used as a proxy for attribution fidelity. The paper identifies two fundamental issues: (1) indiscriminability in insertion masking strategy, which fails to differentiate map quality across visualisation methods, and (2) confounding due to overmasking sparse maps, which introduces random noise effects and severely undermines interpretability.
Modified RISE-eval: Algorithmic Advances
To mitigate the above limitations, Modified RISE-eval is introduced. Insertion masking is omitted, focusing exclusively on deletion masking to more robustly enumerate performance degradation. A threshold mechanism excludes near-zero pixels from masking, preserving the correspondence between attention values and genuine attribution. The masking performance curve is generated by tracking accuracy drop as a function of effective mask ratio, with evaluation scores rescaled to reward early, decisive drops in performance, reflecting high attribution fidelity.
Experimental Design and Setup
A ResNet34-based prototypical network trained on VoxCeleb2 and tested on VoxCeleb1 is used to extract activations and class-guided gradients from four distinct residual blocks. GradCAM and LayerCAM are applied at each layer to generate eight attention map sets per input. Modified RISE-eval quantitatively evaluates these maps by measuring classification accuracy degradation as spectrogram pixels are masked, with energy localisations and class-discriminative ability visually and numerically assessed.
Results: Visualisation and Quantitative Evaluation
Attention maps are visually examined by classifying concatenated spectrograms (two-speaker inputs) as either speaker, demonstrating class discriminative ability at both shallow and deep layers. GradCAM’s last-layer maps most cleanly localise target speaker regions, while LayerCAM yields finer detail at shallow layers but less decisive discrimination at deeper ones.
Masking performance curves generated via Modified RISE-eval show consistent monotonic accuracy drops within effective mask intervals. LayerCAM outperforms GradCAM in quality scores at the first three layers, masking influential regions more rapidly. GradCAM surpasses LayerCAM at the final layer, correlating with stronger class-discriminative attention. Numerical scores (see Table I in the original paper) corroborate these findings, evidencing the nuance in layerwise attribution and method selection.
Implications and Limitations
Practically, the results guide model-tuning and post-hoc interpretability practices in speaker recognition, suggesting LayerCAM be preferred for lower-layer attribution and GradCAM for deep-layer explanations. Theoretically, the study refines quantitative explainability evaluation, but it also highlights unresolved questions in defining ultimate interpretive goals for network attention and the principle utility of attention map explanations.
A bias in Modified RISE-eval is acknowledged: evaluation favours shallower-layer maps due to finer-grained masking and overrepresentation of non-silent regions in spectrogram data, while deeper-layer maps (closer to decision) may localise larger, coarser semantic regions. Addressing this bias is identified as an avenue for further improvement.
Conclusion
This work advances explainable AI in speaker recognition by methodically visualising and evaluating attention maps using CAM-based methods and Improved RISE-eval. GradCAM demonstrates superior attribution fidelity at final network layers, while LayerCAM excels with shallow feature maps. Modified RISE-eval reliably quantifies the extent to which highlighted input regions genuinely influence recognition decisions, improving upon the original by resolving indiscriminability and noise confounds. Future research should refine evaluation protocols and clarify the practical utility of network attention explanations in deployed systems.
