GRIDEX: Grid-Grounded Forensic Explanations for Deepfake Spectrogram Analysis

Abstract: The advancement of speech generation technologies has made artificial speech increasingly realistic. Although modern classification models can achieve high accuracy when it comes to deepfake detection, they do not produce evidences such as indicating where spoof cues appear in the spectrogram and what they imply acoustically, limiting their usefulness in forensic settings. Manual analysis of full spectrograms is resource-intensive, so evidence should narrow attention to the most diagnostic regions. Moreover, existing explainability methods have limited capabilities in connecting contextual attributes to localized evidence, making explanations harder to verify. To overcome this limitation, we propose GRIDEX, a pipeline that, when given a deepfake spectrogram, generates forensic explanations of its anomalies. The pipeline (i) selects top-K anomalous regions in the spectrogram and (ii) produces an explanation for each anomaly. The explanations follow a schema of categorical acoustic fields, including temporal, spectral, phonetic information and interpretation text. To our knowledge, this is the first framework to generate structured forensic explanations using regional grounding for deepfake spectrograms. GRIDEX is trained with a two-stage learning paradigm that combines supervised fine-tuning (SFT) with Group Relative Policy Optimization (GRPO). Experiments on our dataset show improved artifact localization and explanation quality over strong vision-LLM (VLM) baselines. The dataset and code will be released upon publication.

• The paper presents GRIDEX, a grid-grounded framework that articulates region-specific forensic evidence for deepfake audio spectrograms.
• It employs a two-stage vision-language pipeline with fixed-grid segmentation, achieving superior localization with R@3 of 0.386 and mAP of 0.333.
• The framework ensures auditable, explainable outputs through structured metadata and staged optimization, reinforcing forensic traceability.

GRIDEX: A Grid-Grounded Framework for Forensic Explanation of Deepfake Spectrograms

Motivation and Overview

The proliferation of highly realistic speech synthesis systems necessitates forensic approaches that extend beyond simple binary classification to the explicit identification and articulation of evidence of manipulation. Existing models for audio deepfake detection lack the capacity for interpretable, region-specific justifications—an impediment for forensic validation and operational use. GRIDEX addresses this shortcoming by introducing a two-stage vision-LLM (VLM) pipeline that localizes spectrogram regions likely to contain deepfake artifacts and produces structured, attribute-enriched forensic explanations for each identified region.

The GRIDEX framework operates on a fixed spectrogram grid, enabling alignment of model outputs with auditable, spectrotemporal evidence. This design enforces explanations that are both falsifiable and verifiable, suited to forensic analysis where claims must be anchored to observable artifacts.

Figure 1: The GRIDEX framework: grid partitioning and region selection (top left), inference via sequential queries for localization and explanation (top right), and the data pipeline combining VLM interpretation, LLM verification, and human review (bottom).

Data Construction and Region Annotation

To generate detailed region-level supervision, GRIDEX constructs a dataset from the VocV4 corpus, comprising parallel bona fide and vocoder-generated audio samples. Spectrogram difference maps are computed to highlight artifact regions, using smoothed, normalized differences between genuine and fake magnitude spectrograms. Region selection is performed by overlaying uniform grid or alternative segmentation masks, scoring each region by the concentration of artifact pixels identified from the difference map.

Three segmentation strategies are evaluated: uniform grids, SLIC superpixels, and Segment Anything Model (SAM). The $4 \times 4$ grid is ultimately selected for its superior localization reliability and stability compared to superpixels or SAM, which either yield overly coarse or fragmented regions on spectrogram textures.

Figure 2: Spectrogram region segmentation via (from left to right) raw spectrogram, uniform grid, SLIC superpixels, and SAM masks; uniform grids provide stable, interpretable partitioning for regional forensic annotation.

For each selected region, categorical metadata is automatically annotated using forced alignment and deterministic rules: temporal activity (speech/non-speech), frequency band (low/mid/high), and phonetic context (vowel/consonant/unvoiced). Free-form evidence statements are generated in a two-stage protocol involving a VLM (Qwen3-VL-30B-Thinking) for initial narrative and an LLM (Qwen3-235B-Instruct) for consistency and metadata verification; incorrect samples are corrected via manual review to ensure field alignment and explanation accuracy.

Vocoder-specific analysis demonstrates that HiFi-GAN and WaveGlow produce more stable and recurrent artifact localization patterns compared to HN-NSF models, supporting consistent model supervision across training samples.

Figure 4: Overlap in top-3 region selection by vocoder, demonstrating consistent artifact localization within HiFi-GAN and WaveGlow outputs.

Model Architecture and Training Paradigm

GRIDEX is structured as a two-turn VLM system, where:

• Query 1 performs ranked selection of the top-3 suspicious regions from the 16-cell spectrogram grid,
• Query 2 generates, for each region, a structured tuple (Cn, T, F, P, En), representing cell index, temporal activity, frequency band, phonetic category, and a free-form forensic explanation.

The model leverages turn-conditioned parameter-efficient tuning, with separate sets of low-rank adapters for each query. Training employs a staged approach: supervised fine-tuning (SFT) establishes task format, whereas Group Relative Policy Optimization (GRPO) is then applied, using custom reward functions that reinforce set-retrieval accuracy, ranking, field extraction, and explanation–field consistency.

The staged curriculum additionally mitigates the exposure bias that would otherwise result from training Query 2 exclusively on oracle (ground-truth) region IDs, instead conditioning on predicted IDs to better approximate inference-time behavior.

Quantitative Evaluation and Analysis

GRIDEX demonstrates superiority over strong VLM baselines across both localization and explanation tasks on the VocV4 dataset. On localization, GRIDEX achieves R@3 = 0.386, nDCG = 0.411, and mAP = 0.333, compared to the best baseline nDCG of 0.244 and mAP of 0.211. For structured explanations, GRIDEX attains a CovAvg of 0.884 (coverage of extracted field cues in explanation text) and BERTScore F1 of 0.413, with a significant margin over all baseline models.

Further, staged optimization provides substantial gains: GRPO-1 boosts localization R@3 by +0.077 over SFT-1, while GRPO-2 increases field accuracy by +0.209 and explanation–field coverage by +0.809 over SFT-2. These gains are largely attributed to the alignment of reward functions with error modes specific to each query.

Qualitative Interpretation and Error Analysis

GRIDEX explanations consistently recover canonical deepfake artifacts, including fogging, formant attenuation, stochastic detail over-smoothing, and imaging artifacts, all within the predefined coarse taxonomy. Notably, the system also uncovers emergent artifact patterns, such as high-frequency transition discontinuities and mid-band formant inconsistencies, indicating the potential of region-grounded explanation to support artifact discovery beyond initial domain knowledge.

Figure 6: Samples of generated region-grounded explanations aligned with detected prominent artifacts.

Analysis of failure cases reveals bottlenecks at both queries: hotspot bias in Query 1 leads to over-prediction of a subset of region IDs, affecting diversity and ranking quality, while Query 2 errors are primarily due to limitations in accurate field extraction and schema grounding. Nonetheless, explanations are always formatted for audibility and traceability, supporting forensic auditing.

Implications and Future Directions

GRIDEX establishes a new paradigm in explainable deepfake audio forensics by enforcing tightly linked, region-grounded, and structured explanations, supporting both human verifiability and cross-sample artifact aggregation. The fixed-grid protocol and deterministic annotation ensure reproducibility and facilitate cross-method analysis. GRIDEX's structured outputs enable forensic workflows that demand explicit, checkable evidence trails, bridging the gap between black-box detection and operational audit requirements.

Future directions should focus on mitigating localization bias, improving the diversity of selected evidence regions, integrating feedback mechanisms for artifact discovery, and enhancing generalization to unseen datasets and unseen artifact types. Extension to finer-grained or hierarchical region partitioning, as well as dynamic or content-adaptive segmentation, may improve the granularity and adaptability of forensic explanations.

Conclusion

GRIDEX introduces an advanced grid-grounded explainable pipeline for audio deepfake spectrogram analysis, exhibiting substantial improvements in artifact localization and forensic explanation accuracy over prior VLM baselines. Its combination of structured, auditable region-level explanations, robust staged optimization, and deterministic data construction sets a new standard for interpretable audio forensics. Further research should extend this framework towards broader artifact typologies and more robust cross-domain generalization, advancing theory and practice in AI-driven multimedia forensics.

(2606.18738)