GRIDEX: Grid-Based Deepfake Forensic Explanations

• GRIDEX is a structured forensic explanation framework that overlays a fixed grid on spectrograms to localize deepfake artifacts.
• It employs a two-stage learning process with supervised fine-tuning and policy optimization to enhance region retrieval and explanation quality.
• The pipeline outputs structured tuples linking temporal, spectral, and phonetic contexts with free-form evidence, ensuring verifiable analysis.

Searching arXiv for the specified paper to ground the article and citation. GRIDEX is a two-stage, grid-grounded forensic explanation pipeline for deepfake spectrogram analysis that is designed to both localize the most diagnostic regions in a deepfake spectrogram and generate structured, acoustically grounded explanations for each region (Do et al., 17 Jun 2026). It addresses a forensic gap identified by the authors: modern classification models can achieve high accuracy in deepfake detection, but they do not produce evidences such as indicating where spoof cues appear in the spectrogram and what they imply acoustically. In response, GRIDEX overlays a fixed $G\times G$ grid on an input spectrogram $\mathbf S$, uses a vision-LLM to select the top-$K$ most anomalous cells, and then produces, for each selected cell, a tuple that links a region identifier to temporal, spectral, and phonetic context plus a free-form evidence statement (Do et al., 17 Jun 2026).

1. Forensic motivation and conceptual scope

The motivating problem is defined in explicitly forensic terms. Manual analysis of full spectrograms is described as resource-intensive, and evidential support is therefore expected to narrow attention to the most diagnostic regions rather than require exhaustive inspection of the entire image (Do et al., 17 Jun 2026). Existing explainability methods are characterized as having limited capabilities in connecting contextual attributes to localized evidence, which makes explanations harder to verify.

Within that setting, GRIDEX is intended to bridge global deepfake detection and fine-grained forensic analysis by producing verifiable, structured evidence localized to spectrogram regions (Do et al., 17 Jun 2026). The paper presents the method as a pipeline with two coupled objectives: first, identification of the strongest spoof-related artifacts; second, explanation of those artifacts in acoustically meaningful terms.

The authors describe GRIDEX, to their knowledge, as the first framework to generate structured forensic explanations using regional grounding for deepfake spectrograms (Do et al., 17 Jun 2026). This framing is important because the output is not a scalar prediction or an unstructured caption, but a set of region-level explanations anchored to a fixed spatial partition of the spectrogram.

2. Formal problem formulation and grid grounding

The formal input is a magnitude spectrogram $\mathbf S\in\mathbb R^{T\times F}$. The stated goal of deepfake anomaly detection is twofold: to decide which $K$ spectrogram subregions $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$ exhibit the strongest spoof-related artifacts, and to explain each region's anomaly in acoustically meaningful terms (Do et al., 17 Jun 2026).

Supervised localization is derived from bona fide-spoof parallels by constructing a binarized artifact mask through smoothed STFT differences:

$$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$

Each candidate region is then scored by summing mask values over its support:

$$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$

GRIDEX partitions $\mathbf S$ into uniform cells $\mathcal R=\{r_1,\dots,r_{G^2}\}$, with $\mathbf S$0, and selects the top-$\mathbf S$1 cells by sorting these scores, with $\mathbf S$2:

$\mathbf S$3

Because the cells are non-overlapping, no non-maximum suppression or clustering is needed (Do et al., 17 Jun 2026). During training, the predicted top-$\mathbf S$4 cells are supervised by the ground-truth top-$\mathbf S$5 derived from the artifact mask.

This fixed-grid formulation is one of the paper's central design choices. Region annotation was considered with both grid $\mathbf S$6 cells and SLIC superpixels, but the grid was chosen because it yields higher coverage and stability (Do et al., 17 Jun 2026). A plausible implication is that the authors prioritize consistency and verifiability of region references over a more adaptive segmentation scheme.

3. Structured explanation schema

For each selected cell $\mathbf S$7, GRIDEX outputs a 5-tuple $\mathbf S$8 (Do et al., 17 Jun 2026). The fields are defined as follows:

• $\mathbf S$9: cell ID.
• $K$0: temporal activity, with speech assigned if overlap with forced-aligned phones exceeds $K$1 s.
• $K$2: frequency band, based on mean pixel row.
• $K$3: phonetic category, obtained from Montreal Forced Aligner.
• $K$4: a short free-form evidence text.

The paper gives an example of the evidence field: “This region shows smoothed harmonic structure causing muffled vowel timbre.” Because the categorical fields are drawn from a closed vocabulary, the authors argue that explanations are verifiable against the spectrogram region (Do et al., 17 Jun 2026).

This schema combines localization with contextualization. Temporal, spectral, and phonetic labels constrain interpretation to attributes that can be checked against the selected cell, while the free-form evidence statement provides a concise verbal characterization of the anomaly. The paper's emphasis is therefore not on unrestricted natural-language generation, but on a structured explanation format intended to support forensic verification.

4. Model architecture and two-stage learning

The explanation generation model uses Qwen2.5-VL-3B-Instruct as its backbone, together with parameter-efficient fine-tuning through two LoRA adapters, $K$5 for Query 1 and $K$6 for Query 2 (Do et al., 17 Jun 2026). Query 1 takes a spectrogram image with grid overlay and outputs a JSON list of three region IDs. Query 2 takes the same image together with the transcript and the selected IDs and outputs three structured tuples.

The learning paradigm has two stages. In supervised fine-tuning, Stage 1 trains $K$7 on ground-truth top-3 lists, and Stage 2 trains $K$8 on ground-truth structured tuples; weight decay and LoRA regularization are reported as in Table A.1 (Do et al., 17 Jun 2026). The supervised loss for stage $K$9 is

$\mathbf S\in\mathbb R^{T\times F}$0

After supervised fine-tuning, GRIDEX applies Group Relative Policy Optimization (GRPO). During RL, the policy gradient update follows

$\mathbf S\in\mathbb R^{T\times F}$1

where $\mathbf S\in\mathbb R^{T\times F}$2 is stage-specific (Do et al., 17 Jun 2026).

For localization, the GRPO-1 reward combines hit rate, ranking quality, and format validity:

$\mathbf S\in\mathbb R^{T\times F}$3

$\mathbf S\in\mathbb R^{T\times F}$4

For explanation generation, GRPO-2 uses a tuple-level reward involving textual similarity, categorical field accuracy, contextual consistency, and schema validity:

$\mathbf S\in\mathbb R^{T\times F}$5

$\mathbf S\in\mathbb R^{T\times F}$6

$\mathbf S\in\mathbb R^{T\times F}$7

where $\mathbf S\in\mathbb R^{T\times F}$8 if and only if $\mathbf S\in\mathbb R^{T\times F}$9 is correct (Do et al., 17 Jun 2026).

The staged design is central to the method. The paper's discussion indicates that it improves both region retrieval and explanation grounding relative to zero-shot VLM baselines, while also exposing an important dependency: errors in Query 1 propagate directly into Query 2.

5. Dataset construction, supervision, and evaluation protocol

Experiments are conducted on the VocV4 dataset, which contains 10 320 train utterances and 10 192 test utterances, with each utterance re-synthesized by four neural vocoders and with equal genuine versus spoof examples (Do et al., 17 Jun 2026). Spectrograms are STFT magnitudes, and difference maps are smoothed by a separable Gaussian $K$0.

The explanation texts $K$1 are generated by a Qwen3-VL-30B and Qwen3-235B pipeline with manual correction (Do et al., 17 Jun 2026). This indicates that free-form evidence supervision is not obtained directly from raw annotations but through a model-assisted authoring process followed by manual refinement.

The evaluation protocol separates localization, structured-field prediction, grounding quality, and text similarity. The paper reports the following metrics:

Category	Metrics
Localization	R@3, nDCG, mAP
Structured fields	$K$2
Explanation grounding	$K$3
Text similarity	ROUGE-L, BERTScore F1

This metric design reflects the paper's claim that explanation quality is not reducible to text overlap alone. In particular, the distinction between $K$4 and $K$5 separates exact prediction of categorical fields from grounding coverage relative to the selected regions.

The paper also provides a pseudocode outline. Given spectrogram $K$6 and transcript $K$7, the pipeline partitions $K$8 into $K$9 cells, applies Query1 to obtain $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$0, and then, for each selected cell, applies Query2 to produce $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$1, returning both localization and explanations (Do et al., 17 Jun 2026).

6. Empirical results, ablations, and limitations

On Query 1 localization, GRIDEX reports $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$2, $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$3, and $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$4 in the end-to-end setting (Do et al., 17 Jun 2026). The best VLM baseline listed in the paper, InternVL3-78B, reports approximately $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$5, $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$6, and $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$7. On Query 2 end-to-end structured explanations, GRIDEX reports $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$8, $r\subset\{1,\dots,T\}\times\{1,\dots,F\}$9, $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$0, and $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$1, whereas Qwen3-VL-8B end-to-end reports $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$2, $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$3, $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$4, and $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$5 (Do et al., 17 Jun 2026).

The staged optimization ablations isolate the contribution of GRPO after supervised fine-tuning. For localization, SFT-1 to GRPO-1 increases R@3 from $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$6 to $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$7 and nDCG from $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$8 to $$D(t,f)=\frac{\bigl|G(M_s)(t,f)-G(M_b)(t,f)\bigr|}{G(M_b)(t,f)+\epsilon},\qquad \mathrm{Mask}(t,f)=\mathbf1_{D(t,f)>\tau},\;\tau=\mathrm{Quantile}_{0.95}(D).$$9. For explanation generation, SFT-2 to GRPO-2 increases $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$0 from $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$1 to $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$2 and $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$3 from $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$4 to $$A(r;\mathbf S)=\sum_{(t,f)\in r}\mathrm{Mask}(t,f).$$5 (Do et al., 17 Jun 2026). These ablations support the paper's claim that the staged SFT + GRPO training substantially improves region retrieval and explanation grounding.

The limitations identified by the authors are specific. Query 1 can introduce error propagation into Query 2, and there is a “hotspot” bias toward a few frequent regions (Do et al., 17 Jun 2026). The paper also reports lower accuracy on phonetic field extraction. Future directions proposed in the discussion include more diverse region proposals, tighter feedback loops between localization and explanation, cross-dataset generalization, and richer acoustic fields beyond the temporal-spectral-phonetic schema.

A common misconception would be to treat GRIDEX as a generic saliency method for deepfake detection. The paper instead defines it as a structured forensic explanation framework with regional grounding, fixed-cell localization, and tuple-based outputs tied to temporal, spectral, and phonetic attributes (Do et al., 17 Jun 2026). A plausible implication is that its primary contribution lies in evidential organization and verifiability rather than in replacing the underlying deepfake detector with a purely explanatory model.