X-GRAAD Anomaly Scoring Explained

• X-GRAAD anomaly scoring is an explainable method that uses gradient and attention signals to detect deviations from a normal background.
• It employs pixel-wise ELBO gradient analysis in VAEs and token-level attention-gradient measures in transformer models to localize anomalies.
• Experimental results show robust performance with high ROC-AUC for tumor localization and near-zero attack success in NLP backdoor detection.

X-GRAAD (eXplaining Gradient and Attention Anomaly Detection) anomaly scoring encompasses a family of explainable, gradient-based and attention-based scoring mechanisms for detecting data or behavior deviating from an assumed normal background distribution. Originally developed to provide pixel-wise anomaly localization in generative models for medical images and later extended to explainable backdoor trigger detection in neural LLMs, X-GRAAD approaches exploit the concentration of gradients and, where relevant, attention weights caused by anomalous or adversarial inputs. This turns model internals into discriminative signals for flagging out-of-distribution phenomena and malicious artifacts (Das et al., 5 Oct 2025, Zimmerer et al., 2019).

1. Conceptual Foundations and Motivation

X-GRAAD anomaly scoring is rooted in the observation that, for both generative and discriminative neural models, the presence of anomalies can profoundly alter internal model sensitivities with respect to input components. In autoencoding VAEs, regions of input not supported by the training data induce high gradients of the model log-likelihood w.r.t. input pixels. In transformer-based LLMs with backdoor triggers, input tokens that have been co-opted by an attacker cause attention heads and output logits to become highly sensitive to the trigger token, exhibiting both “attention drift” and “gradient dominance” (Das et al., 5 Oct 2025, Zimmerer et al., 2019).

The core objective is to provide a per-component (pixel or token) anomaly score that accurately localizes abnormal structure and a robust sequence- or image-level score for detection and downstream filtering. A central property of X-GRAAD is explainability: the highest scoring component (pixel or token) is interpretable as the likely anomaly source.

2. Mathematical Formulations

2.1. VAE Gradient-Based Anomaly Scoring (Imaging)

Let $x \in \mathbb{R}^D$ be an input (image, typically), $z \in \mathbb{R}^L$ a latent code. The VAE maximizes the evidence lower bound:

$$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$

The X-GRAAD scoring function is the per-pixel (or per-dimension) norm of the ELBO gradient:

$$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$

Approximated via backpropagation through the standard VAE objective and, typically, “SmoothGrad” (Gaussian input noise plus averaging) to suppress artifacts (Zimmerer et al., 2019).

2.2. Attention-Gradient Scoring for Backdoored PLMs

For an input $x = (t_1, ..., t_n)$ to a transformer with $L$ layers and $H$ heads, let $A_{l_i}^{h_j} \in \mathbb{R}^{n \times n}$ denote the softmax attention matrix for head $h_j$ in layer $l_i$. The attention importance for token $z \in \mathbb{R}^L$0:

$z \in \mathbb{R}^L$1

Zero-mean normalization yields:

$z \in \mathbb{R}^L$2

Gradient importance is defined as the L2 norm of the output logit gradient w.r.t. token embedding $z \in \mathbb{R}^L$3:

$z \in \mathbb{R}^L$4

The combined per-token anomaly score (with sentence-level score as the tokenwise maximum):

$z \in \mathbb{R}^L$5

3. Algorithms and Inference-Time Procedures

VAE Anomaly Localization and SmoothGrad

Given learned VAE parameters $z \in \mathbb{R}^L$6, the following routine yields a pixelwise anomaly map:

1. Normalize and (optionally) resize each test input $z \in \mathbb{R}^L$7.
2. For $z \in \mathbb{R}^L$8 (e.g., $z \in \mathbb{R}^L$9–$$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$0):
   • Add small Gaussian noise: $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$1.
   • Forward: obtain encoder posterior, sample $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$2.
   • Compute ELBO $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$3.
   • Backpropagate: $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$4.
3. Aggregate: $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$5.
4. Score: $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$6 (Zimmerer et al., 2019).

Transformer Backdoor Defense Workflow

1. Precompute anomaly scores $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$7 over a clean validation set for threshold estimation.
2. Set detection threshold $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$8 as the $$\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))$$9-th percentile of $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$0 ($$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$1 for BERT-class models; $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$2 for ALBERT).
3. For new input $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$3:
   • Compute $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$4.
   • If $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$5: proceed with standard prediction.
   • Otherwise: locate $$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$6; corrupt the flagged token by random character insertion/replacement and re-evaluate prediction.
4. Output the (possibly sanitized) prediction (Das et al., 5 Oct 2025).

4. Experimental Results and Benchmarks

Imaging (BraTS-2017 Tumor Localization)

Method	Pixelwise ROC-AUC
Denoising AE (recon error)	$$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$7
VAE recon error	$$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$8
Smoothed recon error	$$S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|$$9
VAE sampling variance	$x = (t_1, ..., t_n)$0
Grad (recon term only)	$x = (t_1, ..., t_n)$1
Grad (KL term only)	$x = (t_1, ..., t_n)$2
Full ELBO grad (X-GRAAD)	$x = (t_1, ..., t_n)$3

X-GRAAD (full ELBO gradient) matches or outperforms all prior unsupervised methods for unsupervised tumor localization in MRI (Zimmerer et al., 2019).

Backdoor Detection in PLMs

On SST-2, IMDb, AG's News (datasets) and several backdoor attacks (BadNets, RIPPLES, LWS) for BERT, RoBERTa, DistilBERT, ALBERT:

• X-GRAAD reduces attack success rate (ASR) from nearly $x = (t_1, ..., t_n)$4 to $x = (t_1, ..., t_n)$5–$x = (t_1, ..., t_n)$6 in most settings (compared to $x = (t_1, ..., t_n)$7 for prior methods).
• Maintains clean accuracy within $x = (t_1, ..., t_n)$8–$x = (t_1, ..., t_n)$9 of undefended performance.
• Ablations: Only the combined attention-gradient score achieves ASR $L$0 while preserving clean accuracy (Das et al., 5 Oct 2025).

Example (BERT + SST-2 + BadNets):

Method	ASR	CACC
ONION	0.142	—
RAP	0.002	—
FT	1.0	—
MEFT	0.998	—
PURE	0.292	—
X-GRAAD	0.0	0.923
Undefended	—	0.931

Computational cost: X-GRAAD inference-time scoring on SST-2 requires ≈44–50s and is $L$1 faster than head-pruning methods; no retraining is required (Das et al., 5 Oct 2025).

5. Interpretability, Explainability, and Visualization

A core property of X-GRAAD scores is their attributional clarity:

• In VAEs, per-pixel score maps highlight only out-of-distribution regions (e.g., tumor voxels in brain MRI), with intensity proportional to the magnitude of gradient-based anomaly.
• In transformer models, heatmaps over tokens show only rare trigger tokens (“cf”, “mn”, “tq”) as prominently anomalous: the maximally scoring token is flagged for targeted corruption/remediation.
• The separation between $L$2, $L$3, and the combined $L$4 on clean vs. poisoned data is visualized via histograms; only $L$5 provides strong class separation (Das et al., 5 Oct 2025).
• The decomposition of $L$6 into attention and gradient factors enables fine-grained analysis of whether anomalies stem from syntactic/model-level (attention) or output-sensitivity (gradient) disruptions.

6. Relations to General Anomaly Scoring and Mitigation

X-GRAAD fits within a broader taxonomy of anomaly scoring, which includes statistical (z-score, p-value, meta-rarity), distance-based (Euclidean, Mahalanobis, kNN), density-based (Local Outlier Factor, mass-volume), and reconstruction-based methods (Zohrevand et al., 2019). X-GRAAD is a member of gradient-based and (for text) attention-enhanced reconstruction/density approximation approaches.

Threshold-setting and filtering strategies, such as dynamic/percentile thresholds, ROC/PR curve optimization, and tail modelling (e.g., extreme-value theory), are essential for robust deployment. The percentile threshold in X-GRAAD is empirically tuned by backbone and can be adapted in streaming or changing environments via monitoring of validation-set scores (Zohrevand et al., 2019, Das et al., 5 Oct 2025).

A plausible implication is that X-GRAAD anomaly scores could be integrated with ensemble or hybrid methods—combining, for example, density-based and reconstruction-based scores for further robustness, as recommended in general anomaly detection systems (Zohrevand et al., 2019).

7. Limitations and Prospects for Extension

Limitations of current X-GRAAD methods include:

• Pixel-wise scores can exhibit noise or checkerboarding (imaging) if model or data are not well calibrated; smoothing methods such as “SmoothGrad” are necessary but may introduce blurring (Zimmerer et al., 2019).
• For “far-out” outliers (inputs for which the generative model assigns near-zero density), scores may collapse or become meaningless.
• Slice-wise operation in imaging ignores 3D context; extension to volumetric or spatiotemporal modeling is a proposed direction.
• In NLP, thresholds must be empirically tuned by architecture, and a wrong setting may affect recall or precision.
• The method relies on model gradients and access to internal attention weights, precluding use with closed-box or inflexible models.

Future work includes more expressive base density models (e.g., Glow, PixelCNN++), joint score aggregation across model layers, learned regularization for anomaly map smoothing, and cross-domain score fusion (Zimmerer et al., 2019). For detection in large-scale or streaming settings, adaptive, persistence-based, and ensemble-based filtering procedures are promising avenues to further reduce false alarms and support high-throughput operation (Zohrevand et al., 2019).