---
title: X-GRAAD Anomaly Scoring Explained
url: https://www.emergentmind.com/topics/x-graad-anomaly-scoring
type: topic
---

# X-GRAAD Anomaly Scoring Explained

X-GRAAD (eXplaining Gradient and Attention Anomaly Detection) anomaly scoring encompasses a family of explainable, gradient-based and attention-based scoring mechanisms for detecting data or behavior deviating from an assumed normal background distribution. Originally developed to provide pixel-wise anomaly localization in generative models for medical images and later extended to explainable backdoor trigger detection in neural language models, X-GRAAD approaches exploit the concentration of gradients and, where relevant, attention weights caused by anomalous or adversarial inputs. This turns model internals into discriminative signals for flagging out-of-distribution phenomena and malicious artifacts [2510.04347, 1912.00003].

## 1. Conceptual Foundations and Motivation

X-GRAAD anomaly scoring is rooted in the observation that, for both generative and discriminative neural models, the presence of anomalies can profoundly alter internal model sensitivities with respect to input components. In autoencoding VAEs, regions of input not supported by the training data induce high gradients of the model log-likelihood w.r.t. input pixels. In transformer-based language models with backdoor triggers, input tokens that have been co-opted by an attacker cause attention heads and output logits to become highly sensitive to the trigger token, exhibiting both “attention drift” and “gradient dominance” [2510.04347, 1912.00003].

The core objective is to provide a per-component (pixel or token) anomaly score that accurately localizes abnormal structure and a robust sequence- or image-level score for detection and downstream filtering. A central property of X-GRAAD is explainability: the highest scoring component (pixel or token) is interpretable as the likely anomaly source.

## 2. Mathematical Formulations

### 2.1. VAE Gradient-Based Anomaly Scoring (Imaging)

Let $x \in \mathbb{R}^D$ be an input (image, typically), $z \in \mathbb{R}^L$ a latent code. The VAE maximizes the evidence lower bound:

\[
\log p_\theta(x) \geq \mathcal{L}(x; \theta, \phi) = \mathbb{E}_{z \sim q_\phi(z|x)}[\log p_\theta(x|z)] - D_{\mathrm{KL}}(q_\phi(z|x)\|p(z))
\]

The X-GRAAD scoring function is the per-pixel (or per-dimension) norm of the ELBO gradient:

\[
S_i(x) = \left| \left[ \nabla_x \mathcal{L}(x; \theta, \phi) \right]_i \right|
\]

Approximated via backpropagation through the standard VAE objective and, typically, “SmoothGrad” (Gaussian input noise plus averaging) to suppress artifacts [1912.00003].

### 2.2. Attention-Gradient Scoring for Backdoored PLMs

For an input $x = (t_1, ..., t_n)$ to a transformer with $L$ layers and $H$ heads, let $A_{l_i}^{h_j} \in \mathbb{R}^{n \times n}$ denote the softmax attention matrix for head $h_j$ in layer $l_i$. The attention importance for token $t_k$:

\[
\mathrm{AttnImp}_x(t_k) = \sum_{k'=1}^n \bar a_{k', k} \ , \quad \bar A = \frac{1}{LH} \sum_{i=1}^L \sum_{j=1}^H A_{l_i}^{h_j}
\]

Zero-mean normalization yields:

\[
\mathrm{AttnScore}_x(t_k) = \mathrm{AttnImp}_x(t_k) - \frac{1}{n} \sum_{k'=1}^n \mathrm{AttnImp}_x(t_{k'})
\]

Gradient importance is defined as the L2 norm of the output logit gradient w.r.t. token embedding $e_k$:

\[
\mathrm{GradImp}_x(t_k) = \| \partial \ell / \partial e_k \|_2 \\
\mathrm{GradScore}_x(t_k) = \frac{\mathrm{GradImp}_x(t_k)}{ \frac{1}{n} \sum_{k'=1}^n \mathrm{GradImp}_x(t_{k'}) }
\]

The combined per-token anomaly score (with sentence-level score as the tokenwise maximum):

\[
\mathrm{Score}_x(t_k) = \mathrm{AttnScore}_x(t_k) \cdot \mathrm{GradScore}_x(t_k) \\
\psi(x) = \max_{1 \leq k \leq n} \mathrm{Score}_x(t_k)
\]

## 3. Algorithms and Inference-Time Procedures

### VAE Anomaly Localization and SmoothGrad

Given learned VAE parameters $(\theta, \phi)$, the following routine yields a pixelwise anomaly map:

1. Normalize and (optionally) resize each test input $x$.
2. For $k=1...K$ (e.g., $K=20$–$50$):
   - Add small Gaussian noise: $x_k = x + \mathrm{noise}_k$.
   - Forward: obtain encoder posterior, sample $z_k$.
   - Compute ELBO $\mathcal{L}_k$.
   - Backpropagate: $g_k = \nabla_{x_k} \mathcal{L}_k$.
3. Aggregate: $G = (1/K) \sum_k g_k$.
4. Score: $\mathrm{ScoreMap} = |G|$ [1912.00003].

### Transformer Backdoor Defense Workflow

1. Precompute anomaly scores $\psi(x')$ over a clean validation set for threshold estimation.
2. Set detection threshold $\tau$ as the $p$-th percentile of $\{\psi(x')\}$ ($p=95$ for BERT-class models; $p=65$ for ALBERT).
3. For new input $x$:
   - Compute $\psi(x)$.
   - If $\psi(x)<\tau$: proceed with standard prediction.
   - Otherwise: locate $\arg\max_k \mathrm{Score}_x(t_k)$; corrupt the flagged token by random character insertion/replacement and re-evaluate prediction.
4. Output the (possibly sanitized) prediction [2510.04347].

## 4. Experimental Results and Benchmarks

### Imaging (BraTS-2017 Tumor Localization)

| Method                       | Pixelwise ROC-AUC           |
|------------------------------|-----------------------------|
| Denoising AE (recon error)   | $0.808 \pm 0.009$           |
| VAE recon error              | $0.817 \pm 0.003$           |
| Smoothed recon error         | $0.843 \pm 0.008$           |
| VAE sampling variance        | $0.855 \pm 0.013$           |
| Grad (recon term only)       | $0.894 \pm 0.020$           |
| Grad (KL term only)          | $0.939 \pm 0.007$           |
| Full ELBO grad (X-GRAAD)     | $0.939 \pm 0.008$           |

X-GRAAD (full ELBO gradient) matches or outperforms all prior unsupervised methods for unsupervised tumor localization in MRI [1912.00003].

### Backdoor Detection in PLMs

On SST-2, IMDb, AG's News (datasets) and several backdoor attacks (BadNets, RIPPLES, LWS) for BERT, RoBERTa, DistilBERT, ALBERT:

- X-GRAAD reduces attack success rate (ASR) from nearly $1.0$ to $0.0$–$0.03$ in most settings (compared to $\geq0.15$ for prior methods).
- Maintains clean accuracy within $1$–$2\%$ of undefended performance.
- Ablations: Only the combined attention-gradient score achieves ASR $\approx 0$ while preserving clean accuracy [2510.04347].

Example (BERT + SST-2 + BadNets):

| Method     | ASR     | CACC    |
|------------|---------|---------|
| ONION      | 0.142   | —       |
| RAP        | 0.002   | —       |
| FT         | 1.0     | —       |
| MEFT       | 0.998   | —       |
| PURE       | 0.292   | —       |
| X-GRAAD    | 0.0     | 0.923   |
| Undefended | —       | 0.931   |

Computational cost: X-GRAAD inference-time scoring on SST-2 requires ≈44–50s and is $30\times$ faster than head-pruning methods; no retraining is required [2510.04347].

## 5. Interpretability, Explainability, and Visualization

A core property of X-GRAAD scores is their attributional clarity:

- In VAEs, per-pixel score maps highlight only out-of-distribution regions (e.g., tumor voxels in brain MRI), with intensity proportional to the magnitude of gradient-based anomaly.
- In transformer models, heatmaps over tokens show only rare trigger tokens (“cf”, “mn”, “tq”) as prominently anomalous: the maximally scoring token is flagged for targeted corruption/remediation.
- The separation between $\max \mathrm{AttnScore}$, $\max \mathrm{GradScore}$, and the combined $\psi(x)$ on clean vs. poisoned data is visualized via histograms; only $\psi(x)$ provides strong class separation [2510.04347].
- The decomposition of $\mathrm{Score}_x(t_k)$ into attention and gradient factors enables fine-grained analysis of whether anomalies stem from syntactic/model-level (attention) or output-sensitivity (gradient) disruptions.

## 6. Relations to General Anomaly Scoring and Mitigation

X-GRAAD fits within a broader taxonomy of anomaly scoring, which includes statistical (z-score, p-value, meta-rarity), distance-based (Euclidean, Mahalanobis, kNN), density-based (Local Outlier Factor, mass-volume), and reconstruction-based methods [1904.06646]. X-GRAAD is a member of gradient-based and (for text) attention-enhanced reconstruction/density approximation approaches.

Threshold-setting and filtering strategies, such as dynamic/percentile thresholds, ROC/PR curve optimization, and tail modelling (e.g., extreme-value theory), are essential for robust deployment. The percentile threshold in X-GRAAD is empirically tuned by backbone and can be adapted in streaming or changing environments via monitoring of validation-set scores [1904.06646, 2510.04347].

A plausible implication is that X-GRAAD anomaly scores could be integrated with ensemble or hybrid methods—combining, for example, density-based and reconstruction-based scores for further robustness, as recommended in general anomaly detection systems [1904.06646].

## 7. Limitations and Prospects for Extension

Limitations of current X-GRAAD methods include:

- Pixel-wise scores can exhibit noise or checkerboarding (imaging) if model or data are not well calibrated; smoothing methods such as “SmoothGrad” are necessary but may introduce blurring [1912.00003].
- For “far-out” outliers (inputs for which the generative model assigns near-zero density), scores may collapse or become meaningless.
- Slice-wise operation in imaging ignores 3D context; extension to volumetric or spatiotemporal modeling is a proposed direction.
- In NLP, thresholds must be empirically tuned by architecture, and a wrong setting may affect recall or precision.
- The method relies on model gradients and access to internal attention weights, precluding use with closed-box or inflexible models.

Future work includes more expressive base density models (e.g., Glow, PixelCNN++), joint score aggregation across model layers, learned regularization for anomaly map smoothing, and cross-domain score fusion [1912.00003]. For detection in large-scale or streaming settings, adaptive, persistence-based, and ensemble-based filtering procedures are promising avenues to further reduce false alarms and support high-throughput operation [1904.06646].

Source: https://www.emergentmind.com/topics/x-graad-anomaly-scoring