---
title: 'MRAD-FT: Fine-Tuned Anomaly Detection'
url: https://www.emergentmind.com/topics/mrad-ft
type: topic
---

# MRAD-FT: Fine-Tuned Anomaly Detection

MRAD-FT (Memory-Retrieval Anomaly Detection – Fine-Tuned) refers to a lightweight fine-tuning variant of the MRAD-TF (train-free) model for zero-shot anomaly detection tasks. It is designed to enhance the discrimination between normal and anomalous samples in image-level anomaly classification and pixel-level anomaly segmentation, particularly leveraging large vision-language models such as CLIP while maintaining a frozen backbone and offering state-of-the-art performance with minimal computational overhead [2602.00522].

## 1. Conceptual Framework and Architecture

MRAD-FT builds upon the MRAD-TF paradigm by incorporating a two-level memory bank and a frozen CLIP ViT-L/14-336 backbone. The model separates the CLIP image encoder into two branches: a global branch $\Phi_{\text{cls}}$ yielding the class token, and a local V-V attention branch $\Phi_{\text{vv}}$ producing $R$ patch tokens. The memory structure consists of:

- **Image-level memory:** $K_{\text{cls}} \in \mathbb{R}^{N_c \times d}$ (class tokens), with corresponding one-hot labels $V_{\text{cls}} \in \{0,1\}^{N_c \times 2}$ (normal, anomaly).
- **Pixel-level memory:** $K_{\text{pat}} \in \mathbb{R}^{N_p \times d}$ (patch tokens), with $V_{\text{pat}} \in \{0,1\}^{N_p \times 2}$.

All features are $\ell_2$-normalized.

MRAD-FT introduces two sets of trainable $d \times d$ projection matrices $(W_q^{\text{cls}}, W_k^{\text{cls}}, W_q^{\text{seg}}, W_k^{\text{seg}})$ operating on query and key vectors at image and patch levels, respectively. Given a test image $I$, query features $Q_{\text{cls}}$ (class token) and $Q_{\text{pat}}$ (patch tokens) are extracted. Retrieval logits in a calibrated subspace are computed:

\[
\begin{aligned}
Y_{\text{cls}}^{n/a} &= \mathrm{softmax}\left(\frac{(Q_{\text{cls}} W_q^{\text{cls}})(K_{\text{cls}} W_k^{\text{cls}})^\top}{\tau} + M_p(Q_{\text{cls}}, K_{\text{cls}})\right)V_{\text{cls}}^{n/a}\\
Y_{\text{seg}}^{n/a} &= \mathrm{softmax}\left(\frac{(Q_{\text{pat}} W_q^{\text{seg}})(K_{\text{pat}} W_k^{\text{seg}})^\top}{\tau} + M_p(Q_{\text{pat}}, K_{\text{pat}})\right)V_{\text{pat}}^{n/a}
\end{aligned}
\]

Here, $M_p(\cdot,\cdot)$ denotes a similarity-dropout operator used during training to mask the top-$p\%$ similarities, acting as hard negative mining while preventing trivial retrieval. $\tau$ is the temperature parameter.

## 2. Training Objective and Optimization

MRAD-FT is trained end-to-end on auxiliary datasets by optimizing both image-level and patch-level objectives. Given ground truth image label $y \in \{0,1\}$ and downsampled pixel mask $M \in \{0,1\}^R$:

\[
\mathcal{L} = \underbrace{\mathrm{BCE}(Y_{\text{cls}}, y)}_{\mathcal{L}_{\text{cls}}}
+ \underbrace{\mathrm{Dice}(Y_{\text{seg}}, M) + \mathrm{Focal}(Y_{\text{seg}}, M)}_{\mathcal{L}_{\text{seg}}}
\]

No additional regularization or margin terms are applied. Instead, $M_p$ with $p=5\%$ (classification) and $p=20\%$ (segmentation) acts as hard negative mining. The optimizer is Adam with a learning rate of $5 \times 10^{-4}$, batch size 8, for one training epoch. Only $W_q$ and $W_k$ (total $2d^2$ parameters; $\sim$2.8M for $d=768$) are learned, and the CLIP backbone remains frozen throughout.

## 3. Inference Mechanism and Decision Process

During inference, the similarity-dropout $M_p$ is disabled. The final anomaly score for a test image $I$ is computed as:

\[
S(I) = Y_{\text{cls}}^a + \mathrm{TopKMean}\left(Y_{\text{seg}}^a\right)
\]

where $\mathrm{TopKMean}$ averages the highest $1\%$ of patch-level anomaly scores, emphasizing the most salient regions and suppressing background noise. An image is classified as anomalous if $S(I)$ exceeds a threshold.

## 4. Empirical Performance and Comparative Analysis

Across sixteen industrial and medical benchmarks, including MVTec-AD, VisA, BTAD, MPDD, and others, MRAD-FT consistently surpasses the train-free MRAD-TF baseline and other prompt or CLIP-based methods (AdaCLIP, AnomalyCLIP, FAPrompt):

| Metric                | MRAD-TF        | MRAD-FT        |
|-----------------------|----------------|----------------|
| Pixel AUROC (%)       | 85.5           | 91.9           |
| Pixel PRO (%)         | 64.6           | 78.3           |
| Image AUROC (%)       | 81.0           | 92.0           |
| Image AP (%)          | 83.2           | 91.9           |

Notably, on MVTec-AD, MRAD-FT achieves 92.2% pixel-level AUROC and 92.3% image-level AUROC (versus 86.7% and 79.0% for MRAD-TF, respectively), demonstrating up to a 6-point improvement with the addition of only two linear projection layers.

## 5. Ablation Studies and Design Insights

- **Metric calibration**: The anomaly-on-anomaly versus normal-on-anomaly similarity gap (A$_q$A$_k$ – N$_q$A$_k$) increases from ~0.08 (frozen) to ~0.16 (after fine-tuning), indicating a sharper separation of normal/anomalous features.
- **Two-level memory**: Removing the image-level memory branch degrades image-level AUROC by up to 3 points; discarding pixel-level memory reduces both localization (PRO) and image-level AUROC by 2–4 points. This establishes the complementarity of global and local memories.
- **Memory budget**: Reducing the patch memory from ~3,000 to 100 entries results in ≤1 point AUROC loss; MRAD-FT is robust to memory size reductions.

## 6. Implementation and Engineering Details

- **Auxiliary data:** VisA (2,162 images, 3,093 patches) or MVTec-AD for memory bank construction.
- **Resolution:** Images resized to $518 \times 518$; CLIP ViT-L/14-336 backbone used, always frozen.
- **Feature extraction:** Class tokens ($\Phi_{\text{cls}}$) and $R=14 \times 14$ patch tokens ($\Phi_{\text{vv}}$) are stored with one-hot labels.
- **Hyperparameters:** Temperature $\tau=1$; mask thresholds $p_{\text{cls}}=5\%$, $p_{\text{seg}}=20\%$; top-k pooling fraction $k=1\%$.
- **Computational profile:** Peak memory <8 GB; parameter increase <$3$M; training converges in a single epoch on NVIDIA RTX 3090.
- **Additional techniques:** Similarity-dropout $M_p$ prevents trivial retrieval; V-V attention ($\Phi_{\text{vv}}$) preserves local structure; $\ell_2$ normalization stabilizes similarity computations.

## 7. Significance and Context Within Anomaly Detection

MRAD-FT exemplifies a non-parametric, memory-driven approach that leverages the empirical distribution of auxiliary data, departing from conventional parametric or prompt-tuned anomaly detection strategies. Its architectural simplicity (adding two learned projections), training efficiency (single-epoch convergence), and frozen backbone requirement position it as a compelling solution for scenarios requiring both high statistical efficiency and cross-domain robustness. The framework sets new baselines in both image- and pixel-level anomaly detection and segmentation across heterogeneous datasets without incurring the high computational or modeling cost of alternative approaches [2602.00522].

Source: https://www.emergentmind.com/topics/mrad-ft