---
title: Visual Anomaly Detection Overview
url: https://www.emergentmind.com/topics/visual-anomaly-detection-vad
type: topic
---

# Visual Anomaly Detection Overview

Searching arXiv for recent papers on visual anomaly detection, including image, video, deployment, and interpretability aspects.
Visual anomaly detection (VAD) denotes the identification of images, frames, or regions that deviate from normal patterns, usually under severe anomaly scarcity and often with training performed on normal data only. In still-image inspection, the canonical outputs are **image-level anomaly detection** and **pixel-level anomaly segmentation**; in surveillance literature, the same acronym is also used for video anomaly detection, where the objective is temporal or spatio-temporal identification of abnormal events [2507.12049][2405.19387]. Across industrial inspection, logical anomaly analysis, medical imaging, surveillance, and autonomous driving, the field is shaped by anomaly rarity, noisy or weak supervision, domain gaps, and increasingly by deployment constraints such as memory, latency, bandwidth, and on-device execution [2507.12049][2412.03342][2505.07119].

## 1. Task formulations and problem settings

In the standard image-based setting, a model learns the distribution of normal images and computes an **anomaly score** at test time; high scores indicate a deviation from normal. The two main tasks are deciding whether an image is normal or anomalous and localizing anomalous regions with an anomaly map or pixel-wise mask [2507.12049]. The basic assumption is often **unsupervised** or **one-class** training, but modern formulations also include **noisy unsupervised**, **continual**, **supervised**, **semi-supervised**, **few-shot**, **edge**, and **IoT** scenarios. In the noisy unsupervised case, image-level contamination is explicitly parameterized by
$$
\frac{M}{N+M} \approx C,
$$
where \(N\) normal images and \(M\) anomalous images are mixed into training, making robustness to imperfect curation a first-class evaluation target [2507.12049].

Video work broadens this picture. The generalized taxonomy of video anomaly event detection distinguishes **UVAD** (unsupervised, normal-only), **WAED** (weakly supervised, typically video-level labels), **SVAD** (supervised), and **FVAD** (fully unsupervised, no assumptions on data composition) [2302.05087]. A complementary survey organizes deep video VAD by supervised, weakly supervised, self-supervised, and unsupervised paradigms, and frames anomalies as events or behaviors that notably diverge from anticipated spatio-temporal patterns [2405.19387]. This implies that “normality” may be modeled as appearance regularity, motion regularity, temporal consistency, object-scene compatibility, or some combination thereof.

A further reformulation appears in generalized few-shot VAD. UniVAD targets previously unseen industrial, logical, and medical categories with a **training-free unified model** that uses only a few normal reference images at test time, explicitly rejecting the common “one-category-one-model” paradigm [2412.03342]. A plausible implication is that contemporary VAD is no longer defined solely by normal-only training, but by how flexibly normality can be instantiated under different data regimes.

## 2. Methodological families

Current image-based VAD is usually organized into a small number of recurrent families, each defined by the signal used to characterize normality and compute deviations [2507.12049].

| Family | Representative methods | Core normality signal |
|---|---|---|
| Memory bank-based | PatchCore, CFA, PaDiM | distance to stored features or Gaussian models |
| Student–Teacher | STFPM, PaSTe, RD4AD | feature discrepancy after distillation |
| Reconstruction-based | DRÆM, Ganomaly | reconstruction error |
| Flow-based | FastFlow | inverse log-likelihood |
| Adversarial | SuperSimpleNet | adversarially learned anomaly score |

For memory-style methods, the anomaly score is conceptually a maximum over patch-level distances,
$$
s(x) = \max_{p \in \text{patches}} \text{distance}(f_p(x), \mathcal{M}),
$$
where \(f_p(x)\) is a patch feature and \(\mathcal{M}\) is a memory bank or a Gaussian model. Student–teacher methods instead optimize feature mimicry on normal data with a distillation loss such as
$$
\mathcal{L}_{\text{KD}} = \sum_l \|f_l^T(x) - f_l^S(x)\|^2,
$$
so that test-time deviations in teacher–student features become anomaly signals. Reconstruction-based methods rely on losses of the form
$$
\mathcal{L}_{\text{rec}} = \|x-\hat{x}\|_1 \quad \text{or} \quad \|x-\hat{x}\|_2^2,
$$
whereas FastFlow models normal feature density and scores anomalies through
$$
s(x) = -\log p(f(x)).
$$
The same framework also decouples backbones from anomaly heads, enabling swaps between ResNet18, WideResNet, MobileNetV1/V2, PhiNet, MCUNet, and MicroNet without rewriting the VAD logic [2507.12049].

Video VAD extends these families into explicitly temporal formulations. Weakly supervised methods often use a multiple-instance learning ranking objective so that the highest-scoring segment in an anomalous video exceeds the highest-scoring segment in a normal video, usually with added smoothness and sparsity regularization [2302.05087]. Self-supervised alternatives can replace pixel reconstruction with relation learning: PSTRP predicts patch order and inter-patch similarity in spatial and temporal dimensions, treating anomalies as violations of learned patch relations rather than raw pixel prediction errors [2403.19111]. Likelihood-based video methods further reframe VAD through denoising score matching; the autoregressive DSM approach combines a scene-dependent, motion-aware score function with an appearance term based on PSNR between original and denoised clips to address anomalies near local modes of the learned normal distribution [2506.23282].

Several papers also re-embed semantics into the detector. Object class aware video anomaly detection through image translation learns frame-to-segmentation and frame-to-optical-flow-magnitude mappings, so that deviations in expected semantic layout or motion magnitude become directly explainable anomaly signals [2205.01706]. SlowFastVAD pushes this further by combining a fast masked-autoencoder detector with a slow RAG-enhanced vision-language model, routing only ambiguous segments to the slow branch and fusing both scores for improved accuracy and interpretability [2504.10320]. Earlier ConvLSTM-based video studies showed that variational autoencoders and \(\beta\)-VAE regularization improve discriminative power over plain autoencoders in reconstruction-based video VAD, reinforcing the importance of structured latent spaces in temporal anomaly modeling [2203.03872].

## 3. Benchmarks and evaluation protocols

Image-based VAD is evaluated on a diverse but highly domain-dependent benchmark ecosystem. Integrated libraries now standardize datasets such as **MVTec-AD**, **VisA**, **Real-IAD**, **MIIC**, **MVTec-LOCO**, and **BMAD**, spanning industrial, logical, and medical scenarios [2507.12049]. The MIIC semiconductor benchmark is particularly illustrative of domain dependence: among feature-based methods, **CFA** attains image-level ROC 99.01 and image-level F1 87.96, while **STFPM** gives the highest pixel-level F1 at 77.30; at the same time, the reconstruction-based **IAD+Inpainting** reaches image-level ROC 99.27 and F1 91.23, showing that reconstruction-based VAD remains competitive when abundant normal data are available and the domain gap to ImageNet-style pretraining is large [2505.07576].

The field also relies on standardized pixel-level and image-level metrics. For pixel labels \(y_i \in \{0,1\}\), anomaly scores \(\hat{s}_i\), and threshold \(\tau\), predicted labels are \(\hat{y}_i=\mathbb{1}[\hat{s}_i>\tau]\). Precision, recall, and F1 are then
$$
\text{Precision}(\tau)=\frac{\sum_i \mathbb{1}[\hat{y}_i=1 \land y_i=1]}{\sum_i \mathbb{1}[\hat{y}_i=1]},
$$
$$
\text{Recall}(\tau)=\frac{\sum_i \mathbb{1}[\hat{y}_i=1 \land y_i=1]}{\sum_i \mathbb{1}[y_i=1]},
$$
$$
F1(\tau)=2\cdot \frac{\text{Precision}(\tau)\cdot \text{Recall}(\tau)}{\text{Precision}(\tau)+\text{Recall}(\tau)}.
$$
ROC-AUC, PR-AUC, and PRO complete the standard image-VAD protocol, with analogous image-level variants based on one score per image [2507.12049].

Recent benchmarks deliberately stress failure modes that were largely absent from earlier datasets. M2AD introduces **119,880** high-resolution images from **999 specimens**, captured under **12 synchronized views** and **10 illumination settings**, yielding **120 configurations** per specimen and two protocols: **M2AD-Synergy**, which evaluates fusion across configurations, and **M2AD-Invariant**, which measures single-image robustness under realistic view–illumination interplay [2505.10996]. At \(512\times512/448\times448\), average M2AD-Synergy performance reaches 93.0 O-AUROC, 88.3 I-AUROC, and 93.4 AUPRO for Dinomaly, while M2AD-Invariant remains substantially harder, with the same method averaging 81.3 I-AUROC and 83.3 AUPRO. The benchmark therefore operationalizes a concrete gap between standard laboratory VAD and real-world imaging physics [2505.10996].

Video evaluation uses a different but related benchmark family. The main surveillance datasets include **UCSD Ped1/Ped2**, **CUHK Avenue**, **ShanghaiTech**, **UCF-Crime**, **XD-Violence**, and **NWPU Campus**, while newer studies add **UBnormal** for open-set supervised video anomaly detection and **AnoVox** for autonomous driving anomalies [2302.05087][2405.19387]. AnoVox contains **1,850 frames**, **10 road scenes**, and approximately **14.8%** anomalous camera frames, with image-level and pixel-level labels for content anomalies in road-relevant regions [2604.15291]. Video papers typically report frame-level Micro AUC and Macro AUC, and increasingly also provide pixel-level or region-level metrics when spatial masks are available [2504.10320].

## 4. Systems, efficiency, and deployment

A major contemporary shift is the treatment of VAD as a systems problem rather than only a modeling problem. MoViAD formalizes a modular pipeline
> Dataset → Backbone → VAD Method → Trainer → Evaluation + Profiling → (Quantization / Edge / IoT deployment)
and explicitly includes efficient backbones, quantization and compression utilities, image-level and pixel-level metrics, and profiling tools for model size, memory, and FLOPs [2507.12049]. This reframes reproducibility and deployability as part of the VAD research stack.

Edge-oriented work makes the trade-offs explicit. PaSTe, a partially shared teacher–student variant of STFPM, shows that sharing early teacher/student layers can preserve anomaly quality while substantially reducing resource use. With a MobileNetV2 backbone, STFPM to PaSTe changes inference MACs from 454.4 M to 341.2 M, training MACs from 297.5 M to 198.4 M, and training RAM from 96.15 MB to 22.9 MB, while pixel-level F1 changes from 0.52 to 0.53 [2410.11591]. The broader TinyAD benchmark in the same work also shows order-of-magnitude savings when heavy backbones are replaced by MobileNetV2, MCUNet, PhiNet, or MicroNet [2410.11591].

Bandwidth-constrained IoT deployment introduces a different optimization axis. In an Industrial IoT setting with a **100 kB/s** uplink, sending the original \(224\times224\) image yields payload 60 kB, transmission time 0.60 s, and total latency 0.71 s. By contrast, **WebP (quality 80)** reduces payload to **2 kB**, transmission to **0.02 s**, and total time to **0.14 s**, while preserving image-level ROC AUC almost exactly (0.984 to 0.982) and keeping pixel-level F1 close to baseline (0.570 to 0.555). Raw feature transmission, at **382 kB** and **3.94 s**, is explicitly shown to be dominated by communication cost [2505.07119]. A plausible implication is that, under realistic IIoT bandwidth limits, compression strategy can matter more than detector architecture.

Autonomous-driving deployment sharpens the accuracy–efficiency trade-off further. On AnoVox, **Tiny-Dinomaly** with a DeiT-Tiny backbone achieves **P-ROC 1.00**, **P-F1 0.69**, **P-AP 0.66**, **PRO 0.90**, a **37.6 MB** memory footprint, and **19 ms** inference, while matching Dinomaly’s full-scale localization performance at a fraction of the memory cost [2604.15291]. The same benchmark identifies **PaDiM + MobileNet** and **CFA + MobileNet** as particularly attractive when memory constraints are more severe [2604.15291].

## 5. Interpretability and explanation

Classical VAD explanations are predominantly spatial: anomaly maps indicate **where** something is abnormal, but not necessarily **what** or **why**. Concept-bottleneck VAD addresses this limitation directly. CONVAD introduces a concept dataset, a concept bottleneck branch, a student–teacher visual branch, and a synthetic anomaly generation pipeline so that concept-level explanations and pixel-level localization coexist in the same system [2511.20088]. In the fully supervised CBM setting on MVTec, the CBM branch reaches average **I-AUC = 0.97** and **C-AUC = 0.86**, while the visual branch reaches **I-AUC = 0.96**, **P-AUC = 0.97**, and **P-F1 = 0.50**, close to MobileNet-v2 PatchCore and STFPM baselines. The same framework also supports concept-level intervention: correcting a small number of uncertain concepts can substantially increase image-level F1 [2511.20088].

Video work has introduced a parallel line of semantic interpretability through VLMs. SlowFastVAD uses a fast AED-MAE detector to score all frames and sends only ambiguous segments to a slow RAG-enhanced VLM. The two scores are fused as
$$
S^{\text{fusion}}=\alpha S^{\text{slow}}+(1-\alpha)S^{\text{fast}},
$$
after which temporal smoothing is applied [2504.10320]. On Ped2, the fast detector alone reaches Micro AUC 95.4 at **1655 FPS**, the slow detector reaches Micro AUC 98.4 at **0.5 FPS**, and the hybrid reaches **99.1** at **16 FPS**, while also producing natural-language explanations grounded in retrieved scene-specific patterns [2504.10320]. This establishes an interpretable fast/slow architecture rather than a purely score-based detector.

Explainability can also be built directly into the prediction target. Object class aware video anomaly detection through image translation maps frames to semantic segmentation masks and to masked optical-flow magnitudes, so that appearance anomalies correspond to unexpected object classes and motion anomalies correspond to unexpected motion magnitude. The reported effect is that detections are “completely explainable” and anomalies are localized accurately in the frames, especially when appearance and motion branches are fused [2205.01706].

## 6. Robustness, generalization, and emerging directions

Robustness under real acquisition variability remains a central unsolved issue. M2AD shows that adding more views and more illuminations does not consistently improve performance when methods treat each image independently and fuse by naive score averaging; additional configurations can propagate photometric noise rather than contribute useful evidence [2505.10996]. This suggests that multi-configuration VAD requires feature-level fusion, cross-configuration attention, or explicit physics-aware modeling rather than post-hoc aggregation.

Unified few-shot VAD is another emerging direction. UniVAD proposes a **training-free** model that uses **Contextual Component Clustering**, **Component-Aware Patch Matching**, and **Graph-Enhanced Component Modeling** to handle industrial, logical, and medical anomalies with only a few normal references during testing [2412.03342]. Its motivation is explicit: domain-specific pipelines and the “one-category-one-model” paradigm scale poorly across industrial, logical, and medical settings. The same paper notes that PatchCore reaches **84.1%** image AUROC on MVTec-AD but only **62.0%** on MVTec LOCO, illustrating the difficulty of transferring a domain-specific detector across anomaly semantics [2412.03342].

The field is also moving beyond images and videos into modality transfer. A recent audio study argues that the core principle of modern VAD is **representation-centric** and **modality-agnostic**: PatchCore, PaDiM, CFA, and STFPM can be ported from images to spectrograms, producing time–frequency anomaly maps and making anomaly localization possible in audio anomaly detection as well [2502.18328]. This is consistent with longer-term library plans to extend image-centered VAD toolchains toward multi-view, video, 3D VAD, and audio anomaly detection [2507.12049].

Broader surveys converge on a common research agenda: real rather than staged anomalies, multi-scene and cross-domain generalization, multimodal fusion, multi-view reasoning, online and continual adaptation, and stronger integration of VLMs and foundation models [2302.05087][2405.19387]. A plausible implication is that VAD is evolving from a narrowly defined defect detector into a general framework for open-world perception under distribution shift, with anomaly maps, semantic descriptors, and deployment-aware constraints treated as equally important outputs.

Source: https://www.emergentmind.com/topics/visual-anomaly-detection-vad