Visual Anomaly Detection Overview

• Visual anomaly detection (VAD) is a technique that identifies images, frames, or regions deviating from normal patterns by learning the distribution of normal data.
• It employs diverse methodologies—such as memory bank, student–teacher, reconstruction, and flow-based approaches—to generate image-level scores and pixel-level anomaly maps.
• VAD is critical in industrial inspection, surveillance, medical imaging, and autonomous driving, addressing challenges like data scarcity, noisy supervision, and resource constraints.

Searching arXiv for papers on visual anomaly detection, including image, video, deployment, and interpretability aspects. Visual anomaly detection (VAD) denotes the identification of images, frames, or regions that deviate from normal patterns, usually under severe anomaly scarcity and often with training performed on normal data only. In still-image inspection, the canonical outputs are image-level anomaly detection and pixel-level anomaly segmentation; in surveillance literature, the same acronym is also used for video anomaly detection, where the objective is temporal or spatio-temporal identification of abnormal events (Barusco et al., 16 Jul 2025, Abdalla et al., 2024). Across industrial inspection, logical anomaly analysis, medical imaging, surveillance, and autonomous driving, the field is shaped by anomaly rarity, noisy or weak supervision, domain gaps, and increasingly by deployment constraints such as memory, latency, bandwidth, and on-device execution (Barusco et al., 16 Jul 2025, Gu et al., 2024, Stropeni et al., 11 May 2025).

1. Task formulations and problem settings

In the standard image-based setting, a model learns the distribution of normal images and computes an anomaly score at test time; high scores indicate a deviation from normal. The two main tasks are deciding whether an image is normal or anomalous and localizing anomalous regions with an anomaly map or pixel-wise mask (Barusco et al., 16 Jul 2025). The basic assumption is often unsupervised or one-class training, but modern formulations also include noisy unsupervised, continual, supervised, semi-supervised, few-shot, edge, and IoT scenarios. In the noisy unsupervised case, image-level contamination is explicitly parameterized by

$\frac{M}{N+M} \approx C,$

where $N$ normal images and $M$ anomalous images are mixed into training, making robustness to imperfect curation a first-class evaluation target (Barusco et al., 16 Jul 2025).

Video work broadens this picture. The generalized taxonomy of video anomaly event detection distinguishes UVAD (unsupervised, normal-only), WAED (weakly supervised, typically video-level labels), SVAD (supervised), and FVAD (fully unsupervised, no assumptions on data composition) (Liu et al., 2023). A complementary survey organizes deep video VAD by supervised, weakly supervised, self-supervised, and unsupervised paradigms, and frames anomalies as events or behaviors that notably diverge from anticipated spatio-temporal patterns (Abdalla et al., 2024). This implies that “normality” may be modeled as appearance regularity, motion regularity, temporal consistency, object-scene compatibility, or some combination thereof.

A further reformulation appears in generalized few-shot VAD. UniVAD targets previously unseen industrial, logical, and medical categories with a training-free unified model that uses only a few normal reference images at test time, explicitly rejecting the common “one-category-one-model” paradigm (Gu et al., 2024). A plausible implication is that contemporary VAD is no longer defined solely by normal-only training, but by how flexibly normality can be instantiated under different data regimes.

2. Methodological families

Current image-based VAD is usually organized into a small number of recurrent families, each defined by the signal used to characterize normality and compute deviations (Barusco et al., 16 Jul 2025).

Family	Representative methods	Core normality signal
Memory bank-based	PatchCore, CFA, PaDiM	distance to stored features or Gaussian models
Student–Teacher	STFPM, PaSTe, RD4AD	feature discrepancy after distillation
Reconstruction-based	DRÆM, Ganomaly	reconstruction error
Flow-based	FastFlow	inverse log-likelihood
Adversarial	SuperSimpleNet	adversarially learned anomaly score

For memory-style methods, the anomaly score is conceptually a maximum over patch-level distances,

$$s(x) = \max_{p \in \text{patches}} \text{distance}(f_p(x), \mathcal{M}),$$

where $f_p(x)$ is a patch feature and $\mathcal{M}$ is a memory bank or a Gaussian model. Student–teacher methods instead optimize feature mimicry on normal data with a distillation loss such as

$$\mathcal{L}_{\text{KD}} = \sum_l \|f_l^T(x) - f_l^S(x)\|^2,$$

so that test-time deviations in teacher–student features become anomaly signals. Reconstruction-based methods rely on losses of the form

$$\mathcal{L}_{\text{rec}} = \|x-\hat{x}\|_1 \quad \text{or} \quad \|x-\hat{x}\|_2^2,$$

whereas FastFlow models normal feature density and scores anomalies through

$s(x) = -\log p(f(x)).$

The same framework also decouples backbones from anomaly heads, enabling swaps between ResNet18, WideResNet, MobileNetV1/V2, PhiNet, MCUNet, and MicroNet without rewriting the VAD logic (Barusco et al., 16 Jul 2025).

Video VAD extends these families into explicitly temporal formulations. Weakly supervised methods often use a multiple-instance learning ranking objective so that the highest-scoring segment in an anomalous video exceeds the highest-scoring segment in a normal video, usually with added smoothness and sparsity regularization (Liu et al., 2023). Self-supervised alternatives can replace pixel reconstruction with relation learning: PSTRP predicts patch order and inter-patch similarity in spatial and temporal dimensions, treating anomalies as violations of learned patch relations rather than raw pixel prediction errors (Shen et al., 2024). Likelihood-based video methods further reframe VAD through denoising score matching; the autoregressive DSM approach combines a scene-dependent, motion-aware score function with an appearance term based on PSNR between original and denoised clips to address anomalies near local modes of the learned normal distribution (Zhang et al., 29 Jun 2025).

Several papers also re-embed semantics into the detector. Object class aware video anomaly detection through image translation learns frame-to-segmentation and frame-to-optical-flow-magnitude mappings, so that deviations in expected semantic layout or motion magnitude become directly explainable anomaly signals (Baradaran et al., 2022). SlowFastVAD pushes this further by combining a fast masked-autoencoder detector with a slow RAG-enhanced vision-LLM, routing only ambiguous segments to the slow branch and fusing both scores for improved accuracy and interpretability (Ding et al., 14 Apr 2025). Earlier ConvLSTM-based video studies showed that variational autoencoders and $\beta$-VAE regularization improve discriminative power over plain autoencoders in reconstruction-based video VAD, reinforcing the importance of structured latent spaces in temporal anomaly modeling (Waseem et al., 2022).

3. Benchmarks and evaluation protocols

Image-based VAD is evaluated on a diverse but highly domain-dependent benchmark ecosystem. Integrated libraries now standardize datasets such as MVTec-AD, VisA, Real-IAD, MIIC, MVTec-LOCO, and BMAD, spanning industrial, logical, and medical scenarios (Barusco et al., 16 Jul 2025). The MIIC semiconductor benchmark is particularly illustrative of domain dependence: among feature-based methods, CFA attains image-level ROC 99.01 and image-level F1 87.96, while STFPM gives the highest pixel-level F1 at 77.30; at the same time, the reconstruction-based IAD+Inpainting reaches image-level ROC 99.27 and F1 91.23, showing that reconstruction-based VAD remains competitive when abundant normal data are available and the domain gap to ImageNet-style pretraining is large (Barusco et al., 12 May 2025).

The field also relies on standardized pixel-level and image-level metrics. For pixel labels $N$0, anomaly scores $N$1, and threshold $N$2, predicted labels are $N$3. Precision, recall, and F1 are then

$N$4

$N$5

$N$6

ROC-AUC, PR-AUC, and PRO complete the standard image-VAD protocol, with analogous image-level variants based on one score per image (Barusco et al., 16 Jul 2025).

Recent benchmarks deliberately stress failure modes that were largely absent from earlier datasets. M2AD introduces 119,880 high-resolution images from 999 specimens, captured under 12 synchronized views and 10 illumination settings, yielding 120 configurations per specimen and two protocols: M2AD-Synergy, which evaluates fusion across configurations, and M2AD-Invariant, which measures single-image robustness under realistic view–illumination interplay (Cao et al., 16 May 2025). At $N$7, average M2AD-Synergy performance reaches 93.0 O-AUROC, 88.3 I-AUROC, and 93.4 AUPRO for Dinomaly, while M2AD-Invariant remains substantially harder, with the same method averaging 81.3 I-AUROC and 83.3 AUPRO. The benchmark therefore operationalizes a concrete gap between standard laboratory VAD and real-world imaging physics (Cao et al., 16 May 2025).

Video evaluation uses a different but related benchmark family. The main surveillance datasets include UCSD Ped1/Ped2, CUHK Avenue, ShanghaiTech, UCF-Crime, XD-Violence, and NWPU Campus, while newer studies add UBnormal for open-set supervised video anomaly detection and AnoVox for autonomous driving anomalies (Liu et al., 2023, Abdalla et al., 2024). AnoVox contains 1,850 frames, 10 road scenes, and approximately 14.8% anomalous camera frames, with image-level and pixel-level labels for content anomalies in road-relevant regions (Genilotti et al., 16 Apr 2026). Video papers typically report frame-level Micro AUC and Macro AUC, and increasingly also provide pixel-level or region-level metrics when spatial masks are available (Ding et al., 14 Apr 2025).

4. Systems, efficiency, and deployment

A major contemporary shift is the treatment of VAD as a systems problem rather than only a modeling problem. MoViAD formalizes a modular pipeline

Dataset → Backbone → VAD Method → Trainer → Evaluation + Profiling → (Quantization / Edge / IoT deployment) and explicitly includes efficient backbones, quantization and compression utilities, image-level and pixel-level metrics, and profiling tools for model size, memory, and FLOPs (Barusco et al., 16 Jul 2025). This reframes reproducibility and deployability as part of the VAD research stack.

Edge-oriented work makes the trade-offs explicit. PaSTe, a partially shared teacher–student variant of STFPM, shows that sharing early teacher/student layers can preserve anomaly quality while substantially reducing resource use. With a MobileNetV2 backbone, STFPM to PaSTe changes inference MACs from 454.4 M to 341.2 M, training MACs from 297.5 M to 198.4 M, and training RAM from 96.15 MB to 22.9 MB, while pixel-level F1 changes from 0.52 to 0.53 (Barusco et al., 2024). The broader TinyAD benchmark in the same work also shows order-of-magnitude savings when heavy backbones are replaced by MobileNetV2, MCUNet, PhiNet, or MicroNet (Barusco et al., 2024).

Bandwidth-constrained IoT deployment introduces a different optimization axis. In an Industrial IoT setting with a 100 kB/s uplink, sending the original $N$8 image yields payload 60 kB, transmission time 0.60 s, and total latency 0.71 s. By contrast, WebP (quality 80) reduces payload to 2 kB, transmission to 0.02 s, and total time to 0.14 s, while preserving image-level ROC AUC almost exactly (0.984 to 0.982) and keeping pixel-level F1 close to baseline (0.570 to 0.555). Raw feature transmission, at 382 kB and 3.94 s, is explicitly shown to be dominated by communication cost (Stropeni et al., 11 May 2025). A plausible implication is that, under realistic IIoT bandwidth limits, compression strategy can matter more than detector architecture.

Autonomous-driving deployment sharpens the accuracy–efficiency trade-off further. On AnoVox, Tiny-Dinomaly with a DeiT-Tiny backbone achieves P-ROC 1.00, P-F1 0.69, P-AP 0.66, PRO 0.90, a 37.6 MB memory footprint, and 19 ms inference, while matching Dinomaly’s full-scale localization performance at a fraction of the memory cost (Genilotti et al., 16 Apr 2026). The same benchmark identifies PaDiM + MobileNet and CFA + MobileNet as particularly attractive when memory constraints are more severe (Genilotti et al., 16 Apr 2026).

5. Interpretability and explanation

Classical VAD explanations are predominantly spatial: anomaly maps indicate where something is abnormal, but not necessarily what or why. Concept-bottleneck VAD addresses this limitation directly. CONVAD introduces a concept dataset, a concept bottleneck branch, a student–teacher visual branch, and a synthetic anomaly generation pipeline so that concept-level explanations and pixel-level localization coexist in the same system (Stropeni et al., 25 Nov 2025). In the fully supervised CBM setting on MVTec, the CBM branch reaches average I-AUC = 0.97 and C-AUC = 0.86, while the visual branch reaches I-AUC = 0.96, P-AUC = 0.97, and P-F1 = 0.50, close to MobileNet-v2 PatchCore and STFPM baselines. The same framework also supports concept-level intervention: correcting a small number of uncertain concepts can substantially increase image-level F1 (Stropeni et al., 25 Nov 2025).

Video work has introduced a parallel line of semantic interpretability through VLMs. SlowFastVAD uses a fast AED-MAE detector to score all frames and sends only ambiguous segments to a slow RAG-enhanced VLM. The two scores are fused as

$N$9

after which temporal smoothing is applied (Ding et al., 14 Apr 2025). On Ped2, the fast detector alone reaches Micro AUC 95.4 at 1655 FPS, the slow detector reaches Micro AUC 98.4 at 0.5 FPS, and the hybrid reaches 99.1 at 16 FPS, while also producing natural-language explanations grounded in retrieved scene-specific patterns (Ding et al., 14 Apr 2025). This establishes an interpretable fast/slow architecture rather than a purely score-based detector.

Explainability can also be built directly into the prediction target. Object class aware video anomaly detection through image translation maps frames to semantic segmentation masks and to masked optical-flow magnitudes, so that appearance anomalies correspond to unexpected object classes and motion anomalies correspond to unexpected motion magnitude. The reported effect is that detections are “completely explainable” and anomalies are localized accurately in the frames, especially when appearance and motion branches are fused (Baradaran et al., 2022).

6. Robustness, generalization, and emerging directions

Robustness under real acquisition variability remains a central unsolved issue. M2AD shows that adding more views and more illuminations does not consistently improve performance when methods treat each image independently and fuse by naive score averaging; additional configurations can propagate photometric noise rather than contribute useful evidence (Cao et al., 16 May 2025). This suggests that multi-configuration VAD requires feature-level fusion, cross-configuration attention, or explicit physics-aware modeling rather than post-hoc aggregation.

Unified few-shot VAD is another emerging direction. UniVAD proposes a training-free model that uses Contextual Component Clustering, Component-Aware Patch Matching, and Graph-Enhanced Component Modeling to handle industrial, logical, and medical anomalies with only a few normal references during testing (Gu et al., 2024). Its motivation is explicit: domain-specific pipelines and the “one-category-one-model” paradigm scale poorly across industrial, logical, and medical settings. The same paper notes that PatchCore reaches 84.1% image AUROC on MVTec-AD but only 62.0% on MVTec LOCO, illustrating the difficulty of transferring a domain-specific detector across anomaly semantics (Gu et al., 2024).

The field is also moving beyond images and videos into modality transfer. A recent audio study argues that the core principle of modern VAD is representation-centric and modality-agnostic: PatchCore, PaDiM, CFA, and STFPM can be ported from images to spectrograms, producing time–frequency anomaly maps and making anomaly localization possible in audio anomaly detection as well (Barusco et al., 25 Feb 2025). This is consistent with longer-term library plans to extend image-centered VAD toolchains toward multi-view, video, 3D VAD, and audio anomaly detection (Barusco et al., 16 Jul 2025).

Broader surveys converge on a common research agenda: real rather than staged anomalies, multi-scene and cross-domain generalization, multimodal fusion, multi-view reasoning, online and continual adaptation, and stronger integration of VLMs and foundation models (Liu et al., 2023, Abdalla et al., 2024). A plausible implication is that VAD is evolving from a narrowly defined defect detector into a general framework for open-world perception under distribution shift, with anomaly maps, semantic descriptors, and deployment-aware constraints treated as equally important outputs.