Hyperspectral Anomaly Detection Overview

• Hyperspectral Anomaly Detection (HAD) is the process of identifying unusual spectral signatures in high-dimensional images by modeling backgrounds as low-rank and smooth while detecting sparse, distinct anomalies.
• It employs a range of techniques including RX-based statistical models, low-rank plus sparse decomposition, collaborative reconstruction, and tensor as well as deep learning methods to enhance detection accuracy.
• Recent advances integrate learned priors, self-supervision, and multi-criteria decision frameworks to address challenges such as noise, spatial structure, and the identity mapping problem.

Hyperspectral anomaly detection (HAD) is the task of identifying pixels whose spectra do not fit the dominant background, without prior knowledge of target signatures. In remote sensing, a hyperspectral image (HSI) is commonly modeled as a three-dimensional cube such as $\mathbf{H}\in\mathbb{R}^{L\times M\times N}$ or $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$, where each pixel carries a high-dimensional spectrum across contiguous bands. The operational setting is typically target-unknown, unsupervised, and pixel-wise: only the HSI is given, and the output is a detection map in which higher values indicate higher anomaly likelihood. Across the literature, anomalies are usually small, sparse, and spectrally distinct objects such as airplanes, vehicles, houses, ships, or other man-made targets embedded in complex natural or urban backgrounds (Liu et al., 2024, Lin et al., 6 May 2026).

1. Problem formulation and scene assumptions

HAD is difficult because HSIs combine high dimensionality, strong spectral redundancy, noise and atmospheric artifacts, complex backgrounds, unknown target signatures, small sparse anomalies, and limited labels. Backgrounds can be mixtures of multiple materials and are often structured and non-stationary, while anomalies may occupy only a few pixels yet still exhibit spatial structure rather than isolated impulses (Liu et al., 2024).

A standard decomposition used throughout the field writes the HSI as background plus anomaly,

$$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$

where $\boldsymbol{\mathcal{B}}$ denotes the background tensor and $\boldsymbol{\mathcal{E}}$ denotes the anomaly tensor. In this view, background is treated as low-rank and smooth with strong global spatial-spectral correlations, whereas anomalies are sparse or group-sparse outliers with structured spatial-spectral patterns (Qin et al., 23 May 2025). A common pixel-wise anomaly score is then obtained from the anomaly component by collapsing the spectral dimension, for example

$$T(i_1,i_2)=\sqrt{\sum_{i_3=1}^{n_3}\left|\boldsymbol{\mathcal{E}(i_1,i_2,i_3)}\right|^2},$$

or, in matrix form, $\text{score}(p)=\|\mathbf{a}_p\|_2$ for the $p$-th pixel (Qin et al., 23 May 2025, Liu et al., 2024).

This suggests that HAD is fundamentally a background modeling problem under extreme class imbalance. A plausible implication is that the central design choice is not only how to score deviations, but also how to represent the dominant background so that rare targets remain outside that representation.

2. Classical statistical, representation, and tensor formulations

A canonical statistical baseline is the Reed–Xiaoli detector and its variants. Under a multivariate Gaussian background model, the anomaly score is the Mahalanobis distance

$$R(\bm{x})=(\bm{x}-\bm{\mu})^T\bm{\Sigma}^{-1}(\bm{x}-\bm{\mu}),$$

with global or local estimates of $\bm{\mu}$ and $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$0. RX-type methods are fast and simple, but the Gaussian assumption is often violated in complex scenes, and local covariance estimation is sensitive to noise and outliers (Lin et al., 6 May 2026, Wang et al., 2021).

A second major family is low-rank plus sparse decomposition, often cast as low-rank representation (LRR). A widely used formulation is

$\mathbf{X}\in\mathbb{R}^{H\times W\times C}$1

where background is represented by low-rank coefficients over a dictionary $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$2, while anomalies are encouraged to be sparse through the $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$3-norm. The noted limitations are manual parameter tuning, lack of spatial modeling, and confusion between structured noise or edges and true anomalies (Liu et al., 2024).

Collaborative-representation methods form a related line. In CRD, each pixel is reconstructed from neighboring background samples by ridge regression, and the residual norm becomes the anomaly score. ERCRD replaces the sliding dual window with random sub-sampling and then ensembles several random CRD detectors, thereby removing dual-window tuning and reducing computational complexity while retaining collaborative reconstruction as the anomaly mechanism (Wang et al., 2021).

Tensor methods extend low-rank and sparse modeling to the native three-dimensional structure of HSIs. The literature summarized in HAD-EUNTRFR emphasizes that matrix unfolding can lose intrinsic multidimensional structure, and proposes tensor ring, tensor-SVD, and nonconvex regularization to capture both global spatial-spectral correlations and local smoothness. In that line, background is modeled through tensor ring factors and gradient-domain regularization, while anomalies are handled with generalized nonconvex group sparsity (Qin et al., 23 May 2025).

Family	Representative formulation	Limitations noted
RX / GRX	Mahalanobis distance under Gaussian background	Background often not Gaussian
LRR / low-rank + sparse	$\mathbf{X}\in\mathbb{R}^{H\times W\times C}$4 with $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$5 and $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$6	Manual tuning, no spatial modeling
CRD / ERCRD	Collaborative reconstruction residuals	Dual-window cost in CRD
Tensor low-rank models	$\mathbf{X}\in\mathbb{R}^{H\times W\times C}$7 with tensor priors	Matrix unfolding can lose 3-D structure

These families share a common premise: abundant background is easier to model than rare anomalies. They differ mainly in whether that model is statistical, linear-representational, or tensor-structured.

3. Deep learning, self-supervision, and the identity mapping problem

Deep HAD methods have largely been reconstruction-based. Autoencoders, GANs, CNNs, and transformers learn to reconstruct the input or its background proxy, and reconstruction error serves as anomaly score. In the two-step paradigm, the network is optimized to complete a proxy task such as background reconstruction or generation, and anomaly detection is applied only afterward. The critique advanced by the one-step TDD work is that this can be suboptimal because the model is not optimized directly for anomaly detection and often has poor transferability, meaning that the deep model is trained and tested on the same image (Li et al., 2023).

TDD addresses this by learning a spectral deviation relationship directly. Rather than learning a specific background distribution, it learns the relation that anomalies are spectrally deviating from their surroundings, and trains this relation in an unsupervised manner using anomaly sample simulation together with global self-attention and local self-attention modules (Li et al., 2023). STAD, by contrast, moves detection into a feature space that is argued to be closer to human visual perception, using saliency maps and a small target filter, and then applies a full connected network to convolutional network knowledge distillation strategy to reduce model size for edge deployment (Ma et al., 2024).

A central failure mode in self-supervised deep HAD is the identity mapping problem (IMP). As summarized in the IMP-focused survey and in BiGSeT, self-supervised models may overfit to the entire image, especially with increasing network complexity or prolonged training iterations, so that anomalous pixels are reconstructed almost as well as background. The consequence is that reconstruction errors lose discriminative power (Cui et al., 5 Apr 2025, Liu et al., 2023).

Two recent remedies illustrate the emerging response to IMP. BiGSeT introduces a binary mask-guided separation training strategy with a separation training loss based on a latent binary mask; background is preserved, while potential anomalies are suppressed by a second-order Laplacian of Gaussian operator, and the mask is periodically updated using a robust proportion threshold (Liu et al., 2023). SuperAD frames IMP in a unified optimization perspective with three aspects—perturbation, reconstruction, and regularization—and instantiates them as superpixel pooling and uppooling, error-adaptive convolution, and online background pixel mining (Cui et al., 5 Apr 2025).

This suggests that self-supervised HAD has shifted from merely designing stronger reconstructors to designing mechanisms that prevent reconstructors from becoming too strong on anomalies.

4. Background learning and learned anomaly priors

A major recent direction is to retain explicit background–anomaly decomposition while replacing handcrafted regularizers with learned priors. SAP does this within the LRR framework by redefining the anomaly component using a self-supervised anomaly prior. Its pretext task distinguishes original hyperspectral images from pseudo-anomaly HSIs, where the pseudo-anomaly is generated as a prism with arbitrary polygon bases and arbitrary spectral bands. The learned ResNet-34 features are then combined with Mahalanobis distance, Gaussian smoothing, adaptive thresholding, and an element-wise update

$\mathbf{X}\in\mathbb{R}^{H\times W\times C}$8

so that anomaly estimation becomes a deep, data-driven prior rather than a simple norm penalty (Liu et al., 2024).

The same work also strengthens background modeling through a dual-purified strategy and a plug-and-play background prior. A low-rank embedded network compresses the HSI to 48 bands, a dual-purified dictionary removes likely anomaly contamination while enriching background atoms, and FFDNet replaces the nuclear norm as a plug-and-play denoiser with adaptive noise estimation (Liu et al., 2024). The methodological significance is that both background and anomaly terms become learned priors while the decomposition remains explicit and inspectable.

BSDM approaches background learning differently. It treats background as a learnable pseudo-noise process inside a diffusion model, introduces a statistical offset module for cross-dataset adaptation, and modifies diffusion inference by feeding the original HSI into the denoising network so that background is removed as noise. The output is a background-suppressed HSI on which downstream detectors such as RX, 2S-GLRT, AE, and DFAE achieve improved detection and false-alarm behavior on the reported benchmarks (Ma et al., 2023).

A transport-based alternative is the SCDT-domain method, which models hyperspectral pixels as observations of a template pattern undergoing unknown deformations. In that formulation, background spectra become a convex set in the signed cumulative distribution transform domain, so a simple unsupervised subspace model learned by PCA can represent abundant background signals, and anomalies are scored by their distance from this learned subspace (Rubaiyat et al., 30 Sep 2025).

Across these lines, the background model is no longer restricted to Gaussianity, raw low rank, or plain reconstruction. It may instead be a denoiser prior, a self-supervised anomaly estimator, a diffusion-based suppressor, or a transport-domain subspace.

5. Multi-criteria decision, continual learning, and related extensions

One recent reframing treats HAD as a multi-criteria decision problem rather than a single reconstruction-residual problem. HyFuHAD combines three HAD-based membership functions—morphological, geometrical, and statistical—with a classical fuzzy detector built by Einstein product and Einstein sum, and a quantum detector built from a fuzzy feature aggregation network and a lightweight quantum neural network defuzzifier. The final anomaly map is the element-wise product of the Classical Fuzzy Detection and Quantum Fuzzy Detection maps, $\mathbf{X}\in\mathbb{R}^{H\times W\times C}$9 (Lin et al., 6 May 2026). This is not a background-reconstruction strategy in the usual sense; it aggregates multiple heterogeneous anomaly cues and then fuses them in a decision framework.

Another expansion concerns cross-domain or open-scenario HAD. CL-CaGAN formulates a continual learning setting with sequential HSI scenes $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$0, where a single model must adapt across domains while avoiding catastrophic forgetting. It combines a capsule-enhanced GAN backbone with clustering-based exemplar replay and self-distillation regularization, and evaluates continual performance with ACC and BWT (Wang et al., 17 May 2025). CL-BioGAN develops a biologically-inspired variant for open scenario HAD, combining a self-attention GAN, replay, and a continual learning bio-inspired loss composed of an Active Forgetting Loss and a CL loss (Wang et al., 17 May 2025).

A related but distinct temporal extension is hyperspectral anomaly change detection. ACDA uses two auto-encoder predictors, one for each temporal direction, to model background spectral variation between multi-temporal HSIs; the final anomaly change intensity map is the minimum of the two directional loss maps, which suppresses one-sided noise and highlights anomaly changes (Hu et al., 2020). This suggests that multi-temporal HAD variants inherit the same core concern as single-image HAD: background variation must be modeled more faithfully than anomalies.

6. Evaluation protocols, benchmarks, and recurring limitations

HAD evaluation is dominated by ROC analysis and AUC-style summaries. Beyond the standard ROC of detection probability $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$1 versus false alarm probability $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$2, several papers use 3D ROC analysis with the projections $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$3, $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$4, and $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$5, together with aggregate measures such as

$$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$6

and

$$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$7

Other works use related composites such as $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$8, $$\boldsymbol{\mathcal{M}}=\boldsymbol{\mathcal{B}}+\boldsymbol{\mathcal{E}},$$9, and $\boldsymbol{\mathcal{B}}$0 to balance target detectability against background suppression (Liu et al., 2024, Qin et al., 23 May 2025, Wang et al., 17 May 2025).

Common benchmarks include San Diego, Los Angeles, Texas Coast, Hydice, Airport, Beach, Urban, Pavia, HAD100, HyMap Cooke City, and forest or radiance scenes; anomaly types include airplanes, vehicles, houses, ships, roof pixels, painted metal sheets, and small artificial targets (Liu et al., 2024, Lin et al., 6 May 2026, Ma et al., 2024, Rubaiyat et al., 30 Sep 2025, Liu et al., 2023). This suggests that the field tests detectors across diverse sensors, spatial resolutions, and background complexities, but usually under the same core requirement: anomaly signatures are not given in advance.

Recurring limitations are also consistent across the literature. Handcrafted sparse priors ignore spatial structure and require manual tuning; statistical models rely on Gaussian or related assumptions; matrix-based methods lose native three-dimensional structure; reconstruction-based networks can over-reconstruct anomalies; pseudo-anomaly schemes may not match real anomaly morphology; plug-and-play and deep-prior optimization often lack strict convergence guarantees; cross-domain detectors face catastrophic forgetting; and methods with strong background suppression can still struggle with striping noise, clutter, shadows, or backgrounds that resemble targets (Liu et al., 2024, Qin et al., 23 May 2025, Wang et al., 17 May 2025, Wang et al., 17 May 2025, Ma et al., 2023, Rubaiyat et al., 30 Sep 2025).

Taken together, HAD has evolved from Gaussian detection and low-rank residual modeling toward learned priors, transform-domain geometry, diffusion-based suppression, multi-criteria fusion, and continual adaptation. A plausible implication is that current research no longer treats anomaly detection as a single residual thresholding problem, but as the joint design of background representation, anomaly prior, optimization stability, and operating robustness under scarce labels and heterogeneous scenes.