Anomaly Refinement Decoder (ARD)

• The paper introduces ARD, a neural network module that refines coarse anomaly maps using explicit anomaly-attention and bidirectional feature–score fusion.
• It employs a progressive group-wise test-time training paradigm to adaptively enhance pixel-level anomaly segmentation without retraining the backbone.
• Empirical results show that ARD improves pixel-AP by 3–5 percentage points over baselines, demonstrating its efficiency for industrial visual inspection.

An Anomaly Refinement Decoder (ARD) is a neural network module designed to enhance the spatial granularity and localization accuracy of anomaly detection models, especially in industrial visual inspection and zero-shot anomaly detection (ZSAD) settings. ARD models fuse coarse anomaly maps—such as those produced by vision transformer (ViT)-based patch-level models—with feature tensors to produce refined, high-resolution anomaly segmentation maps. Distinct from conventional decoders, ARD achieves this through explicit conditioning on coarse anomaly scores at every upsampling stage and by employing architectural elements such as anomaly-attention and bidirectional feature–score fusion, allowing integration into existing ZSAD pipelines without labeled anomaly data (Huang et al., 27 Nov 2025).

1. Architecture and Functional Overview

The ARD operates downstream from a ZSAD backbone that extracts both (1) patch-level ViT features $F \in \mathbb{R}^{H/14 \times W/14 \times C}$, and (2) a coarse anomaly score map $A \in \mathbb{R}^{H/14 \times W/14 \times 1}$. The initial stage maps the anomaly score map to a 64-channel auxiliary tensor via 1×1 convolution ($\hat{A}$), and concatenates this with the ViT features along the channel dimension. The resulting tensor passes through a 1×1 convolution to yield a fused 256-channel tensor ($\hat{F}$).

The core of the ARD comprises two stacked Anomaly-aware Refinement (AR) blocks. Each AR block performs bilinear upsampling (factor 2) on both features and score branches, followed by channel reductions and anomaly-attention. Anomaly-attention modules operate on the feature branch, amplifying responses spatially according to a learnable scalar map $\beta$, and ultimately generating a single-channel attention map that gates the convolutional features. Following AR block sequences, a Bidirectional Perception & Interaction (BI) block allows mutual feature–score correction through reciprocal convolutional cross-talk.

Final upsampling brings features to full input resolution, and a shallow convolution (“seg-head”) produces a pixel-wise refined anomaly probability map. No backbone fine-tuning occurs; ARD is trained solely via the refinement decoder weights (Huang et al., 27 Nov 2025).

2. Anomaly-Attention and Feature Fusion Mechanisms

Inside each AR block, the Anomaly-Attention (AA) module forms the principal mechanism for spatially selective feature refinement. Given upsampled features ($Y_{\text{top}}$), background-suppressed features ($Y_{\text{down}}$), and their sum ($Y_{\text{fuse}}$), the AA module scales each location’s feature vector by an exponential of a learnable scalar:

$$\zeta = \exp\left( (\beta - 1) \odot Y_{\text{fuse}} \right)$$

$Y = Y_{\text{fuse}} \odot \zeta$

The resulting refined features undergo two consecutive 3×3 convolutions and a sigmoid activation to yield an attention map $W$. The AR block's output is $\hat{Y} = W \odot Y_{\text{down}}$, transmitted to subsequent blocks and concatenated with the corresponding upsampled score-feature branch ($S_{\text{down}}$) for further refinement. This repeated, spatially targeted fusion sustains the influence of the original anomaly heatmap throughout the upsampling hierarchy.

3. Progressive Group-wise Test-Time Training (PGT) Paradigm

ARD leverages the PGT paradigm for training. The test data is partitioned into sequential groups, and within each group, pseudo-normal samples are selected by minimizing max anomaly scores. Perlin-noise–based anomalies are synthetically superimposed on pseudo-normals to create “pseudo-anomaly” images and segmentation masks. The refinement decoder is trained in each group using a negative Dice coefficient loss:

$$\mathcal{L}_{\mathrm{dice}} = \sum_{i=1}^{r\,l} \mathrm{Dice}(\hat{A}_i^P, M_i^P)$$

$$\mathrm{Dice}(\hat{A}, M) = -\frac{2 \sum_p \hat{A}(p) M(p)}{ \sum_p \hat{A}(p) + \sum_p M(p) }$$

For the first group (before ARD weights are available), the output is the coarse anomaly map. For subsequent groups, only the ARD decoder is fine-tuned, never the backbone. At inference on the next group, ARD outputs are averaged with the original coarse map to balance tradeoffs between missed and spurious detections (Huang et al., 27 Nov 2025).

4. Inference Workflow and Implementation

At inference, the ARD follows a deterministic workflow:

• Extract feature tensor $F$ and coarse anomaly map $A$ from the ViT-based ZSAD backbone.
• For groups trained with ARD, the decoder computes refined map $\bar{A} = \mathrm{ARD}(F, A)$.
• The final anomaly map is $A_{\text{out}} = \frac{1}{2}(A + \bar{A})$.
• Output is a full-resolution pixel-level anomaly probability map.

The ARD is lightweight: with 1–2 million parameters added, and two AR blocks plus BI block contributing minimal computational cost compared to the backbone. All convolutions are standard 2D, activations are ReLU or Sigmoid only, and no normalization is used within ARD. Bilinear upsampling is employed throughout (Huang et al., 27 Nov 2025).

5. Empirical Impact and Comparative Performance

ARD modules demonstrably elevate pixel-level anomaly segmentation performance across industry-standard datasets. When attached to baseline ZSAD models, ARD yields consistent gains of +3 to +5 percentage points (pp) in pixel-AP, surpassing alternative test-time refinement architectures (e.g., U-Net, DeSTSeg, RealNet) that lack explicit spatial guidance from the coarse anomaly map and thus overfit to the synthetic anomaly distribution.

For instance, on the MVTec AD and VisA datasets, AnoRefiner with ARD achieves improvements up to 5.2% in pixel-AP over its baselines (Huang et al., 27 Nov 2025). Unlike fully supervised or synthetic-sample–only approaches, ARD’s progressive group-wise TTT leverages distributional information intrinsic to the (unlabeled) test stream, adapting refinement without exposure to any real annotated anomalies.

6. Comparison with Related Refinement Decoders

ARD exhibits several substantial improvements over prior refinement decoders:

• Spatial Guidance: Prior decoders train on synthetic samples alone and lack spatial guidance; ARD injects the ZSAD-produced coarse anomaly map at every upsampling stage, providing dynamic real-image localization cues.
• Anomaly-Attention: The attentional mechanism adaptively amplifies features at spatial locations with high anomaly likelihood, reducing missed detections of subtle or low-contrast anomalies.
• Bidirectional Correction: The BI block enables feature and score branches to correct each other’s errors via cross-convolutions, enhancing robustness to both imaging artifacts and overfitting to pseudo-anomalies.
• Compatibility and Efficiency: ARD is agnostic to the upstream ZSAD model and introduces only a negligible parameter and FLOP overhead, enabling plug-and-play deployment in large-scale or resource-constrained environments.

A plausible implication is that ARD’s integration of anomaly scores—absent from traditional refinement pipelines—substantially bridges the gap between patch-level transformer outputs and the demands of high-resolution, industrial-grade anomaly localization.

7. Summary Table: ARD Core Operations

Stage	Input Shape	Operation
Initial Fusion	$H/14\times W/14\times C$, $A$	Concatenate and 1×1 Conv to 256 ch
AR Block ×2	$H/s \times W/s \times d$	Upsample ×2, anomaly-attention, concat score branch
Bidirectional (BI)	$h \times w \times d$, $h\times w\times c$	Cross-conv, residual addition
Segmentation Output	$H\times W\times d$	Bilinear upsample, 3×3 Conv, Sigmoid

This structure underpins the ARD’s refinement of coarse anomaly maps into robust, pixel-level segmentations, yielding performance and adaptability advances in ZSAD and industrial inspection contexts (Huang et al., 27 Nov 2025).