---
title: Progressive Group-Wise Test-Time Training (PGT)
url: https://www.emergentmind.com/topics/progressive-group-wise-test-time-training-pgt
type: topic
---

# Progressive Group-Wise Test-Time Training (PGT)

Progressive Group-Wise Test-Time Training (PGT) is a test-time adaptation strategy designed for industrial anomaly segmentation scenarios, particularly in mass-production pipelines where nearly identical parts are inspected sequentially in large quantities. PGT was introduced in the context of zero-shot anomaly detection (ZSAD) approaches, which by design only use unlabeled test-time data and typically operate at coarse, patch-level resolutions. The method targets the refinement of low-resolution anomaly maps into pixel-precise segmentations by progressively adapting a lightweight Anomaly Refinement Decoder (ARD) across groups of incoming product images, thereby improving anomaly localization without access to real anomaly ground truth or labeled data [2511.22595].

## 1. Architecture and Operational Framework

PGT is centered around two modular components: a frozen ZSAD backbone (e.g., APRIL-GAN, VCP-CLIP, MuSc) and a light-weight ARD. The ZSAD backbone extracts Vision Transformer (ViT) patch-level features $F \in \mathbb{R}^{H/14 \times W/14 \times C}$ and generates a coarse anomaly map $A \in \mathbb{R}^{H/14 \times W/14 \times 1}$. The ARD refines these into high-resolution maps $\bar{A}$. Only ARD parameters $\theta$ are updated during test-time adaptation, ensuring the ZSAD model's integrity against catastrophic forgetting or distributional drift. PGT is designed to be compatible with any ZSAD model operating on ViT features.

## 2. Algorithmic Workflow

Given a stream of incoming test images partitioned into $K$ non-overlapping groups $G_1, ..., G_K$, each of size $u$ (empirically $u=30$ yields the best tradeoff), PGT applies an iterative process comprising:

1. **Coarse Anomaly Estimation**: For group $G_i$, compute patch-level anomaly maps $A_j$ and image-level anomaly scores $s_j = \max(A_j)$.
2. **Pseudo-Normal Selection**: Rank $G_i$ by $s_j$ and select the $r$ images with lowest scores as “pseudo-normal” (typically $r=5$).
3. **Pseudo-Anomaly Synthesis**: For each pseudo-normal image $I^+$, generate $l$ synthetic anomalies (usually $l=4$) by masking (Perlin noise), pasting random textures (from DTD), and forming pairs $(\tilde{I}, M)$. 
4. **Test-Time Training**: Fine-tune ARD on these synthetic pairs for $E=20$ epochs using a Dice loss objective:
   $$
   \mathcal{L}(\theta) = \sum_{p=1}^{r\,l} \mathrm{Dice}\left(\bar{A}_p,\, M_p\right), \quad
   \mathrm{Dice}(X,Y) = 1 - \frac{2\sum X\odot Y}{\sum X + \sum Y + \varepsilon}
   $$
5. **Refinement**: For $i>1$, use the updated ARD to refine incoming images in $G_i$, producing final anomaly maps as the average of coarse and refined outputs: $\mathrm{FinalMap}_j = (A_j + \bar{A}_j)/2$.

At every group, this process enables the ARD to incrementally adapt, counteracting the distributional gap between synthetic and real anomalies without introducing overfitting or semantic drift.

## 3. Motivation and Advantages

ZSAD systems do not require annotated anomalies and instead rely on feature-level novelty detection for candidate lesions. However, this produces coarse, low-granularity results due to the inherent spatial resolution of ViT features. Offline-trained, supervised refinement decoders have limited generalizability due to the distribution gap between pseudo and real anomalies. 

PGT is motivated by several needs:
- Leveraging mass-production regularities: Groups of near-identical products in sequential batches.
- Unsupervised adaptation: Utilizing only the unlabeled, streaming test data.
- Distributional robustness: Adapting in small increments per group to reduce both overfitting and training bias from unrepresentative synthetic anomalies.
- Preservation of data flow: Adhering to grouped processing constraints customary in industrial visual inspection.

PGT differentiates itself from single-image adaptation and dataset-level adaptation by operating under realistic industrial constraints and specifically targeting spatially fine segmentation improvement at test-time.

## 4. Hyperparameter Selection and Robustness

Empirical ablations on the MVTec AD and VisA datasets guide preferred hyperparameters:
- Group size $u=30$ (comparable results for $u\in\{20,40\}$, within ±0.5% of best metrics).
- Pseudo-normal selection $r=5$, with performance peaking around this value.
- Number of pseudo-anomalies per normal $l=4$, with diminishing returns beyond that.
- Learning rate $\alpha=10^{-3}$ with SGD; 20 epochs per group (fixed).
- Synthetic mask and texture choices: Perlin noise and DTD textures as default, while variants like CutPaste and SeaS can be substituted with small or slightly improved performance impacts.

PGT's robustness to imperfect pseudo-normal selection is documented: up to 40% contamination with true anomalies among pseudo-normals results in only a 0.8% drop in pixel-AP, reflecting the method's resilience to noisy group composition.

## 5. Empirical Results and Performance

PGT yields measurable improvements in pixel-level anomaly segmentation across multiple ZSAD backbones and datasets:
- VCP-CLIP+PGT on MVTec AD: pixel-AP improves by +2.5%, F1 by +2.8%.
- MuSc+PGT: up to +5.1% gain in pixel-AP.
- VisA dataset: up to +5.2% in segmentation AP.
- Full progressive group-wise adaptation achieves higher scores (pixel-AP=51.6%) than training only on the initial group (pixel-AP=51.0%), confirming the benefit of incremental adaptation.
- Averaging coarse and refined outputs effectively suppresses false positives and negatives.

Independence between per-group pseudo-normals/anomalies is shown to be more effective than sample sharing across groups.

## 6. Integration Strategy and Deployment Considerations

PGT is modular and can be integrated into any ZSAD pipeline with ViT backbone by inserting the ARD downstream of the patch-level anomaly map. The main computational cost arises from retraining ARD for 20 epochs per group, typically processed offline between inspection runs in mass-production settings. Only the ARD undergoes parameter updates, ensuring that the underlying ZSAD remains static and empirically stable. This design limits the risk of catastrophic drift often observed in end-to-end test-time adaptation schemes.

Ablative studies underscore that diverse pseudo-normal sampling outweighs the impact of synthetic anomaly details. The method tolerates high levels of label noise in pseudo-normals and benefits cumulatively from exposure to varied synthetic training conditions across groups.

## 7. Limitations and Prospective Directions

PGT introduces a computational test-time overhead due to episodic ARD fine-tuning. This limitation is mitigated in industrial workflows by offline group processing. The method's efficacy is enhanced by diversity among synthetic anomalies and pseudo-normals; its performance plateaus when synthetic augmentation variation is saturated.

*This suggests* that further benefits may accrue from dynamically selecting augmentation strategies or incorporating domain simulation to reduce the remaining distribution gap. Nevertheless, PGT provides a principled operational endpoint for ZSAD refinement under real-world production constraints, achieving improved fine-grained segmentation with only unlabeled test-time data and without sacrificing image-level anomaly detection performance [2511.22595].

Source: https://www.emergentmind.com/topics/progressive-group-wise-test-time-training-pgt