Sequential PatchCore: Scalable Anomaly Detection

• Sequential PatchCore is a scalable anomaly detection framework that uses an online coreset construction method to reduce memory and compute costs in high-resolution industrial inspection.
• It integrates synthetic photorealistic water-stain generation to simulate real-world surface impurities, enhancing the robustness of defect detection.
• The method enables efficient transfer learning via coreset melding, facilitating continual adaptation on diverse industrial datasets using consumer-grade hardware.

Sequential PatchCore is a scalable anomaly detection framework designed for industrial surface inspection, specifically addressing issues arising from high-resolution image data and the prevalence of surface impurities such as water stains, fingerprints, and stickers. It extends the explicit coreset approach originally popularized by PatchCore by introducing an online, memory-efficient method for coreset construction that enables practical training and inference on large-scale images using consumer-grade hardware. The framework also integrates synthetic impurity modeling—specifically, procedural water-stain generation—to improve robustness against nuisance artifacts that commonly confound visual inspection systems (Mao et al., 16 Jan 2025).

1. Motivation and Limitations in Surface Inspection

Modern automated visual inspection systems typically operate on high-resolution images (e.g., 2–4 K pixels per side), yielding millions of local patches per image when processed with convolutional neural network (CNN) backbones. Traditional explicit coreset-based anomaly detection methods, exemplified by PatchCore, build a feature coreset by extracting embeddings from every patch across all training images and globally subsampling this set to a manageable size. The resource requirements of this approach scale as $\mathcal{O}(N \cdot P)$ in memory—where $N$ is the number of training images and $P$ the number of patches per image—and at least $\mathcal{O}((N \cdot P)^2)$ in compute for the reduction step. These requirements render standard PatchCore infeasible on high-resolution industrial datasets.

Surface impurities (such as water stains, fingerprints, and stickers) further complicate anomaly detection, as their visual characteristics overlap with defect patterns. Existing synthetic data generation pipelines typically focus on ideal defects, neglecting the confounding effects of real-world impurities. Sequential PatchCore addresses these challenges by (i) reformulating coreset construction as an online streaming process to circumvent memory bottlenecks and (ii) explicitly incorporating photorealistic synthetic impurities into the training data to enhance real-world robustness.

2. Mathematical Formulation of Sequential Coreset Construction

Let $\mathcal{M}$ denote the coreset, a selected subset of the patch embedding space with cardinality $|\mathcal{M}| \leq M \ll N \cdot P$. The distance function between embedding vectors is $d(x, y) = \|x - y\|_2$.

The classical PatchCore coreset selection solves

$$\max_{S \subseteq \{\text{all patches}\}, |S|=M} \min_{\substack{x, y \in S \ x \neq y}} d(x, y)$$

through iterative removal of the most redundant embedding (i.e., the vector whose nearest-neighbor distance is minimal).

Sequential PatchCore reformulates this process as follows:

• Initialize $\mathcal{M} \gets \emptyset$.
• For each incoming patch embedding $x$:
  • If $N$0, insert $N$1 into $N$2.
  • Else:
  • 1. Find the nearest neighbor $N$3, with $N$4.
  • 2. Find the pair $N$5 with distance $N$6.
  • 3. If $N$7, replace one of $N$8 (e.g., the lower-indexed) with $N$9.

This greedy, online rule admits patch features into the coreset only if they serve to expand the covered region in feature space, ensuring uniform coreset coverage and maintaining memory usage at $P$0.

3. Algorithmic Workflow and Transfer Learning via Coreset Melding

Sequential PatchCore algorithm:

• Input: Pre-trained feature extractor $P$1 (e.g., ImageNet-initialized MobileNetv3), training images $P$2, coreset size $P$3.
• Output: Coreset $P$4 with $P$5.
• Procedure:
  • For each training image $P$7:
  • a) Extract patch features $P$8.
  • b) For each $P$9:
  • i) If $\mathcal{O}((N \cdot P)^2)$0: $\mathcal{O}((N \cdot P)^2)$1.
  • ii) Else, compute $\mathcal{O}((N \cdot P)^2)$2 (nearest neighbor in $\mathcal{O}((N \cdot P)^2)$3), $\mathcal{O}((N \cdot P)^2)$4 (smallest inter-coreset-pair distance). If $\mathcal{O}((N \cdot P)^2)$5, replace the more redundant coreset member with $\mathcal{O}((N \cdot P)^2)$6.
  • 3. Optionally, build a distance matrix over $\mathcal{O}((N \cdot P)^2)$7 for efficient inference.

Coreset melding for transfer learning:

Given coresets $\mathcal{O}((N \cdot P)^2)$8 and $\mathcal{O}((N \cdot P)^2)$9 from datasets $\mathcal{M}$0 and $\mathcal{M}$1, meld them by streaming elements of $\mathcal{M}$2 through the online replacement process into $\mathcal{M}$3, enabling domain knowledge transfer and continual adaptation without re-extracting all features.

4. Synthetic Photorealistic Water-Stain Generation

Synthetic data generation incorporates photorealistic water stains as perturbed disks embedded in the surface texture. Each stain is parameterized by a center $\mathcal{M}$4, nominal radius $\mathcal{M}$5, and a fractalized boundary induced by Perlin noise: $\mathcal{M}$6 where $\mathcal{M}$7 is the frequency and $\mathcal{M}$8 the amplitude of the noise.

For a surface point $\mathcal{M}$9, the stain assignment is determined by the minimum distance to stain centers (with grid-based partitioning to distribute centers naturally). If a point is within the perturbed radius ($|\mathcal{M}| \leq M \ll N \cdot P$0), reflectance is computed as: $|\mathcal{M}| \leq M \ll N \cdot P$1 where $|\mathcal{M}| \leq M \ll N \cdot P$2 is the base reflectance, $|\mathcal{M}| \leq M \ll N \cdot P$3 is the stain intensity parameter, $|\mathcal{M}| \leq M \ll N \cdot P$4, and $|\mathcal{M}| \leq M \ll N \cdot P$5 is an edge decay parameter. This produces both high-fidelity visual effects and precise binary annotation masks, facilitating robust learning.

5. Comparative Analysis: Sequential Versus Original PatchCore

Aspect	Original PatchCore	Sequential PatchCore
Memory footprint	$|\mathcal{M}| \leq M \ll N \cdot P$6 patch features	$|\mathcal{M}| \leq M \ll N \cdot P$7 coreset features
Reduction algorithm	Top-down global removal	Online replacement rule
Parallelism	Requires global distance matrix	Streaming, pairwise coreset matrix only
Domain transfer	Retrain with combined features	Meld small coresets efficiently
Compute complexity	$|\mathcal{M}| \leq M \ll N \cdot P$8 (reduction)	$|\mathcal{M}| \leq M \ll N \cdot P$9 per epoch

Sequential PatchCore drastically reduces the memory and compute required for coreset construction, trading a moderate increase in required epochs and redundant feature extraction for practical tractability at industrial image scales (e.g., on a single 12 GB GPU).

6. Experimental Results and Performance Evaluation

Experiments were conducted on datasets comprising:

• Real: 18 aluminum plates, 2444×2050 pixels, defects (scratch, bump, dent), and impurities (water stain, fingerprint, sticker), with manual masks.
• Synthetic: 80 clean and 80 water-stained images, with randomized textures and lighting (domain randomization).

Evaluation metrics included pixel-wise precision ($d(x, y) = \|x - y\|_2$0), recall ($d(x, y) = \|x - y\|_2$1), $d(x, y) = \|x - y\|_2$2 score ($d(x, y) = \|x - y\|_2$3), and defect-wise recall ($d(x, y) = \|x - y\|_2$4); $d(x, y) = \|x - y\|_2$5 is the mean recall over defect classes.

Summary of results (see Table 2 in (Mao et al., 16 Jan 2025)):

• Real-only: $d(x, y) = \|x - y\|_2$6
• Synthetic-only: $d(x, y) = \|x - y\|_2$7
• Synth+DR+DA: $d(x, y) = \|x - y\|_2$8
• Synth (ft on real): $d(x, y) = \|x - y\|_2$9
• Melded coreset (Real+Synth): $$\max_{S \subseteq \{\text{all patches}\}, |S|=M} \min_{\substack{x, y \in S \ x \neq y}} d(x, y)$$0

Pre-training on synthetic water-stain data followed by fine-tuning on real data improves both $$\max_{S \subseteq \{\text{all patches}\}, |S|=M} \min_{\substack{x, y \in S \ x \neq y}} d(x, y)$$1 and pixel-wise precision over real-only training. Coreset melding achieves comparable quality to exhaustive fine-tuning while avoiding re-feature extraction. Domain randomization and data augmentation (blur, noise) further enhance recall. This suggests that explicit impurity augmentation and sequential coreset expansion are synergistic for robust industrial anomaly detection.

7. Industrial Considerations and Broader Implications

Annotation ambiguity: Precise pixel-wise annotation is often intractable for subtle or ambiguous defects (notably bumps), with consequent underestimation of model recall at the pixel level. The explicit coreset and defect-wise metrics of Sequential PatchCore enable instance-based evaluation, aligning model assessment with industrial requirements.

Impurity robustness: Surface impurities frequently mimic defect appearance, inflating the false positive rate in typical anomaly detectors. Incorporating synthetic impurity instances in pre-training enables the model to downweight impurity patterns, reducing false alarms without sacrificing sensitivity to true defects.

Resource efficiency: Sequential PatchCore's $$\max_{S \subseteq \{\text{all patches}\}, |S|=M} \min_{\substack{x, y \in S \ x \neq y}} d(x, y)$$2 memory profile allows direct training on megapixel images with a single consumer workstation, removing one of the principal barriers to deploying explicit coreset models in production environments.

Federated and continual learning: Coreset melding enables efficient aggregation of detection expertise across decentralized sites or over time, providing a principled mechanism for knowledge transfer and continual upgrade without the need for raw data transmission or large-scale recomputation.

In summary, Sequential PatchCore extends explicit coreset anomaly detection for surface inspection by providing scalable streaming coreset construction, transfer via coreset melding, and resilience to non-defective surface impurities through targeted synthetic data integration, resulting in a method well-suited for realistic industrial inspection scenarios (Mao et al., 16 Jan 2025).