PointAD: Zero-Shot 3D Anomaly Framework

• PointAD is a CLIP-based framework that performs zero-shot 3D anomaly detection by jointly reasoning over 3D points and 2D renderings.
• It leverages prompt learning to align point and pixel features, combining 3D multiple-instance and 2D multi-task learning for accurate anomaly recognition.
• Enhanced in PointAD+, the integration of explicit geometry modeling and multimodal fusion improves detection on datasets like MVTec3D-AD and Eyecandies.

PointAD is a CLIP-based framework for zero-shot 3D anomaly detection that learns to recognize generic “normal vs. damaged” semantics in 3D point clouds by jointly reasoning over 3D points and 2D renderings. In its original formulation, it renders 3D anomalies into multiple 2D renderings, projects them back into 3D space, and optimizes learnable text prompts from both 3D and 2D through auxiliary point clouds, so that collaboration between point and pixel representations facilitates recognition of underlying 3D anomaly patterns on unseen objects (Zhou et al., 2024). Subsequent work extends this line with explicit geometry-aware representations and hierarchical prompt alignment, broadening PointAD from an implicit point–pixel representation to a more holistic zero-shot 3D anomaly framework (Zhou et al., 3 Sep 2025).

1. Problem setting and conceptual scope

PointAD addresses zero-shot 3D anomaly detection: given a 3D point cloud, and optionally an RGB image of the same object or scene, the system must produce both a global detection score and a local segmentation map, even though no training samples from the target object class are available. Training instead uses auxiliary 3D objects with point-level anomaly labels, and the learned anomaly semantics are transferred to unseen object categories (Zhou et al., 2024).

This setting differs sharply from the dominant unsupervised or few-shot 3D anomaly-detection regime. Earlier 3D methods such as teacher–student schemes, reconstruction methods, or RGB–3D fusion pipelines assume access to normal point clouds of the target object and effectively memorize object-specific normality. PointAD is designed for scenarios in which target training samples are unavailable because of privacy protection, business protection, or the absence of historical data for new products. Its central claim is that CLIP’s strong zero-shot generalization can be redirected from category semantics toward anomaly semantics through prompt learning and 2D–3D alignment (Zhou et al., 2024).

A later formulation, PointAD+, preserves the same zero-shot objective but argues that the original PointAD remains an implicit 3D representation because it relies on rendering-pixel anomalies associated with points and does not explicitly model spatial relations within the point cloud. PointAD+ therefore adds an explicit 3D representation based on geometry-aware aggregation, while retaining the original rendering-based layer (Zhou et al., 3 Sep 2025).

2. Rendering, backprojection, and implicit 3D representation

The original PointAD pipeline begins with a point cloud $x_i^{3d}$ and its point-wise anomaly mask $y_i^{3d}$. For each view $k=1,\dots,K$, a renderer $R^{(k)}$ generates a 2D rendering $x_i^{(k)}$ and a corresponding 2D anomaly mask $y_i^{(k)}$. In the reported implementation, PointAD uses Open3D, CLIP ViT-L/14@336px, and $K=9$ views obtained by rotating the point cloud around the x-axis with angles $$\{-\frac{4}{5}\pi, -\frac{3}{5}\pi, -\frac{2}{5}\pi, -\frac{1}{5}\pi, 0, \frac{1}{5}\pi, \frac{2}{5}\pi, \frac{3}{5}\pi, \frac{4}{5}\pi\}$$ (Zhou et al., 2024).

For each rendering, CLIP’s frozen visual encoder produces a global feature $f_i^{(k)} \in \mathbb{R}^d$ and a local feature map $f_i^{m(k)} \in \mathbb{R}^{h\times w \times d}$. A 3D point $y_i^{3d}$0 is projected to feature-grid coordinates by

$y_i^{3d}$1

The local point feature in view $y_i^{3d}$2 is then defined from the corresponding CLIP patch feature, subject to a visibility constraint that keeps only the frontmost point when multiple points project to the same pixel. This produces view-specific local 3D features $y_i^{3d}$3, while the global 3D representation is simply the set of per-view global CLIP features $y_i^{3d}$4 (Zhou et al., 2024).

This construction is what later work calls an implicit 3D anomaly representation: a point is represented through the rendered pixel features associated with its visible projections. The formulation preserves CLIP’s 2D feature geometry, avoids a learned 3D backbone, and makes dense 3D anomaly prediction computationally tractable because most local computation is performed in 2D feature space rather than directly at all points (Zhou et al., 2024).

3. Prompt learning and hybrid 2D–3D optimization

PointAD adapts CLIP to anomaly semantics by learning object-agnostic normal and abnormal text prompts while keeping both CLIP encoders frozen. The default prompt templates are

$y_i^{3d}$5

$y_i^{3d}$6

where $y_i^{3d}$7 and $y_i^{3d}$8 are learnable embeddings and the prompt length is $y_i^{3d}$9 in the main setting. A class-aware alternative, PointAD-CoOp, replaces “object” with the class name, but the object-agnostic formulation generalizes better to unseen categories (Zhou et al., 2024).

The text encoder maps the prompts to embeddings $k=1,\dots,K$0 and $k=1,\dots,K$1, and anomaly probabilities are computed with a CLIP-style cosine-similarity softmax: $k=1,\dots,K$2 PointAD then combines 3D multiple-instance learning and 2D multi-task learning in a single optimization objective. The 3D branch includes a global MIL loss over view-level global features and a local Dice loss over per-point anomaly maps reconstructed from 2D segmentations. The 2D branch adds view-level global cross-entropy and local Focal-plus-Dice segmentation losses on rendered anomaly masks. The full hybrid objective is

$k=1,\dots,K$3

Only the prompt embeddings are updated; CLIP itself is not fine-tuned (Zhou et al., 2024).

This design couples point and pixel supervision at both global and local scales. The 3D losses force the prompts to encode spatial abnormality as it manifests in point clouds, while the 2D losses regularize the rendered-view features so that anomaly semantics remain stable across views and remain usable when RGB is introduced at test time. This suggests that PointAD’s zero-shot behavior depends less on object identity than on a learned, cross-modal notion of “damagedness” (Zhou et al., 2024).

4. Inference, anomaly scoring, and plug-and-play multimodality

At test time, PointAD renders an unseen point cloud into the same set of views, computes per-view 2D anomaly maps with the learned prompts, and projects those maps back into 3D. The abnormal point-level score map is

$k=1,\dots,K$4

followed by Gaussian smoothing

$k=1,\dots,K$5

The global anomaly score combines the average abnormal probability over views and the peak point-level anomaly score: $k=1,\dots,K$6 Thresholding $k=1,\dots,K$7 yields point-level segmentation, while thresholding $k=1,\dots,K$8 yields object-level detection (Zhou et al., 2024).

A distinctive property of PointAD is that RGB can be integrated in a plug-and-play manner. When aligned RGB images are available, the same frozen CLIP encoder and the same learned prompts are used to produce RGB anomaly maps and RGB global scores, which are then fused with the render-based 3D scores. No retraining is required. This capability relies on the earlier hybrid loss: without the 2D global and local branches, the RGB maps become noisy and multimodal fusion may degrade performance (Zhou et al., 2024).

PointAD+ retains the same inference logic but adds a second, explicit geometry stream. It first aggregates per-point implicit features across visible views,

$k=1,\dots,K$9

then refines them with Euclidean $R^{(k)}$0-nearest neighbors and a Gaussian kernel to obtain spatially aware features $R^{(k)}$1. Geometry prompts $R^{(k)}$2 and $R^{(k)}$3 are applied to these refined point features, and the final anomaly map fuses the original implicit map $R^{(k)}$4 with the explicit geometry map $R^{(k)}$5 (Zhou et al., 3 Sep 2025).

5. Empirical results and ablation structure

PointAD is evaluated on MVTec3D-AD, Eyecandies, and Real3D-AD under both one-vs-rest and cross-dataset zero-shot settings, with metrics including image-level AUROC, Average Precision, point-level AUROC, and AUPRO. On MVTec3D-AD in the point-only zero-shot setting, PointAD reports I-AUROC 82.0, AP 94.2, P-AUROC 95.5, and AUPRO 84.4; in the multimodal RGB+3D setting on the same dataset, it reports I-AUROC 86.9, AP 96.1, P-AUROC 97.2, and AUPRO 90.2 (Zhou et al., 2024).

On Eyecandies, the point-only zero-shot configuration reports I-AUROC 69.1, AP 73.8, P-AUROC 92.1, and AUPRO 71.3, while the multimodal configuration reports I-AUROC 77.7, AP 80.4, P-AUROC 95.3, and AUPRO 84.3. On Real3D-AD, the point-only setting reports I-AUROC 74.8, AP 76.9, and P-AUROC 73.5. In cross-dataset transfer from MVTec3D-AD to Eyecandies, PointAD reports I-AUROC 69.5, AP 74.3, P-AUROC 91.8, and AUPRO 71.4 for 3D, and I-AUROC 78.6, AP 80.8, P-AUROC 94.0, and AUPRO 80.7 for multimodal RGB+3D (Zhou et al., 2024).

PointAD+ improves these results by introducing explicit geometry and cross-hierarchy alignment. On MVTec3D-AD, PointAD+ reports P-AUROC 96.9, AUPRO 88.7, I-AUROC 82.5, and AP 94.8 in the point-only setting, and P-AUROC 98.2, AUPRO 93.1, I-AUROC 87.7, and AP 96.6 in the multimodal setting. On Eyecandies, it reports point-only P-AUROC $R^{(k)}$6 and AUPRO $R^{(k)}$7, and in cross-dataset multimodal transfer from MVTec3D-AD to Eyecandies it reports P-AUROC 94.8, AUPRO 85.2, I-AUROC 80.3, and AP 83.1 (Zhou et al., 3 Sep 2025).

Ablations identify several stable design choices. In the original PointAD, adding the full hybrid loss outperforms partial variants; $R^{(k)}$8–$R^{(k)}$9 renderings provide the best trade-off; prompt length $x_i^{(k)}$0 is optimal; and more auxiliary object classes improve zero-shot performance. The method is reported as robust to rendering blur, lighting changes, and moderate point downsampling, although performance drops when anomalies are occluded or when point density changes drastically. PointAD+ further shows that adding the geometry loss $x_i^{(k)}$1 and the cross-hierarchy contrastive term $x_i^{(k)}$2 produces the best results, and that the explicit geometry layer is particularly helpful for structural anomalies such as bends, dents, and missing pieces (Zhou et al., 2024, Zhou et al., 3 Sep 2025).

6. Relation to later 3D anomaly-detection paradigms and broader “PointAD-style” usage

PointAD occupies one branch of a rapidly expanding 3D anomaly-detection literature. PLANE replaces CLIP-style rendering alignment with a pre-trained point-LLM and dual prompts on both the text and point sides, trains only prompt modules and a point-feature adaptation head while freezing ULIP2-PointBERT, and adopts a multi-class-one-model regime rather than one-model-per-category. On Anomaly-ShapeNet, PLANE reports O-AUROC 0.836 ± 0.006 and P-AUROC 0.821 ± 0.004, and on Real3D-AD it reports O-AUROC 0.777 ± 0.005 and P-AUROC 0.736 ± 0.004 (Wang et al., 16 Feb 2025).

PO3AD represents a different shift: instead of reconstructing clean coordinates from pseudo-anomalous inputs, it predicts point offsets back to normal geometry and uses offset magnitudes as anomaly scores. With normal-vector-guided pseudo anomalies, it reports an average object-level AUC-ROC 83.9% and point-level AUC-ROC 89.8% on Anomaly-ShapeNet, and object-level AUC-ROC 76.5% on Real3D-AD (Ye et al., 2024). These developments indicate that “PointAD” has become associated not only with a specific CLIP-based zero-shot method but also with a broader design space in which point-cloud anomaly detection is reformulated through prompt tuning, cross-modal alignment, or geometry-aware self-supervision.

The term also appears more informally in adjacent point-cloud literature. The GAD framework is explicitly described as a “PointAD-style” design because it combines a global attention-based embedding with dual-domain KNN feature fusion for classification and segmentation tasks, rather than anomaly detection (Li et al., 2024). Similarly, the Point Attention Network has been described as the kind of architecture one might refer to informally as a “Point Attention Design,” combining local graph attention through LAE-Conv with a global point-wise spatial attention module (Feng et al., 2019). These uses are conceptually related but distinct from PointAD proper.

Several limitations recur across the PointAD line. The original PointAD depends on fixed rendering angles, auxiliary point-level anomaly labels, and faithful capture of anomalous geometry; occlusions or severe density changes can degrade performance. PointAD+ notes an additional geometry encoder simplicity issue because its explicit layer relies only on Euclidean kNN and Gaussian weighting rather than a learned generalized point encoder, and it suggests future work on explicit point encoders and fine-grained view selection (Zhou et al., 2024, Zhou et al., 3 Sep 2025). Taken together, these limitations suggest that the long-term trajectory of PointAD research lies in richer geometry modeling, stronger multimodal alignment, and more robust transfer across unseen objects, sensors, and datasets.