HyPCV-Former: Hyperbolic 3D Anomaly Detection

• The paper introduces HyPCV-Former, a novel method leveraging full Lorentz-space operations for hierarchical temporal modeling in 3D point cloud video anomaly detection.
• It integrates a PointNet backbone with hyperbolic embeddings and a spatio-temporal transformer, enabling effective feature extraction and curvature-aware anomaly scoring.
• Empirical evaluations on TIMo and DAD demonstrate significant AUROC improvements and robust detection of complex, temporally unfolding anomalies compared to Euclidean approaches.

Searching arXiv for the HyPCV-Former paper and closely related works mentioned in the provided data. First, locating HyPCV-Former on arXiv. Searching for "HyPCV-Former: Hyperbolic Spatio-Temporal Transformer for 3D Point Cloud Video Anomaly Detection". HyPCV-Former is a hyperbolic spatio-temporal transformer for 3D point cloud video anomaly detection that models temporal evolution directly in Lorentzian hyperbolic space rather than in Euclidean space or through tangent-space approximations. The method targets anomaly detection in point cloud video sequences, with application motivations including public-safety monitoring, indoor building monitoring, and driver monitoring. Its central premise is that many anomalies are not identifiable from a single frame but unfold over time and exhibit hierarchical event structure; accordingly, HyPCV-Former combines native 3D geometry from point cloud videos with hyperbolic geometry for hierarchical temporal reasoning (Cao et al., 1 Aug 2025).

1. Problem setting and conceptual motivation

HyPCV-Former is formulated for 3D point cloud video anomaly detection, where a point cloud video is represented as

$$\mathcal{P}=\{\mathbf{P}_1,\mathbf{P}_2,\dots,\mathbf{P}_T\},\qquad \mathbf{P}_t\in\mathbb{R}^{N\times 3},$$

with each frame containing $N$ 3D points $(x,y,z)$ (Cao et al., 1 Aug 2025). The task is to determine whether a target frame or event is anomalous from the spatio-temporal evolution of the sequence.

The motivating use cases are surveillance and monitoring scenarios in which abnormal events such as violence, medical incidents, or driver abnormalities depend on temporal context before, during, and after the event. In this formulation, the relevant signal is not merely framewise appearance but sustained behavioral evolution. The paper characterizes such anomalies as having a latent hierarchical organization and argues that this structure is poorly matched to standard Euclidean embeddings (Cao et al., 1 Aug 2025).

Two limitations of Euclidean representations are emphasized. First, hierarchical event structure is treated as a central property of anomaly sequences, and hyperbolic space is presented as better suited to tree-like or hierarchical organization because the number of nodes grows exponentially with depth. Second, Euclidean embeddings are described as limited in representing spatio-temporal continuity and the non-Euclidean organization of events evolving over time. The argument is therefore not only about geometric representation of single frames, but about the geometry of event progression.

The method also makes an explicit modality choice. Compared with RGB, point clouds are described as less sensitive to illumination, able to provide explicit 3D geometric information, and more privacy-preserving. Compared with pose or depth-image representations, point clouds are presented as preserving native 3D geometry more directly, whereas pose reconstructs 3D only indirectly from 2D and range or depth images are less suited to complex spatial structures (Cao et al., 1 Aug 2025).

2. Data representation and pipeline organization

At a high level, HyPCV-Former processes a short sequence of point cloud frames through six stages: input sequence formation, per-frame spatial encoding, hyperbolic embedding, hyperbolic spatio-temporal transformation, a task head or decoder, and anomaly scoring (Cao et al., 1 Aug 2025). The method is described as having two stages: a HyPCV-Former encoder and anomaly detection.

Each frame $\mathbf{P}_t$ is first encoded into a Euclidean feature vector $\mathbf{x}_t$ by a point cloud backbone. In the main implementation, the default backbone is PointNet,

$$\mathbf{x}_t = \mathrm{PointNet}(\mathbf{P}_t)\in\mathbb{R}^D.$$

PointNet is chosen explicitly for efficiency and permutation invariance. PointMLP and DGCNN are stated to be compatible alternatives, and the reported experiments compare all three backbones (Cao et al., 1 Aug 2025).

The input representation and preprocessing are specialized to point cloud sequences derived from depth video. Each frame is downsampled to $N=2048$ foreground points after depth-based background subtraction and back-projection from depth video to point cloud. Invalid or missing depth values are removed, and each input sequence contains 3 consecutive frames (Cao et al., 1 Aug 2025). An appendix ablation reports point counts of 512, 1024, 2048, 4096, and 8192, with 2048 selected as the best trade-off between performance and computation.

The overall encoder consists of hyperbolic representation learning and hyperbolic spatio-temporal transformation. Within this organization, the named modules are the per-frame point cloud feature extractor, Euclidean-to-hyperbolic embedding, hyperbolic positional encoding, Hyperbolic Multi-Head Self-Attention (HMHA), HTC and HRC modules borrowed from Hypformer, and hyperbolic anomaly scoring (Cao et al., 1 Aug 2025). The paper identifies three novelties: the first use of hyperbolic geometry for 3D point cloud video anomaly detection, a hyperbolic spatio-temporal transformer with HMHA for frame-level temporal modeling in point cloud videos, and full Lorentz-space computation instead of tangent-space approximation.

A notable architectural claim is that the method performs all feature transformations and anomaly scoring directly within full Lorentzian space. The paper contrasts this with prior hyperbolic video anomaly methods such as HyperVD and DSRL, which are described as defining operations in tangent space. The stated rationale is that tangent-space approximation compromises computational accuracy, whereas full Lorentz-space operations preserve geometry consistency and improve anomaly discrimination (Cao et al., 1 Aug 2025).

3. Lorentzian hyperbolic geometry

The geometric foundation of HyPCV-Former is the Lorentz model, also called the hyperboloid model, of hyperbolic geometry. The $n$-dimensional hyperbolic manifold with curvature $\kappa<0$ is written as

$\mathbb{L}^n_{\kappa}=(\mathcal{L}^n,g_x^\kappa),$

where

$N$0

Because $N$1, this is the hyperboloid constraint with negative Lorentz norm (Cao et al., 1 Aug 2025).

The Lorentzian inner product is defined as

$N$2

The tangent space at $N$3 is

$N$4

The exponential map is given as

$N$5

The logarithmic map is given as

$N$6

These expressions are presented as the operational machinery for mapping between the manifold and tangent spaces, although HyPCV-Former’s design goal is to minimize dependence on tangent-space approximations during transformation and scoring (Cao et al., 1 Aug 2025).

After Euclidean frame features are extracted, they are lifted to the tangent space at the origin

$N$7

with origin

$N$8

and embedded by the exponential map: $N$9 The sequence representation then becomes

$(x,y,z)$0

The paper states that variable curvatures are trainable parameters, following the design principle introduced in Hypformer, so curvature is not treated as globally fixed across all operations (Cao et al., 1 Aug 2025).

Lorentzian intrinsic distance is used for anomaly scoring. The stated expression is

$(x,y,z)$1

The notation is acknowledged in the paper summary as somewhat inconsistent because $(x,y,z)$2 is originally a Euclidean point cloud while the formula invokes a Lorentzian inner product. The intended meaning, however, is explicit: anomaly scoring is performed with Lorentzian intrinsic distance (Cao et al., 1 Aug 2025).

4. Hyperbolic spatio-temporal transformer architecture

The transformer receives Lorentzian frame embeddings and applies hyperbolic positional encoding, HMHA, and hyperbolic non-linear refinement. The best reported configuration uses 4 hyperbolic spatio-temporal transformer layers and 8 heads (Cao et al., 1 Aug 2025). The effective hidden width is not specified unambiguously in the method description, but the parameter analysis identifies 256 channels as the best setting.

Hyperbolic positional encoding is defined through a learnable embedding

$(x,y,z)$3

followed by a manifold-preserving update

$(x,y,z)$4

with $(x,y,z)$5 in experiments (Cao et al., 1 Aug 2025). This operation is presented as a hyperbolic analogue of adding temporal position information and then renormalizing back onto the hyperboloid.

HMHA constructs query, key, and value tensors with HTC-based hyperbolic linear transformations: $(x,y,z)$6

$(x,y,z)$7

$(x,y,z)$8

with

$(x,y,z)$9

The paper states that attention is computed using negative Lorentzian distances, that the full attention matrix is constructed using Lorentzian inner products, that softmax normalization is then applied, and that a learnable scalar bias $\mathbf{P}_t$0 is included. However, the exact attention-score formula is absent from the provided text, and the abstract’s phrase “curvature-aware softmax” is not accompanied by a separate explicit equation in the available body text (Cao et al., 1 Aug 2025).

Attention aggregation uses a Lorentzian midpoint: $\mathbf{P}_t$1 The resulting head outputs $\mathbf{P}_t$2 are concatenated as

$\mathbf{P}_t$3

This makes the aggregation step itself a Lorentz-space operation rather than a tangent-space approximation (Cao et al., 1 Aug 2025).

HTC and HRC are the main fully hyperbolic transformation blocks. The appendix gives HTC as

$\mathbf{P}_t$4

HRC is given as

$\mathbf{P}_t$5

The text states that HRC acts on spatial feature dimensions and preserves Lorentzian causal structure (Cao et al., 1 Aug 2025).

The non-linear transformer operations are defined through HRC: $\mathbf{P}_t$6

$\mathbf{P}_t$7

$\mathbf{P}_t$8

More generally,

$\mathbf{P}_t$9

The paper does not clearly provide an explicit residual formula. This suggests that residual connections may be present implicitly as part of a transformer design, but they are not specified in the provided text and therefore remain undocumented at the level of explicit equations (Cao et al., 1 Aug 2025).

5. Anomaly scoring, supervision, and experimental protocol

HyPCV-Former is evaluated in two benchmark settings: prediction-based anomaly detection on TIMo and classification-based anomaly detection on DAD (Cao et al., 1 Aug 2025). In the prediction setting, the anomaly score for target frame $\mathbf{x}_t$0 is defined as

$\mathbf{x}_t$1

The decoder is an MLP, and Lorentzian intrinsic distance is used for the final score. In the classification setting, the decoder head directly outputs an anomaly score, Lorentzian intrinsic distance is used as a hyperbolic loss, and the anomaly score is derived from distance to normal class prototypes in embedding space. The full prototype-loss formula is not provided in the text (Cao et al., 1 Aug 2025).

The supervision regime is benchmark-dependent. The paper explicitly states that each frame $\mathbf{x}_t$2 has a label $\mathbf{x}_t$3 that can supervise anomaly score training, and it notes that TIMo uses a prediction-based formulation inherited from prior depth-based work, whereas DAD uses class labels or anomaly labels for classification (Cao et al., 1 Aug 2025). The paper therefore does not present a single uniform supervision label such as “unsupervised” or “semi-supervised” across all settings.

Two losses are compared in ablation: Euclidean MSE and Lorentzian intrinsic distance. The total training objective is not written as a single standalone formula, but the comparison establishes the intended geometry-consistent preference. Post-processing applies a moving-average smoothing window with $\mathbf{x}_t$4, where the final score for each frame is the average of the current and previous $\mathbf{x}_t$5 scores (Cao et al., 1 Aug 2025).

The implementation uses PyTorch on 4 NVIDIA RTX 4090 GPUs with CUDA 12.2. Optimization uses AdamW with initial learning rate $\mathbf{x}_t$6, cosine annealing, and 400 training epochs. Sequence length is 3 consecutive frames, point count is 2048 foreground points per frame, transformer depth is 4 layers, attention head count is 8, and curvature is trainable or variable. Batch size is not reported (Cao et al., 1 Aug 2025).

The datasets are both depth-derived rather than native point cloud anomaly datasets. TIMo is an indoor building monitoring dataset captured by an Azure Kinect depth camera, with over 1500 sequences, 909 normal sequences for training, and 679 sequences for testing, among which 569 are anomalous. Its anomaly categories are aggressive behavior, medical issue, and left-behind objects. DAD is a driver anomaly detection dataset in a simulated driving environment, using front-view depth, with 25 subjects for training and 36 test sequences from 6 unseen subjects, including 16 types of anomalous actions not present in training (Cao et al., 1 Aug 2025). Because no dedicated public benchmark for 3D point cloud video anomaly detection exists, the authors convert depth videos to point clouds via back-projection and compare against depth-video anomaly baselines.

6. Empirical performance, interpretation, and limitations

The main evaluation metric is frame-level AUROC (%) (Cao et al., 1 Aug 2025). On TIMo, the best HyPCV-Former configuration uses PointNet and achieves aggressive behavior 80.4, medical issue 75.9, left-behind objects 77.5, total Euclidean-space score (ES) 75.6, and total hyperbolic-space score (HS) 77.3. With PointMLP, the corresponding values are 79.7, 72.7, 75.4, 73.9, and 74.9. With DGCNN, they are 80.0, 72.7, 76.1, 74.4, and 75.5. The best listed TIMo baseline total is P-CAE at 71.4 ES, while the abstract describes the result as a 7% improvement on TIMo (Cao et al., 1 Aug 2025).

On DAD, HyPCV-Former with PointNet attains ES 94.89 and HS 95.55; with PointMLP, ES 88.60 and HS 90.57; and with DGCNN, ES 93.92 and HS 95.08. The best listed baseline is ResNet-18 (pre-trained/post-processed) at 90.20, and the abstract describes the margin as a 5.6% gain on DAD (Cao et al., 1 Aug 2025). In both datasets, the best reported result uses PointNet rather than the alternative backbones.

Ablation studies are designed to separate the contributions of geometry, loss, positional encoding, and transformation block design. For space and loss, the reported AUROC values are 75.4 for Euclidean space with MSE, 77.0 for Euclidean space with Lorentzian intrinsic distance, 77.1 for hyperbolic space with MSE, and 77.3 for hyperbolic space with Lorentzian intrinsic distance (Cao et al., 1 Aug 2025). This indicates that both hyperbolic representation and Lorentzian distance are beneficial, with the combination producing the best result.

For temporal encoding and linear transformation, hyperbolic positional encoding achieves 77.3, standard positional encoding 77.0, and no positional encoding 76.4. HTC-based linear transform yields 77.3, whereas standard linear transform yields 76.8 (Cao et al., 1 Aug 2025). Parameter analysis identifies 4 layers and 256 channels as the best configuration, and frame-count analysis finds that 3 frames are optimal, with longer windows introducing noise and redundancy. Point-count analysis reports 74.2 at 512 points, 75.7 at 1024, 77.3 at 2048, 77.8 at 4096, and 78.1 at 8192; the chosen configuration remains 2048 points as the best trade-off (Cao et al., 1 Aug 2025).

Qualitative evidence includes Isomap visualizations comparing raw point cloud features, Euclidean features trained with MSE, and hyperbolic features trained with Lorentzian distance. The paper reports strong overlap between normal and anomalous samples in raw features, slight improvement with Euclidean training, and clearer separation with full hyperbolic training and Lorentzian distance. Anomaly score curves on TIMo and DAD show that raw scores are noisy and that post-processed scores are smoother and better aligned with anomaly intervals (Cao et al., 1 Aug 2025). Failure cases are not explicitly discussed.

The paper’s interpretation is that hyperbolic geometry is useful because anomalies in video exhibit hierarchical temporal structure and because Lorentzian distance aligns better with the geometry of learned representations than Euclidean scoring does (Cao et al., 1 Aug 2025). This suggests that HyPCV-Former is especially suited to anomalies that are event patterns rather than isolated frame defects, including aggressive behavior, medical incidents, driver abnormal behavior, and left-behind objects whose interpretation depends on context.

Several limitations are explicit or directly inferable from the reported setup. There is no dedicated public benchmark for 3D point cloud video anomaly detection, so the evaluation depends on converting depth datasets to point clouds. The pipeline relies on foreground extraction and depth-to-point-cloud conversion. Computational cost rises with point count, and performance degrades when temporal windows become too long. Real-time deployment is not addressed, and training uses 4 RTX 4090 GPUs. Although PointNet performs best in the reported framework, more advanced local geometric modeling is not deeply explored (Cao et al., 1 Aug 2025). The stated future directions are further exploration of adaptive curvature learning, extension to multi-view or multi-modal 3D video understanding, and application to tasks such as action recognition and behavior forecasting (Cao et al., 1 Aug 2025).