Point-PQAE: Two-View 3D Pre-training

• Point-PQAE is a self-supervised 3D pre-training method that leverages two decoupled partial views to enforce cross-reconstruction.
• It employs a fixed sinusoidal View-Relative Positional Embedding to align independently normalized and rotated views for accurate geometry modeling.
• The approach yields significant performance improvements on benchmarks like ScanObjectNN through a bidirectional reconstruction objective and crop-based view augmentation.

Searching arXiv for the specified paper and closely related point-cloud SSL baselines to ground citations. Point-PQAE is a self-supervised pre-training method for 3D point cloud learning that replaces the standard masked self-reconstruction paradigm with a two-view cross-reconstruction objective. Rather than reconstructing masked points from visible points within a single normalized view, it generates two decoupled partial views from the same raw point cloud and reconstructs each view from the other using explicit relative geometric information. The method was introduced in "Towards More Diverse and Challenging Pre-training for Point Cloud Learning: Self-Supervised Cross Reconstruction with Decoupled Views" (Zhang et al., 1 Sep 2025). Its central claim is that two-view pre-training is more diverse and more challenging than single-view self-reconstruction, and that this increased difficulty yields stronger 3D representations, especially under frozen-feature transfer protocols (Zhang et al., 1 Sep 2025).

1. Conceptual framing and problem setting

Point-PQAE is situated within generative self-supervised learning for point clouds, where the objective is to learn transferable 3D representations without manual labels. The immediate target is the limitation of MAE-style masked point modeling methods such as Point-MAE, Point-BERT, MaskPoint, and Point-M2AE, which reconstruct missing content from visible content in a single point cloud under a shared coordinate system (Zhang et al., 1 Sep 2025). In that setting, the masked region is spatially and semantically close to the visible region and remains under the same pose and normalization. The resulting task is therefore structurally simple and comparatively easy.

The method adopts the intuition, drawn from two-view paradigms in 2D self-supervised learning, that stronger augmentational variance and enforced invariance across views can make pre-training more informative. Point-PQAE transfers that principle into 3D generative SSL by replacing within-view reconstruction with between-view reconstruction. Given two decoupled partial observations of the same object, the model must capture both the intra-view structure of the source view and the inter-view geometry that relates source and target despite independent normalization and rotation (Zhang et al., 1 Sep 2025).

This reframes the pre-training problem. In self-reconstruction, the model can rely heavily on local continuity and shared coordinates. In Point-PQAE, the source and target views are no longer directly comparable in raw coordinates. The task is therefore not merely "part reconstruction" but reconstruction across isolated coordinate systems. A plausible implication is that the encoder must encode more global and more semantically stable structure than in standard masked point modeling.

2. Two-view decoupling and crop-based view generation

The pipeline begins from a raw point cloud $\mathbf{X} \in \mathbb{R}^{p \times 3}$ with $p = 1024$. Point-PQAE samples two crop ratios $r_1, r_2$ uniformly from $[r_m, 1]$, with default minimum crop ratio $r_m = 0.6$, and then selects two random crop centers $\mathbf{C}_1, \mathbf{C}_2$ from the input cloud (Zhang et al., 1 Sep 2025). For each center, it takes the nearest $r_i p$ points in Euclidean distance, producing partial views $\mathbf{X}_1$ and $\mathbf{X}_2$.

A decisive design choice is that each cropped view is normalized independently by min-max normalization centered on its own geometric center $\mathbf{L}_1$ or $p = 1024$0, and each view then undergoes an independent random rotation. The recorded geometric centers remain in the original global frame, but the point coordinates of the views themselves no longer share a common coordinate system (Zhang et al., 1 Sep 2025). The paper designates this as view decoupling: the views are isolated pose-wise and coordinate-wise while still originating from the same object and often partially overlapping.

Patch tokens are then built separately for each view using the Point-BERT and Point-MAE-style grouping pipeline. Farthest Point Sampling selects $p = 1024$1 group centers, with $p = 1024$2 during pre-training, and K-Nearest Neighbors gathers $p = 1024$3 points per patch. Each patch is embedded with a light PointNet front-end before Transformer encoding (Zhang et al., 1 Sep 2025).

The crop mechanism is not treated as a generic augmentation. The ablations indicate that crop alone provides only marginal gains when added to Point-MAE, whereas the full Point-PQAE design yields substantial improvements. Specifically, on ScanObjectNN Full evaluation, Point-MAE without crop gives 92.6 / 91.9 / 88.4, Point-MAE + crop gives 92.9 / 92.1 / 88.8, and Point-PQAE with crop reaches 95.0 / 93.6 / 89.6 across OBJ-BG, OBJ-ONLY, and PB-T50-RS (Zhang et al., 1 Sep 2025). This isolates the contribution of cross-reconstruction and relative position modeling from the crop operation itself.

3. View-Relative Positional Embedding and positional query

The principal technical difficulty in cross-view reconstruction is that the target view must be reconstructed from a source view expressed in a different normalized and rotated coordinate system. Point-PQAE addresses this with a View-Relative Positional Embedding (VRPE) and a positional query block (Zhang et al., 1 Sep 2025).

For reconstructing view 2 from view 1, the method first forms a relative center vector

$p = 1024$4

expressed in the original pre-normalization coordinates. This vector is tiled to patch length $p = 1024$5. It is then concatenated with target-view patch centers $p = 1024$6 to form a patch-wise relative position tensor

$p = 1024$7

The first three dimensions encode local target patch position in the target view’s normalized coordinates; the last three encode global relative displacement between source and target view centers (Zhang et al., 1 Sep 2025).

Instead of using a learnable positional embedding, the method applies a fixed sinusoidal encoding to each of the six scalar coordinates, following a Transformer/MAE-style construction with base $p = 1024$8, and concatenates them into $p = 1024$9, where $r_1, r_2$0 (Zhang et al., 1 Sep 2025). The same process is applied symmetrically for the reverse direction.

VRPE is not added to patch tokens in the ordinary way. It is used as the query in a cross-attention operation, while the encoded source-view features serve as key and value:

$r_1, r_2$1

The output is a target-aligned latent representation $r_1, r_2$2 that is then decoded into the target patches (Zhang et al., 1 Sep 2025). This positional query block is applied once between encoder and decoder.

The ablations make the role of VRPE explicit. Removing positional embedding causes performance to collapse to 84.5 / 85.9 / 79.3 on the three ScanObjectNN variants, compared with 95.0 / 93.6 / 89.6 for sinusoidal VRPE. Absolute positional embedding is weaker at 92.3 / 91.0 / 87.7, and a learnable MLP-based VRPE improves further to 93.4 / 93.1 / 89.1, but still trails the fixed sinusoidal variant (Zhang et al., 1 Sep 2025). The interpretation offered in the paper is that deterministic, geometry-driven relative encoding is better suited to independently normalized and rotated views than absolute or fully learnable alternatives.

4. Cross-reconstruction objective and architecture

The architecture preserves the broad MAE-style encoder-decoder structure of Point-MAE while altering the supervision regime. Each view is encoded separately by a Transformer encoder with 12 blocks, hidden dimension 384, and 6 attention heads. The decoder uses 4 Transformer blocks at the same width and head count (Zhang et al., 1 Sep 2025). The encoder outputs $r_1, r_2$3, and the decoder predicts patch point sets through a linear head reshaped to $r_1, r_2$4.

The core difference from Point-MAE is that there are no mask tokens and no within-view inpainting. Instead, the entire target view is reconstructed from the source view and the relative geometric query. The process is bidirectional: reconstruct view 2 from view 1 and reconstruct view 1 from view 2, using a shared decoder (Zhang et al., 1 Sep 2025).

Training uses symmetric Chamfer distance with squared $r_1, r_2$5 norm:

$r_1, r_2$6

The paper reports that the two-way, or siamese, loss is superior to one-way reconstruction: using only $r_1, r_2$7 yields 93.4 / 92.4 / 89.2 on ScanObjectNN, while the bidirectional objective yields 95.0 / 93.6 / 89.6 (Zhang et al., 1 Sep 2025).

Loss design also matters. Cosine similarity loss gives 90.5 / 89.8 / 85.2, CD-$r_1, r_2$8 gives 93.1 / 91.7 / 89.4, and CD-$r_1, r_2$9 gives the best result at 95.0 / 93.6 / 89.6 (Zhang et al., 1 Sep 2025). The paper attributes this to stronger geometry-reconstruction gradients under squared Chamfer distance.

5. Pre-training protocol and downstream evaluation

Pre-training is conducted on ShapeNet Core, described as approximately 51,300 clean CAD models across 55 categories (Zhang et al., 1 Sep 2025). For each object, 1024 points are sampled by FPS from the full mesh, then two cropped and rotated views are generated with minimum crop ratio $[r_m, 1]$0. Each view contains 64 patches of 32 points. The optimizer is AdamW with initial learning rate $[r_m, 1]$1, cosine decay, 10 warmup epochs, weight decay 0.05, batch size 128, and 300 epochs on a single RTX 3090 (Zhang et al., 1 Sep 2025).

The strongest empirical results appear on ScanObjectNN, especially under frozen-feature evaluation. Under the Full protocol, Point-PQAE improves over Point-MAE† from 92.6 / 91.9 / 88.4 to 95.0 / 93.6 / 89.6, corresponding to gains of +2.4, +1.7, and +1.2 on OBJ-BG, OBJ-ONLY, and PB-T50-RS (Zhang et al., 1 Sep 2025). Under Mlp-Linear, the gains are much larger: Point-MAE† scores 82.8 / 83.2 / 74.1, whereas Point-PQAE reaches 89.3 / 90.2 / 80.8, reported as improvements of 6.5%, 7.0%, and 6.7% (Zhang et al., 1 Sep 2025). Under Mlp-3, Point-PQAE reaches 90.7 / 90.9 / 83.3, approximately 4.4% average gain across the three variants relative to Point-MAE†.

On ModelNet40, the improvements are smaller but consistent. Under Full evaluation with plain Transformers and coordinates only, Point-PQAE reaches 94.0 for 1K points and 94.3 for 8K points, compared with 93.8 / 94.0 for Point-MAE and Point-MAE† (Zhang et al., 1 Sep 2025). On few-shot ModelNet40 protocols, Point-PQAE also improves upon Point-MAE† across Full, Mlp-Linear, and Mlp-3 settings; for example, under Full evaluation it reaches 96.9±3.2 and 98.9±1.0 on 5-way tasks and 94.1±4.2 and 96.3±2.7 on 10-way tasks (Zhang et al., 1 Sep 2025).

For dense prediction, the paper reports improvements on ShapeNetPart and S3DIS. On ShapeNetPart, Point-PQAE achieves 84.6 Cls.mIoU and 86.1 Inst.mIoU, improving Cls.mIoU over Point-MAE by +0.4 while matching Inst.mIoU (Zhang et al., 1 Sep 2025). On S3DIS Area 5, it reaches 70.6 mAcc and 61.4 mIoU, outperforming Point-MAE at 69.9 / 60.8 and a scratch Transformer at 68.6 / 60.0 (Zhang et al., 1 Sep 2025).

6. Ablations, limitations, and position in the literature

The ablation studies support three interacting claims. First, cross-reconstruction is more effective than self-reconstruction. Second, effective decoupling requires crop together with per-view normalization and per-view rotation. Third, relative positional modeling is indispensable (Zhang et al., 1 Sep 2025). The best augmentation setting is rotation only, which yields 95.0 / 93.6 / 89.6, while jitter only, scale only, scale+translate, and rotation+scale+translate all perform worse (Zhang et al., 1 Sep 2025). The crop ratio is also sensitive: if $[r_m, 1]$2 is too small, overlap becomes insufficient; if too large, the two views become too similar. The reported optimum is around $[r_m, 1]$3 (Zhang et al., 1 Sep 2025).

The positional query block is presented as more general than the specific Point-PQAE pre-training setting. In a knowledge-distillation experiment using ReCon as a frozen teacher, a student trained from scratch improves from 83.0 / 84.0 / 79.1 to 93.5 / 91.9 / 88.5 on ScanObjectNN when PQ-based distillation is used (Zhang et al., 1 Sep 2025). This suggests that the query mechanism can function as a broader cross-view or cross-branch alignment module rather than only a component of cross-reconstruction pre-training.

The paper also states several limitations. The backbone remains a vanilla plain Transformer with PointNet patch embedding and is not otherwise specialized for cross-reconstruction. The framework is single-modality, whereas some data-hungry tasks may benefit from multimodal supervision. The relatively small scale of current 3D datasets may cap performance compared with large 2D pre-training corpora. Finally, the crop mechanism depends on nearest-neighbor selection and the hyperparameter $[r_m, 1]$4, to which performance is sensitive (Zhang et al., 1 Sep 2025).

Within the literature, Point-PQAE is positioned against self-reconstruction-based 3D SSL methods including Point-MAE, Point-BERT, MaskPoint, Point-M2AE, PCP-MAE, and Point-FEMAE, as well as against cross-modal methods such as CrossPoint, ACT, I2P-MAE, Joint-MAE, Cross-BERT, TAP, and ReCon (Zhang et al., 1 Sep 2025). The paper characterizes its novelty as threefold: a two-view cross-reconstruction setting for point-cloud SSL, a crop-based view generation mechanism tailored to irregular 3D data, and a relative position-aware positional query block that bridges decoupled coordinate systems (Zhang et al., 1 Sep 2025). In that sense, Point-PQAE marks a shift from reconstructing missing parts in a single frame to modeling correspondence and geometry across decoupled 3D observations.