Utonia: Unified 3D Point Cloud Foundation Model

• Utonia is a self-supervised 3D point cloud model that unifies data from remote sensing, LiDAR, RGB-D scans, CAD models, and video-lifted sources into a single representation space.
• It employs a unified Point Transformer V3 backbone with 3D rotary positional embeddings and perceptual granularity rescaling to maintain cross-domain geometric consistency.
• The model achieves robust performance improvements in indoor/outdoor segmentation, object recognition, and vision-language reasoning, indicating a promising direction for unified 3D foundation models.

Utonia is a large-scale, self-supervised point-cloud foundation model that demonstrates robust transfer capabilities across a diverse range of 3D domains. By leveraging a unified Point Transformer V3 (PTv3) encoder and domain-agnostic design strategies, Utonia achieves a single consistent representation space for point clouds originating from remote sensing, outdoor LiDAR, indoor RGB-D scans, object-centric CAD models, and video-lifted data. Through this unification, Utonia improves the perception capabilities across domains and also supports downstream applications such as vision–language reasoning and embodied robotics, suggesting its potential as an early foundation model for sparse 3D data (Zhang et al., 3 Mar 2026).

1. Unified Encoder Architecture

Utonia’s architecture centers on the Point Transformer V3 backbone. For each input point cloud $P = \{p_i \in \mathbb{R}^3\}_{i=1}^N$, a preprocessing step downsamples the cloud onto a canonical voxel grid before embedding each point—potentially including color ($c_i$) and normal ($n_i$) channels—via a small multilayer perceptron, forming initial features:

$$\mathbf{h}_i^{(0)} = \phi([p_i; c_i; n_i]) \in \mathbb{R}^D.$$

These tokens are processed through $L$ stacked multi-head self-attention (MHSA) transformer blocks, each updating the token set with attention operations:

$$\mathbf{H}' = \mathrm{concat}_{h=1}^H(\mathrm{Attention}_h(\mathbf{Q}, \mathbf{K}, \mathbf{V})), \quad \mathrm{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \mathrm{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^T}{\sqrt{d}}\right)\mathbf{V},$$

where $\mathbf{Q}, \mathbf{K}, \mathbf{V}$ are linear projections of the previous block’s activations. LayerNorm and residual MLPs complete each block.

To enforce geometric consistency independent of dataset-specific coordinate systems, Utonia incorporates 3D rotary positional embeddings (RoPE) directly into MHSA. For each token, coordinates are rescaled to a unified "perceptual granularity", and RoPEs are applied axiswise:

$$\mathrm{RoPE}_{3D}(\mathbf{u}, \bar p_i) = [\mathrm{RoPE}(\mathbf{u}^x, x_i),\ \mathrm{RoPE}(\mathbf{u}^y, y_i),\ \mathrm{RoPE}(\mathbf{u}^z, z_i)],$$

dividing each head’s feature into thirds for the $x$, $y$, and $c_i$0 axes.

2. Cross-Domain Self-Supervised Objective

Utonia employs a teacher–student self-distillation framework, aligning with previous approaches such as Sonata and Concerto but extended across domains. The central loss is feature matching:

$c_i$1

where $c_i$2 is the trainable student encoder, $c_i$3 is its exponential moving average (EMA) teacher, and $c_i$4, $c_i$5 are two augmentations of the same cloud.

For data with paired 2D imagery (RGB-D, video-lifted), an auxiliary loss:

$c_i$6

aligns representations between 3D tokens $c_i$7 and 2D patch tokens $c_i$8 via $c_i$9 or contrastive losses. Notably, no explicit domain-alignment loss is required; backbone sharing, granularity rescaling, RoPE, and modality masking suffice for cross-domain unification.

3. Domain Unification Mechanisms

Three architectural and data handling strategies enable stable joint training across highly diverse point cloud sources:

• Perceptual Granularity Rescale: All point sets are rescaled so that their default voxel grid sizes (e.g., 5 cm for indoor, 0.2 m for Waymo, 1 m for remote sensing) map onto a fixed canonical length ($n_i$0 m). This eliminates scale-induced statistical discrepancies across domains.
• RoPE on Aligned Coordinates: Applying 3D rotary positional embeddings to granularity-normalized coordinates ensures geometric attention operates independently of original dataset biases and adapts to arbitrary sampling densities.
• Causal Modality Blinding: Each sample randomly drops color or normal channels at load time (with probability 0.5), and may mask these per point with low probability $n_i$1, compelling reliance on geometry over auxiliary cues and enhancing robustness to missing modalities.

These mechanisms directly address inconsistent voxelizations, gravity biases, and modality availability, thereby enabling a single representation space for mixed-domain point clouds.

4. Emergent Behaviors and Ablation Insights

Jointly training Utonia on all domains reveals several notable emergent properties:

• Cooperative Learning Across Scales: Simultaneous multi-domain training enhances both indoor and object-centric task performance, suggesting learning synergies rather than competition.
• Selective Gravity Priors: Scene-scale representations maintain weak up-axis alignment, while object-level data achieves near-complete rotation invariance.

Ablation studies highlight the critical role of domain unification components:

Component	Setting	Result (ScanNet200 mIoU, Waymo mIoU)
Grid-size alignment	Per-domain	ScanNet 29.1, Waymo 43.9
	Fixed global	ScanNet 33.5 (+4.4), Waymo 56.6 (+12.7)
3D RoPE	Without	Lower
	With	+1.4 (ScanNet), +2.6 (Waymo), +1.1 (PartNetE)
Modality blinding	None	ScanNet w/o color: 9.3%
	With blinding	ScanNet w/o color: 77.0%
Object-centric augmentation	None	ScanObjectNN: 64.5%
	Full SO(3), scale jitter	ScanObjectNN: 66.9% (+2.4)

These effects underscore the importance of Utonia’s minimalist yet principled design choices in attaining robust and transferable representations.

5. Quantitative Results Across 3D Vision Tasks

Utonia attains or surpasses existing state-of-the-art (SOTA) models such as Concerto and Sonata in standard 3D vision benchmarks:

Task/Benchmark	Concerto (f.t.)	Utonia (f.t.)	Absolute Gain
Indoor Seg. (ScanNet v2 Val, mIoU)	80.7%	81.1%	+0.4
Outdoor Seg. (Waymo Val, mIoU)	69.2%	71.4%	+2.2
Object Part Seg. (PartNetE, mIoU)	60.8%	62.7%	+1.9
Object Class. (ScanObjectNN PB-T50-RS, allAcc)	79.7%	89.9%	+10.2

The consistent performance gain across multiple domains reflects the strength of unified pretraining and robust architectural choices.

6. Multimodal and Robotic Applications

Utonia’s representations extend beyond perception to enhance downstream spatial reasoning and embodied tasks:

• Vision–Language Spatial Reasoning: Incorporating frozen Utonia features into Video-3D-LLM models yields gains for spatial language understanding (e.g., ScanRefer grounding accuracy rises from 52.6→54.0; Multi3DRefer F1, 52.7→54.1; ScanQA EM, 29.6→30.5).
• Robotic Manipulation: Conditioning a vision-language-action policy with Utonia geometry-aware features in simulation (290 Objaverse categories, RGB-D only) results in increased grasp-and-place success (Sonata: 74.7%, Concerto: 80.0%, Utonia: 82.1%).

Qualitative analysis indicates that Utonia features distinctly segment cluttered objects from backgrounds, even under heavy occlusion.

7. Prospects and Future Directions

Utonia is posited as an initial foray toward comprehensive 3D foundation models. Several open challenges remain:

• Task-Conditioned Decoding: The observed gap between linear probing and full fine-tuning on dense prediction tasks suggests the necessity of richer readout mechanisms, such as global class (CLS) tokens, key–query decoders, or promptable register tokens.
• 4D Spatiotemporal Learning: Extending the approach to temporal point cloud sequences requires motion-aware objectives and temporal attention. Preliminary experiments with frame-augmentation suggest that this extension is promising.
• Efficient Backbones: Current reliance on sparse convolutions imposes memory and deployment constraints. Exploring lighter, hardware-efficient point transformers is critical for scalable 4D, cross-domain pretraining.

Utonia demonstrates that with minimal, domain-agnostic adjustments, a single encoder can systematically master heterogeneous point cloud sources, facilitating emergent cross-domain synergies and establishing a path toward unified spatial intelligence (Zhang et al., 3 Mar 2026).