Geometric Pre-training: Methods & Applications

• Geometric pre-training is a self-supervised approach that uses spatial cues and geometric structures to initialize models without human labels.
• Key methodologies include geometric reconstruction, projective consistency, and synthetic geometry to capture spatial relationships.
• Applications span vision, robotics, medical imaging, and molecular modeling, enhancing downstream tasks through improved spatial reasoning.

Geometric Pre-training

Geometric pre-training refers to a collection of methodologies that leverage geometric cues, structures, and priors as the basis for unsupervised or self-supervised model initialization. These approaches exploit explicit or implicit information about spatial structure, topological invariance, projective geometry, or geometric relationships (between points, segments, or physical objects) in lieu of semantic annotation or human labels. Geometric pre-training is widely used across vision, medical imaging, robotics, molecular modeling, physics-informed learning, and document intelligence, facilitating transfer to downstream tasks that require spatial awareness, physical generalization, or spatial reasoning.

1. Foundational Principles and Taxonomy

A unifying principle of geometric pre-training is that models are exposed to data where supervision arises from spatial or geometric relationships, often with no need for human annotation. This supervision can come from:

• Physical priors (e.g., SDFs, occupancy, object boundaries, mesh structure)
• Multi-view geometry (e.g., stereo or multiview consistency)
• Geometric transformations (affine, projective, deformable)
• Synthetic geometric primitives or procedural object generation
• Trajectories, flows, or synthetic dynamics (in physics or temporal domains)
• Multi-modal alignment of visual and geometric representations

Geometric pre-training models may be grouped by their targeted modality:

• 2D Vision: Depth, surface normals, optical flow pre-training (Khan et al., 2023, Lao et al., 2022)
• 3D Vision/Point Clouds: Shape-centric reconstruction, normal/curvature estimation (Tian et al., 2023, Yamada et al., 2024)
• Medical Imaging: 3D topological alignment, volume segmentation with geometric primitives (He et al., 2023, Tadokoro et al., 2024)
• Robotics/Autonomous Driving: Geometry-informed latent spaces, BEV distillation, inductive geometric representations for planning (Wu et al., 2023, Ljungbergh et al., 19 Mar 2025, Zhang et al., 2024, Huang et al., 2023, Xu et al., 2024)
• Document Intelligence: Explicit modeling of 2D geometric constraints between text and layout segments (Luo et al., 2023)
• Physics Modeling: Lifting geometric structure via synthetic dynamics for data efficiency (Wu et al., 23 Feb 2026, Chen et al., 27 Apr 2025)
• Molecular Graphs: Graph–geometry alignment, force/pseudoforce prediction, geometric generative objectives (Wang et al., 2023, Lee et al., 2024)

2. Core Geometric Pre-training Methodologies

2.1 Self-supervised Geometric Reconstruction

Many frameworks directly reconstruct geometric attributes such as 3D coordinates, centroids, normals, or curvatures under heavy masking or dropout, requiring the model to infer spatial structure even with incomplete data (Tian et al., 2023). For example, masked autoencoders are adapted to point clouds by requiring predictions of centroid, occupancy, surface normals, and curvature for each masked region, resulting in finer-grained geometric reasoning compared to regression of raw coordinates alone (Tian et al., 2023).

2.2 Geometric Matching and Context Priors

Explicit geometric alignment objectives are utilized in medical imaging and cross-modal domains. GVSL introduces a geometric matching head that learns to spatially align 3D medical volumes, integrating both global affine transformations and dense local deformations to maximize inter-image similarity via normalized cross-correlation (He et al., 2023). Volume contrast pre-training leverages geometric context priors in 3D medical images through contrastive learning over spatially distinct crops, encoding the inherent geometric context of organs (Wu et al., 2024).

2.3 Projective and Photometric Consistency

Several approaches in vision and robotics adopt view-synthesis, photometric consistency, and projective geometry as unsupervised signals (Khan et al., 2023, Wu et al., 2023, Zhang et al., 2024). In monocular depth estimation, geometric pre-training involves reconstructing one frame from another using predicted depth and relative pose, with losses encompassing photometric reconstruction and geometric consistency (Khan et al., 2023). For visuomotor policy learning, self-supervised geometric modeling via per-pixel reprojection loss forces the encoder to disentangle depth/motion cues from nuisance factors (Wu et al., 2023).

2.4 Synthetic and Procedural Geometry

Procedurally generated geometric primitives serve as an efficient pre-training source in domains where obtaining real labeled geometry is impractical. Primitive Geometry Segment Pre-training generates 3D image volumes composed of randomly parameterized geometric objects and trains segmentation networks to recover their exact masks, equipping models with strong shape priors before exposure to real-world data (Tadokoro et al., 2024). Formula-supervised visual-geometric pre-training synthesizes paired images and point clouds from mathematical (e.g., fractal) formulas to provide tightly aligned, annotation-free visual-geometric supervision (Yamada et al., 2024).

2.5 Trajectory-based Lifting and Synthetic Dynamics

Scaling to physics surrogate tasks, GeoPT innovatively lifts static geometry with synthetic dynamics, self-supervising the model on the evolution of geometric features along randomized velocity fields. The model is trained to map every geometry and synthetic velocity to the resulting trajectory of geometric features, learning geometry-dynamics coupling that transfers efficiently to real fluid mechanics or solid mechanics tasks (Wu et al., 23 Feb 2026). This "lifting" framework is motivated by the mismatch observed when naively pre-training on static geometry alone.

3. Geometric Pre-training in Cross-modal and Downstream Contexts

3.1 Multimodal and Cross-modality Fusion

Unified transformer models are pre-trained on both visual and geometric (point cloud) representations for tasks demanding cross-modal reasoning, such as image and 3D object classification, detection, and segmentation (Yamada et al., 2024). Formula-driven approaches enable perfect cross-modal alignment by construction, allowing a single transformer backbone to learn shared representations.

In document understanding, explicitly modeling geometric relations among text segments boosts performance in relation extraction tasks, with geometric pre-training enforcing direction, distance, and collinearity constraints (Luo et al., 2023).

3.2 Geometric Distillation and Latent Alignment

GLaD demonstrates that distilling geometric priors from a "geometry-aware" vision transformer (pre-trained for depth, normals, point cloud, and pose) into the hidden states of a large multimodal LLM sharpens attention maps, enhances spatial reasoning and significantly improves policy robustness for vision-language-action agents (Guo et al., 10 Dec 2025).

3.3 Pre-training for Physics and Molecules

Emerging methods for molecule modeling pre-train graph neural networks to reconstruct bond lengths, angles, and dihedrals or to predict global/force-level embeddings obtained from