Monocular Geometry Estimation

• Monocular geometry estimation is the process of inferring 3D structural information such as depth, surface normals, and 3D point maps from a single RGB image using learned priors and geometry-based constraints.
• It integrates feedforward networks, generative diffusion models, and geometry-aware loss functions to enhance metric accuracy, edge fidelity, and generalization across diverse scenes.
• The approach is validated by scale-invariant losses and comprehensive benchmarks, showing strong applicability in robotics, AR/VR, autonomous driving, and 3D vision.

Monocular geometry estimation refers to the problem of inferring 3D geometric properties—such as depth, surface normals, or 3D point maps—from a single image, in the absence of stereo or multi-view cues. This highly underconstrained task stands at the core of scene understanding for robotics, AR/VR, autonomous driving, and 3D vision. Recent advances integrate machine learning, geometric reasoning, and self-supervision to mitigate inherent ambiguities and improve accuracy, metricity, and generalization.

1. Foundations and Scope

Monocular geometry estimation encompasses the regression of per-pixel depth maps, 3D surface normals, or point clouds from a single RGB image. While classic geometry dictates that monocular cues alone cannot recover absolute scale or complete 3D structure, deep learning methods leverage priors, context, semantics, and learned features to hallucinate plausible geometry, while self- or weak-supervision and hybridization with geometric constraints mitigate scale ambiguities.

The field now includes not only per-pixel depth estimation, but also broader tasks such as affine-invariant 3D point estimation (Wang et al., 2024), surface normal estimation (Long et al., 2024), omnidirectional (360°) geometry (Li et al., 2022), 3D object pose (Zhang et al., 2021), and camera calibration from monocular cues (Zhu et al., 2023). Foundation models pre-trained on massive data and tailored for open-domain images further broaden its applicability (Wang et al., 2024, Ge et al., 2024).

2. Key Methodological Frameworks

2.1 Discriminative Approaches

The dominant paradigm is to use feedforward encoders (CNNs, ViTs) with decoder heads to regress dense geometry. Encoder–decoder networks (e.g., U-Nets (Ma et al., 2022), DPT (Ge et al., 2024)) are trained with regression or ordinal losses on depth, normals, and sometimes semantic labels. Deterministic pipelines transform images to per-pixel geometry via a fixed set of learned weights, often regularized by affine-invariant or scale-invariant losses (Wang et al., 2024, Ge et al., 2024).

2.2 Generative and Diffusion Models

Generative diffusion models directly approach geometry estimation as a conditional generative modeling problem. Pixel-Perfect Depth (PPD) (Xu et al., 8 Jan 2026) utilizes diffusion transformers operating in the pixel space, denoising depth maps guided by semantic prompts from vision foundation models to achieve artifact-free reconstructions. Diffusion models can outperform latent-space VAEs in preserving object boundaries and sharp details, albeit at higher computational cost (Xu et al., 8 Jan 2026, Ge et al., 2024).

2.3 Geometry-Aware Learning and Constraints

Modern pipelines increasingly fuse explicit 3D geometry:

• Projective Models: Networks encode analytic projective geometry (e.g., depth from 2D box size and 3D dimension for object detection (Zhang et al., 2021)).
• Planar Parallax and Plane+Parallax: Approaches such as DepthP+P (Safadoust et al., 2023), Gamma-from-Mono (Elazab et al., 3 Dec 2025), and MonoPP (Elazab et al., 2024) exploit planar homographies and parallax residuals induced by dominant planes (road, ground) to resolve scale and ensure metric consistency.
• Affine-Invariant Representations: MoGe (Wang et al., 2024) directly predicts a 3D point map up to global scale and shift, with losses enforcing robust affine alignment.
• Geometry-Aware Attention and Multi-Frame Aggregation: Transformer architectures employ spatial and temporal attention informed by geometry to ensure consistency across frames and enforce cycle-consistent or photometric constraints (Ruhkamp et al., 2021).

Table 1 provides a high-level overview of representative methodologies:

Technique	Core Innovation	Representative Papers
Feedforward CNN/Vision Transformer	Discriminative regression/affine-invariant loss	(Ma et al., 2022, Ge et al., 2024)
Pixel-space Diffusion	Generative DiT guided by semantics	(Xu et al., 8 Jan 2026)
Projective/Analytic Models	Closed-form depth/geometric fusion	(Zhang et al., 2021, Safadoust et al., 2023)
Geometry-guided Attention	Depth-aware spatiotemporal attention	(Ruhkamp et al., 2021)
Planar-parallax, Gamma, Teacher-Student	Metricized depth via geometric constraints	(Elazab et al., 3 Dec 2025, Elazab et al., 2024)
Affine-invariant 3D map	Pointwise 3D regression + robust alignment	(Wang et al., 2024, Li et al., 18 Apr 2025)

3. Losses, Priors, and Supervision Strategies

3.1 Supervision Modalities

• Fully Supervised: Direct regression to ground-truth depth, normals, or 3D points, with scale/affine alignment losses if global scale is ambiguous (Wang et al., 2024, Ge et al., 2024).
• Self-Supervised: Photometric reprojection losses, cycle consistency, and synthetic warping establish view synthesis as training signal (Safadoust et al., 2023, Elazab et al., 3 Dec 2025, Ruhkamp et al., 2021).
• Teacher-Student Distillation: Multi-frame or geometry-aware teachers distill metric cues or static scene information to single-frame students (Elazab et al., 2024, Elazab et al., 3 Dec 2025).
• Joint Semantic-Geometry Learning: Semantics are fused during training, not merely as context but also as explicit cross-domain losses (e.g., enforcing that depth edges align with semantic boundaries (Ramirez et al., 2018)).

3.2 Specialized Losses

• Affine/Scale-Invariant L1/L2 Losses: Loss is imposed on depth maps or 3D point clouds only after globally aligning for scale and shift (Wang et al., 2024, Ge et al., 2024).
• Mixture Density and Uncertainty: Learned distributions over depth (e.g., ProbDepthNet's mixture-of-Gaussians (Brickwedde et al., 2019)) quantify aleatoric uncertainty and improve calibration.
• Projective Error Terms: Analytic projective losses use 2D/3D correspondences to enforce epipolar, parallax, or depth-from-box constraints (Zhang et al., 2021, Safadoust et al., 2023, Elazab et al., 3 Dec 2025).
• Adaptive Surface Normal Constraints: Geometry context maps and sampling-based surface normal estimation drive joint depth-normal consistency (Long et al., 2024).

4. Hybridization with Classical Geometry

Integration of explicit geometric reasoning is essential for overcoming the projective ambiguities of monocular inference. Key trends include:

• Planar Priors and Parallax: Robust metric geometry estimation is achieved by exploiting knowledge of camera height and dominant planes (e.g., road) and measuring relative heights via gamma ($\gamma = h/d$) (Elazab et al., 3 Dec 2025).
• Pose Estimation and Scale Recovery: Methods leverage semantic segmentation to isolate static ground regions, refine dynamic object removal, and robustly fit planes to recover global scale during monocular visual odometry or SLAM (Zhang et al., 6 Mar 2025).
• Calibration and Intrinsic Estimation: Monocular geometry estimation is increasingly linked to camera calibration—estimating intrinsics such as focal length and principal point via learned incidence fields and least-squares fitting (Zhu et al., 2023).
• Multi-View Refinement: Monocular cues serve as priors for multi-view 3D reconstruction, compensating for weakly textured or occluded regions where feature matching fails (Li et al., 18 Apr 2025).

5. Benchmarking, Foundation Models, and Generalization

Comprehensive benchmarking campaigns such as GeoBench (Ge et al., 2024) provide unified protocols for comparing discriminative and generative paradigms across diverse datasets (indoor, outdoor, synthetic, real). Salient observations include:

• Discriminative models (ViT+DPT, DINOv2 backbone) can outperform generative diffusion models if fine-tuned on high-quality synthetic data, highlighting the primacy of data quality over data quantity or complexity of model architecture.
• Generative diffusion approaches excel in edge fidelity and detail recovery, with pixel-space transformers (PPD) achieving SOTA “flying pixel”-free reconstructions but at higher computational cost (Xu et al., 8 Jan 2026).
• Affine-invariant pipelines and geometry-aware losses facilitate robust generalization to novel domains, diverse camera intrinsics, and challenging scenes (Wang et al., 2024, Ge et al., 2024).
• Monocular geometry estimation delivers strong zero-shot and cross-dataset performance for downstream tasks, including video geometry (Xu et al., 8 Jan 2026), 3D object detection (Zhang et al., 2021), video odometry (Zhang et al., 6 Mar 2025), and point cloud reconstruction (Li et al., 18 Apr 2025).

6. Dataset Diversity and Evaluation Protocols

To overcome the limitations of restricted real datasets (e.g., NYUv2, KITTI), recent work leverages a mixture of synthetic scenes, real captures, and domain-adaptive pre-processing (Ge et al., 2024, Wang et al., 2024). Evaluations now emphasize affine-invariance, edge-fidelity (edge AbsRel), and 3D point cloud/normal metrics alongside classical depth error rates.

Key benchmarks include:

• Depth: AbsRel, RMSE, δ-threshold accuracy (δ₁, δ₂, δ₃), edge AbsRel.
• Normals: Mean/median angular error, accuracy within angular thresholds.
• Point Maps: Relative error, inlier ratio after optimal affine alignment.
• Field-of-View: Camera FOV estimation, reflecting calibration capabilities.

7. Open Challenges and Prospects

Despite significant advances, several fundamental challenges persist:

• Absolute Scale Ambiguity: While planar-parallax and metricization via structural priors reduce ambiguity, scenes lacking dominant planes, or with inaccurate camera height priors, remain problematic (Elazab et al., 3 Dec 2025, Elazab et al., 2024).
• Dynamic and Non-Rigid Scenes: Robust disentangling of static and moving objects is essential for accurate monocular scale recovery and video geometry (Zhang et al., 6 Mar 2025).
• Edge Fidelity and Thin Structures: Current models navigate the bias-variance trade-off between global consistency and preservation of fine detail, with pixel-space diffusion offering improvement (Xu et al., 8 Jan 2026).
• Generalization and Foundation Models: The integration of discriminative and generative paradigms, informed by high-fidelity synthetic data, is a central avenue for ongoing research (Wang et al., 2024, Ge et al., 2024).
• Computational Cost and Real-Time Constraints: High-precision diffusion models are computationally expensive; distillation and efficient transformer architectures are promising directions (Xu et al., 8 Jan 2026).

This synthesis affirms that monocular geometry estimation is transitioning from isolated, domain-tuned models toward general-purpose, geometry-compatible visual foundation models that combine data, learning, and explicit 3D reasoning in scalable and robust pipelines. The interplay between geometric constraints, strong visual priors, and task-specific supervision remains the linchpin for progress.