SurGe: Improved Surface Geometry in Point Maps

Abstract: Recent feedforward 3D reconstruction methods predict point maps and estimate global 3D geometry remarkably well. However, their predictions still exhibit inaccurate local surface geometry, which is clearly visible qualitatively but only weakly reflected in common metrics. To make these errors more explicit in evaluation, we introduce a point map normal metric that evaluates the local surface orientation induced by neighboring 3D predictions. To reduce these errors, we propose two complementary components: a point gradient matching loss that supervises depth-normalized 3D finite differences, and a Neighborhood Attention Decoder (NAD) that progressively upsamples features and uses Neighborhood Attention for local feature mixing. Across eight zero-shot monocular geometry benchmarks, our model, SurGe, achieves the best average rank for global point map AbsRel and consistently improves local point map and point map normal evaluations.

• The paper introduces a new point map normal metric to measure local surface orientation using mean angular error.
• The paper presents a novel point gradient matching loss and an attention-based decoder that accurately captures thin and complex 3D structures.
• The paper demonstrates that improved local geometric accuracy on standard benchmarks sets a new evaluation paradigm in 3D vision.

SurGe: Improved Surface Geometry in Point Maps – A Technical Analysis

Introduction and Motivation

Monocular 3D scene reconstruction from a single image has rapidly advanced with feedforward architectures that predict dense 3D point maps. While these methods demonstrate competitive global geometric accuracy, they commonly exhibit artifacts and inaccuracies in local surface geometry—manifested as inconsistent surface orientation, distortions on thin structures, and high-frequency surface noise. These pathologies are not adequately captured by standard global pointwise error metrics, thus motivating a shift in both evaluation and modeling focus toward local geometric consistency.

SurGe directly targets this gap. It introduces an improved evaluation paradigm and novel training and architectural innovations, resulting in superior local surface geometry while maintaining competitive global performance.

Methodological Contributions

SurGe makes three principal technical contributions:

1. Point Map Normal Metric: A new evaluation metric that quantifies the mean angular error between local surface normals derived from point map neighborhoods, providing a direct measure of local surface orientation correctness.
2. Point Gradient Matching Loss ($\mathcal{L}_{\text{pgm}}$): A novel, scale-invariant, pairwise loss function supervising the local orientation and magnitude of 3D point gradients. This loss extends ideas from the log-depth gradient matching loss but adapts them coherently for vector-valued (3D) point maps, enforcing local coherence at a finer geometrical level.
3. Neighborhood Attention Decoder (NAD): An attention-based upsampling decoder that, unlike convolutional alternatives, leverages Neighborhood Attention for local, content-adaptive feature mixing at multiple resolutions. This design avoids the limitations of spatial convolution (fixed receptive fields) and pure Transformer decoders (patch-level artifacts), and empirically produces more accurate geometry on thin structures and in regions with high spatial complexity.

Architectural Design

The SurGe pipeline comprises a DINOv2-initialized ViT-Large encoder followed by the NAD. The NAD operates in five progressive upsampling stages, each composed of multiple NAD blocks employing Neighborhood Attention. Stages execute feature mixing and upsampling, culminating in a per-pixel 3D point map output using an $(\xi,\eta,\rho)$ parameterization with exponential scale. The NAD intentionally omits LayerNorm; instead, it relies on QK normalization for training stability, hypothesizing that this better preserves activation scales relevant for regression tasks.

Critically, NAD's local attention combines the spatial sensitivity of convolutions with the adaptivity of feature-driven attention. It enables precise recovery of high-frequency geometric details while remaining computationally tractable at high resolution.

Learning Objective

The supervision scheme combines:

• Global affine-invariant point map loss
• Multi-scale local patch-wise losses (with various spatial support)
• The new point gradient matching loss ($\mathcal{L}_{\text{pgm}}$)

The $\mathcal{L}_{\text{pgm}}$ term matches depth-normalized finite 3D point differences between neighboring pixels, enforcing that local 3D structures are not only positionally accurate but also form locally coherent surfaces. The loss is weighted empirically (factor 10 relative to the global loss) and is masked near occlusion boundaries to avoid irreducible errors due to ground truth ambiguity.

Empirical Results and Analysis

SurGe is evaluated on eight standard zero-shot monocular geometry benchmarks (including NYUv2, KITTI, ETH3D, iBims-1, GSO, Sintel, DDAD, DIODE), employing both established global alignment metrics and the newly proposed point map normal error. Key results include:

• Strongest average rank across global point map metrics versus prior monocular and many-view models.
• Consistent improvement in local surface evaluations, with SurGe yielding the lowest local point map and normal error across all benchmarks.
• Qualitative advances on thin and complex structures, with reduced blockiness, bending, or oscillations versus convolutional and Transformer decoder alternatives.

Ablation studies further elucidate contributions:

• NAD outperforms both convolutional decoders (ConvStack, DPT head) and ViT decoders across all regimes, with the gap especially pronounced in local metrics.
• Replacing $\mathcal{L}_{\text{pgm}}$ with conventional surface losses (normal angle, log-depth gradient) reduces both local and often global geometric fidelity.
• Scaling convolutional decoder capacity narrows the gap but does not close it, emphasizing the efficacy of attention-based local mixing.

There are modest increases in inference latency and memory relative to ConvStack-L, but the computational overhead is marginal compared to encoder costs and justified by accuracy gains.

Implications and Future Directions

SurGe establishes that substantial improvements in local 3D geometry are possible with architectural and loss function innovations tailored to geometric coherence. The point map normal metric concretizes a previously under-explored evaluation axis, likely to influence future model selection and development in geometric vision.

Practically, SurGe's advances improve performance in downstream 3D perception tasks that are sensitive to local geometry, such as SLAM, scene interaction, and robotic manipulation.

Further research directions include:

• Architectural efficiency: Developing lighter-weight attention mechanisms to reduce decoder overhead at high resolution.
• Generalization to sparser or lower-quality labels: Adapting losses and architecture for real-world data regimes.
• Extension to dynamic and sequence data: Investigating temporal consistency and generalizing local surface metrics across time.

Conclusion

SurGe advances monocular 3D reconstruction by directly targeting local surface geometry, marrying a new evaluation metric, a principled pairwise supervision loss, and an attention-based multiscale decoder. The result is a model that achieves state-of-the-art local and global 3D point map accuracy and sets a precedent for prioritizing local geometric fidelity in dense 3D vision tasks. This framework opens avenues for more robust, geometry-aware visual models, with broad implications for future research in computer vision and embodied AI.