InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields (2601.03252v1)

Abstract: Existing depth estimation methods are fundamentally limited to predicting depth on discrete image grids. Such representations restrict their scalability to arbitrary output resolutions and hinder the geometric detail recovery. This paper introduces InfiniDepth, which represents depth as neural implicit fields. Through a simple yet effective local implicit decoder, we can query depth at continuous 2D coordinates, enabling arbitrary-resolution and fine-grained depth estimation. To better assess our method's capabilities, we curate a high-quality 4K synthetic benchmark from five different games, spanning diverse scenes with rich geometric and appearance details. Extensive experiments demonstrate that InfiniDepth achieves state-of-the-art performance on both synthetic and real-world benchmarks across relative and metric depth estimation tasks, particularly excelling in fine-detail regions. It also benefits the task of novel view synthesis under large viewpoint shifts, producing high-quality results with fewer holes and artifacts.

Unread Want to Read Currently Reading Completed

• The paper presents a neural implicit field approach that enables continuous depth estimation, recovering fine-grained geometric details at arbitrary resolutions.
• It leverages a multi-scale local implicit decoder with a vision transformer backbone, achieving state-of-the-art accuracy on benchmarks like Synth4K.
• Adaptive sub-pixel queries ensure uniform 3D point sampling, enhancing downstream applications such as single-view novel view synthesis with reduced artifacts.

InfiniDepth: Neural Implicit Fields for Arbitrary-Resolution and Fine-Grained Depth Estimation

Introduction

InfiniDepth introduces a paradigm shift in monocular depth estimation by modeling the depth map as a neural implicit field rather than a discretized 2D grid. The proposed approach enables continuous depth queries at arbitrary image plane coordinates, inherently supporting upsampling to any target resolution and enhanced recovery of high-frequency geometric details. The method employs a multi-scale local implicit decoder and a vision transformer backbone to hierarchically fuse context and local information, producing depth values robustly at any queried location. To rigorously evaluate high-detail depth recovery, the authors curated Synth4K, a high-resolution synthetic benchmark with complex, diverse visual content.

Figure 1: InfiniDepth architecture: local features are queried at continuous 2D coordinates across ViT encoder scales and hierarchically fused for depth decoding.

Neural Implicit Depth Representation

Traditional monocular depth methods predict depth at discrete pixel grid locations, typically matching the network's output stride to the training image resolution. This rigid approach imposes two primary limitations: (1) generation of high-resolution depth maps necessitates explicit upsampling, which smooths out fine geometric detail; (2) prediction is fundamentally tied to specific discretization, constraining adaptability and limiting utility for high-precision 3D applications.

InfiniDepth formulates depth as a coordinate-conditioned neural field. For any continuous point $(x, y)$, a hierarchical feature pyramid is queried and fused, and a lightweight MLP head regresses the corresponding depth. This localized and resolution-agnostic prediction corresponds with the representational efficacy shown by neural coordinate-based methods in image super-resolution and shape reconstruction. The feature pyramid is constructed via a vision transformer (ViT) encoder with reassembled multi-stage features for optimal spatial and semantic coverage.

Figure 2: Surface normals from implicit differentiation highlight the high-quality 3D geometry encoded by the implicit depth field.

Multi-Scale Local Implicit Decoder

Depth queries at arbitrary $(x, y)$ are mapped to relevant locations in each feature pyramid scale via scaled coordinate transformation and bilinear interpolation of nearby tokens, ensuring smoothness and continuity. This approach differs from convolutional upsampling or patchwise decoding and attains hierarchical aggregation from fine to coarse, detail to context, leveraging residual gated fusion for effective feature blending. Such a design guarantees the receptive field dynamically adapts with pyramid depth, capturing both local variations and global contextual cues.

Infinite Depth Query Strategy for Uniform Surface Sampling

Standard depth-to-point-cloud back-projection from discrete grids introduces density imbalance: variations in depth and surface orientation cause nonuniform 3D sampling density, resulting in artifacts when the point cloud is utilized for rendering (e.g., in novel view synthesis). InfiniDepth proposes an adaptive sub-pixel query allocation mechanism, where each pixel's budget is proportional to its underlying true surface area, accounting for both depth-squared scaling and surface normal alignment. Differentiable computation of local surface normals via autograd on the implicit field supports this estimation. The result is a spatially uniform point cloud advantageous for downstream 3D vision tasks under large viewpoint shifts.

Figure 3: The infinite depth query redistributes sampling density to achieve surface-uniform 3D point clouds, mitigating projection-induced artifacts.

High-Resolution Benchmark: Synth4K

Recognizing deficiencies in existing depth estimation benchmarks—namely low spatial resolution and lack of fine-scale geometric structure—the authors introduce Synth4K, containing RGB-D pairs at 4K resolution from five highly detailed, diverse modern game environments. Multi-scale Laplacian masks are used to highlight high-frequency regions, providing targeted evaluation on geometric detail recovery not available in prior benchmarks.

Figure 4: Synth4K contains 4K-resolution RGB images, ground-truth depth maps, and high-frequency detail masks supporting fine-grained evaluation.

Experimental Results: Relative and Metric Depth Estimation

InfiniDepth is compared with SOTA approaches including DepthAnything, MoGe, Marigold, and diffusion-based models in both relative and metric configurations. On Synth4K, InfiniDepth achieves the highest accuracy across all error thresholds, for both all-pixel and high-frequency masked regions. For example, relative depth $\delta_{0.5}$ scores surpass competing models by 5--10 points in fine-detail regions, and the method maintains a clear lead in $\delta_{0.01}$ for metric cases even under the most stringent error margins. On real-world datasets (KITTI, ETH3D, NYUv2, ScanNet, DIODE), InfiniDepth consistently matches or exceeds SOTA, particularly excelling in preserving scene structure at high resolution.

Figure 5: Qualitative relative depth estimation: InfiniDepth resolves fine details lost by grid-constrained baselines, especially at upsampled resolutions.

Figure 6: Qualitative metric depth comparisons: recovery of high-frequency geometric structures is notably improved.

Point cloud visualizations confirm reduced artifacts and superior geometric fidelity, while ablation studies demonstrate the necessity of implicit representation, multi-scale feature querying, and sub-pixel supervision for high-detail predictions.

Generalization and Downstream Implications

A notable property of InfiniDepth is its robustness in transfer settings: the model, trained exclusively on synthetic data, generalizes to real-world images, maintaining leading performance in zero-shot evaluations. The continuous prediction capability and refined geometric detail positively influence downstream tasks. The integration of a depth query strategy with differentiable geometry computation is especially impactful for single-view novel view synthesis. When combined with Gaussian Splatting, InfiniDepth-generated point clouds produce novel views with significantly fewer holes and geometric distortions under aggressive viewpoint changes, as compared to prior pixel-aligned depth prediction methods.

Figure 7: Single-view NVS using InfiniDepth point clouds produces more complete and stable renderings under large viewpoint shifts than ADGaussian.

Ablations and Design Analysis

Experiments further dissect critical components: representation (implicit fields vs. discrete), feature fusion (multi-scale vs. single-scale), interpolation mechanisms, and compare ViT encoder variants. The studies highlight moderate quantitative gains for relative depth and substantial improvements for metric depth and fine detail across metrics, attributable directly to the implicit encoding and hierarchical querying. Computational cost is kept competitive: the decoder of InfiniDepth is parameter-efficient, and while inference is slightly slower than convolutional baselines, it is much faster than diffusion-based models and delivers significantly higher detail.

Figure 8: Ablation on depth representation: neural implicit fields capture fine-grained geometric regions beyond the reach of grid-based decoders.

Limitations and Future Perspectives

InfiniDepth, as formulated, is specialized for single-view depth estimation and does not impose explicit temporal consistency; flickering may occur across video frames. The technique could be extended to multi-view and video settings with additional constraints, improving temporal stability and broader 3D structure consistency. The principle of coordinate-implicit neural modeling is well-aligned with modern 3D perception pipelines, and its integration constitutes a promising avenue for higher-fidelity geometric reasoning in both autonomous robotics and photorealistic neural rendering.

Conclusion

InfiniDepth precisely models depth as a continuous neural implicit field, supporting arbitrarily fine geometric detail recovery and high-resolution depth prediction. Empirical evidence across both synthetic and real-world datasets confirms dominant performance in both standard and challenging fine-detail settings. The adaptive surface-aware depth query mechanism fundamentally benefits 3D downstream applications, notably in single-view novel view synthesis. The approach sets a technical precedent for high-resolution, detail-preserving, and downstream-interpretable depth estimation, motivating further exploration of implicit field representations in visual perception systems.

---

Citation: "InfiniDepth: Arbitrary-Resolution and Fine-Grained Depth Estimation with Neural Implicit Fields" (2601.03252)