---
title: 'InfiniDepth: Neural Implicit Depth Estimation'
url: https://www.emergentmind.com/topics/infinidepth
type: topic
---

# InfiniDepth: Neural Implicit Depth Estimation

InfiniDepth is a neural architecture for monocular depth estimation that models depth as a continuous implicit field, enabling arbitrary-resolution and fine-grained depth prediction. By replacing traditional discrete image-grid prediction with neural implicit fields, InfiniDepth provides scalability to any output resolution, addresses fine-detail recovery, and facilitates high-quality geometric representations suitable for novel-view synthesis. The method introduces a multi-scale local implicit decoder to efficiently query depth at continuous coordinates and demonstrates state-of-the-art results across synthetic and real-world benchmarks with particular strength in high-frequency, detail-rich regions [2601.03252].

## 1. Mathematical Formulation: Neural Implicit Depth Function

InfiniDepth establishes depth estimation as regression over a continuous 2D domain. The general form of an implicit field is:
\[
\mathbf y = F_\theta(\mathbf x)
\]
where \( F_\theta \) denotes a multi-layer perceptron (MLP). For monocular depth estimation conditioned on an image \( I \), the mapping becomes:
\[
d_I(x, y) = N_\theta( I, (x, y) ),\quad (x, y) \in [0, W] \times [0, H]
\]
Here, \( N_\theta: \mathbb{R}^{H\times W\times 3} \times \mathbb{R}^2 \to \mathbb{R} \) maps an input RGB image and a continuous coordinate to a scalar depth value. The final depth prediction is expressed as:
\[
d_I(x,y) = \mathrm{MLP}(\mathbf h_L)
\]
where \( \mathbf h_L \) is a locally fused feature vector derived through hierarchical multi-scale aggregation.

## 2. Multi-Scale Local Implicit Decoder Architecture

### Input Encoding

- Input: RGB image \( I \in \mathbb{R}^{H\times W \times 3} \)
- Encoding: Processed using a Vision Transformer (ViT-Large, DINOv3)
- Feature Extraction: From transformer layers 4, 11, and 23, token maps are projected to feature maps at resolutions
  \[
  f^k \in \mathbb{R}^{h_k \times w_k \times C^k} \quad (k=1,2,3)
  \]
  with \((h_1, w_1) = 4 \times H/16\), \((h_2, w_2) = 2 \times H/16\), \((h_3, w_3) = 1 \times H/16\) and channels \( C^1 = 256, C^2 = 512, C^3 = 1024 \).

### Continuous Feature Query and Fusion

- For a continuous coordinate \( (x, y) \), corresponding coordinates at each scale are mapped as:
  \[
  (x_k, y_k) = \left(x \cdot \frac{w_k}{W},\; y \cdot \frac{h_k}{H}\right)
  \]
- Local neighborhoods \( \mathcal N_k \) (\(2 \times 2\) area) are bilinearly interpolated to produce local descriptors \( f^k_{(x, y)} \).
- Hierarchical fusion proceeds as:
  \[
  \mathbf h_{k+1} = \mathrm{FFN}_k \left( f^{k+1}_{(x, y)} + \mathbf{g}_k \odot \mathrm{Linear}(\mathbf h_k) \right)
  \]
  - \( \mathrm{Linear} \): Projects \( \mathbf h_k \) to channel space \( C^{k+1} \)
  - \( \mathbf{g}_k \): Learnable channel-wise gate (\(0,1)^{C^{k+1}}\))
  - \( \mathrm{FFN}_k \): Two-layer MLP block, expansion factor 4, ReLU activation, residual connection
- The final fused feature \( \mathbf h_L \in \mathbb{R}^{1024} \) advances through a 3-layer MLP (input 1024, hidden 256, ReLU activations, terminal ELU) to yield the scalar depth value.

## 3. Training Objective and Loss Functions

Supervision is performed on random continuous coordinate–depth tuples:
\[
\{(x_i, y_i, d_i)\}_{i=1}^N
\]
with primary objective:
\[
\mathcal{L} = \frac{1}{N} \sum_{i=1}^N \left|\, \hat d_i - d_i \right|
\]
where \( \hat d_i = d_I(x_i, y_i) \). Ground-truth depths are normalized to log-space as:
\[
d_{\text{norm}} = \frac{d_{\log} - d_{\min}}{d_{\max} - d_{\min}}, \quad d_{\log} = \ln d
\]
with \( d_{\min}, d_{\max} \) the 2nd and 98th percentiles of \( \ln d \) per-image.

For the monocular model, no additional gradient or smoothness regularization is applied. In the context of novel-view synthesis with the Gaussian-Splatting head, an additional \( L_1 \) RGB reconstruction loss and LPIPS perceptual loss are incorporated.

## 4. Inference and Arbitrary-Resolution Depth Querying

Depth inference does not require fixed-grid upsampling or convolution, enabling pointwise continuous prediction:

1. The input image is encoded at native resolution to generate feature pyramids \( \{f^k\}_{k=1}^3 \).
2. To produce a depth map at arbitrary resolution (e.g., 3840×2160), each output point \( (x, y) \) is:
   - Mapped to each scale \( (x_k, y_k) \)
   - Used to bilinearly interpolate local features \( f^k_{(x, y)} \)
   - Processed through hierarchical fusion into \( \mathbf h_L \)
   - Passed to the MLP head for \( \hat d = d_I(x, y) \)

For novel-view synthesis:
- The implicit field is sampled on a grid, and depths are backprojected to 3D.
- Per-pixel surface area weights are computed:
  \[
  w(x, y) = d_I(x, y)^2 |\mathbf n(x, y) \cdot \mathbf v(x, y)| + \varepsilon
  \]
  where \( \mathbf n(x, y) \) is the normal (via autograd) and \( \mathbf v(x, y) \) the view vector.
- Surface samples are drawn by inverse-CDF, then re-queried for a dense, uniform point cloud suitable for Gaussian splatting.

## 5. The Synth4K Benchmark

Benchmarking is conducted on the Synth4K dataset, comprising 4K-resolution frames (3840×2160) from five AAA games with diverse scene geometry and appearance:

| Subset       | Game Title            |
|:-------------|:----------------------|
| Synth4K-1    | CyberPunk 2077        |
| Synth4K-2    | Marvel’s Spider-Man 2 |
| Synth4K-3    | Miles Morales         |
| Synth4K-4    | Dead Island           |
| Synth4K-5    | Watch Dogs            |

Each subset contains hundreds of frames with varied content (indoor/outdoor, lighting, geometry). A high-frequency mask for evaluating fine-detail performance is constructed by applying multi-scale Laplacian energy operators on the depth map, normalizing via the 98th percentile, and sharpening with an exponent \(1/\tau\). Top-k energy pixels define mask regions for detailed evaluation.

Metrics employed include:
- Relative depth: \( \delta_\tau = \% \{\max(\hat d/d^*, d^*/\hat d) < 1.25^\tau\} \), \( \tau \in \{0.5, 1, 2\} \)
- Metric depth: \( \delta_\epsilon \) for \( \epsilon \in \{0.01, 0.02, 0.04\} \)

Zero-shot protocol is enforced: all models are tested without fine-tuning. Test images are input at 504×896; for 4K output, baselines are bilinearly upscaled, whereas InfiniDepth is queried directly at the target resolution.

## 6. Experimental Results and Significance

### Zero-Shot Relative Depth Performance

- On Synth4K, InfiniDepth achieves superior \( \delta_{0.5}, \delta_1, \delta_2 \) scores across all subsets (\(\delta_1 \approx 89\%\); next best: 84–88%).
- In high-frequency regions representing fine details, InfiniDepth attains \( \delta_1 \approx 67.5\% \) versus the second-best at 66.5%.

### Real-World Benchmarks

- On datasets such as KITTI, ETH3D, NYUv2, ScanNet, and DIODE, InfiniDepth is top-3 on all and best on both ETH3D (\(\delta_1 = 99.1\%\)) and DIODE.

### Metric Depth with Sparse LiDAR

- Synth4K with 1.5k sparse LiDAR: InfiniDepth+Prompt (Ours-Metric) produces \( \delta_{0.01} \approx 78\% \) (prior best: 65%). For fine details, \( \delta_{0.01} \approx 33\% \) (prior best: 21%).
- Real data: KITTI (\(\delta_{0.01}\) = 63.9% vs 58.3% for PromptDA), ETH3D (96.7% vs 92.8%).

### Single-View Novel-View Synthesis

With Infinite Depth Query and a Gaussian-Splatting head, InfiniDepth yields syntheses with fewer holes and artifacts under large viewpoint changes than prior pixel-aligned methods (e.g., ADGaussian), as shown in qualitative examples [2601.03252].

## 7. Context and Implications

InfiniDepth replaces conventional grid-based dense prediction with a neural continuous field, directly modeling \( d_I(x, y) = N_\theta(I, (x, y)) \) and leveraging multi-scale fusion for localized detail retrieval. This enables sub-pixel supervision and resolution-agnostic inference, with no explicit upsampling or convolution in the final stages. The approach attains state-of-the-art results both on synthetic 4K data and established real-world datasets, particularly excelling in geometric detail retrieval. A plausible implication is that this architectural paradigm can generalize to other per-pixel prediction tasks where spatial continuity and resolution flexibility are critical [2601.03252].

Source: https://www.emergentmind.com/topics/infinidepth