DepthPro: Vision & Data Depth Methods

• DepthPro is a dual-themed concept combining a foundation model for high-resolution, zero-shot monocular depth estimation with an R package for robust, nonparametric multivariate data analysis.
• The vision model employs a multi-scale transformer architecture and hybrid training on real and synthetic data to achieve state-of-the-art accuracy and sub-second inference times.
• The DepthProc package implements statistical depth functions to deliver robust visualization, inference, and efficient computation for high-dimensional data.

DepthPro refers to two distinct and unrelated concepts in the academic literature: (1) "Depth Pro," a foundation model for zero-shot metric monocular depth estimation in computer vision, and (2) "DepthProc," an R package for robust exploration and statistical inference on multidimensional data via data depth concepts. These systems operate in entirely separate domains—deep monocular depth estimation in vision and nonparametric multivariate statistics, respectively—without overlap in methodology or intended application.

1. Depth Pro: Foundation Model for Monocular Metric Depth

Depth Pro is a foundation model designed for zero-shot, monocular metric depth estimation that produces high-resolution (2.25-megapixel) depth maps with exceptionally sharp object boundaries. The model achieves metric (absolute scale) depth prediction from a single RGB image, independent of camera intrinsics metadata, and can generate 1536×1536 outputs in 0.3 seconds on a standard GPU (Bochkovskii et al., 2024). Its technical contributions consist of:

• An efficient multi-scale vision transformer (ViT) architecture for dense prediction that merges global and local context.
• A hybrid training protocol that leverages both real-world RGB-D datasets and pixel-perfect synthetic data for generalization and sharpness.
• Specialized evaluation metrics for boundary fidelity in depth estimation.
• A focal length regressor that allows for zero-shot metric scaling without external calibration data.

Model Architecture

Depth Pro fuses local and global scene understanding using a dual ViT-based encoding scheme: an "image encoder" computes global context from low-resolution inputs, while a "patch encoder" (with shared weights) processes overlapping 384×384 patches at multiple scales (1×, 0.5×, 0.25× of input size). Feature maps at each scale are rearranged and upsampled, then summed to form a unified spatial feature map at full resolution:

$$F_{\text{patch}} = \sum_{s \in \{1,2,4\}} \mathrm{Upsample}(F_s)$$

This is concatenated with upsampled image-level features and processed via a DPT-style lightweight decoder (comprising 1×1 and 3×3 convolutions with skip-connections) to yield the canonical inverse depth map $C \in \mathbb{R}^{1536 \times 1536}$.

Metric Depth Estimation and Focal Length Recovery

The inverse depth estimate $C(i,j)$ in pixel units is converted to metric depth $D_m(i,j)$ using an automatically regressed focal length $f_{\text{px}}$ (in pixels) and the image width $w$:

$D_m(i,j) = \frac{f_{\text{px}}}{C(i,j)} \times w$

The focal head consists of a convolutional branch over DPT features running in parallel with an independent tiny ViT encoder. Their outputs are concatenated and regressed to $f_{\text{px}}$, trained using an $\ell_2$ loss on log-focal error. This design enables recovery of metric scaling in unseen domains without external camera metadata.

Two-Stage Training Protocol

Stage 1 mixes real (ARKitScenes, ScanNet, Sun-RGBD, etc.) and synthetic (Hypersim, TartanAir, Synscapes, etc.) RGB-D data, using a trimmed mean absolute error (MAE) loss and scale-and-shift-invariant MAE or gradient loss for non-metric and synthetic splits, respectively.

Stage 2 exclusively uses synthetic data (with ground-truth pixelwise depth) and adds first- and second-order multiscale derivative losses—Mean Absolute Gradient Error (MAGE) and Mean Absolute Laplace Error (MALE)—which are critical for learning sharp boundaries in depth output:

$$\mathcal{L}_{\text{MAGE}} = \frac{1}{M} \sum_j \frac{1}{N_j} \sum_i |\nabla_s C^j_i - \nabla_s \hat C^j_i |$$

A curriculum from mixed real–synthetic to synthetic-only fine-tuning is crucial for reconciling generalization and edge sharpness.

Boundary Accuracy Evaluation

Depth Pro introduces dedicated occluding-contour boundary metrics. For a predicted depth map $C \in \mathbb{R}^{1536 \times 1536}$0 and ground-truth $C \in \mathbb{R}^{1536 \times 1536}$1, the occlusion indicator at neighboring pixels $C \in \mathbb{R}^{1536 \times 1536}$2 is:

$C \in \mathbb{R}^{1536 \times 1536}$3

with threshold $C \in \mathbb{R}^{1536 \times 1536}$4 in $C \in \mathbb{R}^{1536 \times 1536}$5 (percent). Precision $C \in \mathbb{R}^{1536 \times 1536}$6, recall $C \in \mathbb{R}^{1536 \times 1536}$7, and the harmonic mean $C \in \mathbb{R}^{1536 \times 1536}$8 are computed and aggregated, with higher thresholds receiving more weight. Mask-based recall is also reported for alpha-matte and segmentation masks.

Performance

Depth Pro demonstrates state-of-the-art accuracy and efficiency on zero-shot metric depth and boundary sharpness metrics. For instance, on the Sun-RGBD dataset $C \in \mathbb{R}^{1536 \times 1536}$9 (pixels within 25% error) is 89.0%, and average processing time for high-resolution outputs remains 0.3 s on a V100 GPU. In boundary F1, Depth Pro outperforms PatchFusion and Marigold, with substantially faster inference (see Table).

Method	Sun-RGBD δ₁	Sintel F1	Runtime (V100 2MP)
Depth Pro	89.0%	0.409	341 ms
PatchFusion	75.6%	0.312	84 s
Marigold	95.8%	0.068	4.4 s

Ablation and Design Analysis

• ViT-L DINOv2 backbones deliver superior generalization ($C(i,j)$0) across benchmarks.
• Multi-scale patch+image encoding is 10–30% faster and achieves sharper boundaries than single high-res ViT.
• Stage 1 gradient losses must be scale-and-shift-invariant applied only to synthetic data to avoid convergence issues.
• Derivative-based losses (MAGE + MALE) are essential for maximizing boundary F1 without compromising global MAE.
• The focal head design, using parallel conv+ViT, yields the best metric accuracy (∼78% $C(i,j)$1 under 25%) compared to other variants.

Depth Pro represents the first system to combine sub-second, high-resolution, sharp-boundary, zero-shot monocular metric depth estimation across visual domains (Bochkovskii et al., 2024).

2. DepthProc: R Package for Robust Multivariate and Functional Data Analysis

DepthProc is an R package providing a comprehensive suite of data depth-based tools for robust, nonparametric analysis of multivariate and functional data, particularly in socio-economic contexts (Kosiorowski et al., 2014). DepthProc leverages statistical depth functions—a class of functionals measuring centrality or outlyingness in multivariate or functional data spaces—and exposes algorithms for depth computation, visualization, multivariate tests, scatter estimation, and robust regression.

Data Depth Concepts

A statistical depth function $C(i,j)$2 maps a point $C(i,j)$3 and a (possibly empirical) distribution $C(i,j)$4 to $C(i,j)$5, with properties including affine invariance, maximality at the center, monotonicity on rays, and vanishing at infinity. DepthProc implements:

• Halfspace (Tukey) depth
• Mahalanobis depth
• Euclidean depth
• Projection depth
• Weighted $C(i,j)$6 depth
• Simplicial depth
• Regression depth
• Student (location-scale) depth

For example, the halfspace depth is given as:

$C(i,j)$7

Core Functionalities

• Computation of depth values, contours, and 3D "perspective" plots for $C(i,j)$8 and $C(i,j)$9
• Multivariate quantile–quantile (DD) plots, DD comparisons with multivariate normal, and scale/asymmetry curves for distributional diagnostics
• Depth-weighted robust covariance and location estimators, e.g., using $D_m(i,j)$0 depth weights
• Depth-based and local Wilcoxon statistical tests for multivariate and functional samples
• Robust regression via deepest regression (maximizing regression depth) and projection-depth-trimmed least squares
• Functional data tools: modified band depth (MBD), Fraiman–Muniz (FM) depth, functional boxplots, and local depth measures

Code is provided for practical use in each functionality, e.g., robust covariance:

$D_m(i,j)$4

Applied Case Studies

DepthProc has been applied to, for instance, exploratory evaluation of UN Fourth Millennium Goal health metrics. Tools such as depth contour plots, DD-plots, depth-weighted scatter, and robust regression provide resistant statistical summaries and meaningful visualizations of location and scale shifts over time. Functional depth and local functional depth have revealed functional modality and structure in time-indexed datasets such as hourly web user trajectories.

Computational Considerations

Projection-type depths are approximated by random direction (default ndir=1000), with DepthProc supporting multi-core parallelism via C++ (RcppArmadillo) and OpenMP. Complexity is controlled (e.g., $D_m(i,j)$1 depth $D_m(i,j)$2). DepthProc generally outperforms the R packages depth and ddalpha for large-scale, high-dimensional computations due to these optimizations.

Customization and Practical Guidance

DepthProc supports flexible parameterization (ndir, method, local depth tuning), offers instant computation of depth medians, and supplies interface functions for integrating with ggplot2 for advanced visualization. Selection of depth function depends on data structure (e.g., projection or Tukey for affine invariance, $D_m(i,j)$3 for speed in high dimensionality), and preprocessing is advised (centering/scaling) when variables are on different scales, unless affine invariance is ensured.

3. Technical Comparison and Scope

Depth Pro and DepthProc serve fundamentally different technical purposes. Depth Pro addresses absolute-scale monocular (single-image) depth estimation in computer vision, introducing transformer-based architectures, synthetic-real transfer, and edge-preserving losses for image-based scene understanding (Bochkovskii et al., 2024). DepthProc exists entirely within nonparametric statistics, providing robust, depth-based methods for exploratory, inferential, and predictive analysis of generic multidimensional datasets (Kosiorowski et al., 2014). No methodological or thematic overlap exists between the two systems.

4. Related Systems and Methodologies

Within their domains, closely related methods and systems can be identified:

For monocular metric depth, related vision architectures include PatchFusion, Metric3D v2, Marigold, and PatchFusion, against which Depth Pro demonstrates substantial improvements in accuracy, boundary sharpness, and runtime.

Within robust multivariate statistics, DepthProc complements packages such as ddalpha and depth, and provides state-of-the-art implementations aligned with the theoretical work of Zuo & Serfling, Lopez-Pintado & Romo, and Paindaveine & Van Bever.

5. Limitations and Future Directions

Depth Pro: Current reliance on synthetic data for boundary sharpening constrains final realism but avoids label noise present in real datasets. Focal estimation depends on the parallel conv+ViT design; alternate designs yielded lower metric accuracy. Extension to broader camera models or multimodal input could further generalize capability (Bochkovskii et al., 2024).

DepthProc: Approximate algorithms for projection/halfspace depth can require increased computation to reach critical inference accuracy, especially in very high dimensions. Users must judiciously choose depth type and direction parameters for efficiency and robustness (Kosiorowski et al., 2014).

A plausible implication is that despite their syntactic similarity, the nomenclature "Depth Pro" in vision and "DepthProc" in statistics describes unrelated developments, each constituting state of the art in their respective research traditions.