CenterDepth: Object-Centric Monocular Depth

• CenterDepth is a point-based, detection-driven paradigm for monocular depth estimation that uses object centers as semantic anchor points to improve efficiency and accuracy.
• The methodology integrates multi-scale backbone features, a keypoint detection head, and a Center FC-CRFs module to refine depth predictions through energy minimization.
• Experimental results on the VirDepth dataset demonstrate significant gains in far-range depth accuracy and computational efficiency compared to traditional pixelwise methods.

CenterDepth is a point-based, detection-driven paradigm for monocular depth estimation, designed to enable high-efficiency and robust performance specifically in the context of automotive perception for urban scenes. By localizing objects through center keypoints and regressing depth only at these semantic anchor points, CenterDepth bypasses traditional pixelwise or fully convolutional depth map predictions, yielding computational and accuracy benefits, especially for far-range and small objects in datasets with severe height-scale imbalance (Tu et al., 26 Apr 2025).

1. Architectural Overview

CenterDepth operates as a one-stage, keypoint-based detector, directly inspired by CenterNet, but extended to monocular depth estimation. The architecture comprises the following main components:

• Backbone: Multiple backbones are supported, including Hourglass-104 (HGL104), ResNet-101 with deformable convolutions (ResNet101-DCN), VoVNet-99, DLA-34, and ResNet-18. Each extracts a multi-scale feature map $F \in \mathbb{R}^{N \times N \times C}$ encoding spatial and semantic cues.
• Detection Head: Outputs a heatmap $H(x, y) \in [0,1]$ for object center localization and two parallel regression heads for object box size $(w, h)$, optionally with sub-pixel offsets.
• Center-anchored Depth Head: Rather than regressing a dense depth map, a scalar depth $D_c$ is predicted exclusively at each detected center $(x_c, y_c)$ via features pooled from a local region.
• Center FC-CRFs Module: A region-restricted, fully connected conditional random field fuses multi-scale features. The FC-CRFs propagate information from all pixels within region $R = \{(x,y)\,|\,|x-x_c|\leq w/2, |y-y_c|\leq h/2\}$ to the corresponding center anchor.
• Joint Detection + Depth: Training is performed end-to-end with a shared backbone; the detector and depth estimation heads are fully integrated, allowing simultaneous learning of semantic localization and depth regression (Tu et al., 26 Apr 2025).

This design enables CenterDepth to remain lightweight, end-to-end, and inherently structured for object-centric downstream tasks.

2. Center FC-CRFs: Depth Refinement via CRF Energy Minimization

The Center FC-CRFs module recasts depth estimation within a detected box as a CRF energy minimization problem. Each pixel in the region $R$ and the center pixel are nodes in the CRF. The model includes the following potential terms:

• Pairwise Potential:

$$\psi_p(D(x_i), D(x_c)) = \omega_{i,c} \| D(x_i) - D(x_c) \|^2$$

where $$\omega_{i,c} = \exp\left( - \| f(x_i) - f(x_c) \|^2 / (2 \sigma_f^2) \right )$$ encodes similarity in feature space, emphasizing dependence between semantically similar pixels [(Tu et al., 26 Apr 2025), Eq. 3–4].

• Unary Potential:

$\psi_u(D(x_i)) = (D(x_i) - \hat{D}(x_i))^2$

aligning estimated depths to local predictions from the depth head.

• Total Energy:

$H(x, y) \in [0,1]$0

[(Tu et al., 26 Apr 2025), Eq. 5].

For inference, a closed-form solution or mean-field updates yield a refined center depth:

$H(x, y) \in [0,1]$1

and optionally update surrounding pixels. This procedure enforces smoothness among depth predictions inside each object proposal while anchoring the final estimate at the detected center (Tu et al., 26 Apr 2025).

3. Detection-Based Depth Estimation Pipeline

The learning objective adopts a multi-task formulation:

• Center Key-point Heatmap Loss (Focal Loss variant):

$H(x, y) \in [0,1]$2

[zhou2019objects].

• Size Regression Loss:

$H(x, y) \in [0,1]$3

• Raw Depth Regression:

$H(x, y) \in [0,1]$4

• Full Multi-task Loss:

$H(x, y) \in [0,1]$5

with weights optimized via cross-validation (e.g., $H(x, y) \in [0,1]$6, $H(x, y) \in [0,1]$7).

During inference, peaks in $H(x, y) \in [0,1]$8 yield object centers, sizes are extracted, and Center FC-CRFs are solved to provide a final depth per center (Tu et al., 26 Apr 2025).

4. VirDepth: Synthetic Benchmark for Monocular Detection and Depth

CenterDepth is benchmarked and trained on the VirDepth dataset, a large-scale, scenario-specific synthetic dataset generated using CARLA and Unreal Engine 4. The data-collection protocol employs three co-located cameras (RGB, depth, semantics), with the ego-vehicle traversing urban environments and capturing:

• RGB images
• Ground-truth depth maps
• Semantic segmentation
• 2D and 3D bounding boxes (CAD-based projection)

VirDepth statistics:

• Five urban scenes, 15,000 images at $H(x, y) \in [0,1]$9 resolution
• Over 180,000 annotated targets, 0–200 m range, object types: cars, vans, trucks, bicycles, pedestrians
• ≥3 objects per image per 50 m bin on average; objects >200 m or occluded are filtered
• Dataset splits: scenes 1–4 for training, scene 5 for validation/test
• Tasks: 2D detection, 3D detection, monocular center-point depth estimation (Tu et al., 26 Apr 2025)

This dataset enables efficient benchmarking of detection-driven depth estimation models across diverse distance and occlusion regimes.

5. Experimental Results and Comparative Evaluation

Extensive evaluation on VirDepth demonstrates substantial improvements in both depth accuracy and computational efficiency:

• Monocular Depth at Object Centers: CenterDepth outperforms global depth methods, particularly at large ranges and for small objects.
  • CenterDepth (Ours, HGL104 backbone): $(w, h)$0, RMSE = 2.106
  • DepthAnythingV2: $(w, h)$1, RMSE = 13.155
  • MonoDepth2: $(w, h)$2, RMSE = 16.802
• 3D Detection Depth (R101 backbone): $(w, h)$3, RMSE = 2.688 vs SMOKE ($(w, h)$4, RMSE = 9.324) and FCOS3D ($(w, h)$5, RMSE = 5.837).
• Distance-binned MAE (150–200 m): CenterDepth achieves MAE = 3.351 while global baselines exceed MAE = 36.
• Speed/Accuracy Efficiency:

| Backbone | Time (ms) | $(w, h)$6 | MAE | RMSE | |-------------|-----------|------------|-------|-------| | HGL104 | 15.6 | 0.989 | 1.394 | 2.106 | | ResNet-101 | 9.5 | 0.983 | 2.834 | 2.688 | | ResNet-18 | 3.4 | 0.909 | 3.415 | 6.436 |

• Far-Target Performance (150–200 m, mAP$(w, h)$7): CenterDepth (512×512) = 45.4% at 6.1 ms, YOLOv8s = 30.0% at 4.3 ms.

Qualitative analyses reveal substantial improvements in small-object recall at long distances and preserve structure critical for autonomous planning tasks via depth-augmented BEV representations (Tu et al., 26 Apr 2025).

6. Pseudocode and Training Workflow

Training and inference pipelines are structured as follows:

Training Loop:

$(w, h)$9

Inference:

$D_c$0

This workflow permits real-time operation and seamless deployment on automotive or robotic platforms (Tu et al., 26 Apr 2025).

7. Relation to Alternative Center-based Approaches

CenterDepth is situated within the broader family of center-based monocular 3D detection and depth estimation methods. Notably, Center3D (Tang et al., 2020) introduces a CenterDepth module pairing explicit 2D-3D center offsets with hybrid depth heads:

• Offset Regression: Learns to bridge the geometric gap between 2D bounding box centers and true projected 3D centers to align depth predictions with physical object locations. The offset $(w, h)$8 is regressed and applied before metric depth backprojection.
• Classification-Regression Fusion: Two depth heads—Linear Increasing Discretization (LID, classification-dominated) and DepJoint (regression-focused with Eigen scaling)—are fused to yield robust per-object depth estimation.
• Full Pipeline: Center3D’s CenterDepth module combines center localization, offset correction, and depth regression, yielding state-of-the-art tradeoffs in 3D AP and BEV AP on KITTI with low computational overhead (Tang et al., 2020).

A plausible implication is that detection-driven, point-based depth estimation offers inherent resilience for imbalanced object scales and long-range detection compared to dense, pixelwise, convolutional monocular depth regression. This is evidenced in scenarios such as path planning, where per-object center depths facilitate explicit BEV reconstructions for downstream tasks [(Tu et al., 26 Apr 2025), Fig. 7].

---

References:

• Depth as Points: Center Point-based Depth Estimation (Tu et al., 26 Apr 2025)
• Center3D: Center-based Monocular 3D Object Detection with Joint Depth Understanding (Tang et al., 2020)