---
title: 'FlexDepth: Adaptive Monocular Depth Models'
url: https://www.emergentmind.com/topics/flexdepth
type: topic
---

# FlexDepth: Adaptive Monocular Depth Models

FlexDepth is a scale-driven and flexible family of self-supervised monocular depth estimation models tailored for challenging road scenarios. In its explicit formulation, it addresses two recurrent weaknesses of self-supervised driving depth systems: fixed decoder designs that do not scale well across model sizes, and the failure of static-scene photometric assumptions in the presence of dynamic traffic participants. The framework combines a two-stage static-dynamic decoupled training strategy with a Scale-Driven Decoder (SDD), and is instantiated as a model family ranging from Flex-Nano to Flex-X-Large [2607.00736]. A broader methodological reading is also suggested by adjacent work: “FlexDepth” can denote a class of depth systems that adapt depth representations across scales, tasks, domains, modalities, and temporal regimes, rather than treating depth as a single fixed output format.

## 1. Definition and problem setting

FlexDepth addresses self-supervised monocular depth estimation from driving videos. The training signal comes from unlabeled monocular sequences: given a target frame \(I_t\), the network predicts a dense depth or disparity map \(D_t\) or \(\text{disp}_t\), and supervision is induced by view synthesis and photometric consistency with adjacent frames. The design is explicitly motivated by two limitations of prior self-supervised MDE systems. First, many methods use a fixed encoder-decoder pipeline with hand-designed upsampling that compresses already low-resolution deep features before spatial recovery, which degrades high-level cues and harms boundary quality, especially in lightweight models. Second, the rigid-scene assumption behind photometric reprojection is violated by moving vehicles, cyclists, pedestrians, occlusions, and disocclusions, while many dynamic-scene remedies rely on semantics, optical flow, pseudo-labels, or heavy multi-frame pipelines that are difficult to deploy on edge hardware [2607.00736].

The framework defines a model family—Flex-Nano, Flex-Small, Flex-Medium, Flex-Large, and Flex-X-Large—built around three components: a YOLOv11-derived multi-scale encoder, a scale-aware decoder that changes with model size, and a two-stage training procedure that first learns standard self-supervised depth and then re-trains the depth network with an adaptive static-dynamic mask. The same design is intended to cover multiple accuracy-efficiency operating points, rather than optimizing a single architecture for a single budget [2607.00736].

## 2. Self-supervised formulation and static-dynamic decoupling

In Stage 1, FlexDepth jointly trains a depth network and a PoseNet. PoseNet uses a ResNet-18 encoder with a 4-layer decoder to estimate the relative pose \(P_{t\to s}\) between the target frame \(I_t\) and source frames \(I_s\). The target image is reconstructed by differentiable warping,
$$
\hat{I}_t = I_s\big\langle \text{proj}(\hat{D}_t, P_{t\to s}, K)\big\rangle,
$$
where \(K\) is the camera intrinsic matrix. The photometric loss is
$$
L_p = \alpha \frac{1 - \text{SSIM}(\hat{I}_t, I_t)}{2} + (1-\alpha)\|\hat{I}_t - I_t\|_1,
$$
with \(\alpha = 0.85\). For multiple source views, FlexDepth uses minimum reprojection with auto-masking,
$$
L_r = \mu \cdot \min_s L_p(\hat{I}_t, I_t),
$$
and combines it with edge-aware smoothness on mean-normalized inverse depth,
$$
L_{\text{smooth}} = |\partial_x d_t^*| e^{-|\partial_x I_t|} + |\partial_y d_t^*| e^{-|\partial_y I_t|}.
$$
The Stage-1 multi-scale objective is
$$
L_{\text{raw}} = \frac{1}{4} \sum_{s} \big( L_r + \lambda L_{\text{smooth}} \big), \quad \lambda = 0.001.
$$
After Stage 1, early-stage and late-stage depth snapshots are retained, producing \(\text{disp}_i\) and \(\text{disp}_l\) [2607.00736].

The central observation behind Stage 2 is that dynamic regions are less stable across epochs than static background regions. FlexDepth therefore defines a static-dynamic decoupled mask
$$
M = \big[\,|\text{disp}_i^\beta - \text{disp}_l^\beta| < g(\phi(f_{s_i}, f_{s_l}))\,\big],
$$
with \(\beta = 0.7\). Here \(f_{s_i}\) and \(f_{s_l}\) are intermediate feature maps from early and late stages, \(\phi(f_{s_i}, f_{s_l})\) is a discrepancy representation, and \(g(\cdot)\) is a small CNN that predicts an image-specific threshold. Stage 2 freezes PoseNet and re-trains only the depth network using a masked probabilistic loss
$$
L_m = M \cdot \left[ \log(\sigma + 1) + \frac{\lambda L_r^2}{2\sigma^2} \right],
$$
with \(\lambda = 0.5\), together with anisotropic geometric depth smoothing,
$$
L_{\text{GDS}} = |\partial_x \hat{d}_t| e^{-|\partial_x I_t|} + [\eta (1 - M) + M]\, |\partial_y \hat{d}_t| e^{-|\partial_y I_t|},
$$
where \(\eta = 100\). The Stage-2 loss is
$$
L_{\text{total}} = \frac{1}{HW} \sum_t \big( L_m + \gamma L_{\text{GDS}} \big).
$$

This decoupling is empirically nontrivial. On KITTI, Flex-X-Large without Stage 2 reports AbsRel \(0.100\), SqRel \(0.729\), RMSE \(4.431\), and \(\delta_1=0.900\); with Stage 2 and the adaptive mask, these become AbsRel \(0.093\), SqRel \(0.605\), RMSE \(4.114\), and \(\delta_1=0.910\). In dynamic-region analysis, it reports moving-object AbsRel \(0.159\) versus \(0.177\) for DSI-MonoViT, and moving-object SqRel \(1.484\) versus \(1.918\) [2607.00736].

## 3. Scale-Driven Decoder

The architectural core of FlexDepth is the Scale-Driven Decoder. At level \(i\), the decoder is written as
$$
D_i = \mathcal{P} \left\{ \mathcal{C}\left( \text{Concat}\big[\mathcal{U}(\mathbf{F}_{i+1}), \mathbf{F}_{enc,i}\big] \right) \right\},
$$
where \(\mathbf{F}_{enc,i}\) is the encoder feature, \(\mathbf{F}_{i+1}\) is the coarser decoder feature, \(\mathcal{U}\) is the upsampling operator, \(\mathcal{C}\) is the post-convolution module, and \(\mathcal{P}\) is the prediction head. A defining design choice is the removal of the pre-convolution stage found in many U-Net-like decoders, on the grounds that deep features are already information-sparse and should not be compressed before spatial recovery [2607.00736].

For small models, \(\mathcal{C}\) is instantiated as the High-Efficiency Bottleneck (HEB). Given \(X\in\mathbb{R}^{C\times H \times W}\), a convolution and channel split produce
$$
[f_a, f_b] = \mathcal{S}(\text{Conv}(X)),
$$
followed by a dense sequence
$$
F_1 = B_1(f_a),\ F_2 = B_2(F_1),\dots, F_n = B_n(F_{n-1}),
$$
and an output fusion
$$
X_{\text{out}} = \text{Conv}(\text{Concat}[f_a, f_b, F_1,\dots,F_n]).
$$
For larger models, \(\mathcal{C}\) becomes the High-Performance Bottleneck (HPB), using a CSP-style structure,
$$
B^k(x) = \text{Conv}\Big( \text{Concat}\big[ \prod_{j=1}^n B_j'( \text{Conv}(x)),\ \text{Conv}(x) \big] \Big).
$$
The decoder therefore changes with scale: HEB emphasizes optimization and dense gradient flow in low-FLOP regimes, whereas HPB prioritizes richer local feature modeling in higher-capacity models [2607.00736].

Upsampling is likewise content-aware. Rather than fixed bilinear interpolation, FlexDepth predicts sampling offsets and applies GridSample after PixelShuffle:
$$
\mathcal{U}(X) = \text{GS}\left(X,\, \text{PS}\left( \frac{2\cdot(\mathbf{C} + \Delta \mathbf{P})}{\text{Norm} - 1}, s \right)\right),
$$
where \(\mathbf{C}\) is the regular sampling grid, \(\Delta \mathbf{P}\) are learned offsets, and \(s\) is the upsampling scale. The prediction head is also scale-dependent: smaller models use a conventional “predict then upsample” head, while larger ones use an inverted head that upsamples features first and predicts depth afterward. This scale-aware decoder is not confined to the FlexDepth family; when plugged into MonoViT, the reported configuration reduces computation from \(59.7\) GFLOPs to \(33.2\) GFLOPs and improves KITTI AbsRel from \(0.099\) to \(0.095\) [2607.00736].

## 4. Model family, efficiency, and empirical performance

FlexDepth is explicitly organized as a deployment-oriented family rather than a single network.

| Variant | GFLOPs | Representative figures |
|---|---:|---|
| Flex-Nano | 0.718 | 1.52 M parameters; 37.6 FPS on Snapdragon 8 Elite |
| Flex-Small | 2.78 | 6.06 M parameters; 18.6 FPS on Snapdragon 8 Elite |
| Flex-Medium | 9.99 | 12.7 M parameters; 5.8 FPS on Snapdragon 8 Elite |
| Flex-Large | 11.52 | 15.2 M parameters |
| Flex-X-Large | 24.56 | 32.3 M parameters; 3.0 FPS on Snapdragon 8 Elite |

On KITTI, Flex-Nano reports AbsRel \(0.110\) at \(0.7\) GFLOPs, Flex-Small reports AbsRel \(0.104\), SqRel \(0.713\), RMSE \(4.458\), and \(\delta_1=0.890\) at \(2.8\) GFLOPs, Flex-Medium reports AbsRel \(0.096\), and Flex-Large reports AbsRel \(0.095\). Flex-X-Large reports AbsRel \(0.093\), SqRel \(0.605\), RMSE \(4.114\), and \(\delta_1=0.910\), outperforming reported self-supervised baselines such as MonoViT, FGTO, DSI-MonoViT, and Self-Distillation while using substantially fewer FLOPs. On Cityscapes, Flex-X-Large reports AbsRel \(0.086\), SqRel \(0.877\), RMSE \(5.268\), and \(\delta_1=0.926\), again surpassing prior dynamic-scene methods such as ProDepth, FGTO, and DSI-MonoViT at lower computational cost [2607.00736].

The family also emphasizes zero-shot behavior. On Make3D, KITTI-trained Flex-X-Large reports AbsRel \(0.279\), RMSE \(6.599\), and RMSElog \(0.144\), better than the self-supervised baselines listed in the comparison. Under fair LS alignment and improved KITTI ground truth, Flex-X-Large at \(640\times192\) reports AbsRel \(0.063\), \(\delta_1=0.952\), \(32\) M parameters, and \(25\) GFLOPs, compared with Depth Anything v2 ViT-L at \(1722\times518\), which reports AbsRel \(0.070\), \(\delta_1=0.956\), \(335\) M parameters, and \(1947\) GFLOPs. This comparison suggests that in-domain self-supervision can rival or exceed zero-shot foundation depth models at much lower cost in the driving regime [2607.00736].

## 5. Broader “FlexDepth” interpretations in depth research

A broader interpretation of FlexDepth is suggested by several adjacent systems: depth is treated as a representation that should be adaptable across domains, tasks, temporal horizons, resolutions, and conditioning signals, rather than as a fixed dense map predicted in one shot.

| Work | Setting | Flexibility axis |
|---|---|---|
| PopNet [2212.05370] | Source-free depth for object pop-out and segmentation | Frozen depth model adapted to SOD/COD via OPN and contact surface |
| PDDM [2405.09342] | Image-guided depth completion | Scene-adaptive depth bins, progressive decoupling, multi-scale supervision |
| FlashDepth [2504.07093] | Streaming video depth estimation | 2K, 24 FPS, high/low-resolution hybrid with temporal alignment |
| Amodal Depth Anything [2412.02336] | Object-conditioned amodal relative depth | Occluded geometry from \(I_o, D_o, M_a\) |
| DD3Dv2 [2210.02493] | Monocular 3D detection | In-domain depth adaptation using LiDAR or RGB videos |
| Depth Any Video [2410.10815] | Variable-length video depth estimation | Mixed-duration training and keyframe interpolation up to 150 frames |

In "Source-free Depth for Object Pop-out" [2212.05370], a frozen DPT+Boosting depth network is adapted to salient and camouflaged object detection through an Object Popping Network and a learned contact surface \(D_c\), enabling a depth-based pseudo-mask \(S_s = \sigma(\sigma_o (D_{po}-D_c))\). In "Progressive Depth Decoupling and Modulating for Flexible Depth Completion" [2405.09342], flexibility is identified with scene-adaptive discretization: the Bins Initializing Module derives seed bins from sparse depth, an incremental depth decoupling branch refines them from global to local, and a modulating branch refines probability maps from coarse to fine; the reported ablation shows PDDM improving from UD \(0.016/0.106\) and SID \(0.015/0.105\) to AbsRel \(0.014\) and RMSE \(0.095\).

In video, "FlashDepth" [2504.07093] converts a pretrained single-image model into a streaming 2K depth system, running at \(2044\times1148\) and \(24\) FPS by inserting a small Mamba temporal module into decoder features and pairing a high-resolution small model with a low-resolution large model via cross-attention. "Depth Any Video with Scalable Synthetic Data" [2410.10815] extends flexibility differently: it builds a synthetic dataset of \(40{,}000\) video clips and about \(6\)M frames, adopts mixed-duration training with frame dropout and RoPE, and uses keyframe-based interpolation to handle videos up to about \(150\) frames. "Amodal Depth Anything" [2412.02336] defines amodal depth estimation as object-conditioned prediction of invisible geometry from \((I_o, D_o, M_a)\), introduces the \(564\)K-image ADIW dataset, and reports a \(69.5\%\) improvement in accuracy over the previous SoTA on ADIW. "Depth Is All You Need for Monocular 3D Detection" [2210.02493] treats depth as the central transferable representation for 3D detection and shows DD3Dv2 improving nuScenes test performance from mAP \(41.8\), NDS \(0.48\) to mAP \(46.1\), NDS \(0.51\) when depth is adapted in-domain with LiDAR supervision.

Taken together, these systems suggest that “FlexDepth” is not only a model family name but also a design stance: depth models should be target-adaptive, modular, and able to bend toward different downstream constraints without discarding pretrained geometric knowledge.

## 6. Limitations and research directions

As a concrete self-supervised driving system, FlexDepth remains driving-centric. Its reported limitations include limited applicability to indoor scenes with irregular textures, clutter, and close-range occlusions; no explicit treatment of severe weather, night-time, lens dirt, or sensor noise; an assumption of fixed camera intrinsics and reasonably calibrated pinhole geometry; and the standard scale ambiguity of self-supervised monocular depth, which still requires alignment for metric evaluation [2607.00736].

The broader literature suggests several plausible extensions. One direction is stronger temporal and streaming support: FlashDepth emphasizes recurrent temporal alignment at real-time 2K resolution, while Depth Any Video emphasizes mixed-duration training and long-sequence interpolation [2504.07093][2410.10815]. Another is extending beyond visible surfaces: Amodal Depth Anything frames occluded geometry as a relative-depth problem conditioned on amodal masks, and PopNet frames foreground extraction as reasoning over a contact surface in 3D [2412.02336][2212.05370]. A third is deeper task adaptation: DD3Dv2 uses depth as an in-domain auxiliary task for monocular 3D detection, and PDDM uses scene-adaptive discretization to tailor depth completion to the observed depth distribution [2210.02493][2405.09342]. A plausible implication is that future FlexDepth systems will combine scale-aware self-supervised training with temporal adaptation, amodal conditioning, and scene-adaptive depth representations, while preserving the deployment discipline that motivated the original FlexDepth family.

Source: https://www.emergentmind.com/topics/flexdepth