Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation

Abstract: Self-Supervised Monocular Depth Estimation (MDE) has garnered attention in recent years due to its independence from ground truth. However, most existing models are limited to a single scale and exhibit considerable performance degradation in complex driving environments. Networks specifically designed to handle dynamic traffic participants tend to be overly complex, hindering their deployment on resource-constrained automotive edge devices. To address these limitations and move towards robust driving perception, we propose FlexDepth, a scale-driven and flexible family of self-supervised MDE models tailored for challenging road scenarios. FlexDepth employs a two-stage static-dynamic decoupled training strategy, enabling the independent assessment of confidence for both static backgrounds and dynamic road objects. Furthermore, it introduces a meticulously designed Scale-Driven Decoder (SDD) to dynamically select components based on scale size, facilitating efficient feature fusion and the output of high-precision depth maps. Extensive experiments on standard driving benchmarks demonstrate that without any auxiliary information, our model achieves state-of-the-art performance across arbitrary scales with minimal computational overhead. Our smallest model, Flex-Nano, requires only 0.7 GFLOPs and achieves 37.6 FPS on mobile platforms, ensuring reliable real-time perception while maintaining excellent zero-shot generalization.Our source code is avalible: https://github.com/startnew/flexdepth

• The paper introduces FlexDepth, a versatile architecture leveraging a Scale-Driven Decoder and dynamic masking for improved self-supervised monocular depth estimation in dynamic driving scenarios.
• It decomposes training into static and dynamic stages, enabling robust gradient suppression for moving objects without reliance on semantic labels.
• The approach demonstrates state-of-the-art efficiency and accuracy across benchmarks like KITTI, Cityscapes, and Make3D, making it deployable on diverse hardware.

FlexDepth: Scale-Driven Self-Supervised Monocular Depth Estimation for Robust Driving Perception

Introduction

The paper "Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation" (2607.00736) presents FlexDepth, a modular family of self-supervised monocular depth estimation (MDE) architectures tailored for real-world autonomous driving and robotic navigation scenarios. The critical innovations address two dominant issues: (i) robustness in dynamic environments with moving agents, and (ii) efficient, scalable inference suitable for deployment on edge hardware, without compromising generalization or accuracy.

Self-supervised monocular depth estimation, which avoids reliance on explicit ground-truth depth supervision, dominates practical applications but has faced two persistent bottlenecks. First, model design has traditionally focused on encoders, leaving decoders underexplored, often leading to suboptimal trade-offs between efficiency and accuracy. Second, robust performance in highly dynamic road scenes remains unsolved, as the implicit static-world assumptions of reprojection objectives are violated by moving objects. FlexDepth is engineered to systematically address both fronts.

Methodology

Scale-Driven Decoder Architecture

FlexDepth introduces the Scale-Driven Decoder (SDD), allowing models to adapt their decoder structure according to designated resource/accuracy constraints. The decoder's architecture is parameterized by three major components: post-convolution module $\mathcal{C}$, upsampling module $\mathcal{U}$, and prediction head $\mathcal{P}$.

• For lightweight variants ("Nano"/"Small"), High-Efficiency Bottleneck (HEB) modules are used with dynamic upsampling and a conventional prediction head.
• Larger variants ("Medium"/"Large"/"X-Large") employ High-Performance Bottleneck (HPB) modules with Cross Stage Partial (CSP) fusion, more aggressive dynamic upsampling, and an inverted prediction head (upsampling follows prediction), which increases representational richness for high-resolution features.

FlexDepth modifies the canonical U-Net fusion paradigm by removing pre-convolutions prior to scale restoration, thus maintaining global context before upsampling, which is critical for edge and structure preservation in lightweight models. The result is a scalable family (Nano—X-Large) that can be deployed on diverse hardware budgets while maintaining exceptional boundary accuracy and minimal computational overhead.

Figure 1: FlexDepth system overview, illustrating the two-stage static-dynamic training pipeline with modular decoder.

Two-Stage Static-Dynamic Decoupled Training

To address the static-scene violation induced by dynamic objects, FlexDepth decomposes the learning procedure into two stages:

1. Standard Self-Supervised Training: Trains the depth network and PoseNet jointly using photometric reprojection under the assumption of mostly static geometry.
2. Dynamic-Region Adaptive Refinement: After initial convergence, the model analyses temporal disparities between early and late-stage predictions to identify training-time instability, characteristic of dynamic regions. An adaptive mask is generated through a differentiable thresholding network that produces a scene-specific confidence threshold ($\mu$), which is applied to decouple static and dynamic areas.

This mask $M$ is then used to selectively apply loss terms during retraining, allowing the network to focus supervision on reliable regions and robustly suppress gradients from unpredictable, non-rigid or moving entities—without any reliance on semantic/flow labels or motion priors.

Figure 2: Adaptive Dynamic Masking network generates per-frame $\mu$ for dynamic masking, enhancing robustness under real-world dynamics.

Figure 3: Examples of dynamic mask generation showing scene-dependent $\mu$ threshold selection.

Training Objectives

FlexDepth leverages conventional photometric losses (with auto-masking for trivial identity projections) and introduces a masked normal distribution loss with dynamic uncertainty in the second training stage, as well as a geometric depth smoothing penalty that is spatially modulated by the adaptive mask.

Experimental Results

Efficiency/Accuracy Trade-off

Across all standard benchmarks (KITTI, Cityscapes, Make3D), the FlexDepth family achieves state-of-the-art (SOTA) accuracy at every measured complexity point. Notable results include:

• Flex-Nano: 0.7 GFLOPs, 37.6 FPS on Snapdragon 8 Elite, surpassing prior Tiny/Small models in accuracy at less than 20% of the compute.
• Flex-X-Large: 24.6 GFLOPs delivers SOTA accuracy across dynamic and static region metrics with a 44% reduction in compute versus transformer-based baselines such as MonoViT.

Figure 4: Accuracy vs. cost comparison on KITTI and representative visual outcomes on Cityscapes showing fine boundary and dynamic region fidelity.

Robustness in Dynamic Environments

Decoupled dynamic-region evaluation shows that FlexDepth outperforms all published self-supervised and pseudo-label-augmented SOTA models on dynamic segments, with substantial gains in Abs Rel and $\delta < 1.25$ for moving objects (see main/appendix tables).

Figure 5: Qualitative evaluation on Cityscapes; error maps highlight robust predictions on moving agents in complex scenes.

Figure 6: KITTI comparison: FlexDepth demonstrates edge preservation and reduced failure in areas with significant motion compared to prior SOTA.

Generalization and Foundation Model Comparison

FlexDepth exhibits robust zero-shot transfer to out-of-domain datasets, e.g., Make3D, Cityscapes, DDAD, consistently outperforming prior SOTA by a wide margin on cross-domain metrics (see Appendix). A head-to-head comparison—under foundation model protocols—with Depth Anything V2 (DA2) shows that domain-adapted FlexDepth can achieve lower Abs Rel with an order-of-magnitude fewer parameters/FLOPs than DA2, and with superior far-range object completeness.

Analysis and Visualization

Qualitative analyses highlight that FlexDepth recovers thin, distant objects (e.g., tree trunks, traffic signs, pedestrians) lost by large foundation/transformer models in zero-shot mode. The dynamic masking strategy further allows the architecture to deliver consistent predictions in highly dynamic traffic scenes devoid of strong motion cues.

Figure 7: KITTI scenes; depth maps from FlexDepth with sharp structural boundaries and excellent depth gradients.

Figure 8: Mid/long-range vehicle detection; FlexDepth accurately recovers dynamic agents lost by prior SOTA in sparse-pixel regions.

Figure 9: Occlusion-heavy tailgating: dynamic adaptive masking eliminates motion artifacts, ensuring artifact-free close-range prediction.

Figure 10: Pedestrian and cyclist handling in dynamic scenes, surpassing both temporal and single-frame baselines in error reduction.

Implications and Future Directions

Practical Implications: FlexDepth enables deployment of robust monocular depth perception on the full spectrum of automotive/mobile hardware, providing a principled path for industry transition from heavy supervised/foundation paradigms to efficient, data-adaptive, self-supervised learning. Its architecture allows low-latency, high-fidelity inference that is robust to the non-stationarity and unpredictability of real-world environments.

Theoretical Implications: The modular SDD and adaptive dynamic masking paradigm demonstrate that architectural choices in decoders and training pipelines are at least as critical as backbone scaling. The successes in decoupling dynamic supervision point towards the value of training-time instability as an unsupervised source of supervisory signal—a principle potentially applicable beyond MDE to temporal semantic segmentation or ego-motion estimation.

Future Directions: Key open research avenues include:

• Integration of spatial-temporal decoupling for improved handling of extended occlusions and non-Lambertian effects.
• Extension to indoor and highly irregular scenes, currently a known limitation due to the emphasis on outdoor driving domain.
• Unified multi-task decoders leveraging the SDD principle across semantic, instance, and flow domains.
• Hybridization with lightweight foundation models, leveraging large-scale priors without sacrificing adaptation to domain-specific distributions.

Conclusion

FlexDepth represents a comprehensive advancement in scalable, robust, and efficient self-supervised monocular depth estimation. Its scale-driven decoder and two-stage static-dynamic training pipeline push the limits of both computational efficiency and real-world adaptability, setting a new standard for deployment-ready MDE architectures in dynamic environments and across arbitrary hardware budgets. The paradigm offers a complementary path to foundation models, demonstrating that with strategic architectural and training innovations, self-supervision remains highly competitive even in the foundation model era.