DepthMaster: Taming Diffusion Models for Monocular Depth Estimation (2501.02576v1)

Abstract: Monocular depth estimation within the diffusion-denoising paradigm demonstrates impressive generalization ability but suffers from low inference speed. Recent methods adopt a single-step deterministic paradigm to improve inference efficiency while maintaining comparable performance. However, they overlook the gap between generative and discriminative features, leading to suboptimal results. In this work, we propose DepthMaster, a single-step diffusion model designed to adapt generative features for the discriminative depth estimation task. First, to mitigate overfitting to texture details introduced by generative features, we propose a Feature Alignment module, which incorporates high-quality semantic features to enhance the denoising network's representation capability. Second, to address the lack of fine-grained details in the single-step deterministic framework, we propose a Fourier Enhancement module to adaptively balance low-frequency structure and high-frequency details. We adopt a two-stage training strategy to fully leverage the potential of the two modules. In the first stage, we focus on learning the global scene structure with the Feature Alignment module, while in the second stage, we exploit the Fourier Enhancement module to improve the visual quality. Through these efforts, our model achieves state-of-the-art performance in terms of generalization and detail preservation, outperforming other diffusion-based methods across various datasets. Our project page can be found at https://indu1ge.github.io/DepthMaster_page.

• The paper introduces DepthMaster, a model that pioneers a single-step deterministic diffusion approach for efficient and precise monocular depth estimation.
• It employs a feature alignment module to integrate external semantic features, reducing texture overfitting and enhancing scene representation.
• The paper presents a Fourier enhancement module and a two-stage training strategy that balance global structure with fine details, outperforming prior diffusion-based models.

"DepthMaster: Taming Diffusion Models for Monocular Depth Estimation" is an innovative approach that leverages diffusion models to achieve state-of-the-art performance in monocular depth estimation. The paper introduces a model called DepthMaster, which utilizes a single-step deterministic paradigm to enhance inference efficiency while maintaining high performance. The key advancements in this work focus on bridging the generative features from diffusion models with the discriminative task of depth estimation.

Key Contributions

1. DepthMaster Model Design:
   • Single-Step Deterministic Paradigm: Unlike traditional diffusion models that rely on iterative denoising processes, which can be slow, DepthMaster employs a single-step approach. This significantly enhances inference speed without compromising on accuracy. The model directly converts an image from the latent space to a depth map using a simplified process, improving computational efficiency.
2. Feature Alignment Module:
   • This module addresses the challenge of diffusion models overemphasizing texture details, which can lead to unrealistic textures in depth predictions. The feature alignment module incorporates high-quality external representations (e.g., from DINOv2) to align the feature distributions of the diffusion model with those of an external encoder. This integration of semantic information enhances the model's ability to represent scenes accurately by reducing overfitting to irrelevant textures.
3. Fourier Enhancement Module:
   • To overcome the limitations of the single-step paradigm in capturing fine-grained details, DepthMaster introduces a Fourier Enhancement Module. This module operates in the frequency domain and provides a balance between low-frequency structure capture and high-frequency detail enhancement, effectively simulating the iterative nature of traditional diffusion processes.
4. Two-Stage Training Strategy:
   • DepthMaster adopts a two-stage training strategy to fully leverage the generative capabilities of its architecture. The first stage focuses on learning the scene structure using latent-space supervision and the feature alignment module. In the second stage, the model is fine-tuned on pixel-level details, utilizing the Fourier enhancement module and a weighted multi-directional gradient loss to ensure sharp and detailed depth predictions.

Experimental Validation

• Performance: DepthMaster achieves exceptional generalization and detail preservation across various datasets, surpassing other diffusion-based methods and performing competitively compared to data-driven approaches.
• Efficiency: The deterministic approach allows to shift the denoising models' focus from unnecessary texture detail to structural detail, which is essential for accurate depth estimation.

Experimental Results

The DepthMaster approach was evaluated across various datasets, including KITTI, NYUv2, ETH3D, ScanNet, and DIODE, against both data-driven and other diffusion-based models. Remarkably, DepthMaster achieved top ranks in terms of average performance, demonstrating that the model successfully bridges the gap between data-driven and diffusion-based depth estimation approaches. It excelled in situations requiring generalization across diverse datasets and showed impressive detail-preserving capabilities, crucial for accurately capturing the true scene geometry. DepthMaster outperformed other diffusion-based methods and even some data-driven models, confirming its robustness and efficiency for zero-shot monocular depth estimation.

conclusive Performance and Efficiency

In summary, DepthMaster provides a robust and computationally efficient solution for zero-shot monocular depth estimation. Its novel integration of generative diffusion features into a discriminative task, along with its ability to preserve fine-grained details, highlights the potential of diffusion models when suitably adapted for tasks outside their traditional domains of use. The inclusion of both feature alignment and frequency domain processing stands out as a comprehensive approach to enhancing depth estimation quality, while its single-step paradigm offers a significant improvement in processing speed.