PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes (2210.01612v3)

Abstract: Multiple near frontal-parallel planes based depth representation demonstrated impressive results in self-supervised monocular depth estimation (MDE). Whereas, such a representation would cause the discontinuity of the ground as it is perpendicular to the frontal-parallel planes, which is detrimental to the identification of drivable space in autonomous driving. In this paper, we propose the PlaneDepth, a novel orthogonal planes based presentation, including vertical planes and ground planes. PlaneDepth estimates the depth distribution using a Laplacian Mixture Model based on orthogonal planes for an input image. These planes are used to synthesize a reference view to provide the self-supervision signal. Further, we find that the widely used resizing and cropping data augmentation breaks the orthogonality assumptions, leading to inferior plane predictions. We address this problem by explicitly constructing the resizing cropping transformation to rectify the predefined planes and predicted camera pose. Moreover, we propose an augmented self-distillation loss supervised with a bilateral occlusion mask to boost the robustness of orthogonal planes representation for occlusions. Thanks to our orthogonal planes representation, we can extract the ground plane in an unsupervised manner, which is important for autonomous driving. Extensive experiments on the KITTI dataset demonstrate the effectiveness and efficiency of our method. The code is available at https://github.com/svip-lab/PlaneDepth.

• The paper introduces PlaneDepth, a self-supervised monocular depth estimation method using orthogonal vertical and ground planes, which improves accuracy compared to traditional frontal-parallel methods.
• Key technical innovations include a Laplacian Mixture Model for depth distribution, a novel data augmentation preserving plane orthogonality, and an augmented self-distillation loss for handling occlusions.
• Evaluations on the KITTI dataset demonstrate PlaneDepth's superior performance, yielding smoother ground depth and detailed object edges, which is crucial for applications like autonomous navigation.

Self-supervised Depth Estimation with PlaneDepth: A Detailed Analysis

The paper "PlaneDepth: Self-supervised Depth Estimation via Orthogonal Planes" introduces a novel methodology for monocular depth estimation (MDE) that leverages orthogonal planes to enhance depth representation. Traditional approaches to MDE have predominantly utilized frontal-parallel planes for depth representation, which tend to produce inaccuracies when modeling the ground plane due to their perpendicular positioning relative to frontal-parallel planes. In contrast, the PlaneDepth framework proposes using a set of orthogonal planes—comprising vertical and ground planes—for more precise depth estimation.

Methodology and Innovations

PlaneDepth introduces key technical advancements to the self-supervised training of MDE:

1. Orthogonal Plane Representation:
   • The proposed method uses orthogonal planes to more effectively capture the depth and geometry of both vertical and ground planes compared to standard frontal-parallel approaches.
   • This orthogonal configuration allows for the unsupervised extraction of ground planes, which is crucial in applications like autonomous driving.
2. Laplacian Mixture Model:
   • Depth distribution is estimated using a Laplacian Mixture Model centered on the orthogonal planes. This probabilistic model refines the depth computation by leveraging Laplacian distributions, resulting in a more deterministic optimization process and improved depth accuracy.
3. Data Augmentation and Transformation:
   • A novel resizing and cropping transformation is developed to maintain the orthogonality of the predefined planes, mitigating the distortion effects that traditional augmentations can introduce.
   • Neural Positional Encoding (NPE) is incorporated to enhance the network's robustness to these transformations.
4. Augmented Self-distillation Loss:
   • An adaptive self-distillation loss is incorporated, using a bilateral occlusion mask to address depth estimation in occluded regions.
   • This approach fine-tunes the model to achieve robustness against occlusions, yielding more accurate depth predictions.

Evaluation and Results

The methodology is rigorously evaluated on the KITTI dataset, a standard benchmark for MDE tasks. The results underscore the efficacy of the PlaneDepth model in producing smoother ground depth predictions and finely detailed object edges, achieving superior performance metrics compared to existing methods.

Implications and Future Directions

The implications of PlaneDepth span both theoretical and practical domains. Theoretically, this work advances the understanding of plane-based depth estimation by demonstrating the impact of orthogonal configurations on depth accuracy. Practically, the ability to accurately model ground planes has direct applications in autonomous navigation and robotic vision systems, which require precise environmental awareness.

The continued development of self-supervised frameworks, such as PlaneDepth, reflects a broader trend in AI towards reducing reliance on labeled datasets, which are costly and time-consuming to produce. Future work could explore further enhancements in plane geometry representations and adaptive strategies for dynamic environments, potentially integrating additional sensory inputs beyond monocular vision.

In summary, PlaneDepth advances the state-of-the-art in self-supervised MDE by proposing a thoughtful redesign of the depth representation paradigm, effectively addressing longstanding limitations of traditional methods and opening avenues for further innovation in vision-based AI systems.