Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving (2203.02112v1)

Abstract: Pseudo-LiDAR 3D detectors have made remarkable progress in monocular 3D detection by enhancing the capability of perceiving depth with depth estimation networks, and using LiDAR-based 3D detection architectures. The advanced stereo 3D detectors can also accurately localize 3D objects. The gap in image-to-image generation for stereo views is much smaller than that in image-to-LiDAR generation. Motivated by this, we propose a Pseudo-Stereo 3D detection framework with three novel virtual view generation methods, including image-level generation, feature-level generation, and feature-clone, for detecting 3D objects from a single image. Our analysis of depth-aware learning shows that the depth loss is effective in only feature-level virtual view generation and the estimated depth map is effective in both image-level and feature-level in our framework. We propose a disparity-wise dynamic convolution with dynamic kernels sampled from the disparity feature map to filter the features adaptively from a single image for generating virtual image features, which eases the feature degradation caused by the depth estimation errors. Till submission (November 18, 2021), our Pseudo-Stereo 3D detection framework ranks 1st on car, pedestrian, and cyclist among the monocular 3D detectors with publications on the KITTI-3D benchmark. The code is released at https://github.com/revisitq/Pseudo-Stereo-3D.

• The paper introduces a novel Pseudo-Stereo framework that transforms monocular images into virtual stereo pairs to enhance 3D detection accuracy in autonomous driving.
• It employs a three-tiered process—image-level, feature-level, and feature cloning—to overcome the depth estimation challenges of traditional pseudo-LiDAR methods.
• Experiments on the KITTI-3D benchmark show significant improvements in Average Precision for cars, pedestrians, and cyclists, validating its robust performance.

Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving

The paper "Pseudo-Stereo for Monocular 3D Object Detection in Autonomous Driving" introduces a novel framework for detecting 3D objects using monocular images specifically aimed at enhancing autonomous driving applications. This research leverages the advantages of stereo depth perception and modifies existing strategies to develop a Pseudo-Stereo 3D detection framework. The three-tiered virtual view generation, encompassing image-level generation, feature-level generation, and feature cloning, distinguishes this work and represents an advancement over prior monocular detection methods.

Technical Summary

The Pseudo-Stereo framework is motivated by the need to bridge the gap between monocular and stereo imaging for 3D perception. Traditional Pseudo-LiDAR approaches convert monocular images to per-point depth estimations, which are then fed into LiDAR-based architectures. However, these techniques face challenges in accurately representing depth due to the substantial gap in image-to-LiDAR generation. The Pseudo-Stereo method circumvents this limitation by focusing on image-to-image transformations with reduced modality conversion loss.

1. Image-Level Generation: This method transforms the monocular image into a stereo pair by generating a virtual stereo image using estimated disparity maps derived from depth mapping. It involves warping techniques that are computationally intensive.
2. Feature-Level Generation: This approach generates a virtual stereo feature directly from the image features and disparity features, using a disparity-wise dynamic convolution that filters the left image features. This convolution uses dynamic kernels sampled from disparity feature maps, intended to adaptively filter out features to foster better virtual feature generation.
3. Feature Cloning: The simplest approach serves as a baseline wherein the left features are directly cloned as the right stereo features. This method, despite being simplistic, doesn't require additional depth estimation steps, thus enhancing generalization capability.

The investigation into depth-aware learning revealed that while estimated depth maps could enhance detection when used at both image and feature levels, the depth loss effectively supported feature-level generation only. The framework’s efficacy is reflected in its first-place ranking on the KITTI-3D benchmark for simultaneous detection of cars, pedestrians, and cyclists.

Results and Implications

The paper quantitatively demonstrates the performance improvements of the Pseudo-Stereo framework. Through experiments conducted on the KITTI-3D dataset, the feature-level generation method notably surpasses other monocular 3D detection methods, including DD3D and MonoFlex, across various object classes. The results indicate significant improvements in Average Precision (AP) metrics, especially in $AP_{3D}$ and $AP_{BEV}$, showcasing its robustness and applicability in real-world scenarios.

Additionally, feature-level generation is highlighted for its computational efficiency and adaptive learning capabilities, marking a significant methodological departure from the manual depth alignment used in image-level generation. The disparity-wise dynamic convolution notably contributes to mitigating feature degradation resulting from depth estimation errors, thus improving overall monocular 3D detection performance.

Future Directions

This research opens several avenues for future development in monocular 3D object detection systems. The findings suggest that further examination of virtual view generation methods can enhance the depth-aware learning processes inherent in this framework. Additionally, adapting Pseudo-Stereo generation techniques to different neural architectures or different sensor modalities could yield further improvements in accuracy and computational efficiency. In the context of autonomous driving, integrating these methods with other environmental perception systems could dramatically increase the capability and reliability of autonomous navigation algorthims.

In conclusion, the Pseudo-Stereo framework presents a compelling, technically sophisticated approach to overcoming the limitations of monocular 3D detection by harnessing stereo-like depth perception. This research significantly impacts the domain of autonomous driving and 3D computer vision, paving the way for more accurate and cost-effective solutions.