RRPN: Radar Region Proposal Network for Object Detection in Autonomous Vehicles (1905.00526v2)

Abstract: Region proposal algorithms play an important role in most state-of-the-art two-stage object detection networks by hypothesizing object locations in the image. Nonetheless, region proposal algorithms are known to be the bottleneck in most two-stage object detection networks, increasing the processing time for each image and resulting in slow networks not suitable for real-time applications such as autonomous driving vehicles. In this paper we introduce RRPN, a Radar-based real-time region proposal algorithm for object detection in autonomous driving vehicles. RRPN generates object proposals by mapping Radar detections to the image coordinate system and generating pre-defined anchor boxes for each mapped Radar detection point. These anchor boxes are then transformed and scaled based on the object's distance from the vehicle, to provide more accurate proposals for the detected objects. We evaluate our method on the newly released NuScenes dataset [1] using the Fast R-CNN object detection network [2]. Compared to the Selective Search object proposal algorithm [3], our model operates more than 100x faster while at the same time achieves higher detection precision and recall. Code has been made publicly available at https://github.com/mrnabati/RRPN .

RRPN: A Novel Approach for Real-Time Object Detection in Autonomous Vehicles

The paper, authored by Ramin Nabati and Hairong Qi, presents an innovative approach to object detection in autonomous vehicles through the introduction of Radar Region Proposal Network (RRPN). The primary objective of RRPN is to enhance the real-time performance of two-stage object detection systems by leveraging radar data, thus mitigating the inherent latency typically introduced by region proposal algorithms in traditional CNN-based systems.

Overview

RRPN stands as a significant contribution to the domain of autonomous driving, where rapid and accurate perception is paramount. The proposed system specifically targets the reduction of computational bottleneck introduced by traditional vision-based region proposal mechanisms, offering a radar-based alternative that operates substantially faster — reportedly over 100 times faster than conventional Selective Search algorithms.

Methodology

The paper outlines the RRPN framework through several integral components:

1. Perspective Transformation: This step involves mapping radar detections from the vehicle's coordinates to the camera's image coordinates. The transformation allows for the integration of radar data with image-based perception, which is crucial for precise localization of objects.
2. Anchor Generation: RRPN improves upon anchor-based region proposals by generating multiple bounding boxes with varying sizes and aspect ratios for each radar detection. Notably, it addresses the potential misalignment of radar detections by generating translated anchors.
3. Distance Compensation: Objects are scaled in relation to their distance from the vehicle using a formula that integrates radar-provided range information. This allows for more accurate estimations of object sizes in images, crucial for effective bounding box generation.

Results and Performance

The authors implement RRPN in conjunction with a Fast R-CNN object detection network, utilizing two backbone configurations: ResNet-101 and ResNeXt-101. Results are benchmarked on the NuScenes dataset, which provides a comprehensive set of synchronized radar and camera data, including challenging driving scenarios. RRPN demonstrates superior mean Average Precision (AP) and mean Average Recall (AR) compared to Selective Search, alongside remarkable computational efficiency, generating proposals in a fraction of the time required by traditional methods.

The paper provides detailed per-class analysis, revealing substantial improvements in detection precision across various object classes, particularly for persons, motorcycles, and bicycles, suggesting improved robustness in complex scenarios.

Implications and Future Directions

RRPN's contribution to autonomous vehicle perception lies in its dual role as both a region proposal network and an implicit sensor fusion method. The integration of radar data enhances the system's attention mechanism, improving focus on objects that are critical for safety and navigation, such as vehicles and pedestrians on the road.

The paper opens pathways for further research into sensor fusion methods that leverage radar, LiDAR, and camera data. Future work could explore extending the RRPN framework to three-dimensional object detection or integrating advanced radar signal processing techniques to enhance detection accuracy. Additionally, leveraging this approach in other domains requiring real-time object detection could also be explored, such as robotic vision and augmented reality applications.

In conclusion, the RRPN method proposed by Nabati and Qi provides a valuable enhancement to object detection in autonomous driving, offering both practical benefits in computational efficiency and theoretical insights into radar-vision data fusion.