A Comprehensive Study of the Robustness for LiDAR-based 3D Object Detectors against Adversarial Attacks (2212.10230v3)

Abstract: Recent years have witnessed significant advancements in deep learning-based 3D object detection, leading to its widespread adoption in numerous applications. As 3D object detectors become increasingly crucial for security-critical tasks, it is imperative to understand their robustness against adversarial attacks. This paper presents the first comprehensive evaluation and analysis of the robustness of LiDAR-based 3D detectors under adversarial attacks. Specifically, we extend three distinct adversarial attacks to the 3D object detection task, benchmarking the robustness of state-of-the-art LiDAR-based 3D object detectors against attacks on the KITTI and Waymo datasets. We further analyze the relationship between robustness and detector properties. Additionally, we explore the transferability of cross-model, cross-task, and cross-data attacks. Thorough experiments on defensive strategies for 3D detectors are conducted, demonstrating that simple transformations like flipping provide little help in improving robustness when the applied transformation strategy is exposed to attackers. \revise{Finally, we propose balanced adversarial focal training, based on conventional adversarial training, to strike a balance between accuracy and robustness.} Our findings will facilitate investigations into understanding and defending against adversarial attacks on LiDAR-based 3D object detectors, thus advancing the field. The source code is publicly available at \url{https://github.com/Eaphan/Robust3DOD}.

• The paper shows that point perturbation most effectively disrupts LiDAR 3D detectors, significantly reducing mAP on datasets like KITTI and Waymo.
• The study reveals that adversarial examples transfer across models, exposing vulnerabilities in various detectors such as SECOND, Voxel-RCNN, and PointPillar.
• The research introduces Balanced Adversarial Focal Training (BAFT) as a promising defense that balances detection accuracy with enhanced robustness against adaptive attacks.

Overview of LiDAR-based 3D Object Detectors under Adversarial Attacks

The paper "A Comprehensive Study of the Robustness for LiDAR-based 3D Object Detectors against Adversarial Attacks" presents a meticulous evaluation of the vulnerabilities and defenses of LiDAR-based 3D object detectors when confronted with adversarial attacks. As the deployment of these detectors in safety-critical applications like autonomous driving becomes increasingly prevalent, understanding their robustness becomes imperative.

Key Findings and Methodological Approach

1. Adversarial Attacks on 3D Object Detectors: The paper extends three prominent adversarial attack methods to evaluate their effect on 3D object detection tasks. These include point perturbation, detachment, and attachment:
   • Point Perturbation: This attack involves slight shifts in the 3D coordinates of points in the input point cloud, and it is found to be the most effective in disrupting detectors.
   • Point Detachment: Points are selectively removed based on a saliency map derived from point-wise gradients.
   • Point Attachment: New points are introduced strategically to compromise detection accuracy.
2. Evaluation Metrics and Detectors: The effectiveness of these attacks is evaluated using a robust set of metrics, particularly the mAP ratio derived from standard datasets, KITTI and Waymo. The paper scrutinizes a variety of state-of-the-art 3D detectors including SECOND, Voxel-RCNN, and PointPillar, providing a comprehensive view of how design choices affect robustness.
3. Transferability of Attacks: The investigation reveals that adversarial examples have notable cross-model, cross-task, and cross-domain transferability. Notably, examples from more vulnerable point-based detectors like PointRCNN effectively compromise other detectors, highlighting significant cross-model weaknesses.
4. Defense Mechanisms: The paper examines defenses like simple data transformations and more advanced strategies, concluding that many existing defenses offer limited efficacy against adaptive attacks, where attackers are aware of defense mechanisms. However, adversarial training, particularly the proposed Balanced Adversarial Focal Training (BAFT), strikes a promising balance between maintaining detection accuracy on clean data and improving robustness to adversarial attacks.

Implications and Future Directions

The implications of this paper are multifaceted. Practically, improvements in the robustness of 3D object detectors can significantly enhance the reliability and safety of autonomous systems. Theoretically, the insights into attack mechanisms and defense strategies underscore the need for continued advancements in adversarial robustness. Future research could build on the BAFT by further refining training methodologies to enhance real-world applicability and robustness. Moreover, as the paper highlights the limitations of specific defenses against adaptive attacks, there is room for the development of novel, more robust defense frameworks that can generalize across various models and attack types.

In summary, this research reveals critical insights into the current state of LiDAR-based 3D object detection systems under adversarial conditions, presenting both challenges and opportunities for future advancements in the domain.