SmartCooper: Vehicular Collaborative Perception with Adaptive Fusion and Judger Mechanism (2402.00321v3)
Abstract: In recent years, autonomous driving has garnered significant attention due to its potential for improving road safety through collaborative perception among connected and autonomous vehicles (CAVs). However, time-varying channel variations in vehicular transmission environments demand dynamic allocation of communication resources. Moreover, in the context of collaborative perception, it is important to recognize that not all CAVs contribute valuable data, and some CAV data even have detrimental effects on collaborative perception. In this paper, we introduce SmartCooper, an adaptive collaborative perception framework that incorporates communication optimization and a judger mechanism to facilitate CAV data fusion. Our approach begins with optimizing the connectivity of vehicles while considering communication constraints. We then train a learnable encoder to dynamically adjust the compression ratio based on the channel state information (CSI). Subsequently, we devise a judger mechanism to filter the detrimental image data reconstructed by adaptive decoders. We evaluate the effectiveness of our proposed algorithm on the OpenCOOD platform. Our results demonstrate a substantial reduction in communication costs by 23.10\% compared to the non-judger scheme. Additionally, we achieve a significant improvement on the average precision of Intersection over Union (AP@IoU) by 7.15\% compared with state-of-the-art schemes.
- Physical layer evaluation of V2X communications technologies: 5G NR-V2X, LTE-V2X, IEEE 802.11bd, and IEEE 802.11p. In IEEE 90th Vehicular Technology Conference (VTC2019-Fall), pages 1–7, Honolulu, HI, USA, Sep. 2019.
- nuScenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 11621–11631, Seattle, USA, Jun. 2020.
- F-cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds. In Proceedings of the Fourth ACM/IEEE Symposium on Edge Computing (SEC), pages 88–100, Washington DC, USA, Nov. 2019.
- Vehicle as a service (VaaS): Leverage vehicles to build service networks and capabilities for smart cities. arXiv preprint arXiv:2304.11397, 2023.
- Cooperative perception technology of autonomous driving in the internet of vehicles environment: A review. Sensors, 22(15):5535, Jul. 2022.
- CARLA: An open urban driving simulator. In Conference on Robot Learning (CoRL), pages 1–16, Mountain View, CA, USA, Oct. 2017.
- Age of information in energy harvesting aided massive multiple access networks. IEEE Journal on Selected Areas in Communications, 40(5):1441–1456, May 2022.
- Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research, 32(11):1231–1237, 2013.
- Dynamic adaptive DNN surgery for inference acceleration on the edge. In IEEE Conference on Computer Communications (INFOCOM), pages 1423–1431, Paris, France, Apr. 2019.
- Where2comm: Communication-efficient collaborative perception via spatial confidence maps. Advances in neural information processing systems (NeurlPS), 35:4874–4886, 2022.
- Multivehicle cooperative driving using cooperative perception: Design and experimental validation. IEEE Transactions on Intelligent Transportation Systems (TITS), 16(2):663–680, Aug. 2014.
- Pointpillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 12697–12705, Long Beach, CA, USA, Jun. 2019.
- Deep learning for lidar point clouds in autonomous driving: A review. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3412–3432, Aug. 2020.
- Tracking and transmission design in terahertz V2I networks. IEEE Transactions on Wireless Communications, 22(6):3586–3598, Jun. 2022.
- Who2com: Collaborative perception via learnable handshake communication. In IEEE International Conference on Robotics and Automation (ICRA), pages 6876–6883, Paris, France, May 2020.
- Distributed graph-based optimization of multicast data dissemination for internet of vehicles. IEEE Transactions on Intelligent Transportation Systems (TITS), 24(3):3117–3128, Dec. 2022.
- An enhanced information sharing roadside unit allocation scheme for vehicular networks. IEEE Transactions on Intelligent Transportation Systems (TITS), 23(9):15462––15475, Jan. 2022.
- SHIFT: A synthetic driving dataset for continuous multi-task domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pages 21371–21382, New Orleans, LA, USA, Jun. 2022.
- A Talebpour and H. S Mahmassani. Influence of connected and autonomous vehicles on traffic flow stability and throughput. Transportation research part C: emerging technologies, 71:143–163, 2016.
- V2VNet: Vehicle-to-vehicle communication for joint perception and prediction. In European Conference on Computer Vision, pages 605–621, Glasgow, UK, Aug. 2020.
- 3U: Joint design of UAV-USV-UUV networks for cooperative target hunting. IEEE Transactions on Vehicular Technology, 72(3):4085–4090, Mar. 2022.
- PointFusion: Deep sensor fusion for 3d bounding box estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pages 244–253, Salt Lake City, UT, USA, Jun. 2018.
- OPV2V: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication. In 2022 International Conference on Robotics and Automation (ICRA), pages 2583–2589, Philadelphia, PA, USA, May 2022.
- CoBEVT: Cooperative bird’s eye view semantic segmentation with sparse transformers. In Conference on Robot Learning (CoRL), pages 989–1000, Atlanta, GA, USA, Mar. 2023.
- Variable rate deep image compression with modulated autoencoder. IEEE Signal Processing Letters, 27:331–335, Jul. 2020.