V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving (2202.08449v2)

Abstract: Vehicle-to-everything (V2X) communication techniques enable the collaboration between vehicles and many other entities in the neighboring environment, which could fundamentally improve the perception system for autonomous driving. However, the lack of a public dataset significantly restricts the research progress of collaborative perception. To fill this gap, we present V2X-Sim, a comprehensive simulated multi-agent perception dataset for V2X-aided autonomous driving. V2X-Sim provides: (1) \hl{multi-agent} sensor recordings from the road-side unit (RSU) and multiple vehicles that enable collaborative perception, (2) multi-modality sensor streams that facilitate multi-modality perception, and (3) diverse ground truths that support various perception tasks. Meanwhile, we build an open-source testbed and provide a benchmark for the state-of-the-art collaborative perception algorithms on three tasks, including detection, tracking and segmentation. V2X-Sim seeks to stimulate collaborative perception research for autonomous driving before realistic datasets become widely available. Our dataset and code are available at \url{https://ai4ce.github.io/V2X-Sim/}.

• The paper presents a novel multi-agent collaborative perception dataset that overcomes single-vehicle limitations by leveraging V2X communication.
• It benchmarks state-of-the-art algorithms like DiscoNet and V2VNet, demonstrating significant improvements in detection, tracking, and segmentation.
• The dataset, generated via CARLA and SUMO co-simulation, provides a robust platform for advancing collaborative autonomous driving research.

Overview of V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving

The paper “V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving” presents a novel multi-agent perception dataset aimed at facilitating autonomous driving research by leveraging Vehicle-to-Everything (V2X) communication systems. The authors recognize the limitations of single-vehicle perception due to occlusions and range constraints, proposing a multi-agent collaboration approach using V2X communication as a pivotal mechanism. They address the gap in publicly available collaborative perception datasets by introducing V2X-Sim, a synthetic dataset generated through CARLA and SUMO co-simulation, containing sensor data from vehicles and roadside units (RSU).

Key Contributions

1. Dataset Features: V2X-Sim is characterized by multi-modality sensor recordings that include RGB cameras, depth cameras, LiDAR, and semantic segmentation cameras on vehicles and RSUs. The authors provide diverse ground-truth annotations, supporting tasks such as detection, tracking, and segmentation. This is pivotal for autonomous driving research, allowing for comprehensive perception beyond individual vehicle capabilities.
2. Simulation Setup: The dataset was generated using a co-simulation platform involving CARLA, an open-source driving simulator, and SUMO, a traffic flow simulator, ensuring realistic multi-agent driving scenarios. These tools enabled the synthesis of traffic environments and interactions at various crossroads and junctions.
3. Benchmark Evaluations: The authors implement several state-of-the-art collaborative perception algorithms as benchmarks, including DiscoNet, V2VNet, When2com, and Who2com. These benchmarks serve as reference points for assessing collaborative perception tasks within the dataset, showing appreciable improvements in detection, tracking, and segmentation tasks through multi-agent collaboration relative to single-agent performance.

Experimental Insights

The empirical results demonstrate significant enhancements in perception accuracy when employing V2X-Sim for collaborative tasks compared to isolated single-agent perception. Specifically, in detection tasks, frameworks such as DiscoNet and V2VNet exhibit improved average precision, substantiating the assertion that multi-agent collaboration enriches environmental understanding.

The paper also examines robustness against noise and communication constraints, showing stability across variations in pose noise and compression ratios during data transmission. This underscores the practicality of V2X systems in real-world scenarios where data fidelity may be variable.

Implications for Future Research

The introduction of V2X-Sim provides a foundation for further exploration into collaborative perception strategies, high-density sensor data integration, and more complex interaction models in autonomous driving contexts. With the dataset publicly available, researchers can develop and test novel collaborative algorithms, optimizing information sharing and decision-making processes across vehicular networks.

In terms of immediate advancements, incorporating real-world data alongside synthetic simulations would enhance the dataset’s applicability, bridging the gap between simulated environments and real-life road conditions. Furthermore, expansion into different urban environments could add to the dataset’s diversity, offering comprehensive scenarios for testing autonomous systems’ adaptability.

Conclusion

V2X-Sim represents a substantial step forward in collaborative autonomous driving research, providing crucial tools and benchmarks necessary for driving innovation. It sets the stage for developing advanced V2X communication-enabled perception systems, promoting safer and more efficient autonomous vehicle operation. Future adaptations and expansions of the dataset will undoubtedly contribute to the evolution of collaborative perception frameworks, advancing the state-of-the-art in autonomous driving technology.