Flexible Ad Hoc Collaboration Across Heterogeneous V2X Agents Without Joint Training

Develop a flexible and scalable cooperative perception framework for vehicle-to-everything (V2X) that enables vehicles to collaborate on the fly with arbitrary encountered agents despite heterogeneity in sensor modalities, sensor configurations, and model architectures, without requiring prior joint training or sharing of data/models, while maintaining robust fusion performance.

Background

Most current methods for handling heterogeneity assume that all participating agents can share data and models for joint training, an unrealistic assumption in practice due to privacy, bandwidth, and autonomy constraints. Real deployments will involve agents with diverse sensors and proprietary models meeting on the fly.

The authors envision a plug-and-play paradigm where vehicles collaborate without prior joint training, necessitating mechanisms for aligning heterogeneous representations and fusing information reliably in dynamic, ad hoc settings. Achieving this paradigm is identified as an open challenge.

References

How to pave the way toward such a flexible and scalable cooperative framework remains an open and challenging question.

— Wireless Communication as an Information Sensor for Multi-agent Cooperative Perception: A Survey  (2505.00747 - Song et al., 30 Apr 2025) in Section 3: Information Fusion, Subsection: Heterogeneity

The development of large-scale multi-agent perception datasets is an open challenge for the collaborative perception community, and future evaluation of our framework will depend on the availability of such datasets.

— CoAdapt: An LLM-based Framework for Adaptive Collaborative Perception in IIoT Robotic Swarms  (2609.16852 - Hassan et al., 15 Sep 2026) in Section 5, subsection “Limitations,” paragraph “Dataset Constraints”