Flexible Ad Hoc Collaboration Across Heterogeneous V2X Agents Without Joint Training

Develop a flexible and scalable cooperative perception framework for vehicle-to-everything (V2X) that enables vehicles to collaborate on the fly with arbitrary encountered agents despite heterogeneity in sensor modalities, sensor configurations, and model architectures, without requiring prior joint training or sharing of data/models, while maintaining robust fusion performance.

Background

Most current methods for handling heterogeneity assume that all participating agents can share data and models for joint training, an unrealistic assumption in practice due to privacy, bandwidth, and autonomy constraints. Real deployments will involve agents with diverse sensors and proprietary models meeting on the fly.

The authors envision a plug-and-play paradigm where vehicles collaborate without prior joint training, necessitating mechanisms for aligning heterogeneous representations and fusing information reliably in dynamic, ad hoc settings. Achieving this paradigm is identified as an open challenge.