Dice Question Streamline Icon: https://streamlinehq.com

Real-world datasets for vision-aided mmWave beam training

Develop and release realistic, large-scale datasets that pair camera imagery (and potentially other sensors) with mmWave channel/beam measurements in real environments to enable training and benchmarking of deep learning methods for vision-aided beam selection, tracking, and handover.

Information Square Streamline Icon: https://streamlinehq.com

Background

Vision-aided beam training methods rely on deep learning using camera images to predict beams, but most existing datasets come from ray-traced environments.

The paper explicitly notes the need for real-world datasets to bridge the sim-to-real gap and advance practical deployment of vision-aided mmWave beamforming.

Such datasets would support model generalization, multi-modal fusion (e.g., with LiDAR, GPS), and robust operation in diverse lighting and weather conditions.

References

Although one such dataset is provided in , where the authors obtained the images of the scenes from ray tracing environments, creating datasets in realistic environments is an open challenge.

— The Integrated Sensing and Communication Revolution for 6G: Vision, Techniques, and Applications (2405.01816 - González-Prelcic et al., 3 May 2024) in Sensing-assisted communication, Vision-aided beam training (Section: Technologies for sensing assisted communication)