BeamVLM for Low-altitude Economy: Generative Beam Prediction via Vision-language Models

Abstract: For low-altitude economy (LAE), fast and accurate beam prediction between high-mobility unmanned aerial vehicles (UAVs) and ground base stations is of paramount importance, which ensures seamless coverage and reliable communications. However, existing deep learning-based beam prediction methods lack high-level semantic understanding of dynamic environments, resulting in poor generalization. On the other hand, the emerging LLM based approaches show promise in enhancing generalization, but they typically lack rich environmental perception, thereby failing to capture fine-grained spatial semantics essential for precise beam alignment. To tackle these limitations, we propose in this correspondence a novel end-to-end generative framework for beam prediction, called BeamVLM, which treats beam prediction as a vision question answering task capitalizing on powerful existing vision-LLMs (VLMs). By projecting raw visual patches directly into the language domain and judiciously designing an instructional prompt, the proposed BeamVLM enables the VLM to jointly reason over UAV trajectories and environmental context. Last, experimental results on real-world datasets demonstrate that the proposed BeamVLM outperforms state-of-the-art methods in prediction accuracy and also exhibits superior generalization for other scenarios such as vehicle-to-infrastructure (V2I) beam prediction.

• The paper introduces BeamVLM, a generative vision-language model that predicts future mmWave beam indices directly from eight raw camera frames using LoRA fine-tuning and temporal physical prompts.
• BeamVLM achieves 83.3% UAV Top-1 accuracy at the next time step, 71.4% at five steps ahead, and over 93% V2I Top-3 accuracy across horizons, outperforming LSTM and BeamLLM baselines.
• The method improves visual beam prediction but requires 95 ms per inference and evaluation remains limited to within-scene splits, leaving latency, resource demands, and cross-scene generalization as key deployment challenges.

Motivation and problem setting

Beam management for high-mobility UAVs in millimeter-wave (mmWave) bands is constrained by short channel coherence times, making exhaustive beam search prohibitively expensive. The paper addresses this by casting beam prediction as a vision-language task. The authors identify two complementary deficiencies in prior work: discriminative deep learning approaches (CNN/RNN-based classifiers) overfit low-level pixel statistics and generalize poorly to unseen environments, while recent LLM-based methods such as BeamLLM and BP-LLM compress visual scenes into sparse proxies—bounding boxes or numerical AoD/beam histories—thereby discarding environmental context (building geometry, urban canyon structure, blockers) that is physically coupled to the wireless channel.

The system model considers a downlink ground-to-UAV link with an $N$-antenna ULA at the base station (BS) augmented with an RGB camera. Beam prediction is performed over a $90^\circ$ azimuth sector using an oversampled DFT codebook ($N=16$, $M=32$), and the objective is to learn a mapping $f_\Theta: \mathbf{U}[t] \rightarrow m^\star[t]$ from raw images to the optimal beam index $$m^\star[t] = \arg\max_m |\mathbf{h}^H[t]\mathbf{f}_m|^2$$, evaluated on DeepSense 6G Scenario 23.

Architecture and method

BeamVLM is built on Qwen2.5-VL (3B parameters) and reformulates beam prediction as generative visual question answering rather than classification. Raw frames are resized to $224\times224$, encoded by a ViT into patch embeddings, projected to the language dimension, and fused with tokenized text in an autoregressive decoder that emits beam indices directly as text tokens. Fine-tuning uses LoRA ($r=8$, $\alpha=16$) restricted to attention projections, yielding only 42.2M trainable parameters out of 3.1B total. Training minimizes teacher-forced cross-entropy over generated beam-index sequences; inference de-tokenizes output strings back onto the codebook.

A notable design element is the structured instructional prompt, which comprises three static blocks: dataset definition (8 input frames, codebook size 32), task instruction with strict output formatting (exactly five comma-separated integers for slots $t+1$ through $90^\circ$0), and a physical-prior context hint stating that beam indices evolve smoothly over time, encouraging temporally consistent predictions aligned with UAV inertia.

Experimental results

Experiments use a sliding window of 13 samples (8 input frames, 5 prediction steps) with a 70/30 train-test split within a single scene. Key results:

Metric	BeamVLM	Best baseline	Gain
UAV Top-1 @ $90^\circ$1	83.3%	LSTM	+10.8%
UAV Top-1 @ $90^\circ$2	71.4%	—	—
V2I Top-1 @ $90^\circ$3	72.1%	BeamLLM	+11.1%
V2I Top-3 across horizons	>93.0%	BeamLLM/LSTM drop to 91.0%/84.0% @ $90^\circ$4	—

In the V2I generalization test (Scenario 8, with only prompt semantics adjusted), BeamVLM leads RNN by 26.0% in Top-1 accuracy at $90^\circ$5. An ablation removing the textual prompt degrades initial Top-1 accuracy by 3.76% (UAV) and 3.60% (V2I), yet the prompt-free variant still surpasses BeamLLM and LSTM—evidence that the generative multimodal architecture itself, not merely prompt engineering, drives the improvement.

Limitations and open questions

The paper's efficiency profile is a substantive trade-off: BeamVLM requires 95 ms per inference versus 7.2 μs for LSTM and 2.3 ms for BeamLLM, and its 3.1B parameter footprint raises deployment questions for latency-critical LAE links that the paper does not resolve. Evaluation is confined to single-scene splits of two DeepSense scenarios; cross-scene zero-shot generalization—often the central claim motivating semantic approaches—is not tested. The framework also assumes BS-side camera coverage with a fixed $90^\circ$6 field of view, and multi-UAV operation is deferred to independent per-UAV prediction without addressing interference or coordination. Whether the smoothness prior embedded in the prompt remains beneficial under aggressive maneuvers or NLOS transitions is left unexamined.

Conclusion

BeamVLM demonstrates that treating mmWave beam prediction as a generative VQA task over raw visual patches, fine-tuned via LoRA and guided by physics-informed prompts, yields consistent accuracy gains over both discriminative and LLM-based baselines in real-world UAV and V2I datasets. Its principal open issues are inference latency, computational cost, and validation of generalization beyond within-scene splits.