---
title: LiDAR-Aided mmWave Beam Tracking
url: https://www.emergentmind.com/topics/lidar-aided-mmwave-beam-tracking
type: topic
---

# LiDAR-Aided mmWave Beam Tracking

LiDAR-aided mmWave beam tracking is the use of LiDAR sensing as out-of-band information for estimating, predicting, or tracking the beam configuration of millimeter-wave links. In the works considered here, LiDAR is used both as a geometric sensor that reconstructs indoor reflective structure for non-line-of-sight beam steering and as a sequential sensing modality for current and future beam prediction in vehicle-to-infrastructure settings. The common objective is to reduce the overhead of exhaustive beam sweeps while maintaining alignment under blockage, mobility, or NLoS propagation [2511.09840][2203.05548][2412.05427][2509.19092].

## 1. Problem formulation and sensing representations

In mmWave systems, initial beam acquisition and beam tracking are distinct tasks. Initial acquisition, also called beam selection, runs a full sweep over a fixed codebook of narrow beams to find the best index once. Beam tracking instead treats alignment as a sequential estimation problem: once an initial beam index is known, the system predicts future beam indices as the user or vehicle moves, thereby avoiding repeated exhaustive re-sweeps [2412.05427]. A representative formulation seeks
$$
i_t^*=\arg\max_{i\in\{1\ldots M\}} |w_i^H H_t f_i|^2,
$$
with performance often evaluated through instantaneous spectral efficiency
$$
R_t=\log_2\!\left(1+\frac{|w_{\hat i_t}^H H_t f_{\hat i_t}|^2}{\sigma^2}\right).
$$

The LiDAR observation supplied to the tracker varies by system design. In real-world V2I beam prediction, raw point clouds are projected onto a 1D range image, producing a quantized-angle range vector $\mathbf{x}[t]\in\mathbb{R}^D$ synchronized with mmWave receive-power measurements over $M=64$ beams [2203.05548]. In multimodal V2I tracking, the LiDAR input is a point cloud $P_t\in\mathbb{R}^{M\times 3}$, preprocessed by range-normalization, ground-removal, and voxelization into a 2D grid $L_t\in\mathbb{R}^{V\times V}$ [2412.05427]. In indoor NLoS beam steering, LiDAR is not merely a feature source for a predictor; it is used to segment planar surfaces, estimate normals, and infer viable specular bounce paths [2511.09840].

This range of representations reflects two operational regimes. One regime uses LiDAR to infer environment geometry explicitly and then computes a departure angle from reflection physics. The other uses LiDAR-derived features as inputs to recurrent or hybrid deep networks that output beam probabilities over time. Both regimes are motivated by the same mmWave constraint: large beam codebooks and rapid channel variation make exhaustive beam training costly.

## 2. Geometry-driven indoor tracking with specular reflectors

The indoor framework "See and Beam" couples a co-located LiDAR sensor and mmWave radio to exploit surfaces that reflect both light and mmWave signals [2511.09840]. The reported hardware consists of a Quanergy M8 LiDAR acquiring 3D point clouds at $\ge 10$ Hz and a 60 GHz Sivers EVK02001 transceiver with an 8-element phased ULA steering beams over a $\pm 15^\circ$ azimuth range. Its processing pipeline is explicit: raw point-cloud acquisition, preprocessing by voxel downsampling and outlier removal, planar surface segmentation via RANSAC-based clustering, normal estimation, computation of the specular bounce path, mapping of the inferred departure angle to the nearest quantized steering vector, and periodic updates as the user or environment moves.

Plane fitting is posed on inlier points $\{\mathbf{p}_i\}$ using
$$
a\,x+b\,y+c\,z+d=0,\qquad \sqrt{a^2+b^2+c^2}=1,
$$
followed by least-squares refinement, with unit normal $\mathbf{n}=(a,b,c)^T$. Given an incident unit vector $\mathbf{u}$ from transmitter to reflector, the reflected unit vector is
$$
\mathbf{u}_{\rm refl}=\mathbf{u}-2(\mathbf{n}\cdot\mathbf{u})\mathbf{n}.
$$
The azimuth departure angle $\hat\theta$ extracted from the LiDAR-estimated reflection geometry is then quantized to the nearest codebook direction in $\{-15^\circ,\dots,15^\circ\}$. For the 8-element ULA, the steering vector is
$$
\mathbf{a}(\theta)=
\bigl[
1,\,
e^{-j\frac{2\pi}{\lambda}d\sin\theta},\,
\dots,\,
e^{-j\frac{2\pi}{\lambda}d(N-1)\sin\theta}
\bigr]^T,
$$
with $d=\lambda/2$ and $N=8$, and the beamforming weight is set to $\mathbf{w}=\mathbf{a}(\theta)$.

The proof-of-concept deployment uses an L-shaped corridor, 2.5 m wide, with the transmitter 3.8 m from the reflector at 1.5 m height. The reflectors are $0.3\times 0.9$ m$^2$ panels mounted at $45^\circ$: a mirror, a glossy silver sheet, and copper. Evaluation includes minimum RSS across 102 receiver grid points in deep NLoS, the CCDF of RSS, beam search time, and the angular error $\Delta\theta$ between the LiDAR estimate and the exhaustive-search optimum. The reported minimum-RSS gain for LiDAR-guided reflection with a mirror is greater than 20 dB relative to no reflector. At $\Pr=1$, the CCDF gain is +12 dB for the silver reflector, +9 dB for the mirror, and +6 dB for copper. Exhaustive scan requires 11 beam directions with 9 ms dwell, approximately 100 ms, whereas LiDAR-guided operation uses 1 direction plus 2 local refinements, approximately 10 ms. The median angular error is approximately $1.2^\circ$.

The same measurements also show that LiDAR reflectivity and mmWave reflectivity are not interchangeable. LiDAR detection rates are 95% for the mirror, 70% for glossy silver, and 4% for copper, even though copper is described as strong for mmWave and poor for LiDAR. Semi-specular surfaces such as glossy silver lose LiDAR returns beyond approximately 5 m, forcing fallback to fixed beams. Receiver-side multipath is reported to cause small residual performance gaps.

## 3. Recurrent prediction of current and future beams in real-world V2I

A complementary line of work treats LiDAR-aided beam tracking as a sequence-learning problem over synchronized sensing and communication data [2203.05548]. In "LiDAR Aided Future Beam Prediction in Real-World Millimeter Wave V2I Communications," the LiDAR snapshot at time $t$ is represented by a quantized-angle range vector $\mathbf{x}[t]\in\mathbb{R}^D$, embedded as $\tilde{\mathbf{x}}[t]=W_e\mathbf{x}[t]+b_e$, and processed by a stack of single-layer GRUs. With hidden dimension $H=64$, the recurrent state evolves as
$$
\mathbf{h}_t=\mathrm{GRU}\bigl(\tilde{\mathbf{x}}[t],\mathbf{h}_{t-1}\bigr),
$$
and a fully connected layer with softmax outputs a probability vector over $M=64$ beam indices. The optimal beam label is aligned to the LiDAR frame through
$$
p^\star[t]=\arg\max_{m=1,\dots,M}|y_m[t]|^2,
$$
where $y_m[t]$ is the receive power on beam $m$.

Training uses the DeepSense-6G dataset with an 80%/20% split over nonoverlapping vehicle runs. The observation window is $W=8$ LiDAR frames, and the model predicts the current beam plus future beams at horizons $t+1$, $t+2$, and $t+3$. The reported hyperparameters are LiDAR embedding size $D_e=64$, beam embedding size $M_e=64$, GRU hidden size $H=64$, batch size 256, Adam with learning rate $10^{-3}$, and 50–100 epochs until convergence. No data augmentation is reported.

The evaluation emphasizes top-$k$ beam prediction accuracy and beam-training overhead reduction. For current-beam prediction, the LiDAR model achieves Top-1 accuracy of 57.5%, Top-2 accuracy of 80.6%, Top-3 accuracy of 89.5%, and Top-5 accuracy of 95.6%. For future prediction at horizon +1, the Top-5 accuracy is 95.0%. The corresponding Top-5 values at +2 and +3 are 94.5% and 93.3%. If beam training is limited to the top-5 predicted beams rather than all 64 beams, the fractional reduction is
$$
\Delta O = 1-\frac{5}{64}\approx 92.2\%.
$$
The beam-history baseline assumes perfect knowledge of the previous 8 optimal beams, yet the LiDAR approach remains within 2–3 percentage points of that baseline.

Two characteristics are especially notable. First, the LiDAR-aided model does not use previous beams; it relies only on up-to-date LiDAR frames, and the paper states that no beam calibration or pilot overhead is needed. Second, its average Top-5 accuracy remains essentially flat as the operation window length increases, whereas the beam-history baseline decays quickly unless retraining is extremely frequent. The reported limitations are equally explicit: the method assumes clear line-of-sight between UE and BS so that LiDAR can observe the vehicle, degrades if LiDAR is corrupted by heavy rain or occlusion, and requires retraining when the environment or LiDAR mounting geometry changes significantly.

## 4. Multimodal V2I datasets and ResNet–LSTM tracking

"Machine Learning-Based mmWave MIMO Beam Tracking in V2I Scenarios: Algorithms and Datasets" formalizes beam tracking as multimodal sequential classification and contributes a new public dataset containing images, LiDAR information, and GNSS positioning [2412.05427]. The channel is modeled as
$$
H_t=\sum_{\ell=1}^L \alpha_{\ell,t}\, a_r(\phi_{\ell,t}^A)\, a_t(\phi_{\ell,t}^D)^H,
$$
with fixed DFT codebooks at transmitter and receiver. The best beam index is again the maximizer of beamformed received power, but the tracking model is driven by a history of LiDAR, GNSS, and previous beam indices.

The public dataset has two scenarios with 20 ms-spaced scenes: t001 "Rosslyn," an urban canyon with 10 episodes, 5000 scenes per episode, and 2 receivers; and t002 "Porto," a roundabout with 20 episodes, 250 scenes per episode, and 5 receivers. Each scene includes a LiDAR point cloud of approximately 100k points in global GNSS coordinates. After range-normalization and ground-removal, the point cloud is voxelized into a 2D grid $L_t\in\mathbb{R}^{V\times V}$ in which cells are quantized as $-1$ for obstacle, $0$ for empty, $-2$ for BS, and $-3$ for UE. GNSS trajectories are rasterized into a coordinate matrix $C_t\in\mathbb{R}^{V\times V}$ carrying 10 for Tx, a gradient $[3\to 8]$ along UE heading, and 1 for static scatterers. Past beam indices are included with window size $W_b=3$ and one-hot encoding. Camera images are optional, though the reported focus is LiDAR+GNSS.

The network combines a ResNet-style CNN and an LSTM. Residual blocks satisfy
$$
x_{l+1}=x_l+F(x_l;W_l),
$$
where $F$ denotes two conv-batchnorm-ReLU layers. Three such blocks produce a feature vector $z_t$, concatenated with embeddings of previous beams and then passed through an LSTM. A final fully connected layer with softmax yields beam probabilities. Training is performed per scenario with 70% of episodes for training, 10% for validation, and 20% for test, using batch size 64, 50 epochs with early stopping, Adam with $\alpha=10^{-4}$, $\beta_1=0.9$, $\beta_2=0.999$, dropout 0.3 in fully connected layers, $\ell_2$ weight decay $10^{-5}$, and LiDAR augmentation by random rotations of $\pm 5^\circ$ around the vertical axis plus Gaussian jitter on point positions.

The reported Top-1 accuracies show substantial variation across scenario and model. In t001, a baseline selection CNN using LiDAR attains 59%, Salehi et al. [multimodal selection] attains 78%, and the proposed tracking model using LiDAR plus past beams attains 84%. In t002, the baselines are at or below 40%, while the proposed tracking model reaches 97%. These gains are stated to translate directly into higher average $R_t$ and much lower misalignment, approximately 2–3% versus greater than 20%. The paper also identifies the main operational trade-offs: out-of-band sensing removes pilot overhead on the mmWave link, LiDAR is described as robust to NLOS and multipath because it captures obstacle geometry directly, and a 20 ms update rate matches coherence times of vehicular channels; against this, high-resolution LiDAR introduces cost and complexity, dense urban canyons can cause occlusion, and broad generalization requires diverse dataset collection to avoid site-specific bias.

## 5. Model compression and data-free knowledge distillation

As LiDAR-aided beam tracking systems become more dependent on deep models, complexity and data requirements become their own bottlenecks. "Data-Free Knowledge Distillation for LiDAR-Aided Beam Tracking in MmWave Systems" addresses this by distilling a compact student model from a pre-trained teacher without using real LiDAR data during distillation [2509.19092]. The setting is a single-cell mmWave system with a fixed BS equipped with a ULA and onboard LiDAR, a single-antenna mobile vehicle, and a codebook
$$
\mathcal{W}=\{\mathbf{w}_1,\dots,\mathbf{w}_M\},\qquad \|\mathbf{w}_m\|_2=1.
$$
The optimal beam index is
$$
m^\star(t)=\arg\max_{m\in\{1,\dots,M\}} |\mathbf{h}(t)^\mathsf{H}\mathbf{w}_m|^2.
$$
The LiDAR input is a sequence of $L$ past frames, $\mathcal{X}(t;L)=\{\mathbf{x}(t-L+1),\dots,\mathbf{x}(t)\}$.

The teacher model is an LSTM or GRU encoder over the $L$-frame sequence with a fully connected decoder, hidden-state dimension 128, and outputs for the current slot and $V$ future slots. The student has the same high-level architecture but hidden-state dimension 32, reducing parameter count from approximately 96 K to approximately 25 K. Synthetic LiDAR sequences are produced by a two-layer generator fed by Gaussian noise of dimension $Q=500$, with 64 ReLU units in the first layer and $L\times D$ tanh outputs in the second, followed by zero-padding of length $V$ to match the teacher input format.

The generator objective combines metadata loss, activation loss, and entropy loss. Metadata loss matches the mean and variance of a designated hidden-layer activation of the teacher on synthetic versus real data. Activation loss encourages large-norm intermediate activations, and entropy loss encourages confident teacher outputs on synthetic samples. The total generator loss is
$$
\mathcal{L}^G=\mathcal{L}_{\mathrm{meta}}^G+\alpha\,\mathcal{L}_{\mathrm{act}}^G+\beta\,\mathcal{L}_{\mathrm{ent}}^G,
$$
with $\alpha=10^{-4}$ and $\beta=10^{-2}$. Student training then uses either KL-divergence on softened teacher and student outputs or an MSE loss directly on logits. The reported hyperparameters are batch size 32, Adam with learning rate $10^{-3}$ for both generator and student, $E_G=500$, $E_S=500$, and temperature $T=5$ when KL is used.

The simulation setup uses DeepSense V2I Scenario 8 at 60 GHz with vehicle speed 3–18 km/h, a BS with a 16-element ULA, codebook size $M=64$, observation window $L=8$, and future horizon $V=3$. The teacher’s current-slot accuracy is approximately 68% Top-1 and approximately 92% Top-5. Data-free KD with the full weighted generator loss and KL recovers approximately 99% of teacher performance. With the same generator and student training by MSE on logits, the distilled student slightly outperforms the teacher by approximately 3.8% in Top-1 accuracy, which the paper attributes to implicit regularization. In ablations, metadata loss alone yields approximately 62% Top-1, compared with approximately 50% for activation only and approximately 45% for entropy only; the full weighted generator loss yields approximately 64%. A. Zakeri et al. further report that MSE can fully replace temperature-tuned KL loss while requiring fewer fine-tuned hyperparameters, and that on future slots 2–3 it yields up to +2% over KL.

## 6. Comparative performance, limitations, and extensions

Reported LiDAR-aided mmWave beam tracking results differ by propagation regime, sensing representation, and inference mechanism. The following summary organizes representative outcomes already reported in the cited works [2511.09840][2203.05548][2412.05427][2509.19092].

| Setting | LiDAR role | Representative reported result |
|---|---|---|
| Indoor deep NLoS with passive reflectors | Geometric mapping of co-reflective surfaces and specular bounce inference | Minimum RSS improved by over 20 dB; search time reduced from ∼100 ms to ∼10 ms |
| Real-world V2I future beam prediction | GRU-based prediction from synchronized LiDAR range vectors | Current Top-5 accuracy 95.6%; future +1 Top-5 accuracy 95.0%; $\Delta O\approx 92.2\%$ |
| Multimodal V2I tracking | ResNet–LSTM using LiDAR, GNSS, and past beams | Top-1 accuracy 84% in t001 and 97% in t002 |
| Data-free model compression | Synthetic LiDAR generation for teacher–student distillation | Student slightly outperforms teacher by ≈ 3.8% in Top-1 |

Several constraints recur across these results. LiDAR assistance is not uniformly reliable across materials: in indoor reflection steering, copper is strong for mmWave but poor for LiDAR and is detected only 4% of the time, while glossy silver has a 70% LiDAR detection rate and loses returns beyond approximately 5 m [2511.09840]. LiDAR assistance is also not uniformly calibration-free across formulations: the GRU-based V2I predictor explicitly states that it uses no previous optimal beam information and no beam calibration, whereas the multimodal ResNet–LSTM tracker incorporates past beam indices and GNSS, and the distillation framework assumes a pre-trained teacher model [2203.05548][2412.05427][2509.19092]. Environmental sensitivity remains central: heavy rain, occlusion, site-specific geometry, and changes in LiDAR mounting geometry are all reported as factors that can degrade performance or require retraining [2203.05548][2412.05427].

The extensions proposed across the cited works indicate where the field is moving. For indoor NLoS systems, the same geometric pipeline is reported to extend to multiple reflectors by clustering all planar patches and selecting the one minimizing the bistatic angle $\beta$; with GPU-accelerated point-cloud processing, end-to-end latency can drop below 50 ms for real-time VR/AR; and future integration with electronically steerable metasurfaces would use LiDAR to locate the user and configure the RIS phase profile [2511.09840]. For vehicular tracking, proposed directions include multisensor fusion with camera and radar, online adaptation under day/night or rain, domain-adaptive training across cities, and lightweight architectures for edge deployment [2412.05427]. For multi-user indoor reflection steering, one stated objective is joint minimization of
$$
\sum_k d_{\mathrm{Tx}\to R_k\to U_k}^{-2}
$$
through selection of reflecting zones on a large curved panel [2511.09840].

Taken together, these works define LiDAR-aided mmWave beam tracking as a family of techniques rather than a single algorithmic template. Some systems compute beams from explicit geometry and specular reflection laws; others infer beam probabilities from learned temporal structure in LiDAR, GNSS, and beam histories; still others compress those predictors through synthetic-data distillation. What unifies them is the use of optical spatial sensing to reduce or bypass mmWave beam training overhead while preserving beam alignment under mobility, blockage, or NLoS operation.

Source: https://www.emergentmind.com/topics/lidar-aided-mmwave-beam-tracking