Intensity-to-LiDAR Online Distillation

• The paper introduces a distillation framework that leverages dense LiDAR intensity maps as a teacher to enhance semantic map quality in LiDAR-only networks.
• It employs a teacher–student architecture with both feature-level and logit-level distillation to align multi-scale BEV features and segmentation outputs.
• Quantitative results on nuScenes demonstrate significant mIoU improvements and enhanced detection of fine semantic classes under adverse conditions.

Intensity-to-LiDAR Online Distillation is an online knowledge distillation paradigm for semantic map construction in autonomous driving, in which semantic knowledge from camera images (or, specifically, dense LiDAR intensity maps) is leveraged during training to compensate for the semantic sparsity and weak texture cues in raw LiDAR-based BEV (bird’s-eye view) representations. At inference time, only the LiDAR branch is deployed, maintaining the robust spatial precision and real-time efficiency of pure-LiDAR methods, but enhanced with semantics distilled from richer modalities during training. The approach is exemplified by “LiDAR2Map” (Wang et al., 2023) and “LIE” (Mazumder et al., 2 May 2026), unified by the use of teacher–student architectures where the teacher branch fuses LiDAR and camera or intensity data, and the student branch is LiDAR-only.

1. Motivation and Core Rationale

LiDAR sensors provide precise 3D geometric structure vital for reliable autonomous vehicle mapping and localization. However, BEV features derived from raw LiDAR exhibit inherent semantic deficiencies: spatial features contain little texture, color, or class-discriminative information, resulting in map segmentation outputs that are geometrically accurate but semantically noisy (Wang et al., 2023). In contrast, semantic cues in wide-FoV camera images or dense LiDAR intensity maps can resolve this ambiguity, but these modalities alone are insufficient—cameras lack reliable depth and are susceptible to projection distortion and lighting, while raw intensity patterns vanish at range and with atmospheric attenuation.

Intensity-to-LiDAR online distillation addresses this gap by employing auxiliary modalities during training as semantic “teachers,” transferring knowledge via feature-level and logit-level distillation into the LiDAR “student.” This results in a LiDAR-only model that leverages image-derived semantics without imposing extra sensor or latency costs during deployment (Mazumder et al., 2 May 2026).

2. Architectural Framework and Distillation Mechanisms

Both LiDAR2Map (Wang et al., 2023) and LIE (Mazumder et al., 2 May 2026) employ a teacher–student network design, with parallel branches:

• The student branch processes only raw LiDAR point clouds (including per-point intensity and timestamp), encodes them via a PointPillars-like or VoxelNet encoder, and produces BEV feature maps and final segmentation outputs.
• The teacher branch fuses the student’s LiDAR features with either camera-derived BEV pseudo-images (LiDAR2Map) or preprocessed dense 2D intensity map tiles (LIE), employing specialized cross-modal fusion modules.

Distillation proceeds via two pathways:

• Feature-level distillation: Multi-scale BEV features are aligned between the teacher and student at several resolutions ($\tilde{F}_i^{BEV}$). In LiDAR2Map, a learnable tree filter processes each feature map under low-level guidance before summing L₁ distances across scales. In LIE, global affinity matrices at each scale are extracted via the Latent Tree Filter and matched via L₁ loss.
• Logit-level distillation: Softmax probability maps (per-class BEV segmentation outputs) from both branches are aligned by minimizing KL divergence between teacher ($P_{Teacher}$) and student ($P_{Student}$) logits.

In LIE, the overall loss function is

$$\mathcal{L} = \mathcal{L}_{fusion\_seg} + \mathcal{L}_{seg} + \alpha\,\mathcal{L}_{feature} + \beta\,\mathcal{L}_{logit}$$

where $\alpha=0.4$ and $\beta=1.5$ (Mazumder et al., 2 May 2026). All terms are optimized jointly, and distillation branches participate only during training.

3. Input Modalities, Fusion Strategy, and Preprocessing

Student input consists solely of a temporally local LiDAR sweep, including point coordinates, intensity, and timestamp information, within a restricted BEV ROI (e.g., ±85 m × ±35 m).

Teacher input leverages dense, preprocessed semantic cues:

• In LiDAR2Map, the teacher uses camera images, transformed by a backbone (ResNet/EfficientNet/Swin-Tiny), projected to BEV via depth-prediction and pooling. Perspective–to–BEV lifting is critical for spatial alignment (Wang et al., 2023).
• In LIE, dense 2D intensity maps are constructed by rasterizing accumulated ground-segmented past LiDAR sweeps at 0.1 m resolution, cropping a BEV-aligned tile centered on the ego-vehicle, and enhancing with local maxima and smoothing to emphasize features such as lane markings (Mazumder et al., 2 May 2026).

Feature Fusion in the teacher branch is performed by specialized modules:

• LiDAR2Map uses the Position-Guided Feature Fusion Module (PGF²M) that concatenates BEV features, applies convolutions with position encodings, and multiplicative MLP-based reweighting.
• LIE applies the Position-Guided Cross-Modal Fusion (PGxMF) with positional embeddings and Attentional Feature Fusion (AFF) blocks that generate context-aware attention over both modalities.

4. Training Regimes and Implementation Specifics

Both methods train end-to-end using standard semantic segmentation objectives (cross-entropy and Lovász-Softmax losses) augmented by the distillation terms. Training specifics include:

• Optimizer: Adam with weight decay ($1 \times 10^{-7}$)
• Learning rate schedule: Step decay ($2 \times 10^{-3}$ initial, $\times 0.1$ at epoch 20)
• Epochs: 30 on nuScenes dataset (700 train, 150 val scenes)
• Data augmentation: Random global rotations, scaling, translation jitter, and flipping for LiDAR; color jitter and resizing for camera/intensity images
• Hardware: Typically 4× NVIDIA A40 GPUs; batch size as per LiDAR2Map’s default for fair comparison (Wang et al., 2023, Mazumder et al., 2 May 2026)

All fusion and distillation components are present only during training. At inference, only the LiDAR branch and BEV decoder are deployed, so there is no increase in runtime or sensor requirements.

5. Empirical Performance and Robustness

Intensity-to-LiDAR online distillation demonstrates significant gains over prior LiDAR-only and camera/LiDAR fusion approaches.

Quantitative results (nuScenes, 60 m × 30 m BEV):

Method	Modality	Lane Divider (%)	Ped Crossing (%)	Boundary (%)	mIoU (%)
HDMapNet–LiDAR	L	26.7	17.3	44.6	29.5
LiDAR2Map (KD cam)	L	60.4	45.5	66.4	57.4
LIE (KD intensity)	L (KD from intensity)	61.0	46.6	66.3	58.0

On long-range BEV (120 m × 60 m), LIE achieves 55.7% mIoU versus 40.9% for the best camera-priors model (Mazumder et al., 2 May 2026).

Ablation studies confirm that logit-level distillation typically yields ~0.3–0.6 mIoU gain, encoder/decoder feature-level distillation yields 0.7–1.4 mIoU, and the full fusion–distillation pipeline achieves the best segmentation. Improvements concentrate in “thin” semantic classes (lane dividers, crossings) that manifest as high-contrast features in the intensity channel.

Robustness is enhanced: LIE delivers competitive mIoU under rainy/night conditions and adapts efficiently to Argoverse2, achieving 50.8% mIoU after fine-tuning on just 10% of the target dataset—surpassing camera-based models trained on the full set (Mazumder et al., 2 May 2026).

At inference, both LiDAR2Map and LIE retain high throughput (LIE: 36 FPS on RTX 4090), as all auxiliary intensity/camera inputs and distillation computations are omitted.

6. Comparative Analysis and Broader Implications

Intensity-to-LiDAR online distillation outperforms previous BEV-space knowledge distillation frameworks such as MonoDistill, MGD, xMUDA, and 2DPASS on nuScenes benchmarks (Wang et al., 2023). The approach narrows the semantic gap between LiDAR-only and camera-based/fusion models while preserving the spatial fidelity and operational advantages of pure-LiDAR pipelines.

A critical insight is that distilling from dense, high-SNR intensity cues extracted from LiDAR point clouds forgoes the modality mismatch inherent with RGB cameras, resulting in student networks that are less susceptible to modality-induced domain gaps, cross-weather failures, and visual artifacts (Mazumder et al., 2 May 2026). This suggests that future distillation frameworks may benefit from blending or hierarchically fusing multiple auxiliary modalities during training and from further refining cross-modal fusion blocks.

A plausible implication is more robust, efficient, and generalizable LiDAR perception stacks for real-world autonomous driving applications, as pure-LiDAR models equipped with distilled semantics can operate effectively across cities, sensor variations, and adverse environments.

7. Limitations and Future Perspectives

While intensity-to-LiDAR distillation closes much of the semantic gap between modalities, limitations persist. The method’s efficacy depends on the quality of the intensity source maps; insufficiently dense or noisy reflectivity data reduces its benefit for classes that require fine semantic discrimination. Both LiDAR2Map and LIE focus on the nuScenes and Argoverse2 datasets; transfer to older, sparser LiDAR sensors or to domains without consistent high-contrast features is not yet resolved.

Future directions include: leveraging online scene-adaptive intensity map construction, integrating additional auxiliary modalities (e.g., radar, panoptic labels), and extending distillation to instance and panoptic segmentation tasks. Finally, as sensor fusion and cross-modal supervision advance, direct exploitation of native LiDAR return intensity and higher-order physical priors for semantic distillation remains an open research frontier.

---

References

• LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation (Wang et al., 2023)
• LIE: LiDAR-only HD Map Construction with Intensity Enhancement via Online Knowledge Distillation (Mazumder et al., 2 May 2026)