LiRaFusion: LiDAR-Radar 3D Detection Fusion

• LiRaFusion is a LiDAR-radar fusion framework that jointly encodes voxel features for effective multi-modal 3D object detection.
• It combines early joint voxel encoding with adaptive gated BEV aggregation to exploit complementary sensing strengths.
• The framework enhances detection performance in varied weather and long-range scenarios by integrating Doppler velocity and dense geometry.

LiRaFusion is a backbone-level fusion framework for LiDAR-radar 3D object detection introduced to improve the use of radar in nuScenes-style detection pipelines. It combines an early fusion module for joint voxel feature encoding with a middle fusion module that adaptively fuses bird’s-eye-view (BEV) feature maps via a gated network, and is designed to exploit LiDAR’s dense and precise geometry together with radar’s robustness, long-range sensing, and Doppler velocity (Song et al., 2024).

1. Problem setting and design rationale

LiRaFusion addresses automated-driving 3D object detection under conditions in which neither LiDAR nor radar is sufficient alone. The formulation in the original work is explicitly motivated by the complementary sensing characteristics of the two modalities: LiDAR provides dense, geometrically accurate point clouds in good conditions, but can degrade in adverse weather and at long range where returns become sparse; automotive radar is robust to rain, fog, and low light, offers longer effective range, and provides Doppler measurements, but radar point clouds are much sparser and noisier, contain lower-resolution geometry, and often include artifacts such as ghosts and multipath (Song et al., 2024).

The method is framed as a response to several limitations observed in prior LiDAR-radar detectors on nuScenes. The underlying radar representation is the nuScenes radar object list, obtained after on-board CFAR filtering, and earlier fusion systems either exhibited degraded performance when radar was added, or relied on constraints such as restricting evaluation to moving classes or short ranges. LiRaFusion therefore targets the “performance gap of existing LiDAR-radar detectors” by modifying fusion at two distinct levels: voxel-level feature construction and BEV-level feature aggregation.

Architecturally, the method is not a stand-alone detector head. It is a backbone-level fusion framework that plugs into standard backbone-neck-head designs. The high-level pipeline consists of: LiDAR point cloud and radar point cloud input; preprocessing with multi-sweep accumulation, coordinate alignment, ego-motion compensation, and zero-padding into a unified schema; early fusion by joint voxel feature encoding; backbone extraction through sparse 3D convolutions and VoxelNet for the LiDAR stream and PointPillars for the radar stream; middle fusion by adaptive gated BEV fusion; and final prediction through an FPN neck and either a transformer-based TransHead or a CenterPoint-style CenterHead (Song et al., 2024).

2. Unified input representation and preprocessing

LiRaFusion uses LiDAR point clouds and radar point clouds derived from the nuScenes object lists. Multiple sweeps are accumulated into the current sample frame, and a time-difference channel is appended for each point: $At_\ell$ for LiDAR and $At_r$ for radar. LiDAR and radar points are transformed from their sensor frames to the current ego frame using the calibration and ego-pose information supplied by nuScenes. Radar velocities use the compensated values $Vxcomp$ and $Vycomp$ from the radar object lists (Song et al., 2024).

A central preprocessing step is the construction of a shared 9-dimensional point schema by zero-padding modality-specific attributes. The two point types are represented as

• LiDAR point: $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$
• Radar point: $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$

This representation preserves LiDAR intensity and time offset, and radar RCS, compensated Doppler velocities, and time offset, while allowing both modalities to be stacked before voxelization (Song et al., 2024).

The paper follows MMDetection3D’s VoxelNet encoder and standard sparse convolutions with BEV flattening. Exact voxel size, grid extents, and normalization are inherited from the hosting detector configuration rather than fixed in the paper. This means that LiRaFusion is structurally coupled to the host framework’s voxelization regime, even though its fusion logic is defined independently of a particular head.

3. Early fusion through joint voxel feature encoding

The early fusion module performs joint voxel feature encoding on the stacked LiDAR-radar point set. After voxelization, LiRaFusion uses a simplified VoxelNet-style voxel feature encoder with output dimensionality equal to the 9-dimensional input schema. For each non-empty voxel $v$, the feature construction is explicitly split into shared geometry, LiDAR-specific attributes, and radar-specific attributes (Song et al., 2024).

The first three dimensions are the centroid coordinates obtained by mean pooling over all points in the voxel, across both modalities:

$$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$

The next two dimensions are LiDAR attributes averaged only over LiDAR points in the voxel:

$$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$

If $N_{v,L}=0$, these terms are set to zero. Radar attributes are handled analogously, but with an additional learnable projection. Let the mean radar attribute vector be

$At_r$0

LiRaFusion applies a learnable $At_r$1 linear map,

$At_r$2

and if $At_r$3, the radar portion is zero. The full voxel feature is therefore

$At_r$4

These features are processed by sparse 3D convolutions and a VoxelNet backbone to produce a BEV feature map that the paper refers to as the LiDAR feature map, even though radar information has already been injected through joint voxel encoding. In parallel, radar points are also processed by a PointPillars backbone to produce a second BEV feature map. The paper states that pillarization is preferable for radar because of noisy radar height (Song et al., 2024).

This early-fusion design differs from post hoc feature concatenation: radar attributes are embedded at the voxel-feature level before the main LiDAR backbone, so radar cues affect the learned geometric representation rather than being introduced only after modality-specific backbones have diverged.

4. Middle fusion through adaptive gated BEV aggregation

The middle fusion module adaptively fuses the two expert BEV feature maps. Let the early-fused LiDAR-stream feature map be $At_r$5 and the radar-stream feature map be $At_r$6. LiRaFusion concatenates them to form

$At_r$7

Modality-specific gates are then produced by $At_r$8 convolutions followed by sigmoids:

$At_r$9

where $Vxcomp$0 outputs $Vxcomp$1 channels and $Vxcomp$2 outputs $Vxcomp$3 channels. The fused representation is

$Vxcomp$4

The gates are channel-specific and spatially resolved, with shape $Vxcomp$5, rather than channel-constant. The paper reports that this channel-specific design outperforms channel-constant gating for BEV features: channel-constant gating yields NDS $Vxcomp$6 and mAP $Vxcomp$7, whereas channel-specific gating yields NDS $Vxcomp$8 and mAP $Vxcomp$9 (Song et al., 2024).

The interpretive analysis of the gate maps is also part of the method’s contribution. Learned gate maps assign higher LiDAR weights near the ego vehicle, where LiDAR returns are dense, and increased radar weights at far ranges, where radar’s longer effective range becomes more useful. This depth-dependent weighting is consistent with the paper’s reported gains at long range and in rainy scenes. The true-positive metrics similarly indicate that LiRaFusion improves motion-related and orientation-related estimates: compared with FUTR3D-LO, LiRaFusion reports mAOE $Vycomp$0 versus $Vycomp$1, mAVE $Vycomp$2 versus $Vycomp$3, and mAAE $Vycomp$4 versus $Vycomp$5 (Song et al., 2024).

5. Detector integration, training protocol, and computational profile

LiRaFusion is implemented within MMDetection3D and is evaluated with two heads. The first is TransHead, the transformer-based detection head associated with FUTR3D; the second is CenterHead, the center-based BEV head associated with CenterPoint. LiRaFusion does not redefine these heads. Instead, it supplies fused BEV features to the host neck and head, and training follows the head-specific objectives already used by FUTR3D and CenterPoint, namely classification and box regression losses, with direction, velocity, and attribute terms when the host head supports them (Song et al., 2024).

The evaluation uses the nuScenes official split of $Vycomp$6 training scenes and $Vycomp$7 validation scenes. Multi-sweep accumulation is used throughout. Radar-specific preprocessing remains close to the underlying dataset representation: the method uses the nuScenes radar object lists, including RCS and compensated velocities, and does not introduce additional handcrafted ghost removal beyond vendor processing. CenterHead experiments use test-time augmentation for consistency with EZFusion in the 7-class moving-object setting, and the reported results do not use model ensembling (Song et al., 2024).

The runtime overhead is modest but measurable. On an RTX A6000 GPU, the TransHead-based LiRaFusion model runs at $Vycomp$8 FPS on nuScenes validation, compared with $Vycomp$9 FPS for FUTR3D-LO, $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$0 FPS for FUTR3D-LR, and $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$1 FPS for EZFusion-LR*. This positions LiRaFusion as slightly slower than the LiDAR-only baseline while still remaining in the same operating regime (Song et al., 2024).

6. Quantitative performance and ablation evidence

The main empirical claim is that LiRaFusion improves both TransHead-based and CenterHead-based detectors on nuScenes. In the full 10-class TransHead setting on nuScenes validation, LiRaFusion reaches NDS $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$2 and mAP $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$3, compared with FUTR3D-LO at NDS $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$4, mAP $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$5; FUTR3D-LR at NDS $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$6, mAP $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$7; and EZFusion-LR* at NDS $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$8, mAP $[x, y, z, intensity, At_\ell, 0, 0, 0, 0]$9. On nuScenes test, the same configuration reports NDS $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$0 and mAP $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$1 (Song et al., 2024).

Setting	Baseline(s)	LiRaFusion
TransHead, nuScenes val, 10 classes	FUTR3D-LO: 65.74 / 59.39; FUTR3D-LR: 65.37 / 58.08; EZFusion-LR*: 65.77 / 59.24	66.69 / 60.11
CenterHead, nuScenes val, 7 moving classes	CenterPoint-7: 69.41 / 61.38; EZFusion-LR*: 69.85 / 61.85	72.16 / 65.18

In the 7-class moving-object CenterHead setting, LiRaFusion reports NDS $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$2 and mAP $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$3, exceeding CenterPoint-7 at NDS $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$4, mAP $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$5 and EZFusion-LR* at NDS $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$6, mAP $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$7. The class-wise gains are especially large for bicycle and motorcycle: bicycle AP rises to $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$8 from $[x, y, z, 0, 0, rcs, Vxcomp, Vycomp, Atr]$9 for CenterPoint-7 and $v$0 for EZFusion-LR*, while motorcycle AP rises to $v$1 from $v$2 and $v$3 respectively (Song et al., 2024).

The range- and weather-conditioned analyses reinforce the same pattern. Relative to FUTR3D-LO, LiRaFusion improves mAP from $v$4 to $v$5 in the $v$6–$v$7 m band, from $v$8 to $v$9 in the $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$0–$$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$1 m band, and from $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$2 to $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$3 in the $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$4–$$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$5 m band. Under sunny conditions, NDS/mAP increase from $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$6 to $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$7; under rainy conditions, they increase from $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$8 to $$\mu_x(v)=\frac{1}{N_v}\sum_i x_i,\qquad \mu_y(v)=\frac{1}{N_v}\sum_i y_i,\qquad \mu_z(v)=\frac{1}{N_v}\sum_i z_i.$$9. The larger rainy-scene gain suggests that radar’s robustness is being exploited effectively, although the paper does not introduce an explicit weather-aware module (Song et al., 2024).

Module ablations isolate the contribution of each fusion stage. Starting from FUTR3D-LO at NDS $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$0, mAP $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$1, mAVE $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$2, and AP(car) $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$3, LiRaFusion-early yields $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$4, $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$5, $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$6, and $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$7; LiRaFusion-middle yields $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$8, $$\bar I_L(v)=\frac{1}{N_{v,L}}\sum_j intensity_j,\qquad \bar A_{t,L}(v)=\frac{1}{N_{v,L}}\sum_j At_{\ell,j}.$$9, $N_{v,L}=0$0, and $N_{v,L}=0$1; full LiRaFusion yields $N_{v,L}=0$2, $N_{v,L}=0$3, $N_{v,L}=0$4, and $N_{v,L}=0$5. The ablation indicates that the middle gated module contributes more strongly to mAP and mAVE than early fusion alone, while the full configuration combines both effects (Song et al., 2024).

7. Position in the literature, naming ambiguity, and limitations

Within LiDAR-radar perception, LiRaFusion occupies the 3D object-detection segment of a broader literature on multimodal BEV fusion. Bi-LRFusion addresses dynamic-object detection through query-based height feature fusion, query-based BEV feature fusion, and a subsequent radar-to-LiDAR BEV fusion stage, with particular emphasis on radar height ambiguity and sparsity (Wang et al., 2023). RaLiBEV instead formulates LiDAR-radar fusion as anchor-free BEV object detection from radar range-azimuth heatmaps and LiDAR pillars, using the dense query map-based interactive transformer for BEV fusion and GACHIPS label assignment (Yang et al., 2022). LiRaNet uses early, feature-level fusion of radar, LiDAR, and HD maps in BEV for end-to-end trajectory prediction rather than 3D box detection (Shah et al., 2020). LRFusionPR is a polar-BEV LiDAR-radar fusion network for place recognition, not for detection, and explicitly distinguishes itself from LiRaFusion by focusing on descriptor retrieval with cross-attention and structure-aware distillation (Qi et al., 27 Apr 2025).

There is also a naming ambiguity outside autonomous driving. A later diffusion-model work uses “LiRaFusion” to denote a dynamic training-free fusion framework for subject and style LoRAs in image generation, which is unrelated to LiDAR-radar 3D perception (Cao et al., 17 Feb 2026). In technical contexts, disambiguation therefore depends on domain: in autonomous-driving perception, LiRaFusion usually denotes the LiDAR-radar detector introduced by Song et al.; in diffusion-model literature, the same label may refer to LoRA fusion.

The principal limitations reported for the LiDAR-radar detector are architectural and data-related rather than conceptual. LiRaFusion adds modest overhead relative to FUTR3D-LO; it depends on the nuScenes radar object lists rather than raw radar data; ghost and multipath handling are learned implicitly rather than modeled explicitly; and exact voxelization hyperparameters are inherited from the host frameworks and may affect performance (Song et al., 2024). A plausible implication is that part of the method’s effectiveness arises from a favorable alignment between its fusion design and the host detector’s representation choices. The paper identifies broader LiDAR-camera-radar integration, temporal fusion beyond multi-sweep stacking, transformer-based cross-modal fusion in BEV, and uncertainty modeling for radar measurements as natural future directions (Song et al., 2024).