LDRFusion: Multimodal Fusion in 3D Detection

Updated 7 July 2026

LDRFusion is a polysemous term covering varied 3D detection fusion methods, notably a LiDAR-dominant two-stage detector using pseudo point clouds for refinement.
The framework employs hierarchical pseudo point residual encoding and dual-stage loss optimization to enhance proposal precision and robustness on benchmarks like KITTI.
Additional interpretations include LiDAR–radar anchor-free BEV detection and minimum-information-loss fusion in multi-target tracking, underscoring its versatile applications.

Searching arXiv for papers using the term “LDRFusion” and close variants to ground the article. LDRFusion denotes multiple technically distinct fusion methods in recent arXiv literature. In its most specific and title-level usage, it refers to a LiDAR-dominant multimodal refinement framework for 3D object detection that stages LiDAR-only proposal generation ahead of LiDAR–image refinement with pseudo point clouds (Wang et al., 22 Jul 2025). In adjacent literature, the same label or near-homonymous shorthand has also been used for an anchor-box-free LiDAR–radar BEV detector derived from RaLiBEV (Yang et al., 2022), for minimum-information-loss fusion of labeled random finite set densities in distributed multitarget tracking (Gao et al., 2019), and as an informal shorthand in some LiDAR–radar place-recognition and detection summaries. The term therefore spans both embodied perception and Bayesian multi-object fusion, with the 2025 LiDAR-dominant detector providing the most explicit standalone use of the name in the cited corpus (Wang et al., 22 Jul 2025).

1. LiDAR-dominant two-stage detector

In "LDRFusion: A LiDAR-Dominant multimodal refinement framework for 3D object detection" (Wang et al., 22 Jul 2025), the framework is organized as a two-stage pipeline. The first stage relies solely on raw 3D point clouds. Points are voxelized and processed by a Voxel Feature Encoder followed by a sparse-conv 3D backbone to produce a feature volume $S^L\in\mathbb{R}^{C\times X\times Y\times Z}$ . A Region Proposal Network predicts per-anchor objectness and box offsets, yielding $N$ initial RoIs $\{o_i^L\}_{i=1}^N$ . RoI pooling extracts fixed-size features $f_i^L$ , and two MLP heads produce a box residual $r_i^L=R^L(f_i^L)$ and a classification score $c_i^L=C^L(f_i^L)$ . A Part-Sensitive Warping module refines scores via an auxiliary classification map $\mathcal X$ (Wang et al., 22 Jul 2025).

The stage-one formulation is given as

$f^{L}_i = \mathrm{Pooling}(S^L,\,o^{L}_i),\quad r^L_i = R^L\bigl(f^L_i\bigr),\quad c^L_i = C^L\bigl(f^L_i\bigr)+\mathrm{PSW}\bigl(\mathcal X,b^L_i\bigr),$

with detections $d^L_i = (r^L_i,c^L_i)$ for $i=1\ldots N$ (Wang et al., 22 Jul 2025). The first-stage losses are Focal Loss for classification and Smooth $N$ 0 for regression.

The second stage introduces RGB-derived pseudo point clouds only after LiDAR has already produced accurately localized proposals. Given an image $N$ 1, a depth-completion network $N$ 2 estimates a dense depth map $N$ 3, which is back-projected through camera intrinsics and extrinsics into a dense pseudo point cloud $N$ 4. The first-stage detections $N$ 5 are reused as cascaded RoIs $N$ 6 (Wang et al., 22 Jul 2025).

For each cascaded RoI, pseudo points are cropped, encoded by a hierarchical pseudo point residual module, pooled jointly with LiDAR features, and fused at RoI level:

$N$ 7

Final heads then produce

$N$ 8

At inference, stage-one and stage-two outputs are merged by a weighted sum

$N$ 9

followed by standard 3D NMS (Wang et al., 22 Jul 2025).

This design assigns distinct functional roles to the two modalities. LiDAR is used first for accurate localization, while pseudo point clouds are deferred to a refinement stage aimed at detecting challenging instances. The paper explicitly frames this as a response to the noise introduced by pseudo points when they are used too early in proposal-refinement pipelines (Wang et al., 22 Jul 2025).

2. Hierarchical pseudo point residual encoding

A central module in the 2025 detector is the Hierarchical Pseudo Point Residual Encoding module, abbreviated HPR (Wang et al., 22 Jul 2025). Each pseudo point $\{o_i^L\}_{i=1}^N$ 0 carries 9-dimensional attributes $\{o_i^L\}_{i=1}^N$ 1. Around each point, a neighborhood $\{o_i^L\}_{i=1}^N$ 2 is built in image space by offsets $\{o_i^L\}_{i=1}^N$ 3, with $\{o_i^L\}_{i=1}^N$ 4 (Wang et al., 22 Jul 2025).

Initialization is performed by a small MLP:

$\{o_i^L\}_{i=1}^N$ 5

At each iteration $\{o_i^L\}_{i=1}^N$ 6, the module computes both feature and positional residuals:

$\{o_i^L\}_{i=1}^N$ 7

A learned weighting $\{o_i^L\}_{i=1}^N$ 8 is then applied, and neighbor features are updated as

$\{o_i^L\}_{i=1}^N$ 9

Aggregation across the neighborhood uses another MLP:

$f_i^L$ 0

The final pseudo-point representation concatenates the outputs of all iterations:

$f_i^L$ 1

The paper states that HPR is intended to enhance the representation of local structures in pseudo point clouds by encoding neighborhood sets using both feature and positional residuals (Wang et al., 22 Jul 2025). A plausible implication is that this residualized neighborhood modeling is designed to compensate for the structural brittleness of depth-completed pseudo geometry relative to native LiDAR measurements.

3. Optimization, losses, and proposal fusion

The second-stage training objective in the 2025 framework includes auxiliary LiDAR and pseudo-point losses together with the multimodal fusion loss. The auxiliary losses $f_i^L$ 2 and $f_i^L$ 3 each use Focal Loss for classification and Smooth $f_i^L$ 4 for regression. The multimodal fusion loss $f_i^L$ 5 uses Focal for classification and Smooth $f_i^L$ 6 plus GIoU for regression (Wang et al., 22 Jul 2025).

The full multi-stage loss is

$f_i^L$ 7

with $f_i^L$ 8 (Wang et al., 22 Jul 2025).

Proposal fusion is deliberately instance-level rather than an earlier dense cross-modal blending. The paper gives the final merging rule as

$f_i^L$ 9

followed by standard 3D non-maximum suppression (Wang et al., 22 Jul 2025). This makes the framework “LiDAR-dominant” not only in nomenclature but in optimization topology: proposals are anchored by the first stage, and multimodal refinement operates downstream of that anchor.

The authors contrast this strategy with prior LiDAR–camera fusion methods that construct spatial pseudo point clouds as auxiliary input and adopt a proposal-refinement framework throughout. Their stated concern is that introducing pseudo points inevitably brings noise, potentially resulting in inaccurate predictions (Wang et al., 22 Jul 2025). The architecture responds by restricting the role of pseudo points to second-stage refinement rather than first-stage proposal genesis.

4. Empirical results on KITTI

The 2025 LDRFusion paper reports results on KITTI across single-class and multi-class settings (Wang et al., 22 Jul 2025). For single-class car detection on the test set using 3D AP at $r_i^L=R^L(f_i^L)$ 0, the reported numbers are:

Method	Easy	Mod.	Hard
Voxel R-CNN	90.90	81.62	77.06
SFD (fusion)	91.73	84.76	77.92
LDRFusion (Ours)	91.92	85.47	80.43

On the validation set, the paper reports both $r_i^L=R^L(f_i^L)$ 1 and $r_i^L=R^L(f_i^L)$ 2 car AP:

Metric	Easy	Mod.	Hard
3D-AP ( $r_i^L=R^L(f_i^L)$ 3)	96.00	89.06	86.47
3D-AP ( $r_i^L=R^L(f_i^L)$ 4)	90.24	87.72	86.21

For multi-class validation at $r_i^L=R^L(f_i^L)$ 5, LDRFusion reports car AP of 95.86, 88.77, and 86.39 for Easy, Moderate, and Hard; pedestrian AP of 73.67, 66.12, and 60.06; cyclist AP of 91.32, 75.42, and 70.75; and an overall mAP of 78.71 (Wang et al., 22 Jul 2025).

The paper also reports ablations showing that a cascade refinement added to the baseline raises 3D-AP mAP from 89.92 to 90.35, and that adding HPR alone yields 90.18, while “+ Cascade + HPR (Ours)” reaches 90.51 (Wang et al., 22 Jul 2025). It further states that on “Hard” car instances, LDRFusion improves over LiDAR-only by approximately 3.3 percentage points, from 77.06 to 80.43 on the test set.

Inference throughput is reported as approximately 10 FPS on a pair of 3090 GPUs (Wang et al., 22 Jul 2025). This positions the method as a refinement-oriented detector that attempts to preserve real-time viability while retaining a two-stage design.

5. Other uses of the name in LiDAR–radar perception

The label “LDRFusion” is also used in a different sense in the expanded description of "RaLiBEV: Radar and LiDAR BEV Fusion Learning for Anchor Box Free Object Detection Systems" (Yang et al., 2022). There, “LDRFusion” refers to an anchor-box-free BEV fusion detector using compact BEV-raster inputs, anchor-free single-point label assignment via GACHIPS, and a symmetric interactive transformer denoted DQMITBF (Yang et al., 2022).

In that formulation, radar range–azimuth heatmaps are reprojected into Cartesian BEV on a fixed $r_i^L=R^L(f_i^L)$ 6 grid, forming $r_i^L=R^L(f_i^L)$ 7, while LiDAR point clouds are pillarized into a $r_i^L=R^L(f_i^L)$ 8 grid with $r_i^L=R^L(f_i^L)$ 9-resolution $c_i^L=C^L(f_i^L)$ 0, yielding $c_i^L=C^L(f_i^L)$ 1 (Yang et al., 2022). The detector attaches a multi-scale head in the style of YOLOv4/Centripoint and predicts a 1-channel center heatmap and a 6-channel regression output encoding $c_i^L=C^L(f_i^L)$ 2 (Yang et al., 2022).

The same source describes four label-assignment strategies—DIPS, GAHPS, GAHIPS, and GACHIPS—of which GACHIPS is presented as the consistent-cost formulation that forces the same single grid cell to own both classification and box-regression loss (Yang et al., 2022). Under the hardest setting, train = Clear+Fog and test = Fog at IoU = 0.8, “LDRFusion with GACHIPS alone achieves AP = 93.7%, and with the full DQMITBF fusion it reaches AP = 94.1%,” while the cited prior art ST-MVDNet++ scores 75.1% at the same operating point (Yang et al., 2022).

This usage differs categorically from the 2025 LiDAR–camera LDRFusion. It is single-stage rather than two-stage, LiDAR–radar rather than LiDAR–image, and center-based anchor-free rather than proposal-refinement-based. The shared name therefore does not denote a single canonical architecture across object-detection literature.

6. Minimum-information-loss LDRFusion in labeled RFS tracking

A further and conceptually separate use appears in "Fusion of labeled RFS densities with minimum information loss" (Gao et al., 2019). Here LDRFusion denotes a minimum-information-loss fusion method for labeled random finite set densities in distributed multi-sensor multitarget tracking rather than deep neural perception.

The method seeks a fused density $c_i^L=C^L(f_i^L)$ 3 minimizing weighted information loss:

$c_i^L=C^L(f_i^L)$ 4

with $c_i^L=C^L(f_i^L)$ 5, $c_i^L=C^L(f_i^L)$ 6, and

$c_i^L=C^L(f_i^L)$ 7

The unconstrained solution is the linear opinion pool

$c_i^L=C^L(f_i^L)$ 8

(Gao et al., 2019).

To preserve conjugacy, the paper constrains the fused density to remain in the same family as the local densities. For M $c_i^L=C^L(f_i^L)$ 9-GLMB densities, the fused JEP is

$\mathcal X$ 0

with track-wise mixture weights

$\mathcal X$ 1

and fused per-label densities

$\mathcal X$ 2

For LMB densities, the fused parameters are

$\mathcal X$ 3

with

$\mathcal X$ 4

(Gao et al., 2019).

The same framework handles differing fields of view through subspace decomposition of the global label space and addresses label mismatching via rank assignment optimization. For two LMBs, a cost matrix is constructed from track-to-track divergences such as Jensen–Shannon divergence or Cauchy–Schwarz divergence, and the linear assignment is solved by the Hungarian algorithm (Gao et al., 2019).

Simulation results reported in that source indicate that, in same-FoV settings, MIL and GCI are comparable at high $\mathcal X$ 5, while at moderate or low $\mathcal X$ 6 of $\mathcal X$ 7– $\mathcal X$ 8, MIL outperforms GCI in localization and cardinality accuracy. In different-FoV settings, GCI “zeros out” tracks outside the common FoV, whereas MIL preserves exclusive-FoV tracks via subspace decomposition and RAO (Gao et al., 2019).

This probabilistic LDRFusion is unrelated to the 2025 LiDAR-dominant detector except by name. Its domain is consensus multitarget tracking, its objects are labeled-RFS densities, and its operators are KLD minimization, family-preserving pooling, and divergence-based label assignment.

7. Scope, ambiguity, and relation to adjacent fusion research

The term’s ambiguity is amplified by nearby literature that does not always use “LDRFusion” as an official paper title but employs closely related LiDAR–radar fusion formulations. "Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection" introduces bidirectional interaction through LiDAR→Radar query-based height features and Radar→LiDAR query-based BEV fusion, then merges enriched BEV maps before a CenterPoint-style head (Wang et al., 2023). On nuScenes val, it reports 67.5% mAP and 69.8% NDS versus 62.0% mAP and 66.1% NDS for CenterPoint, with the largest gains on dynamic classes (Wang et al., 2023). "L4DR: LiDAR-4DRadar Fusion for Weather-Robust 3D Object Detection" instead emphasizes Multi-Modal Encoding, Foreground-Aware Denoising, an IM $\mathcal X$ 9 backbone, and Multi-Scale Gated Fusion, reporting up to 20.0% 3D mAP improvement over LiDAR-only under simulated fog on VoD (Huang et al., 2024).

Likewise, "DeepFusion: A Robust and Modular 3D Object Detector for Lidars, Cameras and Radars" adopts a modular BEV alignment-and-addition scheme for LiDAR, camera, and radar feature maps (Drews et al., 2022), while "LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving" uses early BEV fusion and mid-level gated fusion for weather-type recognition over nine classes on MSU-4S (Albashir et al., 23 Mar 2026). These works form the broader technical context in which the 2025 LDRFusion detector situates its LiDAR-first, pseudo-point-later design.

A common misconception would be to treat LDRFusion as a single standardized architecture across autonomous-driving perception and multitarget tracking. The cited literature does not support that reading. Instead, the label is polysemous: it names at least one LiDAR-dominant two-stage detector (Wang et al., 22 Jul 2025), one minimum-information-loss labeled-RFS fusion rule (Gao et al., 2019), and a LiDAR–radar anchor-free BEV detector in an expanded RaLiBEV description (Yang et al., 2022). This suggests that, in technical discussion, “LDRFusion” should be disambiguated by domain and citation rather than assumed to designate a unique method.