Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving

Abstract: Long-range 3D object detection is critical for safe autonomous driving at highway speeds, yet existing radar-camera fusion methods remain limited at extended ranges. BEV-based methods capture scene-level context but incur rapidly growing computation and often lose fine-grained object detail, while query-based methods are efficient but provide limited scene-level context. Temporal fusion further requires both multi-frame accumulation for sparse distant observations and object-level motion modeling for fast-moving objects. We propose Horizon3D, a sparse radar-camera fusion framework for long-range 3D object detection that combines Gaussian primitives with sparse BEV features. Horizon3D initializes Gaussian primitives at radar- and camera-estimated object keypoints using Keypoint-Guided Gaussian Initialization, refines them through Object-Centric Sparse Fusion, and splats them onto the BEV plane to fuse object-level detail with sparse radar BEV context. It further introduces Dual-Path Temporal Fusion, which aggregates temporal cues through a BEV path for scene-level accumulation and a Gaussian path for object-level motion propagation. Experiments on TruckScenes show that Horizon3D achieves state-of-the-art radar-camera 3D detection performance. On the validation set, it outperforms the previous best method by +3.0 NDS and +1.6 mAP while maintaining competitive inference speed.

• The paper introduces a hybrid representation combining Gaussian primitives with sparse BEV features to improve long-range 3D object detection.
• It employs keypoint-guided Gaussian initialization and dual-path temporal fusion for efficient sensor fusion and precise object-level motion modeling.
• Experimental results on TruckScenes show state-of-the-art accuracy with competitive inference speed, validating its robustness for highway autonomous driving.

Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving

Motivation and Problem Formulation

Long-range 3D object detection underpins the operational safety of autonomous vehicles, specifically in highway environments where early detection is critical given extended stopping distances. The primary challenge with radar-camera fusion at large ranges is sensor sparsity: distant objects yield few radar points and occupy minimal image pixels, rendering per-frame information insufficient. BEV (Bird’s Eye View)-based methods provide scene-level context but entail high computational and memory cost, scaling poorly with distance due to dense spatial grids. Conversely, query-centric methods furnish efficient object-level encoding but compromise on scene-level context. Temporal fusion strategies further compound these issues, as fast-moving objects require precise motion compensation, and sparsity demands robust long-range information accumulation.

Horizon3D Architecture

The Horizon3D framework introduces a hybrid representation merging Gaussian primitives and sparse BEV features. The system comprises three principal modules:

1. Keypoint-Guided Gaussian Initialization (KGGI): This module estimates object keypoints from both radar and camera modalities, judiciously initializing Gaussian primitives at these locations. Unlike prior work that deploys numerous Gaussians across the scene, KGGI places Gaussians only at estimated object centers, providing efficient object-centric aggregation.
2. Object-Centric Sparse Fusion (OCSF): OCSF iteratively refines Gaussian parameters via multimodal cross-attention. Refined Gaussians are splatted onto the BEV plane to generate an object-centric BEV representation. This sparse BEV map is fused with radar BEV features, capturing both object characteristics and broader scene context.
3. Dual-Path Temporal Fusion (DPTF): Temporal information is integrated via two complementary strategies: a BEV path aggregates multi-frame BEV features to mitigate sparsity, while a Gaussian path propagates and warps Gaussian primitives across frames for explicit object-level motion modeling. Both paths leverage velocity predictions for compensation, ensuring temporal alignment of fast-moving objects.

   Figure 1: Overall architecture of Horizon3D, illustrating the multi-modal encoding, keypoint-guided Gaussian initialization, sparse fusion, and dual-path temporal fusion pipeline.

Key Technical Innovations

Gaussian-Based Sparse Object Representation

Horizon3D’s Gaussians are initialized based on radar and camera keypoint extraction, using VoxelNeXt for radar and dedicated image encoders for camera inputs. Each Gaussian encapsulates location, scale, orientation, opacity, velocity, and features, with random and historical Gaussians supplementing coverage. Feature refinement occurs through deformable cross-attention, aggregating modality-specific context at the Gaussian’s spatial extent, and producing a compact yet expressive set representing foreground objects.

Sparse Gaussian Splatting and BEV Fusion

Refined Gaussians are projected onto the BEV plane, with their contributions forming a supervised occupancy map. Gaussian-weighted pooling provides object-centric BEV features, subsequently fused with sparse radar features via a gating network and submanifold sparse convolutions. This design achieves the dual objective of scene-level context and object-level detail without the inefficiency of dense BEV construction.

Figure 2: Object-centric splatting and velocity-guided temporal alignment in Horizon3D, highlighting BEV and Gaussian temporal fusion mechanisms.

Dual-Path Temporal Fusion

The BEV path employs occupancy thresholds to identify object-centric cells, storing features and cell-wise velocities in memory. Historical frames are aligned to current coordinates using ego-motion and per-cell velocity prediction, and temporally fused via sparse convolutions. The Gaussian path uses farthest point sampling to maintain spatial and object diversity in memory, propagating Gaussians across frames and warping them based on predicted velocities and ego-motion. Temporal self-attention within the Gaussian encoder further facilitates cross-frame information exchange, addressing residual misalignment.

Experimental Evaluation

Benchmarking on TruckScenes

Extensive evaluations on the TruckScenes dataset demonstrate Horizon3D’s superiority over prior radar-camera fusion and even state-of-the-art LiDAR-based approaches. On the validation split, Horizon3D with ResNet-50 backbone achieves an NDS of 37.4 and mAP of 23.6, outperforming BEVFusion (NDS 30.4, mAP 18.2), CRT-Fusion, and RCTrans. With VoVNet-99, Horizon3D surpasses SpaRC by +3.0 NDS and +1.6 mAP, reflecting both architectural flexibility and robustness with varying backbones.

Distance-wise analysis reveals that Horizon3D maintains top performance across all ranges, especially for 25–50m (+6.5 mAP vs. BEVFusion) and 100–150m (+1.5 mAP, +4.5 NDS vs. BEVFusion), validating effective compensation for long-range sparsity.

Efficiency and Ablation Analyses

Horizon3D sustains competitive inference speed (8.5 FPS, 3.24 GB memory) despite high accuracy, owing to its selective sparse processing. Ablation studies highlight that all modules (KGGI, OCSF, DPTF) are complementary: Gaussian initialization at sensor-guided keypoints, bounded box enlargement for occupancy supervision, velocity compensation in both fusion paths, and optimal sampling strategies (farthest point sampling) all yield measurable improvements in detection metrics.

Qualitative Results

Visualizations show that Horizon3D focuses activations on true object regions, yielding accurate temporal alignment and velocity estimation. Dense BEV methods (CRT-Fusion) diffused activations, misaligning predictions over background regions. The hybrid representation preserves the spatial precision necessary for high-speed, long-range scenarios.

Figure 3: Qualitative comparison of BEV feature maps and velocity predictions, demonstrating Horizon3D’s precise object localization and motion modeling.

Practical and Theoretical Implications

Horizon3D effectively reconciles the dichotomy between scene-level context and object-level detail in radar-camera fusion. By leveraging hybrid representations and sparse temporal aggregation, the method enables efficient, scalable, and accurate perception—key traits for highway-speed autonomous driving. The design is agnostic to backbone selection and adapts to sparse, noisy sensor regimes, promoting extensibility to future sensor modalities or broader application domains.

Theoretically, the use of keypoint-guided Gaussian initialization, object-centric splatting, dual-path temporal fusion, and explicit velocity compensation represent an overview of emerging themes in hybrid representation learning and motion-aware multi-modal fusion. Potential future directions include hierarchical temporal aggregation, unsupervised keypoint refinement, tighter integration with tracking pipelines, and extension to multi-agent or multi-sensor settings.

Conclusion

Horizon3D constitutes a sparse radar-camera fusion architecture built for long-range 3D perception in autonomous driving. By merging Gaussian primitives with sparse BEV features through the KGGI, OCSF, and DPTF modules, the framework achieves state-of-the-art accuracy and efficiency on TruckScenes. Its dual-path temporal fusion with explicit velocity compensation addresses both per-frame sparsity and object-level motion, enabling robust detection of small and fast-moving objects at extended ranges. Horizon3D sets a new standard for radar-camera fusion, laying the foundation for safe, scalable, and high-performance autonomous driving perception systems (2606.31096).