GSFusion: Multi-Modal Fusion in 3D Mapping & SLAM

Updated 7 July 2026

GSFusion is a multi-modal fusion paradigm that integrates LiDAR, camera, and inertial data to enhance 3D semantic occupancy and mapping.
It is applied to tasks such as LiDAR-camera semantic occupancy prediction, online RGB-D mapping with TSDF and Gaussian splatting, and globally optimized LiDAR-Inertial-Visual SLAM.
Innovations in GSFusion include explicit cross-modal feature fusion, dual-map strategies, and global optimization techniques that improve efficiency and rendering quality.

Searching arXiv for exact and related uses of “GSFusion” to ground the article in current papers. Searching arXiv for "GSFusion". GSFusion is a name used for multiple research artifacts rather than a single unified method family. In the literature provided here, it appears in at least three distinct senses: as a Geometric- and Semantic-aware Fusion module inside a LiDAR-camera 3D semantic occupancy predictor, as an online RGB-D mapping system that combines 3D Gaussian splatting with TSDF fusion, and as a globally optimized LiDAR-Inertial-Visual mapping system for Gaussian splatting (Pan et al., 2024, Wei et al., 2024, Park et al., 31 Jul 2025). The shared motif is fusion across heterogeneous representations or sensing modalities, but the technical meaning of the term varies substantially across autonomous driving, RGB-D reconstruction, and SLAM.

1. Terminological scope and major usages

In "Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction" (Pan et al., 2024), GSFusion denotes a Geometric- and Semantic-aware Fusion module. The paper states that it is used to "explicitly enhance LiDAR features by incorporating neighboring camera features through a K-nearest neighbors (KNN) search" within a LiDAR-camera 3D semantic occupancy prediction framework (Pan et al., 2024).

In "GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion" (Wei et al., 2024), GSFusion denotes a full hybrid online RGB-D mapping system. Its central design combines a TSDF volumetric map with a 3D Gaussian map, using quadtree-based image subdivision and TSDF-guided Gaussian insertion to reduce the number of initialized splats while maintaining rendering quality (Wei et al., 2024).

In "GSFusion: Globally Optimized LiDAR-Inertial-Visual Mapping for Gaussian Splatting" (Park et al., 31 Jul 2025), GSFusion denotes an online LiDAR-Inertial-Visual mapping system for 3D Gaussian splatting. Its defining components are a surfel-to-surfel constraint in global pose-graph optimization, a pixel-aware Gaussian initialization strategy, and a bounded sigmoid constraint for Gaussian scale control (Park et al., 31 Jul 2025).

This multiplicity of uses means that the term is best understood as a label reused across subfields rather than as a stable acronym with one canonical expansion. A common misconception is to assume that all papers titled or labeled "GSFusion" are direct variants of one another; the record here indicates instead that they address different tasks, use different sensor suites, and optimize different objectives.

Within Co-Occ, the broader problem is multi-modal semantic occupancy prediction for autonomous driving, specifically LiDAR-camera 3D semantic occupancy prediction (Pan et al., 2024). The paper motivates the method by citing "modality heterogeneity, modality misalignment, and insufficient modality interactions" during fusion, which can lead to the loss of important geometric and semantic information (Pan et al., 2024).

The Co-Occ framework couples "explicit LiDAR-camera feature fusion with implicit volume rendering regularization" (Pan et al., 2024). In this design, GSFusion is the explicit fusion component. The abstract states that the method first proposes a Geometric- and Semantic-aware Fusion (GSFusion) module and that the module enhances LiDAR features by incorporating neighboring camera features via a K-nearest neighbors (KNN) search (Pan et al., 2024). Afterward, the fused feature is projected back to the image planes using volume rendering in order to reconstruct color and depth maps, which are supervised respectively by camera images and LiDAR-derived depth estimations (Pan et al., 2024).

The same abstract attributes a second function to volume rendering: it "can proficiently bridge the gap between 3D LiDAR sweeps and 2D images while serving as a physical regularization to enhance LiDAR-camera fused volumetric representation" (Pan et al., 2024). This suggests that, in Co-Occ, GSFusion should not be read as an isolated fusion block but as one half of a two-part design in which explicit feature coupling is complemented by image-plane reconstruction constraints.

A limitation of the present record is that it does not provide the module equations, losses, or ablation tables for GSFusion inside Co-Occ. The concrete claims available are therefore restricted to the role stated in the abstract: LiDAR-feature enhancement through neighboring camera features and KNN-based cross-modal interaction (Pan et al., 2024).

3. GSFusion as online RGB-D mapping where Gaussian splatting meets TSDF fusion

The RGB-D version of GSFusion addresses online RGB-D mapping by combining the structural advantages of traditional volumetric fusion with the visualization quality of 3D Gaussian splatting (Wei et al., 2024). The paper explicitly argues that volumetric fusion preserves spatial structure but lacks realism, whereas Gaussian-based reconstruction often suffers from artifacts, inconsistencies with the underlying 3D structure, and difficulty with real-time optimization because of the massive number of Gaussian parameters that must be updated (Wei et al., 2024).

Its central idea is to avoid treating 3D Gaussians as a standalone representation. Instead, GSFusion maintains two parallel maps: an octree-based TSDF grid and a 3D Gaussian map (Wei et al., 2024). The TSDF side is built on Supereight2, with each voxel storing a TSDF value and a weight, while the Gaussian side uses standard oriented ellipsoidal primitives with center, rotation, scale, opacity, and spherical harmonics coefficients (Wei et al., 2024).

The system processes an RGB-D stream frame by frame. First, depth is integrated into the TSDF. Second, new Gaussians are initialized from a quadtree decomposition of the RGB image based on contrast. Third, the Gaussian map is optimized online using a photometric loss between the rendered image and the input RGB frame (Wei et al., 2024). The Gaussian initialization step is the paper’s main efficiency mechanism. Rather than initializing splats densely over pixels, GSFusion back-projects quadtree cell centers and checks the nearest TSDF voxel; if the voxel weight equals 1, the location is treated as newly allocated and a Gaussian is inserted (Wei et al., 2024).

The rendering formulation follows projected 2D Gaussian splats with front-to-back alpha compositing. For pixel $\mathbf{u}$ , the rendered color is

$\hat{\mathbf{I}}_k[\mathbf{u}] = \sum_{i=1}^{N} \mathbf{c}_i \alpha_i \hat{G}_i(\mathbf{u}) \prod_{j=1}^{i-1} (1 - \alpha_j \hat{G}_j(\mathbf{u})).$

The online optimization objective is an $L_1$ photometric loss,

$L = \|\mathbf{I}_k - \hat{\mathbf{I}}_k\|_1,$

with keyframe maintenance used to allocate more optimization to informative frames and randomly revisited keyframes in order to reduce forgetting (Wei et al., 2024).

A notable feature of this formulation is that geometry is not merely auxiliary; TSDF structure constrains where splats are inserted and thereby suppresses redundancy. The paper frames this as the main reason the approach produces a compact Gaussian map with fewer artifacts (Wei et al., 2024).

4. GSFusion as globally optimized LiDAR-Inertial-Visual Gaussian mapping

The LiDAR-Inertial-Visual version of GSFusion targets online SLAM and mapping with 3D Gaussian splatting in large, unbounded, real-world environments (Park et al., 31 Jul 2025). The motivation differs from the RGB-D system. Here the authors emphasize limitations of camera-based Gaussian SLAM in low-texture areas, bad lighting, outdoors, and short-range operation, while also arguing that LiDAR-based Gaussian systems still lack sufficiently dense global alignment for photorealistic quality (Park et al., 31 Jul 2025).

The architecture is a dual-map system consisting of a sparse surfel map for localization and a dense Gaussian map for rendering (Park et al., 31 Jul 2025). At runtime, synchronized LiDAR, camera, and IMU streams feed a front-end that uses IMU preintegration, geometric LiDAR constraints, and photometric visual constraints to estimate motion and locally refine the map. The back-end maintains a global pose graph, performs loop closure, runs incremental global optimization with iSAM2, and updates the affected Gaussians after pose correction (Park et al., 31 Jul 2025).

The odometry objective combines point-to-plane ICP and photometric terms:

$L_{odom} = \sum\|e_{icp}\|^{2} + \lambda\sum\|e_{photo}\|^{2}.$

For global consistency, the system adds a surfel-to-surfel alignment constraint to the pose graph, motivated by the claim that ordinary pose-graph optimization is too coarse for photorealistic 3DGS (Park et al., 31 Jul 2025). The global objective is

$L_{\text{global}} = \sum_{i, j} \| e_{\text{pose}(i, j)} \|^2 + \lambda_{\text{surfel}} \sum_{i, j} \| e_{\text{surfel}(i, j)} \|^2.$

Two additional mechanisms are central. The first is pixel-aware Gaussian initialization. Instead of deriving Gaussian scale only from local 3D point density, the method computes a projected 2D covariance and an effective projected radius

$r_{eff} = \sqrt[4]{\det(\boldsymbol\Sigma_{2D})},$

then rescales the Gaussian so that its projected footprint matches a target pixel size (Park et al., 31 Jul 2025). The second is a bounded sigmoid scale constraint

$\sigma_{\text{bounded}} = \sigma_{\min} + (\sigma_{\max} - \sigma_{\min}) \cdot \text{sigmoid}(s),$

introduced to prevent uncontrolled Gaussian growth in poorly observed regions such as sky or background (Park et al., 31 Jul 2025).

This version of GSFusion therefore uses the term "fusion" in a broader systems sense: LiDAR provides geometry, IMU provides motion propagation, and the camera provides appearance, while the map itself integrates surfel-based structure and Gaussian-based rendering (Park et al., 31 Jul 2025).

5. Comparative structure of the three usages

Although the three systems solve different problems, they can be compared along the axes of task, sensing, representation, and optimization.

Usage of GSFusion	Task	Core fusion object
Co-Occ (Pan et al., 2024)	3D semantic occupancy prediction	LiDAR features enhanced with neighboring camera features
RGB-D GSFusion (Wei et al., 2024)	Online RGB-D mapping	TSDF volumetric map combined with 3D Gaussian map
LiDAR-Inertial-Visual GSFusion (Park et al., 31 Jul 2025)	Online Gaussian SLAM/mapping	LiDAR, IMU, camera, surfel map, and Gaussian map

In Co-Occ, the fusion target is a volumetric semantic representation regularized through feature-space volume rendering (Pan et al., 2024). In the RGB-D mapping system, the fusion target is a hybrid map representation in which TSDF preserves geometry and 3DGS supplies photorealistic rendering (Wei et al., 2024). In the LiDAR-Inertial-Visual system, the fusion target is both sensor-level and representation-level, since the method jointly optimizes multi-sensor constraints and updates a dense Gaussian map from globally corrected poses (Park et al., 31 Jul 2025).

A plausible implication is that the repeated reuse of the term reflects a broader trend: "fusion" in contemporary 3D vision increasingly means not only combining sensors, but also coupling explicit geometric representations with appearance-oriented renderers or semantic predictors.

6. Quantitative results and empirical claims

The Co-Occ abstract reports that "extensive experiments on the popular nuScenes and SemanticKITTI benchmarks verify the effectiveness of our Co-Occ for 3D semantic occupancy prediction," but no numerical results for the GSFusion module itself are available in the supplied record (Pan et al., 2024).

The RGB-D GSFusion paper reports strong efficiency gains relative to Gaussian SLAM baselines. On ScanNet++, the paper reports 6.14 FPS, 29.3 MB model size, and 2810 MB GPU memory for GSFusion, compared with 0.19 FPS, 206.3 MB, and 4417 MB GPU memory for SplaTAM, and 1.29 FPS, 111.8 MB, and 3906 MB GPU memory for RTG-SLAM (Wei et al., 2024). On ScanNet++ with global optimization, representative rendering metrics are 28.84 PSNR / 0.897 SSIM / 0.138 LPIPS on training views and 25.45 PSNR / 0.848 SSIM / 0.216 LPIPS on novel views (Wei et al., 2024). On Replica, the paper reports 34.65 PSNR / 0.949 SSIM / 0.056 LPIPS, exceeding the cited RTG-SLAM results 33.38 / 0.929 / 0.069 (Wei et al., 2024).

The LiDAR-Inertial-Visual GSFusion paper emphasizes rendering quality, compactness, and map-building efficiency. It reports that Ours(1pixel) achieves PSNR values of 27.5 on CBD02, 29.2 on Retail, 27.8 on SYSU01, and 20.0 on airport03, while Ours(5pixel) often delivers nearly comparable PSNR with much smaller maps (Park et al., 31 Jul 2025). It reports map sizes such as 31.0 MB on Retail, 24.7 MB on SYSU01, and 26.8 MB on Reception for the 5-pixel setting, and cites 96 seconds on Reception as the fastest runtime among compared methods on the custom datasets (Park et al., 31 Jul 2025).

These empirical sections reveal that the two mapping papers use "GSFusion" to denote systems explicitly optimized for the quality-efficiency trade-off in online Gaussian reconstruction. By contrast, Co-Occ’s GSFusion is a module-level contribution inside a semantic occupancy framework rather than a standalone mapper (Pan et al., 2024).

Several nearby acronyms in the record are not exact instances of GSFusion but are relevant to disambiguation. "Gate-Shift-Fuse for Video Action Recognition" introduces GSF, sometimes described in the supplied details as GSFusion in some contexts, but the paper itself defines the module as Gate-Shift-Fuse rather than GSFusion (Sudhakaran et al., 2022). "GS-FUSE: Granger-Supervised Gated Fusion and Multi-Granularity Alignment for Event-Driven Financial Forecasting" is another distinct term, hyphenated and used for multimodal financial forecasting rather than 3D perception (Zhang et al., 27 May 2026). The seismic foundation model paper is explicitly named GSFM, not GSFusion (Cheng et al., 3 Feb 2025).

This naming overlap can obscure the technical content if the term is cited without context. In 3D vision specifically, however, the exact GSFusion label is associated in the supplied record with methods that attempt to reconcile complementary strengths: LiDAR and camera in semantic occupancy prediction (Pan et al., 2024), TSDF geometry and Gaussian rendering in RGB-D mapping (Wei et al., 2024), and LiDAR, inertial, visual, surfel, and Gaussian representations in globally optimized SLAM (Park et al., 31 Jul 2025).

Taken together, these usages show that GSFusion functions less as a single canonical algorithm than as a recurring design pattern centered on structured fusion. In one case, the emphasis is geometric-semantic cross-modal feature injection; in another, volumetric-geometric scaffolding for compact Gaussian mapping; in another, globally consistent multi-sensor Gaussian SLAM. The term is therefore best interpreted with its immediate paper context, task definition, and representation choices made explicit.