---
title: 'GSFusion: Multi-Modal Fusion in 3D Mapping & SLAM'
url: https://www.emergentmind.com/topics/gsfusion
type: topic
---

# GSFusion: Multi-Modal Fusion in 3D Mapping & SLAM

Searching arXiv for exact and related uses of “GSFusion” to ground the article in current papers.
Searching arXiv for "GSFusion".
GSFusion is a name used for multiple research artifacts rather than a single unified method family. In the literature provided here, it appears in at least three distinct senses: as a **Geometric- and Semantic-aware Fusion** module inside a LiDAR-camera 3D semantic occupancy predictor, as an **online RGB-D mapping** system that combines 3D Gaussian splatting with TSDF fusion, and as a **globally optimized LiDAR-Inertial-Visual mapping** system for Gaussian splatting [2404.04561], [2408.12677], [2507.23273]. The shared motif is fusion across heterogeneous representations or sensing modalities, but the technical meaning of the term varies substantially across autonomous driving, RGB-D reconstruction, and SLAM.

## 1. Terminological scope and major usages

In "Co-Occ: Coupling Explicit Feature Fusion with Volume Rendering Regularization for Multi-Modal 3D Semantic Occupancy Prediction" [2404.04561], GSFusion denotes a **Geometric- and Semantic-aware Fusion** module. The paper states that it is used to "explicitly enhance LiDAR features by incorporating neighboring camera features through a K-nearest neighbors (KNN) search" within a LiDAR-camera 3D semantic occupancy prediction framework [2404.04561].

In "GSFusion: Online RGB-D Mapping Where Gaussian Splatting Meets TSDF Fusion" [2408.12677], GSFusion denotes a full **hybrid online RGB-D mapping system**. Its central design combines a TSDF volumetric map with a 3D Gaussian map, using quadtree-based image subdivision and TSDF-guided Gaussian insertion to reduce the number of initialized splats while maintaining rendering quality [2408.12677].

In "GSFusion: Globally Optimized LiDAR-Inertial-Visual Mapping for Gaussian Splatting" [2507.23273], GSFusion denotes an **online LiDAR-Inertial-Visual mapping system** for 3D Gaussian splatting. Its defining components are a surfel-to-surfel constraint in global pose-graph optimization, a pixel-aware Gaussian initialization strategy, and a bounded sigmoid constraint for Gaussian scale control [2507.23273].

This multiplicity of uses means that the term is best understood as a label reused across subfields rather than as a stable acronym with one canonical expansion. A common misconception is to assume that all papers titled or labeled "GSFusion" are direct variants of one another; the record here indicates instead that they address different tasks, use different sensor suites, and optimize different objectives.

## 2. GSFusion in multi-modal 3D semantic occupancy prediction

Within Co-Occ, the broader problem is **multi-modal semantic occupancy prediction** for autonomous driving, specifically LiDAR-camera 3D semantic occupancy prediction [2404.04561]. The paper motivates the method by citing "modality heterogeneity, modality misalignment, and insufficient modality interactions" during fusion, which can lead to the loss of important geometric and semantic information [2404.04561].

The Co-Occ framework couples "explicit LiDAR-camera feature fusion with implicit volume rendering regularization" [2404.04561]. In this design, GSFusion is the explicit fusion component. The abstract states that the method first proposes a **Geometric- and Semantic-aware Fusion (GSFusion) module** and that the module enhances LiDAR features by incorporating neighboring camera features via a **K-nearest neighbors (KNN) search** [2404.04561]. Afterward, the fused feature is projected back to the image planes using volume rendering in order to reconstruct color and depth maps, which are supervised respectively by camera images and LiDAR-derived depth estimations [2404.04561].

The same abstract attributes a second function to volume rendering: it "can proficiently bridge the gap between 3D LiDAR sweeps and 2D images while serving as a physical regularization to enhance LiDAR-camera fused volumetric representation" [2404.04561]. This suggests that, in Co-Occ, GSFusion should not be read as an isolated fusion block but as one half of a two-part design in which explicit feature coupling is complemented by image-plane reconstruction constraints.

A limitation of the present record is that it does not provide the module equations, losses, or ablation tables for GSFusion inside Co-Occ. The concrete claims available are therefore restricted to the role stated in the abstract: LiDAR-feature enhancement through neighboring camera features and KNN-based cross-modal interaction [2404.04561].

## 3. GSFusion as online RGB-D mapping where Gaussian splatting meets TSDF fusion

The RGB-D version of GSFusion addresses **online RGB-D mapping** by combining the structural advantages of traditional volumetric fusion with the visualization quality of 3D Gaussian splatting [2408.12677]. The paper explicitly argues that volumetric fusion preserves spatial structure but lacks realism, whereas Gaussian-based reconstruction often suffers from artifacts, inconsistencies with the underlying 3D structure, and difficulty with real-time optimization because of the massive number of Gaussian parameters that must be updated [2408.12677].

Its central idea is to avoid treating 3D Gaussians as a standalone representation. Instead, GSFusion maintains two parallel maps: an **octree-based TSDF grid** and a **3D Gaussian map** [2408.12677]. The TSDF side is built on Supereight2, with each voxel storing a TSDF value and a weight, while the Gaussian side uses standard oriented ellipsoidal primitives with center, rotation, scale, opacity, and spherical harmonics coefficients [2408.12677].

The system processes an RGB-D stream frame by frame. First, depth is integrated into the TSDF. Second, new Gaussians are initialized from a **quadtree decomposition** of the RGB image based on contrast. Third, the Gaussian map is optimized online using a photometric loss between the rendered image and the input RGB frame [2408.12677]. The Gaussian initialization step is the paper’s main efficiency mechanism. Rather than initializing splats densely over pixels, GSFusion back-projects quadtree cell centers and checks the nearest TSDF voxel; if the voxel weight equals 1, the location is treated as newly allocated and a Gaussian is inserted [2408.12677].

The rendering formulation follows projected 2D Gaussian splats with front-to-back alpha compositing. For pixel $\mathbf{u}$, the rendered color is
$$
\hat{\mathbf{I}}_k[\mathbf{u}] = \sum_{i=1}^{N} \mathbf{c}_i \alpha_i \hat{G}_i(\mathbf{u}) \prod_{j=1}^{i-1} (1 - \alpha_j \hat{G}_j(\mathbf{u})).
$$
The online optimization objective is an $L_1$ photometric loss,
$$
L = \|\mathbf{I}_k - \hat{\mathbf{I}}_k\|_1,
$$
with keyframe maintenance used to allocate more optimization to informative frames and randomly revisited keyframes in order to reduce forgetting [2408.12677].

A notable feature of this formulation is that geometry is not merely auxiliary; TSDF structure constrains where splats are inserted and thereby suppresses redundancy. The paper frames this as the main reason the approach produces a compact Gaussian map with fewer artifacts [2408.12677].

## 4. GSFusion as globally optimized LiDAR-Inertial-Visual Gaussian mapping

The LiDAR-Inertial-Visual version of GSFusion targets **online SLAM and mapping with 3D Gaussian splatting in large, unbounded, real-world environments** [2507.23273]. The motivation differs from the RGB-D system. Here the authors emphasize limitations of camera-based Gaussian SLAM in low-texture areas, bad lighting, outdoors, and short-range operation, while also arguing that LiDAR-based Gaussian systems still lack sufficiently dense global alignment for photorealistic quality [2507.23273].

The architecture is a **dual-map system** consisting of a sparse surfel map for localization and a dense Gaussian map for rendering [2507.23273]. At runtime, synchronized LiDAR, camera, and IMU streams feed a front-end that uses IMU preintegration, geometric LiDAR constraints, and photometric visual constraints to estimate motion and locally refine the map. The back-end maintains a global pose graph, performs loop closure, runs incremental global optimization with iSAM2, and updates the affected Gaussians after pose correction [2507.23273].

The odometry objective combines point-to-plane ICP and photometric terms:
$$
L_{odom} = \sum\|e_{icp}\|^{2} + \lambda\sum\|e_{photo}\|^{2}.
$$
For global consistency, the system adds a **surfel-to-surfel alignment constraint** to the pose graph, motivated by the claim that ordinary pose-graph optimization is too coarse for photorealistic 3DGS [2507.23273]. The global objective is
$$
L_{\text{global}} = \sum_{i, j} \| e_{\text{pose}(i, j)} \|^2 + \lambda_{\text{surfel}} \sum_{i, j} \| e_{\text{surfel}(i, j)} \|^2.
$$

Two additional mechanisms are central. The first is **pixel-aware Gaussian initialization**. Instead of deriving Gaussian scale only from local 3D point density, the method computes a projected 2D covariance and an effective projected radius
$$
r_{eff} = \sqrt[4]{\det(\boldsymbol\Sigma_{2D})},
$$
then rescales the Gaussian so that its projected footprint matches a target pixel size [2507.23273]. The second is a **bounded sigmoid scale constraint**
$$
\sigma_{\text{bounded}} = \sigma_{\min} + (\sigma_{\max} - \sigma_{\min}) \cdot \text{sigmoid}(s),
$$
introduced to prevent uncontrolled Gaussian growth in poorly observed regions such as sky or background [2507.23273].

This version of GSFusion therefore uses the term "fusion" in a broader systems sense: LiDAR provides geometry, IMU provides motion propagation, and the camera provides appearance, while the map itself integrates surfel-based structure and Gaussian-based rendering [2507.23273].

## 5. Comparative structure of the three usages

Although the three systems solve different problems, they can be compared along the axes of task, sensing, representation, and optimization.

| Usage of GSFusion | Task | Core fusion object |
|---|---|---|
| Co-Occ [2404.04561] | 3D semantic occupancy prediction | LiDAR features enhanced with neighboring camera features |
| RGB-D GSFusion [2408.12677] | Online RGB-D mapping | TSDF volumetric map combined with 3D Gaussian map |
| LiDAR-Inertial-Visual GSFusion [2507.23273] | Online Gaussian SLAM/mapping | LiDAR, IMU, camera, surfel map, and Gaussian map |

In Co-Occ, the fusion target is a **volumetric semantic representation** regularized through feature-space volume rendering [2404.04561]. In the RGB-D mapping system, the fusion target is a **hybrid map representation** in which TSDF preserves geometry and 3DGS supplies photorealistic rendering [2408.12677]. In the LiDAR-Inertial-Visual system, the fusion target is both **sensor-level and representation-level**, since the method jointly optimizes multi-sensor constraints and updates a dense Gaussian map from globally corrected poses [2507.23273].

A plausible implication is that the repeated reuse of the term reflects a broader trend: "fusion" in contemporary 3D vision increasingly means not only combining sensors, but also coupling explicit geometric representations with appearance-oriented renderers or semantic predictors.

## 6. Quantitative results and empirical claims

The Co-Occ abstract reports that "extensive experiments on the popular nuScenes and SemanticKITTI benchmarks verify the effectiveness of our Co-Occ for 3D semantic occupancy prediction," but no numerical results for the GSFusion module itself are available in the supplied record [2404.04561].

The RGB-D GSFusion paper reports strong efficiency gains relative to Gaussian SLAM baselines. On ScanNet++, the paper reports **6.14 FPS**, **29.3 MB** model size, and **2810 MB GPU memory** for GSFusion, compared with **0.19 FPS**, **206.3 MB**, and **4417 MB GPU memory** for SplaTAM, and **1.29 FPS**, **111.8 MB**, and **3906 MB GPU memory** for RTG-SLAM [2408.12677]. On ScanNet++ with global optimization, representative rendering metrics are **28.84 PSNR / 0.897 SSIM / 0.138 LPIPS** on training views and **25.45 PSNR / 0.848 SSIM / 0.216 LPIPS** on novel views [2408.12677]. On Replica, the paper reports **34.65 PSNR / 0.949 SSIM / 0.056 LPIPS**, exceeding the cited RTG-SLAM results **33.38 / 0.929 / 0.069** [2408.12677].

The LiDAR-Inertial-Visual GSFusion paper emphasizes rendering quality, compactness, and map-building efficiency. It reports that **Ours(1pixel)** achieves PSNR values of **27.5** on CBD02, **29.2** on Retail, **27.8** on SYSU01, and **20.0** on airport03, while **Ours(5pixel)** often delivers nearly comparable PSNR with much smaller maps [2507.23273]. It reports map sizes such as **31.0 MB** on Retail, **24.7 MB** on SYSU01, and **26.8 MB** on Reception for the 5-pixel setting, and cites **96 seconds on Reception** as the fastest runtime among compared methods on the custom datasets [2507.23273].

These empirical sections reveal that the two mapping papers use "GSFusion" to denote systems explicitly optimized for the quality-efficiency trade-off in online Gaussian reconstruction. By contrast, Co-Occ’s GSFusion is a module-level contribution inside a semantic occupancy framework rather than a standalone mapper [2404.04561].

## 7. Related naming ambiguities and broader context

Several nearby acronyms in the record are not exact instances of GSFusion but are relevant to disambiguation. "Gate-Shift-Fuse for Video Action Recognition" introduces **GSF**, sometimes described in the supplied details as **GSFusion** in some contexts, but the paper itself defines the module as Gate-Shift-Fuse rather than GSFusion [2203.08897]. "GS-FUSE: Granger-Supervised Gated Fusion and Multi-Granularity Alignment for Event-Driven Financial Forecasting" is another distinct term, hyphenated and used for multimodal financial forecasting rather than 3D perception [2605.28520]. The seismic foundation model paper is explicitly named **GSFM**, not GSFusion [2502.01111].

This naming overlap can obscure the technical content if the term is cited without context. In 3D vision specifically, however, the exact GSFusion label is associated in the supplied record with methods that attempt to reconcile complementary strengths: LiDAR and camera in semantic occupancy prediction [2404.04561], TSDF geometry and Gaussian rendering in RGB-D mapping [2408.12677], and LiDAR, inertial, visual, surfel, and Gaussian representations in globally optimized SLAM [2507.23273].

Taken together, these usages show that GSFusion functions less as a single canonical algorithm than as a recurring design pattern centered on structured fusion. In one case, the emphasis is **geometric-semantic cross-modal feature injection**; in another, **volumetric-geometric scaffolding for compact Gaussian mapping**; in another, **globally consistent multi-sensor Gaussian SLAM**. The term is therefore best interpreted with its immediate paper context, task definition, and representation choices made explicit.

Source: https://www.emergentmind.com/topics/gsfusion