VIM-GS: Visual-Inertial Gaussian Splatting

Updated 10 July 2026

The paper introduces a hybrid depth reconstruction framework that leverages visual-inertial SfM and object-guided propagation to refine coarse LFM depth into metric-dense depths for novel-view synthesis.
It employs a three-level segmentation pyramid with RANSAC-based primitive fitting and affine calibration to accurately bridge sparse, high-confidence SfM points with dense but coarse monocular depth estimates.
By integrating dynamic depth refinement within the Gaussian Splatting pipeline, VIM-GS enhances rendering quality, sharpens distant scene details, and accelerates convergence in large-scale settings.

VIM-GS, short for “Visual-Inertial Monocular Gaussian Splatting via Object-level Guidance in Large Scenes,” is a Gaussian Splatting framework using monocular images for novel-view synthesis in large scenes. Its central objective is to generate dense, accurate depth images from monocular RGB inputs for high-definite GS rendering by leveraging the accurate but sparse depth from visual-inertial Structure-from-Motion to refine the dense but coarse depth from large foundation models, then using object-segmented depth propagation and a dynamic depth refinement module to support high-fidelity monocular GS rendering in long-range, texture-ambiguous environments (Zhang et al., 8 Sep 2025).

1. Problem formulation and scope

VIM-GS is defined against a specific limitation of Gaussian Splatting. GS typically requires accurate depth to initiate Gaussian ellipsoids using RGB-D/stereo cameras, yet their limited depth sensing range makes it difficult for GS to work in large scenes. Monocular images avoid that sensing-range constraint, but they lack depth to guide the learning and lead to inferior novel-view synthesis results. VIM-GS addresses this gap by treating depth recovery, rather than splat rendering alone, as the critical bottleneck in large-scene monocular GS (Zhang et al., 8 Sep 2025).

The method is motivated by three failure modes of large foundation models for monocular depth estimation in large scenes: cross-frame inconsistency, inaccuracy for distant scenes, and ambiguity in deceptive texture cues. In the formulation used by VIM-GS, dense LFM depth is not discarded; instead, it is treated as dense but coarse geometry that must be metrically corrected by visual-inertial Structure-from-Motion. This leads to a hybrid depth construction pipeline in which VI-SfM provides camera poses, metric scale, gravity alignment, and sparse but accurate point depths, while object-level propagation bridges the gap between sparse constraints and dense outputs (Zhang et al., 8 Sep 2025).

The framework therefore occupies a specific point in the GS design space. It is monocular rather than RGB-D or stereo, visual-inertial rather than purely photometric, and object-guided rather than purely pixelwise. A plausible implication is that VIM-GS is best understood not as a rendering-only system, but as a geometry-conditioning framework for monocular GS in large scenes.

2. Visual-inertial substrate and system inputs

VIM-GS takes as inputs monocular RGB frames $I_t$ and synchronized IMU measurements, together with dense LFM depth $r_t(u)$ and object segmentation masks $\{M_k\}$ from SAM. Its visual-inertial substrate is a VI-SfM front-end that outputs camera intrinsics $K$ , camera poses $T_t$ in metric scale, sparse $3$D points $P_i$ with per-point absolute depth $Z_i^{abs}$ , per-point confidence $\omega_i \in [0,1]$ , and keyframe selection (Zhang et al., 8 Sep 2025).

The VI front-end is technically explicit. Features are extracted by SuperPoint, matching is performed by LightGlue, and outlier rejection uses USAC-PROSAC with an 8-point model. Initial triangulation is performed by SVD, followed by non-linear least squares refinement with analytic Jacobians and Huber loss. Per-point confidence $\omega$ is derived from multi-frame reprojection consistency. IMU integration provides metric scale and gravity alignment, stabilizing pose estimation and triangulation over long trajectories; the description further states that IMU helps maintain metric scale and robustness against motion blur and low-texture stretches (Zhang et al., 8 Sep 2025).

This visual-inertial foundation is not a peripheral component. In VIM-GS it supplies the only metrically reliable geometric anchors available in a monocular pipeline. The dense depth map used by GS is therefore not predicted directly by a monocular model; it is reconstructed through the interaction of VI-SfM constraints, segmentation, and structured propagation.

3. Object-segmented depth propagation

The core mechanism for turning sparse metric depth into dense depth is object-segmented depth propagation. VIM-GS builds a 3-level segmentation pyramid with thresholds $r_t(u)$ 0, where child masks inherit pixels from parent masks and under-supervised masks are merged with neighbors if depths are consistent or promoted to a coarser level. Morphological opening/closing and removal of tiny components are used to denoise masks. This hierarchy is intended to improve robustness on small objects and boundaries while increasing supervision density (Zhang et al., 8 Sep 2025).

Within each mask $r_t(u)$ 1, the method fits candidate geometric models $r_t(u)$ 2 to SfM points by minimal-sample RANSAC. Hypotheses are scored by inlier ratio $r_t(u)$ 3 and residual variance $r_t(u)$ 4 through

$r_t(u)$ 5

The accepted model is the one with highest confidence, subject to the stated inlier criterion, and adjacent masks may be merged when their points are model-consistent. The residual for an implicit model $r_t(u)$ 6 is

$r_t(u)$ 7

This structure-aware stage is what allows VIM-GS to address deceptive textures: textured flat surfaces can be regularized toward planar geometry rather than interpreted as false relief (Zhang et al., 8 Sep 2025).

Dense depth inside a mask is then represented by a mask-wise affine calibration of LFM depth,

$r_t(u)$ 8

where $r_t(u)$ 9 is the LFM relative depth and $\{M_k\}$ 0 are mask-level scale and shift. The per-mask objective is

$\{M_k\}$ 1

with

$\{M_k\}$ 2

$\{M_k\}$ 3

$\{M_k\}$ 4

and

$\{M_k\}$ 5

The solver is IRLS, with weighted normal equations and the closed-form updates

$\{M_k\}$ 6

$\{M_k\}$ 7

Edge-aware weights suppress smoothing across strong image gradients or different masks, thereby preserving object boundaries while aligning LFM depth to the metric scale set by VI-SfM (Zhang et al., 8 Sep 2025).

VIM-GS supplements propagation with a dynamic depth refinement module to handle the crippled SfM depth of dynamic objects and refine the coarse LFM depth. For each mask $\{M_k\}$ 8, inter-frame translation $\{M_k\}$ 9 is estimated from VI front-end geometry using two cosine-law constraints for each $K$ 0D point $K$ 1 and its translated correspondence $K$ 2: $K$ 3

$K$ 4

$K$ 5 is estimated by RANSAC and LM refinement. If $K$ 6, the segment is labeled dynamic and all its pixels are occluded from downstream reconstruction and losses: no Gaussian initialization from these pixels, and no photometric or depth supervision from them (Zhang et al., 8 Sep 2025).

The GS stage uses standard $K$ 7D Gaussian primitives $K$ 8, where $K$ 9, $T_t$ 0, $T_t$ 1, and $T_t$ 2. After projection, the $T_t$ 3D covariance is

$T_t$ 4

and the elliptical footprint density is

$T_t$ 5

Per-pixel opacity is

$T_t$ 6

with front-to-back compositing defined by $T_t$ 7, $T_t$ 8, $T_t$ 9, and $3$0. The rendered expected depth is

$3$1

Refined dense depth is back-projected through

$3$2

and used to initialize Gaussian means and covariances (Zhang et al., 8 Sep 2025).

Optimization is governed by a photometric loss

$3$3

and a robust, visibility-weighted depth-consistency loss

$3$4

where $3$5 is the static-valid mask and $3$6. The total objective is

$3$7

Training follows a geometry-to-appearance schedule: it starts with large $3$8 to settle geometry, then cosine-anneals it to a floor while $3$9 dominates. This design indicates that VIM-GS treats depth reliability as a precondition for stable radiance optimization rather than merely an auxiliary regularizer (Zhang et al., 8 Sep 2025).

5. Experimental characterization, ablations, and limitations

The evaluation described for VIM-GS uses outdoor, large-scale sequences derived from KITTI and KITTI-360, with synchronized RGB, IMU, and semi-dense LiDAR depths aligned to images. Depth is evaluated with AbsRel, RMSE, and $P_i$ 0-thresholds; novel-view synthesis is evaluated with PSNR, SSIM, and LPIPS. Comparisons are reported against Depth Anything V2, Depth Pro, ZoeDepth, and DepthSplat for depth, and against monocular GS systems such as VINGS-Mono, Intern-GS, and MonoGS++ for NVS, under identical hyperparameters and splits (Zhang et al., 8 Sep 2025).

The supplied description reports qualitative rather than tabulated quantitative conclusions. VIM-GS is stated to improve large-scene NVS quality over monocular-supported baselines and accelerate convergence due to superior geometric initialization. Qualitative results show sharper distant structures, fewer floaters/streaks, and better boundary preservation, while dynamic gating reduces artifacts induced by moving agents. The quality–resource discussion further states that VIM-GS achieves target PSNR with fewer splats or fewer steps compared to monocular baselines. The provided manuscript text does not include explicit numeric values in the tables; the reported results are summarized qualitatively (Zhang et al., 8 Sep 2025).

The ablation narrative isolates the role of each major component. Removing the segmentation pyramid or adjacency merging degrades boundary and small-object depth and slows GS convergence. Disabling IRLS affine calibration or primitive fitting causes LFM-relative scale drift and texture-induced relief. Turning off dynamic refinement increases errors and introduces transient artifacts in NVS. These statements place the object-level propagation mechanism, rather than the renderer alone, at the center of the framework’s empirical behavior (Zhang et al., 8 Sep 2025).

The stated limitations are equally specific. VIM-GS depends on segmentation quality; sparse SfM coverage limits supervision in textureless regions; highly nonrigid or fast-moving objects may evade segment-wise gating or lead to nearby static regions being incorrectly masked; and extremely far depths remain numerically difficult even with VI guidance. Proposed future improvements include richer primitive libraries and data-driven model selection, tighter temporal consistency with uncertainty modeling, adaptive segmentation, loop closure and global consistency in the VI backend, and end-to-end training that jointly optimizes depth propagation and GS (Zhang et al., 8 Sep 2025).

Although VIM-GS is explicitly the title of the monocular large-scene method above, the surrounding literature uses similar terminology more broadly for visual-inertial mapping or SLAM with Gaussian Splatting. This broader usage does not change the primary meaning of VIM-GS in (Zhang et al., 8 Sep 2025), but it does help situate the method within a growing family of dense visual-inertial GS systems.

Work	Explicit use	Relation to VIM-GS
“VIM-GS: Visual-Inertial Monocular Gaussian Splatting via Object-level Guidance in Large Scenes” (Zhang et al., 8 Sep 2025)	Explicit title	Monocular NVS in large scenes via VI-SfM-guided depth refinement
“VIGS-SLAM: Visual Inertial Gaussian Splatting SLAM” (Zhu et al., 2 Dec 2025)	Described as a concrete realization of “Visual-Inertial Mapping/SLAM with Gaussian Splatting”	Real-time SLAM with tightly coupled IMU and 3DGS
“GS-GVINS: A Tightly-integrated GNSS-Visual-Inertial Navigation System Augmented by 3D Gaussian Splatting” (Zhou et al., 16 Feb 2025)	Tailored in the supplied explanation to the “VIM-GS” query	GNSS–visual–inertial navigation augmented by 3DGS
“GeVI-SLAM: Gravity-Enhanced Stereo Visual Inertial SLAM for Underwater Robots” (Shen et al., 28 Oct 2025)	Presented in the supplied explanation as fitting a gravity-stabilized VI mapping/SLAM framework	Gravity-enhanced stereo VI SLAM under underwater degeneracy

This pattern suggests a broader umbrella sense in which VIM-GS can denote visual-inertial mapping or SLAM systems that use Gaussian Splatting or related dense map representations. Within that broader context, (Zhang et al., 8 Sep 2025) is the monocular large-scene depth-refinement formulation, while (Zhu et al., 2 Dec 2025) is a tightly coupled visual-inertial 3DGS SLAM system, (Zhou et al., 16 Feb 2025) extends dense differentiable Gaussian maps into tightly integrated GNSS–visual–inertial navigation, and (Shen et al., 28 Oct 2025) is positioned as a gravity-stabilized VI mapping/SLAM framework in underwater settings.

The acronym also requires disambiguation outside this subfield. “ViM” in out-of-distribution detection denotes “Virtual-logit Matching” (Wang et al., 2022), and “TD-VIM” in speaker verification denotes “Time-Domain Voice Identity Morphing” (PN et al., 7 Apr 2026). Those usages are unrelated to Gaussian Splatting and visual-inertial geometry. In the large-scene monocular rendering literature, VIM-GS refers specifically to the fusion of visual-inertial SfM, object-guided dense depth refinement, and Gaussian Splatting for high-definite rendering in scenes beyond the effective range of RGB-D and stereo sensors (Zhang et al., 8 Sep 2025).