GloSplat: Joint SfM & Gaussian Splatting Reconstruction

• GloSplat is a 3D reconstruction framework that integrates global SfM and 3D Gaussian Splatting to jointly optimize camera poses and appearance for improved efficiency and accuracy.
• The method demonstrates state-of-the-art performance with up to +1.37 dB PSNR gains and lower pose errors compared to traditional COLMAP-based and free methods across multiple datasets.
• It preserves explicit SfM feature tracks as separate, optimizable parameters from Gaussian primitives, using a joint bundle-adjustment loss to effectively reduce pose drift during training.

GloSplat is a 3D reconstruction and novel view synthesis framework that tightly couples global Structure-from-Motion with 3D Gaussian Splatting by performing joint pose-appearance optimization during 3DGS training. Its defining architectural choice is to preserve explicit SfM feature tracks as first-class entities throughout optimization: track 3D points are maintained as separate optimizable parameters from Gaussian primitives, and a reprojection loss operates alongside photometric supervision to refine camera poses continuously while constraining drift (Xiong et al., 5 Mar 2026). In the surrounding literature, the name is sometimes also used more loosely to denote broader “GloSplat-style” goals such as global, semantic, or ray-traced Gaussian pipelines; however, the specific method titled “GloSplat” is a pose-and-appearance optimization framework for faster and more accurate 3D reconstruction (Xiong et al., 5 Mar 2026).

1. Definition and scope

GloSplat addresses the standard modular separation between feature extraction, matching, SfM, and novel view synthesis. In conventional pipelines, SfM estimates camera poses from feature correspondences, those poses are frozen, and a NeRF or 3DGS model is then trained on top. The consequence, as described in the paper, is that radiance-field optimization cannot correct geometric errors inherited from pose estimation, even though rendering exposes a much richer photometric signal than the sparse correspondences used by SfM (Xiong et al., 5 Mar 2026).

The framework therefore begins from unposed multi-view images, runs a global SfM pipeline to obtain initial camera poses and sparse 3D tracks, converts SfM points into an initial 3D Gaussian set, and then trains a 3DGS model while keeping SfM feature tracks and their 3D points as separate learnable variables. Camera poses receive gradients from both dense photometric supervision and a reprojection-based bundle-adjustment objective. This differs from prior joint methods such as BARF, NeRF--, and 3RGS, which refine poses using photometric gradients alone (Xiong et al., 5 Mar 2026).

Two variants are defined. GloSplat-F is a COLMAP-free variant using retrieval-based pair selection for efficient reconstruction, with XFeat and LightGlue, and is optimized for speed and scalability. GloSplat-A uses SIFT and exhaustive matching, matching the standard COLMAP setup for maximum quality and for comparison under the same matching budget (Xiong et al., 5 Mar 2026).

2. Core representation and optimization objective

The scene representation is standard 3D Gaussian Splatting. Each Gaussian has a center $\boldsymbol{\mu}_j \in \mathbb{R}^3$, a covariance $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$ represented through scale and rotation, an opacity $\alpha_j \in (0,1)$, and color encoded with spherical harmonics coefficients $\mathbf{c}_j$ up to degree $3$ (Xiong et al., 5 Mar 2026). Rendering is performed with gsplat using differentiable splatting and front-to-back alpha compositing.

The photometric objective is the usual L1-plus-SSIM form:

$$\mathcal{L}_{\text{photo}} = (1 - \lambda_{\text{SSIM}})\,\|\hat{\mathbf{I}} - \mathbf{I}\|_1 + \lambda_{\text{SSIM}} (1 - \text{SSIM}(\hat{\mathbf{I}}, \mathbf{I})),$$

with $\lambda_{\text{SSIM}} = 0.2$ (Xiong et al., 5 Mar 2026).

GloSplat’s key addition is a joint bundle-adjustment loss on explicit track points:

$$\mathcal{L}_{\text{BA}}^{\text{joint}} = \sum_{k} \sum_{(i, p) \in \mathcal{T}_k} \rho_{\text{Huber}}\!\left( \left\| \pi_i(\mathbf{X}_k) - \mathbf{x}_{i,p} \right\|^2; \delta \right),$$

where $\delta = 1.0$ pixel, $\mathbf{X}_k$ is the 3D point for track $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$0, and $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$1 is the camera projection model (Xiong et al., 5 Mar 2026). The total training objective is

$\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$2

A crucial detail is that track points $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$3 remain distinct from Gaussian centers $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$4. Gaussians represent the volumetric appearance and geometry used in rendering, while track points serve as persistent geometric anchors. The paper reports that merging tracks with Gaussian centers degrades performance by approximately $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$5 dB PSNR for both variants, which is presented as evidence that the separation is important (Xiong et al., 5 Mar 2026).

This design specifically targets early-stage pose drift. The paper argues that when Gaussians are still sparse and colors inaccurate, purely photometric gradients are noisy and can move cameras toward poor local minima. The reprojection-based BA term constrains camera updates through multi-view consistency even before the radiance field has matured, while later photometric gradients can still provide fine-scale refinement (Xiong et al., 5 Mar 2026).

3. Pipeline architecture

The pipeline begins with frozen feature extraction and matching, followed by global SfM. The global SfM stage builds a view graph, estimates pairwise relative poses, solves for absolute rotations with rotation averaging, then estimates translations and 3D points, and finally performs bundle adjustment (Xiong et al., 5 Mar 2026). The positioning stage uses

$\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$6

where $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$7 is the unit bearing and $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$8 is the camera center (Xiong et al., 5 Mar 2026).

SfM points are then converted into an initial Gaussian set. Each SfM 3D point becomes one Gaussian with $\boldsymbol{\Sigma}_j \in \mathbb{R}^{3\times 3}$9, scale derived from the mean distance to the $\alpha_j \in (0,1)$0 nearest SfM neighbors, opacity initialized to $\alpha_j \in (0,1)$1, and SH color initialized from the point’s observed colors. The system then applies MCMC-3DGS densification up to approximately $\alpha_j \in (0,1)$2M Gaussians (Xiong et al., 5 Mar 2026).

The two variants diverge mainly in matching strategy. GloSplat-F uses MegaLoc retrieval, selecting top-$\alpha_j \in (0,1)$3 neighbors for each image, with $\alpha_j \in (0,1)$4 reported as the default. It uses XFeat with up to $\alpha_j \in (0,1)$5 keypoints and $\alpha_j \in (0,1)$6-dimensional descriptors, then LightGlue matching, producing a view graph whose number of edges scales linearly with the number of images (Xiong et al., 5 Mar 2026). GloSplat-A instead performs exhaustive pairwise matching with SIFT and nearest-neighbor matching, yielding a denser graph and higher-quality initialization at a higher computational cost (Xiong et al., 5 Mar 2026).

During joint optimization, the learnable variables are camera extrinsics, track 3D points, and Gaussian parameters including positions, scales, rotations, opacities, and SH coefficients. Rendering uses gsplat, and the optimizer is Adam. The reported learning rates are $\alpha_j \in (0,1)$7 for Gaussian positions, scaled by scene extent; $\alpha_j \in (0,1)$8 for scales; $\alpha_j \in (0,1)$9 for rotations; $\mathbf{c}_j$0 for opacities; $\mathbf{c}_j$1 for SH coefficients; and $\mathbf{c}_j$2 for camera extrinsics, with poses parameterized on $\mathbf{c}_j$3 and updates applied via composition through PyPose (Xiong et al., 5 Mar 2026).

4. Empirical performance

GloSplat is evaluated on MipNeRF360, Tanks and Temples, CO3Dv2, and a ScanNet subset, using PSNR, SSIM, LPIPS, rotation error, and ATE (Xiong et al., 5 Mar 2026).

For COLMAP-free reconstruction, GloSplat-F is reported as state of the art among COLMAP-free methods. On MipNeRF360 it achieves 27.77 PSNR, 0.818 SSIM, and 0.164 LPIPS; on Tanks and Temples 25.82 PSNR, 0.869 SSIM, and 0.151 LPIPS; and on CO3Dv2 32.71 PSNR, 0.936 SSIM, and 0.088 LPIPS (Xiong et al., 5 Mar 2026). The paper states that this is best among COLMAP-free methods on all three datasets and that on MipNeRF360 it improves over VGGT-X by +1.37 dB PSNR, +0.036 SSIM, and −0.013 LPIPS (Xiong et al., 5 Mar 2026).

For COLMAP-based comparison, GloSplat-A achieves 28.86 PSNR, 0.862 SSIM, and 0.139 LPIPS on MipNeRF360, exceeding the reported numbers of Improved-GS, Perceptual-GS, 3DGS-MCMC, and other listed baselines (Xiong et al., 5 Mar 2026). The reported margin over Improved-GS is +0.67 dB PSNR, +0.026 SSIM, and a reduction in LPIPS from 0.186 to 0.139 (Xiong et al., 5 Mar 2026).

Pose quality is evaluated on ScanNet. On scene 0079_00, GloSplat-F reports 2.12 degrees rotation error, 0.011 m ATE, and 33.42 PSNR; on 0301_00, 7.82 degrees, 0.007 m, and 31.24 PSNR; on 0418_00, 3.78 degrees, 0.010 m, and 32.35 PSNR (Xiong et al., 5 Mar 2026). In all three examples, those values are lower in rotation error and ATE and higher in PSNR than the COLMAP and 3RGS baselines listed in the same comparison (Xiong et al., 5 Mar 2026).

The speed claims are also central. On the Courthouse scene with 1000 images, GloSplat-F is reported as 13.3× faster than COLMAP + 3DGS and slightly faster than VGGT-X at that scale. Under exhaustive matching, GloSplat-A is reported as approximately 3.2× faster than COLMAP on the same scene (Xiong et al., 5 Mar 2026).

5. Ablations and technical significance

The ablation study is used to separate the contributions of global SfM and joint optimization. On MipNeRF360, removing the joint BA term lowers GloSplat-F from 27.77 dB to 26.96 dB, a drop of 0.81 dB (Xiong et al., 5 Mar 2026). Freezing poses after SfM reduces performance to 19.18 dB, a reported drop of 8.59 dB, which the paper presents as evidence that pose refinement during 3DGS training is essential rather than optional (Xiong et al., 5 Mar 2026).

A second decomposition compares COLMAP initialization with and without joint BA under exhaustive matching. The paper reports 28.52 dB for COLMAP + joint BA versus 27.91 dB for MCMC$\mathbf{c}_j$4, attributing +0.61 dB to joint optimization. The remaining gain from 28.52 dB to 28.86 dB for GloSplat-A is +0.34 dB, attributed to replacing incremental COLMAP with the global SfM stage (Xiong et al., 5 Mar 2026). The authors summarize this as approximately 64% of the gain from joint optimization and 36% from global SfM (Xiong et al., 5 Mar 2026).

The paper also reports that removing MCMC densification costs −1.75 dB, indicating that densification remains important but orthogonal to the joint pose-appearance design (Xiong et al., 5 Mar 2026). This suggests that GloSplat should be understood less as a replacement for mature 3DGS optimization machinery and more as a coupling mechanism between geometric estimation and splat training.

A common misconception, reflected implicitly by comparison with BARF, NeRF--, and 3RGS, is that “joint pose optimization” in radiance fields is equivalent to adding camera parameters to a photometric objective. GloSplat’s contribution is narrower and more specific: it keeps explicit track geometry alive after SfM, rather than discarding it once an initialization has been obtained (Xiong et al., 5 Mar 2026). A plausible implication is that the framework inherits some of the robustness of classical multi-view geometry while still exploiting dense differentiable rendering.

6. Position within the broader Gaussian-splatting literature

Within the immediate 3DGS optimization literature, GloSplat is primarily a reconstruction pipeline. It does not modify the Gaussian representation itself, the differentiable rasterization equations, or the MCMC densification mechanism; it changes how camera poses and track geometry are optimized during training (Xiong et al., 5 Mar 2026). This makes it complementary to renderer-level systems such as “FlashGS,” which is described as an open-source CUDA Python library for efficient differentiable rasterization of standard 3D Gaussian Splatting through algorithmic and kernel-level optimizations and is best understood as a drop-in renderer for standard 3DGS scenes (Feng et al., 2024). A plausible implication is that a GloSplat-style training pipeline could use a FlashGS-style backend to improve large-scene and high-resolution scaling without altering GloSplat’s joint objective.

The name “GloSplat” also appears more broadly in later papers as shorthand for ambitions beyond pose refinement. “Instant Colorization of Gaussian Splats” describes the need for an “instant GloSplat projector” that projects arbitrary 2D signals onto a fixed 3D Gaussian scene through visibility-weighted least squares, thereby turning 2D feature maps or semantic probabilities into globally consistent per-splat attributes (Lieber et al., 18 Apr 2026). “SplatBus” is presented as solving integration and viewing problems for any GloSplat-like renderer by exposing GPU framebuffers through NVIDIA IPC to Unity, Blender, Unreal Engine, and OpenGL viewers (Xu et al., 21 Jan 2026). “REdiSplats” frames “GloSplat” as a broad name for GS methods with GI or ray tracing, while its own contribution is a ray-traced, mesh-parameterized, editable Gaussian formulation rather than full global illumination (Byrski et al., 15 Mar 2025). “GreenhouseSplat” does not present a new splatting algorithm such as GloSplat, but its authors explicitly position it as a greenhouse-scale radiance-field pipeline that could support global splat-map use in robotics (Tabaa et al., 2 Oct 2025).

These usages indicate that “GloSplat” has acquired a secondary, looser meaning in the surrounding literature: a global, richly optimized, or semantically enriched Gaussian-splatting system. The specific paper titled “GloSplat,” however, remains more concrete. It is a framework in which global SfM initialization and joint photometric-geometric optimization are integrated so that explicit feature tracks stabilize and refine camera poses throughout 3DGS training (Xiong et al., 5 Mar 2026).

7. Limitations and prospective extensions

The paper states several limitations. Feature extraction, retrieval, and matching are frozen; there is no end-to-end differentiation through XFeat, LightGlue, MegaLoc, or SIFT (Xiong et al., 5 Mar 2026). If retrieval misses overlapping views, global SfM quality degrades, and the joint stage cannot fully recover from that initialization. The gap between GloSplat-F and GloSplat-A, reported as 1.09 dB on MipNeRF360, is attributed mainly to the sparser matching graph and to learned-versus-SIFT feature differences (Xiong et al., 5 Mar 2026).

The framework is also restricted to static scenes. Dynamic or non-rigid cases are not addressed (Xiong et al., 5 Mar 2026). Exhaustive matching in GloSplat-A remains quadratic in the number of images, even though the global SfM implementation is faster than COLMAP. GloSplat-F scales linearly in matching complexity, but that scalability depends on retrieval quality (Xiong et al., 5 Mar 2026).

The paper suggests several future directions: end-to-end differentiable feature and SfM pipelines; dynamic or non-rigid extensions using Gaussian dynamics; integration with larger-scale 3DGS systems such as CityGaussian and VastGaussian; and improved retrieval or matching through stronger learned priors such as MASt3R or DUSt3R (Xiong et al., 5 Mar 2026). This suggests that GloSplat is best viewed not as a terminal formulation but as an interface layer between classical global geometry and modern Gaussian-based rendering.