SplitGS-Loc: Gaussian Splatting for Localization

• SplitGS-Loc is a localization framework that refines photometric 3D Gaussian Splatting by splitting Gaussians to yield precise, one-to-one 2D–3D correspondences.
• It exploits covariance-derived scales, opacity, and composition weights to directly lift discriminative multiview features without rendering-based blurring.
• Empirical results demonstrate increased PnP inlier counts and reduced translation errors, making SplitGS-Loc competitive for robust visual localization.

Searching arXiv for the primary paper and closely related Gaussian-splatting localization work mentioned in the provided data. SplitGS-Loc is a localization-specialized framework for constructing Gaussian Splatting-based Feature Fields (GSFFs) from a photometrically trained 3D Gaussian Splatting (3DGS) scene, with the explicit objective of supporting robust direct 2D–3D matching and PnP-based camera pose estimation without any per-scene feature training or iterative pose refinement. Its central premise is that photometrically optimized GSFFs are structurally misaligned with the requirements of point-based pose solvers: the volumetric extent of Gaussians induces ambiguous many-to-one pixel-to-point correspondences, and photometric optimization introduces superfluous Gaussians that lack multi-view consistency. SplitGS-Loc addresses both issues by exploiting Gaussian attributes—especially covariance-derived scales, opacity, and rasterization composition weights—to produce compact, discriminative, and multi-view-consistent feature fields for visual localization (Lee et al., 8 May 2026).

1. Problem formulation and the mismatch between photometric GSFFs and PnP

In structure-based visual localization, the task is to estimate camera pose $(\mathbf{R}, \mathbf{t})$ for a query image by establishing 2D–3D correspondences $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$ and solving PnP with RANSAC. The optimization target is written as

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$

where $\pi$ denotes camera projection (Lee et al., 8 May 2026).

GSFFs attach learnable descriptors $\mathbf{z}_n$ to Gaussians in a 3DGS scene, so that query image features can be matched directly to Gaussians, yielding 2D–3D correspondences. SplitGS-Loc is motivated by the observation that GSFFs obtained by photometric training, or by rendering-based feature losses, are inherently ill-suited to this use case. A single 3D Gaussian models a spatial volume rather than an isolated point; once projected into an image, that Gaussian may cover many pixels. When the Gaussian is later reduced to a single 3D representative, typically its center $\mu_n$, many distinct pixels become associated with the same 3D location. The resulting many-to-one correspondence pattern is geometrically inconsistent with the assumptions of PnP.

The ambiguity is not merely representational. PnP with RANSAC depends on a set of mutually consistent point-to-point correspondences. When multiple pixels that originate from different surfaces or depths are mapped to one Gaussian center, reprojection errors increase, the inlier set becomes less coherent, and RANSAC may reject otherwise plausible hypotheses, require substantially more iterations, or converge to suboptimal poses. The paper further identifies a second source of degradation: photometric 3DGS optimization is free to introduce Gaussians that explain view-dependent effects or localized appearance variations, even when these Gaussians contribute only in a few training views and exhibit weak multi-view support. Such Gaussians are useful for rendering fidelity but are poorly aligned with the requirements of stable novel-view localization.

A common misconception is that a photometrically strong GSFF is automatically suitable for direct 2D–3D localization. SplitGS-Loc explicitly argues against that view. Its contribution lies in treating localization as a distinct design target rather than as a by-product of photometric scene representation.

2. 3D Gaussian Splatting and feature-field construction

A 3DGS scene is represented as a set of Gaussians

$\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$

where each Gaussian $g_n$ has a mean $\mu_n \in \mathbb{R}^3$, covariance

$\Sigma_n = R_n S_n S_n^\top R_n^\top,$

with rotation $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$0 and diagonal scale matrix $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$1, opacity $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$2, and spherical-harmonic color coefficients $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$3 (Lee et al., 8 May 2026).

For pixel $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$4 in image $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$5, rendered color is computed as

$(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$6

where $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$7 is the composition weight. This weight reflects both the projected Gaussian footprint in the image plane and the accumulated transmittance along the ray; it is analogous to volumetric alpha compositing weights. GSFFs augment each Gaussian with a feature vector $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$8, giving a rendered feature

$(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$9

In conventional GSFF construction, $$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$0 is often optimized with a rendering loss of the form

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$1

where $$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$2 are 2D image features from an encoder. SplitGS-Loc identifies a limitation of this strategy: gradients from a ray are distributed across multiple Gaussians, which leads to feature blurring and weakens per-Gaussian distinctiveness. The method therefore avoids rendering-based feature optimization and instead performs direct feature lifting from multi-view pixel descriptors, so that descriptors attached to Gaussians remain aligned with strong pixel-level evidence rather than with a diffuse reconstruction objective (Lee et al., 8 May 2026).

This design decision is conceptually important. It reframes the feature field not as an auxiliary rendering layer but as a localization-oriented structure whose descriptors must support correspondence extraction under PnP.

3. Mixture-of-Gaussians-based splitting

The defining mechanism of SplitGS-Loc is Mixture-of-Gaussians-based splitting. The objective is to replace ambiguous many-to-one pixel-to-point mappings with more precise one-to-one correspondences by decomposing each Gaussian into several smaller Gaussians. If a single large Gaussian spans an extended volumetric region, approximating that region with multiple finer Gaussians reduces the number of pixels that each component significantly influences and tightens the geometric interpretation of each 2D–3D match (Lee et al., 8 May 2026).

The method begins from a Gaussian distribution

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$3

and replaces it by a mixture

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$4

which in the 3DGS setting corresponds to child Gaussians with opacities $$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$5. To make the construction tractable, the derivation factorizes the Gaussian along its principal axes and performs splitting along the major axis, that is, the axis with the largest scale. In canonical coordinates, the scalar coordinate along this axis is modeled as

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$6

and approximated by a symmetric three-component mixture

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$7

with

$$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$8

subject to $$\min_{\mathbf{R},\mathbf{t}} \sum_k \left\| \pi\big(\mathbf{R}\mathbf{X}_k + \mathbf{t}\big) - \mathbf{u}_k\right\|^2,$$9.

The parameters are determined by moment matching up to the fourth order, using

$\pi$0

for $\pi$1. Under the balanced-splitting constraint $\pi$2, the derived solution is

$\pi$3

Positive semi-definiteness requires

$\pi$4

and the implementation chooses $\pi$5, which the paper reports as empirically best for localization performance.

In 3D, the parent Gaussian is split along its major axis into three child Gaussians located at offsets $\pi$6 and $\pi$7 in canonical coordinates, with reduced variance along that axis, unchanged covariance components along the other axes, and opacities redistributed as $\pi$8, $\pi$9, and $\mathbf{z}_n$0. Rotating the construction back to world coordinates via the original rotation $\mathbf{z}_n$1 yields three child Gaussians that preserve the overall volumetric distribution while narrowing the individual projected footprints. The paper reports that localization accuracy improves monotonically as $\mathbf{z}_n$2 increases within the feasible interval, which it interprets as evidence that wider separation of child components better reduces pixel–Gaussian ambiguity (Lee et al., 8 May 2026).

The significance of the splitting step is not simply densification. The method’s own ablations distinguish Mixture-of-Gaussians splitting from geometry-only densification, showing that nearest-neighbor upsampling remains inferior. This suggests that covariance-aware decomposition, rather than an arbitrary increase in point count, is the relevant mechanism.

4. Composition weights, salient Gaussian selection, and feature lifting

SplitGS-Loc builds on PlugGS-Loc, a training-free feature-lifting pipeline that uses composition weights from GS rasterization to select Gaussians that contribute significantly and consistently across views and to aggregate discriminative multi-view features to those Gaussians (Lee et al., 8 May 2026). The key observation is that composition weights encode not only projected support but also opacity and transmittance, making them an occlusion-aware indicator of how strongly a Gaussian explains a pixel.

For each training image $\mathbf{z}_n$3 and pixel $\mathbf{z}_n$4, the method defines the set of primary Gaussians as

$\mathbf{z}_n$5

with a fixed threshold $\mathbf{z}_n$6. This eliminates weak pixel–Gaussian associations and retains only contributors with strong explanatory power. Rather than storing all pixel-level weights, the method compresses per-image contribution into a representative image–Gaussian weight

$\mathbf{z}_n$7

and for each Gaussian collects the informative nonzero weights as

$\mathbf{z}_n$8

Gaussians with $\mathbf{z}_n$9 are discarded.

Importance is then scored by

$\mu_n$0

which rewards high visibility, high opacity or transmittance, and multi-view consistency. To keep the map compact while preserving spatial coverage, Gaussians are grouped into local regions by anchor sampling and $\mu_n$1-nearest-neighbor grouping in 3D, and the most important Gaussian in each region is retained. This produces a set of anchor Gaussians that are uniformly distributed and strongly supported across views.

Feature construction proceeds by direct multi-view aggregation. Given image features $\mu_n$2 from an encoder—SuperPoint in the experiments—the method defines

$\mu_n$3

namely the set of pixels where Gaussian $\mu_n$4 attains maximal weight in each contributing image. With softmax-normalized weights $\mu_n$5 over $\mu_n$6 and channel-normalized pixel descriptors $\mu_n$7, the Gaussian feature is

$\mu_n$8

This aggregation strategy is central to SplitGS-Loc’s localization behavior. It uses strong pixel–Gaussian associations as soft assignments, avoids rendering-loss optimization entirely, and thereby prevents the feature blurring associated with rendering-based GSFF learning. The paper’s Gaussian-attribute ablation supports this interpretation: a simple Projection+Average strategy, which ignores composition weights, yields markedly worse localization than the weight-aware construction, indicating that opacity- and transmittance-aware selection is a primary source of robustness (Lee et al., 8 May 2026).

5. End-to-end construction pipeline and PnP-based localization

The full SplitGS-Loc pipeline begins from a standard RGB 3DGS model trained for photometric reconstruction only. That base training takes approximately $\mu_n$9 minutes per scene. The additional localization-specific construction comprises Mixture-of-Gaussians splitting, pre-filtering, weight aggregation, Gaussian sampling, and feature registration, with total runtime of approximately $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$0 seconds per scene; the breakdown is Gaussian Splitting $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$1 s, Pre-filtering $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$2 s, Weight Aggregation $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$3 s, Gaussian Sampling $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$4 s, and Feature Registration $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$5 s (Lee et al., 8 May 2026).

After splitting, sampling must remain aware of parent–child structure. Scoring each child independently and then collapsing to a single representative would negate the benefit of decomposition. The method therefore computes per-child scores and aggregates them into a parent-level quantity

$\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$6

then performs spatial grouping and sampling on parent centers $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$7 using $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$8, while retaining all three child Gaussians for each selected parent. This preserves the local multi-Gaussian structure that splitting introduced.

At query time, SuperPoint extracts 2D keypoints and descriptors from the input image. Each retained Gaussian is treated as a 3D point at its center $\mathcal{G} = \{ g_n \}_{n=1}^{N_g},$9 with descriptor $g_n$0, and query descriptors are matched to Gaussian descriptors, following STDLoc’s matching and PoseLib settings. The resulting correspondences have the form

$g_n$1

PoseLib is then used for PnP with RANSAC, conceptually minimizing

$g_n$2

where $g_n$3 is the inlier-robust loss induced by RANSAC.

The empirical claim of the method is that splitting materially alters the correspondence distribution seen by PnP. On Cambridge, the number of many-to-one correspondences per PnP is reduced from approximately $g_n$4 to $g_n$5. Under different RANSAC settings, the number of valid PnP inliers increases from $g_n$6 to $g_n$7 or from $g_n$8 to $g_n$9 when switching from STDLoc to SplitGS-Loc. Median translation error improves from $\mu_n \in \mathbb{R}^3$0 cm to $\mu_n \in \mathbb{R}^3$1 cm, and the method remains stable under more efficient RANSAC limits: STDLoc degrades from $\mu_n \in \mathbb{R}^3$2 cm to $\mu_n \in \mathbb{R}^3$3 cm, whereas SplitGS-Loc remains at approximately $\mu_n \in \mathbb{R}^3$4–$\mu_n \in \mathbb{R}^3$5 cm (Lee et al., 8 May 2026).

Query-time performance is correspondingly efficient. With a moderate RANSAC setting of $\mu_n \in \mathbb{R}^3$6–$\mu_n \in \mathbb{R}^3$7 iterations, SplitGS-Loc reaches $\mu_n \in \mathbb{R}^3$8 cm / $\mu_n \in \mathbb{R}^3$9 in about $\Sigma_n = R_n S_n S_n^\top R_n^\top,$0 ms per query on Cambridge. With a higher-accuracy setting of $\Sigma_n = R_n S_n S_n^\top R_n^\top,$1–$\Sigma_n = R_n S_n S_n^\top R_n^\top,$2 iterations, it reaches $\Sigma_n = R_n S_n S_n^\top R_n^\top,$3 cm / $\Sigma_n = R_n S_n S_n^\top R_n^\top,$4 in about $\Sigma_n = R_n S_n S_n^\top R_n^\top,$5 ms. The method’s pose estimation pipeline therefore remains direct: no dense refinement is required to achieve the reported state of the art, although an optional GS-CPR refinement produces the SplitGS-Loc$\Sigma_n = R_n S_n S_n^\top R_n^\top,$6 variant on 7Scenes (Lee et al., 8 May 2026).

6. Empirical performance, ablations, and limitations

Evaluation is reported on Cambridge Landmarks, comprising five outdoor scenes up to approximately $\Sigma_n = R_n S_n S_n^\top R_n^\top,$7, and on 7Scenes, comprising seven indoor scenes spanning $\Sigma_n = R_n S_n S_n^\top R_n^\top,$8–$\Sigma_n = R_n S_n S_n^\top R_n^\top,$9. Metrics are median translation error in centimeters and rotation error in degrees per scene, using SfM pseudo-ground truth from Brachmann et al. 2021 rather than the original D-SLAM ground truth (Lee et al., 8 May 2026).

Dataset / variant	Map size	Average error
Cambridge Landmarks, SplitGS-Loc	97 MB	8.7 cm / 0.15°
7Scenes, SplitGS-Loc	88 MB	1.0 cm / 0.26°
7Scenes, SplitGS-Loc$(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$00	~175 MB	0.6 cm / 0.19°

On Cambridge Landmarks, SplitGS-Loc uses a mapping time of $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$01 minutes, consisting of $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$02 minutes for 3DGS training and approximately $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$03 minute for GSFF construction. The per-scene results are GreatCourt $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$04 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$05, King’s College $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$06 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$07, Old Hospital $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$08 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$09, Shop Facade $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$10 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$11, and St. Mary’s Church $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$12 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$13. The paper compares these results against classical feature matching, scene coordinate regression, NeRF-based localization, and GS-based methods including GSplatLoc, GSFFs-PR, STDLoc, GSVisLoc, RAP, GS-RelocNet, and PlugGS-Loc, and reports that SplitGS-Loc is competitive with or better than both training-free and per-scene-trained alternatives while avoiding hours of mapping time and dense pose refinement in its base form (Lee et al., 8 May 2026).

On 7Scenes, the base SplitGS-Loc variant also uses a $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$14 minute mapping time and achieves $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$15 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$16 with an $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$17 MB map. The refined SplitGS-Loc$(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$18 variant reaches $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$19 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$20 with a map size of approximately $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$21 MB, matching or surpassing the best per-scene-trained GS and NeRF methods listed in the comparison. A plausible implication is that the correspondence quality produced by the base feature field is already sufficiently strong that refinement acts as an optional enhancer rather than as a structural necessity.

The ablation study clarifies the source of the gains. Varying the splitting factor $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$22 shows monotonically improving accuracy as $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$23 increases within the feasible range, supporting the interpretation that wider separation of child Gaussians reduces correspondence ambiguity. Varying the primary-Gaussian threshold $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$24 shows that values that are too low admit too many weak associations and worsen feature blurring, whereas values that are too high reduce coverage; the method fixes $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$25 and reports robustness in the range $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$26–$(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$27. Component ablations report $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$28 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$29 for “STDLoc GSFFs + cosine similarity sampling,” $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$30 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$31 when replacing the GSFFs with PlugGS-Loc’s feature lifting under the same sampling, and $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$32 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$33 after adding SplitGS-Loc’s weight-based sampling. A separate attribute ablation reports $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$34 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$35 for Projection+Average, $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$36 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$37 for PlugGS-Loc, $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$38 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$39 for nearest-neighbor upsampling, and $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$40 cm / $(\mathbf{u}_k \leftrightarrow \mathbf{X}_k)$41 for SplitGS-Loc. These results are presented as evidence that both composition weights and covariance-guided splitting are essential, and that increasing Gaussian count alone is insufficient (Lee et al., 8 May 2026).

The principal limitation identified by the paper is dependence on the quality of the underlying photometric 3DGS. Challenging scenes with severe illumination changes, dynamic content, or very sparse coverage in large-scale environments remain difficult when the base Gaussian scene fails to reconstruct reliably. SplitGS-Loc does not solve that upstream failure mode; it inherits the reconstruction limits of the photometric representation on which it is built. This suggests that future progress is likely to depend on advances in neural scene representations for difficult capture conditions as much as on further improvements in correspondence disambiguation itself (Lee et al., 8 May 2026).