Disambiguating 2D-3D Correspondences in Gaussian Splatting-based Feature Fields for Visual Localization

Abstract: While Gaussian Splatting-based Feature Fields (GSFFs) have shown promise for visual localization, this paper highlights that photometrically optimized GSFFs are inherently ill-suited for 2D-3D matching. The volumetric extent of each Gaussian induces many-to-one pixel-to-point mappings that destabilize PnP-based pose estimation, while photometric optimization gives rise to superfluous Gaussians devoid of multi-view consistency. To address these issues, we propose SplitGS-Loc, a localization-specialized GSFFs construction framework that disambiguates 2D-3D correspondences by exploiting Gaussian attributes. Our key design, Mixture-of-Gaussians-based splitting, decomposes each Gaussian into smaller Gaussians, replacing ambiguous many-to-one with precise one-to-one correspondences. In parallel, we exploit composition weights from GS rasterization to select Gaussians that significantly and consistently contribute across multiple views and aggregate discriminative features through strong pixel-Gaussian associations, enforcing multi-view consistency. The resulting compact yet discriminative feature fields enable stable PnP convergence, achieving state-of-the-art performance on localization benchmarks. Extensive experiments validate that SplitGS-Loc extends the utility of photometric GSFFs to accurate and efficient localization by exploiting Gaussian attributes, without per-scene training or iterative pose refinement.

• The paper introduces SplitGS-Loc, which disambiguates many-to-one 2D-3D correspondences in Gaussian Splatting-based Feature Fields to stabilize PnP and improve RANSAC inlier selection.
• It employs a deterministic Mixture-of-Gaussians splitting strategy combined with composition-weight-based feature aggregation to enhance spatial resolution and localization accuracy.
• Empirical results on benchmarks such as Cambridge Landmarks demonstrate that SplitGS-Loc achieves superior pose estimation efficiency and robustness compared to traditional photometric GSFF methods.

Disambiguating Many-to-One 2D-3D Correspondences in Gaussian Splatting for Visual Localization

Introduction

Gaussian Splatting-based Feature Fields (GSFFs) have rapidly gained traction in visual localization tasks, offering a volumetric 3D representation that bridges geometric and learned feature modalities. However, conventional GSFFs constructed via photometric optimization are not directly suited for 2D-3D matching: the volumetric support of each Gaussian induces a many-to-one mapping from 2D pixels to single 3D points, which destabilizes PnP-based pose estimation and undermines inlier selection during RANSAC. Additionally, superfluous Gaussians with poor multi-view consistency proliferate due to the reliance on color-centric objectives rather than direct correspondence fidelity.

This work introduces SplitGS-Loc, a GSFFs construction framework specialized for visual localization, which disambiguates 2D-3D correspondences by both algorithmic splitting of Gaussians and robust feature assignment using composition weights. The method yields state-of-the-art accuracy and efficiency without per-scene training or iterative pose refinement.

Figure 1: (a) Many-to-one pixel correspondences occur when each Gaussian covers a large volumetric zone; splitting Gaussians resolves this. (b) The proposed method balances mapping efficiency with localization accuracy.

Limitations of Photometric GSFFs for 2D-3D Matching

Current GSFFs built atop photometrically optimized Gaussian Splatting are inherently ambiguous for correspondence-based localization. Each Gaussian spans a volumetric region, so multiple image pixels are associated via rasterization with the same 3D point, resulting in non-injective mappings that violate the assumptions of PnP solvers. Additionally, the photometric loss spawns redundant Gaussians capturing view-dependent effects without enforcing multi-view feature consistency. Prior approaches, such as STDLoc, mitigate some of these deficiencies via dense refinement but require scene-specific training, pose-side detectors, and large memory budgets, detracting from deployment practicality.

Figure 2: PlugGS-Loc pipeline leverages composition weights for efficient feature association and Gaussian selection in GSFFs.

PlugGS-Loc: Efficient Feature Lifting via Composition Weights

To mitigate feature blurring from rendering-based optimization, PlugGS-Loc directly aggregates image features onto Gaussians by thresholding composition weights, capturing only those Gaussians that significantly and consistently contribute to rasterization across multiple views. Each Gaussian is assigned a multi-view aggregated feature reflecting robust pixel-level associations, bypassing the need for feature rendering loss and sidestepping the pathologies of gradient diffusion.

Gaussian sampling is informed by the multi-view composition-weight statistics, favoring Gaussians that maintain salient visibility across training images and enabling a compact yet discriminative GSFF suitable for robust correspondence formation.

SplitGS-Loc: Disambiguation through Mixture-of-Gaussians-Based Decomposition

SplitGS-Loc generalizes PlugGS-Loc by structurally addressing the core ambiguity: each Gaussian is decomposed into several finer Gaussians along its major axis using a deterministic Mixture-of-Gaussians split. This process increases the spatial resolution and injectivity of the 2D-3D correspondence mapping while distributing opacities and inherited features in a manner that respects the statistical properties of the parent.

The parameters for splitting (e.g., weights, variances, positional offsets) are derived via four-moment matching, guaranteeing that the children collectively preserve the spatial support and mass distribution of the original Gaussian. This splitting is computationally efficient and maintains compatibility with volumetric rendering, as each child Gaussian remains differentiable and amenable to standard pipeline processing.

Figure 3: Ablation shows that increasing the splitting factor $\beta$ consistently improves localization performance, and judicious selection of the pixel-Gaussian weight threshold controls feature sharpness and coverage.

Following splitting, assignment of features and composition-weight-based sampling proceeds as in PlugGS-Loc, but the increased granularity substantially reduces many-to-one correspondences, as evidenced by more robust PnP convergence and larger valid inlier sets during RANSAC (see Section PnP Convergence).

Figure 4: SplitGS-Loc prunes redundant Gaussians and focuses multi-view feature aggregation on discriminative components, enhancing correspondence quality.

Empirical Results and Comparative Analysis

SplitGS-Loc establishes strong numerical results across standard benchmarks, including Cambridge Landmarks and 7Scenes. On the largest scene in Cambridge Landmarks (GreatCourt), the method outperforms both training-intensive and training-free baselines, achieving lower median translation and rotation errors compared to STDLoc, GS-RelocNet, and NeRF-based pipelines—without iterative refinement or per-scene model adaptation.

PlugGS-Loc, while slightly less accurate, still outperforms several state-of-the-art methods with substantially reduced computational and memory overhead, validating the composition-weight-based aggregation as a robust alternative to rendering-based GSFF optimization.

Ablation studies indicate that the composition weight threshold $\tau$ and the Gaussian splitting factor $\beta$ are critical hyperparameters: moderate pruning yields sharp features and accurate localizations, while overly aggressive pruning reduces map coverage and robustness. The splitting strategy demonstrates monotonic gains with respect to $\beta$, constrained only by the positive semi-definiteness of the covariance matrix.

Figure 5: Visual comparison of PlugGS-Loc and SplitGS-Loc camera pose predictions on GreatCourt. SplitGS-Loc stabilizes pose estimation even in challenging, highly ambiguous foreground regions.

PnP Convergence and Map Scalability

By eliminating many-to-one matches, SplitGS-Loc increases the number of valid inliers input to PnP solvers and accelerates RANSAC convergence. Experiments confirm that under equivalent compute budgets, SplitGS-Loc yields both better accuracy and runtime compared to STDLoc—a result persisting even under reduced RANSAC iteration regimes.

Performance scales with map size, but unlike naive upsampling or nearest neighbor interpolation, Mixture-of-Gaussians-based splitting ensures that map densification preserves feature distinctiveness and geometric plausibility.

Theoretical and Practical Implications

This work demonstrates that direct exploitation of volumetric Gaussian attributes—composition weights and principal axis factorization—enables construction of GSFFs that are inherently suited for 2D-3D correspondence tasks in visual localization. The SplitGS-Loc framework obviates the need for multi-stage per-scene optimization pipelines and renders direct matching both scalable and accurate.

The methodology reframes the use of neural scene representations: instead of merely being compact archives for view synthesis or refinement, GSFFs built via these mechanisms become deployable assets for efficient, on-the-fly localization in both controlled and unconstrained scenarios.

Future Outlook

While the framework is limited by the initial quality of the photometric GS training, particularly under varying illumination or severely undersampled scenes, its adaptability to advances in GS scene representations and differentiable rendering makes it likely to remain state-of-the-art as neural representation capabilities expand. Future extensions may target better robustness to real-world degradations and explore integration with multi-modal feature lifting (e.g., language-guided GSFFs).

Conclusion

SplitGS-Loc introduces a principled, deployment-oriented approach for constructing localization-specialized Gaussian Splatting-based Feature Fields, resolving many-to-one 2D-3D correspondence ambiguities that plague prior GSFFs. By coupling Mixture-of-Gaussians decomposition with composition-weight-informed feature assignment, the method ensures stable PnP convergence, efficient map utilization, and strong empirical results, laying the foundation for practical, high-precision localization pipelines without per-scene retraining or computationally intensive refinement (2605.07351).