S3PO-GS: 3D Gaussian SLAM for Outdoor Scenes

• The paper introduces S3PO-GS, a monocular outdoor SLAM system that jointly optimizes camera poses and a 3D Gaussian pointmap to ensure scale consistency.
• It employs a novel patch-based dynamic mapping module that aligns pre-trained pointmap priors with the 3D Gaussian map, reducing iterations and minimizing scale drift.
• Empirical results on datasets like Waymo, KITTI, and DL3DV show significant improvements in tracking accuracy and novel view synthesis compared to prior 3DGS SLAM systems.

S3PO-GS is a monocular, RGB-only SLAM system for outdoor scenes built on 3D Gaussian Splatting (3DGS), designed to jointly optimize camera poses and a 3D Gaussian map that supports real-time, high-fidelity novel view synthesis in challenging large-scale environments with strong viewpoint changes (Cheng et al., 4 Jul 2025). It was introduced to address three limitations identified in prior 3DGS SLAM formulations for outdoor monocular settings: differentiable rendering–only tracking without explicit geometric priors, separate tracking modules whose scale must be continuously re-aligned to the 3DGS representation, and monocular scale ambiguity that can induce scale drift in long trajectories (Cheng et al., 4 Jul 2025). Its central premise is that pose estimation should be anchored to the 3DGS map’s own scale, while pre-trained pointmap priors should be injected in a scale-consistent manner through patch-based dynamic mapping (Cheng et al., 4 Jul 2025).

1. Problem setting and design objective

S3PO-GS targets outdoor navigation scenarios involving streets, cars, vegetation, large rotations, long trajectories, and large viewpoint changes (Cheng et al., 4 Jul 2025). In this regime, globally consistent metric geometry is required for correct relative distances between scene elements, while robust tracking must persist across sharp turns and substantial camera displacement (Cheng et al., 4 Jul 2025). The paper argues that if the scale of the reconstructed Gaussian map drifts, then PnP or re-localization to that map becomes unreliable and novel view synthesis quality deteriorates through ghosting, stretching, and wrong parallax (Cheng et al., 4 Jul 2025).

The method is positioned against two broad classes of prior 3DGS SLAM systems. Differentiable rendering–only tracking, exemplified in the paper by MonoGS, estimates pose by minimizing a photometric loss via differentiable 3DGS rendering, but in outdoor monocular RGB-only setups there are no explicit geometric priors, texture sparsity and repetitive patterns can induce local minima, and convergence requires many iterations per frame while remaining fragile under large viewpoint changes (Cheng et al., 4 Jul 2025). Separate tracking modules with external geometry, such as Photo-SLAM, OpenGS-SLAM, and MGS-SLAM, rely on external depth or VO networks and then align that module’s scale to the 3DGS map; in long outdoor trajectories, small alignment errors accumulate and cause scale drift in both map and trajectory (Cheng et al., 4 Jul 2025).

S3PO-GS addresses this by introducing two coupled components: a self-consistent tracking module anchored in a 3DGS pointmap, and a patch-based pointmap dynamic mapping module (Cheng et al., 4 Jul 2025). This suggests that the system should be understood not merely as a rendering-centric SLAM variant, but as a 3DGS pipeline in which geometric consistency is structurally enforced at both the tracking and mapping levels.

2. Core representation: the 3D Gaussian pointmap

A key concept in S3PO-GS is the “3D Gaussian pointmap,” defined as a per-pixel 3D point map rendered from the Gaussian scene rather than a traditional unordered point cloud (Cheng et al., 4 Jul 2025). From a given camera pose, the system renders a depth map $D \in \mathbb{R}^{W\times H}$ via volumetric alpha compositing of Gaussians along each pixel ray, then back-projects depth pixels using camera intrinsics $K$ to produce a dense 3D pointmap: $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$ This pointmap is defined in the 3DGS scene coordinate system and therefore shares the same global scale as the Gaussian map (Cheng et al., 4 Jul 2025).

The pointmap serves two roles. For tracking, it is the 3D reference for PnP. For mapping, it is the initial geometric template for inserting new Gaussians and supervising geometry refinement (Cheng et al., 4 Jul 2025). This dual use is central to the system’s scale-consistency claim: the 3DGS map is treated as the single global object to which both pose estimation and geometric supervision are anchored.

The underlying Gaussian representation follows standard 3DGS structure, with each Gaussian $i$ parameterized by a mean $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i = R_i S_i S_i^T R_i^T$ with $R_i \in SO(3)$ and diagonal $S_i$, color $c_i \in \mathbb{R}^3$, and opacity $\alpha_i \in [0,1]$ (Cheng et al., 4 Jul 2025). Unlike original 3DGS, S3PO-GS drops spherical harmonics and stores per-Gaussian RGB color directly, a simplification described as suitable for SLAM speed and complexity (Cheng et al., 4 Jul 2025). Rendering uses alpha compositing for both color and depth: $K$0 and these rendered quantities are reused for pointmap construction, tracking, and geometry supervision (Cheng et al., 4 Jul 2025).

3. Tracking: Pointmap-Anchored Pose Estimation and photometric refinement

The tracking module is termed Pointmap-Anchored Pose Estimation (PAPE) and estimates the pose $K$1 of the current frame in the 3DGS map’s scale (Cheng et al., 4 Jul 2025). The system first selects an adjacent keyframe $K$2 with known pose $K$3, then renders a depth map $K$4 and pointmap $K$5 from the current Gaussian map at that pose (Cheng et al., 4 Jul 2025). It next applies a pre-trained pointmap network, specifically MASt3R or DuSt3R, to the image pair $K$6, obtaining pointmaps $K$7, $K$8 and per-pixel confidences $K$9 in the pre-trained model’s own scene-agnostic coordinate system (Cheng et al., 4 Jul 2025).

Correspondences are established by nearest-neighbor search with a confidence threshold, and these image-space correspondences are then lifted to scale-consistent 2D–3D correspondences by associating pixels in the adjacent keyframe with rendered 3D points from $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$0 (Cheng et al., 4 Jul 2025). Pose is then estimated via PnP + RANSAC. Because the 3D points $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$1 used in PnP are in the 3DGS map coordinate system, the estimated pose is directly consistent with the map scale; the pre-trained pointmap network contributes correspondences only and never sets metric scale (Cheng et al., 4 Jul 2025).

The initial PnP estimate is refined by differentiable rendering. Pose $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$2 is represented via Lie algebra $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$3, and the photometric objective is

$$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$4

with analytic pose gradients implemented in CUDA following MonoGS (Cheng et al., 4 Jul 2025). Non-edge or invalid regions are down-weighted so that optimization focuses on informative pixels (Cheng et al., 4 Jul 2025).

A notable empirical claim of the paper is the reduction in required tracking iterations. MonoGS typically needs 50–100 pose iterations for stable convergence in outdoor scenes, whereas S3PO-GS reaches near-optimal accuracy with only 5 iterations because PAPE provides a strong pose initialization and geometric consistency (Cheng et al., 4 Jul 2025). On Waymo_405841, the paper reports that MonoGS fails to converge reliably below 50 iterations, OpenGS-SLAM degrades below 30 iterations, and S3PO-GS attains ATE $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$5 m already at 5 iterations with minimal subsequent improvement (Cheng et al., 4 Jul 2025). This supports the interpretation that the system’s robustness derives less from replacing photometric tracking than from constraining it to a small-refinement regime.

4. Mapping: patch-based scale alignment, point replacement, and geometric supervision

The mapping module is designed to inject geometric priors while avoiding scale ambiguity (Cheng et al., 4 Jul 2025). At each keyframe, two pointmaps are available: the rendered pointmap $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$6, which is in 3DGS scene coordinates and thus has correct global scale but may be noisy or incomplete, and the pre-trained pointmap $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$7 from MASt3R, which offers accurate local geometry but has unknown or drifting global scale relative to 3DGS (Cheng et al., 4 Jul 2025). The task is to estimate a scale factor $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$8 such that $$X^r(i,j) = K^{-1} \begin{bmatrix} iD(i,j) \ jD(i,j) \ D(i,j) \end{bmatrix} \in \mathbb{R}^3.$$9 matches $i$0 in metric scale (Cheng et al., 4 Jul 2025).

Instead of global normalization, S3PO-GS performs scale alignment locally in patches. Both pointmaps are split into $i$1 patches, and each patch is summarized by mean and standard deviation of depth or point values (Cheng et al., 4 Jul 2025). Candidate patches satisfy

$i$2

with example values $i$3 (Cheng et al., 4 Jul 2025). Within candidate patches, values are normalized,

$i$4

and a pixel is marked as a correct point if

$i$5

with $i$6 (Cheng et al., 4 Jul 2025). If the set of correct points $i$7 is sufficiently large, the scale update is

$i$8

The system iterates this process up to a maximum of three times (Cheng et al., 4 Jul 2025). If the number of correct points is too low, it uses nearest-neighbor matches to an already aligned adjacent-keyframe pointmap $i$9 to obtain an alternative scale estimate before optionally running one more patch-based iteration (Cheng et al., 4 Jul 2025).

After scale alignment, S3PO-GS constructs a hybrid pointmap $\mu_i \in \mathbb{R}^3$0 by replacing incorrect points in the rendered pointmap with aligned pointmap values: $\mu_i \in \mathbb{R}^3$1 with $\mu_i \in \mathbb{R}^3$2 (Cheng et al., 4 Jul 2025). The paper reports that when the current view is well covered by the Gaussian map, only about 10% of points are replaced, whereas for large viewpoint changes or sparse coverage, 30–50% may be replaced (Cheng et al., 4 Jul 2025). The hybrid pointmap is then randomly downsampled and used to insert or update Gaussians, with means initialized at $\mu_i \in \mathbb{R}^3$3, colors taken from the keyframe image, and covariances or scales initialized isotropically or via heuristics (Cheng et al., 4 Jul 2025).

Within a local keyframe window $\mu_i \in \mathbb{R}^3$4, the system jointly optimizes camera poses and Gaussian parameters using three losses: a photometric loss, a geometry loss, and isotropic regularization (Cheng et al., 4 Jul 2025). The geometry supervision is

$\mu_i \in \mathbb{R}^3$5

and the full objective is

$\mu_i \in \mathbb{R}^3$6

with $\mu_i \in \mathbb{R}^3$7 and $\mu_i \in \mathbb{R}^3$8 (Cheng et al., 4 Jul 2025). A plausible implication is that S3PO-GS treats pre-trained geometry not as an external geometric reconstruction to be fused wholesale, but as a locally aligned supervisory signal constrained by the current Gaussian map.

5. End-to-end pipeline and scale-consistency mechanism

The system begins by initializing a 3D Gaussian map through optimization of MASt3R’s pointmap for 1000 steps, using the pre-trained pointmap to initialize a base set of Gaussians and yielding an initial metric-scale 3DGS scene (Cheng et al., 4 Jul 2025). For each incoming frame, the nearest keyframe is selected using covisibility-based keyframe management from MonoGS (Cheng et al., 4 Jul 2025). Tracking proceeds through pointmap rendering, MASt3R or DuSt3R matching, PnP + RANSAC, and roughly five iterations of Gauss-Newton pose refinement on $\mu_i \in \mathbb{R}^3$9 (Cheng et al., 4 Jul 2025).

When a frame is chosen as a keyframe based on covisibility and translation thresholds, the mapping stage performs scale alignment, point replacement, Gaussian insertion or update, and local map optimization over a sliding window (Cheng et al., 4 Jul 2025). The paper explicitly states that there is no loop closure or global bundle adjustment; S3PO-GS is a local optimization system, with global pose-graph optimization and global 3DGS refinement left to future work (Cheng et al., 4 Jul 2025).

The paper’s explanation for why scale does not drift is structural. Pose tracking always uses 2D–3D PnP with 3D points $\Sigma_i = R_i S_i S_i^T R_i^T$0 rendered from the 3DGS scene, whose scale is fixed at initialization (Cheng et al., 4 Jul 2025). Mapping always aligns pre-trained pointmaps to the current 3DGS scale before those pointmaps are used for correction or supervision (Cheng et al., 4 Jul 2025). There is therefore no independent external trajectory in another scale. This suggests that scale consistency in S3PO-GS is not an emergent property of optimization, but a design constraint imposed by the data flow between modules.

6. Empirical performance, ablations, and limitations

Experiments were conducted on three outdoor datasets: Waymo Open Dataset with 9 sequences of 200 frames each, KITTI with 8 sequences of 200 frames each, and DL3DV with 3 sequences of 300 frames each (Cheng et al., 4 Jul 2025). Tracking was evaluated with ATE RMSE in meters, and novel view synthesis with PSNR, SSIM, and LPIPS on non-keyframes (Cheng et al., 4 Jul 2025). Baselines included NeRF-SLAM, NICER-SLAM, GlORIE-SLAM, MonoGS, Photo-SLAM, and OpenGS-SLAM, with supplementary comparisons to DROID-SLAM, MASt3R-SLAM, CF-3DGS, and Splat-SLAM on KITTI (Cheng et al., 4 Jul 2025).

Dataset	Tracking result	NVS result
Waymo	S3PO-GS ATE: 0.622; GlORIE-SLAM: 0.589	S3PO-GS PSNR: 26.73, SSIM: 0.845, LPIPS: 0.360
KITTI	S3PO-GS ATE: 1.048	S3PO-GS PSNR: 20.03; SSIM / LPIPS best
DL3DV	S3PO-GS ATE: 0.032	S3PO-GS PSNR: 29.97, SSIM: 0.893, LPIPS: 0.108

On Waymo, S3PO-GS achieved the best NVS results and an ATE of 0.622, second to GlORIE-SLAM’s 0.589 but ahead of OpenGS-SLAM’s 0.839 and MonoGS’s 8.529 (Cheng et al., 4 Jul 2025). On KITTI, it obtained the best ATE at 1.048, outperforming GlORIE-SLAM at 1.134, OpenGS-SLAM at 3.224, and MonoGS at 9.493, while also achieving the best NVS performance (Cheng et al., 4 Jul 2025). On DL3DV, it achieved an ATE of 0.032, compared with 0.141 for OpenGS-SLAM, 0.492 for GlORIE-SLAM, and 0.274 for MonoGS, with PSNR 29.97, SSIM 0.893, and LPIPS 0.108 (Cheng et al., 4 Jul 2025). The paper also reports relative ATE reductions over OpenGS-SLAM of approximately 67.5% on KITTI and 77.3% on DL3DV, and PSNR improvements of $\Sigma_i = R_i S_i S_i^T R_i^T$1, $\Sigma_i = R_i S_i S_i^T R_i^T$2, and $\Sigma_i = R_i S_i S_i^T R_i^T$3 over the previously best 3DGS SLAM on Waymo, KITTI, and DL3DV respectively (Cheng et al., 4 Jul 2025).

The ablation studies attribute these gains to the interaction of all major components. Without PAPE, ATE increases dramatically and sensitivity to iteration count returns; with PAPE, ATE remains around 0.55 m from 5 to 100 iterations on Waymo (Cheng et al., 4 Jul 2025). Without pose refinement, ATE rises to 1.79 and PSNR drops to 24.45, compared with 0.62 and 26.73 with refinement (Cheng et al., 4 Jul 2025). Without scale alignment, ATE becomes 3.50 and PSNR 23.49; without point replacement, ATE is 1.35 and PSNR 25.59; without geometry loss, ATE is 3.73 and PSNR 25.70 (Cheng et al., 4 Jul 2025). A direct “MonoGS + MASt3R” configuration yields ATE 3.84 and PSNR 23.09, accompanied by severe blurring and ghosting due to scale drift if pointmaps are used naively (Cheng et al., 4 Jul 2025). This directly addresses a possible misconception: pre-trained pointmaps alone do not eliminate monocular scale ambiguity, and improper integration can worsen both tracking and reconstruction.

In supplementary KITTI comparisons, S3PO-GS on KITTI-07 is reported at ATE 0.55 m, PSNR 20.6, GPU memory around 9.5 GB, and full SLAM runtime around 5 minutes for the sequence, excluding an additional 10-minute color refinement used for all 3DGS methods in those comparisons (Cheng et al., 4 Jul 2025). Memory is described as comparable to MonoGS and OpenGS-SLAM, runtime as similar to MonoGS and much faster than Splat-SLAM or CF-3DGS (Cheng et al., 4 Jul 2025).

The authors explicitly note two limitations: no dynamic object handling and no loop closure or global bundle adjustment (Cheng et al., 4 Jul 2025). The pipeline assumes static scenes, so moving vehicles or pedestrians are not handled explicitly, and long sequences with large loops could benefit from loop detection, global pose graph optimization, and global 3DGS refinement (Cheng et al., 4 Jul 2025). Other implicit constraints named in the paper are reliance on a strong pre-trained pointmap model such as MASt3R and a heavy GPU requirement for real-time-like operation on large outdoor sequences (Cheng et al., 4 Jul 2025).