GlORIE-SLAM: Globally Optimized RGB-Only SLAM

• GlORIE-SLAM is a monocular dense SLAM approach characterized by a deformable neural point cloud that separates geometric re-anchoring from fixed learned features.
• The system introduces a DSPO layer that jointly optimizes camera poses, high-error disparities, and monocular depth scale/shift to efficiently address depth ambiguity and noise.
• Evaluated on datasets like Replica, TUM-RGBD, and ScanNet, it demonstrates competitive rendering, reconstruction, and tracking metrics while managing RGB-only limitations.

Searching arXiv for GlORIE-SLAM and closely related SLAM papers to ground the article with current citations. GlORIE-SLAM, short for “Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM,” is a monocular dense SLAM framework that combines a globally optimized tracking backend with a deformable neural point cloud map. It is designed for RGB-only operation, without a depth sensor, and targets two coupled difficulties in monocular dense SLAM: maintaining a globally consistent dense map and camera trajectory over long sequences, with loop closures and global bundle adjustment, and handling depth and scale ambiguity efficiently. Its central design choice is to represent the scene as a neural point cloud whose 3D positions can be re-anchored when keyframe poses or depths are refined, while its learned features remain fixed; this avoids costly backpropagation through large grid encodings during global corrections. The system further introduces a DSPO layer that jointly optimizes camera poses, per-keyframe disparity maps, and monocular depth scale and shift with the aid of a monocular depth estimator (Zhang et al., 2024).

1. Problem setting and defining contributions

GlORIE-SLAM addresses RGB-only dense SLAM in a regime where dense reconstruction, rendering, and long-horizon trajectory consistency must be achieved from monocular video alone. The system is explicitly positioned against RGB-only dense SLAM methods that predominantly use grid-based neural implicit encodings and/or struggle to efficiently realize global map and pose consistency. Its proposed alternative is an efficient RGB-only dense SLAM system using a flexible neural point cloud scene representation that adapts to keyframe poses and depth updates without needing costly backpropagation (Zhang et al., 2024).

Two contributions define the method. First, it uses a deformable neural point cloud for RGB-only dense SLAM. Each point stores learned geometric and color features, while its 3D location remains tied to the keyframe pose and depth from which it was initialized. When loop closure or bundle adjustment changes the geometry, the point positions are re-anchored rather than re-trained. Second, it introduces the DSPO layer, standing for Disparity, Scale and Pose Optimization. DSPO uses a monocular depth prior per keyframe, splits each disparity map into low-error and high-error pixels via multi-view consistency, and jointly optimizes camera poses, disparity at high-error pixels, and monocular depth scale and shift in an alternating optimization scheme (Zhang et al., 2024).

A recurrent misconception in dense neural SLAM is that global map consistency in monocular settings effectively requires grid or hash-grid features to be re-optimized after loop closure. GlORIE-SLAM is presented as a counterexample to that assumption. In the formulation described by its authors, map corrections are purely geometric: only point locations are updated, while the neural features remain fixed. This suggests a deliberate separation between appearance encoding and geometric anchoring, with the latter handled by classical optimization rather than repeated neural retraining (Zhang et al., 2024).

2. Tracking backend, keyframes, and factor-graph organization

The online pipeline is divided into a tracker and a mapper, connected through keyframes and depth. The input is an RGB video stream. For each incoming frame, optical flow to the last keyframe is computed via a RAFT-like recurrent optical flow network; if the mean flow magnitude exceeds a threshold $\tau$, the frame is promoted to a keyframe and inserted into a factor graph. Each keyframe carries an initial pose estimate, a semi-dense disparity map predicted by the tracking module, and an RGB image (Zhang et al., 2024).

The tracker maintains a factor graph $\mathcal{G}(\mathcal{V}, \mathcal{E})$. Nodes $\mathcal{V}$ correspond to keyframes, each parameterized by a pose $\omega_i \in SE(3)$ and a disparity map $d_i$. Edges $\mathcal{E}$ encode optical-flow constraints between keyframe pairs. Tracking proceeds through local dense bundle adjustment over a sliding window, alternating with DSPO, and is augmented by occasional loop-closure edges and global bundle adjustment. In this design, the factor graph is not only a trajectory-optimization structure but also the mechanism through which depth updates propagate to the map (Zhang et al., 2024).

The dense bundle-adjustment baseline follows the DROID-SLAM-style formulation in which poses and disparities are optimized from optical-flow reprojection residuals. Let $\tilde{p}_{ij}$ denote the predicted pixel coordinates when keyframe $i$’s pixel grid is projected into keyframe $j$ using optical flow. With disparity as inverse depth, the objective is

$$\arg\min_{\{\omega_i, d_i\}} \sum_{(i,j)\in\mathcal{E}} \left\| \tilde{p}_{ij} - K \omega_j^{-1}\big(\omega_i (1/d_i) K^{-1}[p_i,1]^T\big) \right\|^2_{\Sigma_{ij}}.$$

This is solved with Gauss–Newton and a Schur-complement reduction over poses and disparities. GlORIE-SLAM then alternates this DBA stage with DSPO rather than treating dense bundle adjustment as sufficient on its own (Zhang et al., 2024).

3. Deformable neural point cloud and implicit rendering

The scene map is a neural point cloud

$\mathcal{G}(\mathcal{V}, \mathcal{E})$0

where $\mathcal{G}(\mathcal{V}, \mathcal{E})$1 is the point position in world coordinates, $\mathcal{G}(\mathcal{V}, \mathcal{E})$2 is a geometric feature, $\mathcal{G}(\mathcal{V}, \mathcal{E})$3 is a color feature, $\mathcal{G}(\mathcal{V}, \mathcal{E})$4 is the index of the keyframe that anchored the point, $\mathcal{G}(\mathcal{V}, \mathcal{E})$5 is the originating pixel coordinate in that keyframe, and $\mathcal{G}(\mathcal{V}, \mathcal{E})$6 is the depth at initialization. The point features act as implicit encodings: at rendering time, features from nearby points are interpolated and fed into small MLP decoders (Zhang et al., 2024).

Point insertion depends on a proxy depth map $\mathcal{G}(\mathcal{V}, \mathcal{E})$7 constructed for each keyframe. New points are added by sampling $\mathcal{G}(\mathcal{V}, \mathcal{E})$8 pixels uniformly and $\mathcal{G}(\mathcal{V}, \mathcal{E})$9 pixels among the top $\mathcal{V}$0 pixels by color-gradient magnitude $\mathcal{V}$1, thereby balancing spatial coverage and local detail. For each sampled pixel, the system projects the proxy depth into 3D and searches for existing points within a dynamic radius

$\mathcal{V}$2

If no point is found, three points are initialized along the viewing ray at depths $\mathcal{V}$3, $\mathcal{V}$4, and $\mathcal{V}$5. The radius therefore adapts both to unknown monocular scale and to local texture (Zhang et al., 2024).

The proxy depth map itself fuses two sources: multi-view filtered keyframe depths from the tracker, denoted $\mathcal{V}$6, and a monocular depth prior $\mathcal{V}$7 from an Omnidata-based mono-depth network. The procedure first applies two-view consistency filtering, then projects all valid pixels into a fused point cloud and reprojects that cloud into each keyframe to obtain a near-dense depth map $\mathcal{V}$8. Because monocular depth is only defined up to scale and shift, each keyframe estimates alignment parameters by least squares,

$\mathcal{V}$9

and fills missing regions of the fused map with the aligned monocular prior. The resulting proxy depth is almost dense and scale-corrected, and it is used both for mapping supervision and for point creation (Zhang et al., 2024).

Feature interpolation is local and explicitly point-based. For a query point in space, neighbors are searched within radius $\omega_i \in SE(3)$0; at least two neighbors are required, otherwise occupancy is set to zero. Up to eight nearest neighbors are used, with inverse-square distance weights, to form interpolated geometric and color features. These are decoded by two MLPs: an occupancy decoder $\omega_i \in SE(3)$1 and a color decoder $\omega_i \in SE(3)$2. The occupancy decoder is reused from Point-SLAM and kept fixed, while only the color decoder parameters $\omega_i \in SE(3)$3 and the point features are optimized online (Zhang et al., 2024).

Rendering is depth-guided rather than based on dense ray marching. For each pixel, the system samples $\omega_i \in SE(3)$4 points along the ray in the interval $\omega_i \in SE(3)$5, predicts occupancy $\omega_i \in SE(3)$6 and color $\omega_i \in SE(3)$7 for each sample, and uses standard volume-rendering weights

$\omega_i \in SE(3)$8

Rendered depth and color are then accumulated as weighted sums over the samples. This sparse depth-guided sampling is used to keep rendering efficient while concentrating computation near the estimated surface (Zhang et al., 2024).

4. DSPO: disparity, scale, and pose optimization

The DSPO layer is the mechanism through which GlORIE-SLAM introduces a geometric prior into an RGB-only SLAM pipeline. Dense bundle adjustment alone yields noisy disparity maps, and scale estimation can drift. DSPO therefore splits each keyframe disparity map into low-error pixels $\omega_i \in SE(3)$9, judged reliable by multi-view consistency, and high-error pixels $d_i$0, judged unreliable. The optimization then treats only the high-error disparities as free variables, while using both subsets to align the monocular prior (Zhang et al., 2024).

The DSPO objective jointly optimizes high-error disparities together with the monocular depth scale $d_i$1 and shift $d_i$2 for each keyframe. In the notation of the method,

$d_i$3

with $d_i$4 so that reliable pixels exert stronger influence on scale and shift. The first term is a reprojection error over high-error pixels, the second regularizes those pixels toward monocular inverse depth, and the third uses reliable pixels to refine the monocular alignment (Zhang et al., 2024).

Optimization is alternating rather than fully joint. The system alternates between DBA, which optimizes poses $d_i$5 and full disparities $d_i$6, and DSPO, which optimizes high-error disparities $d_i$7, scale $d_i$8, and shift $d_i$9 while holding the poses fixed. The authors explicitly note that simultaneous optimization of poses, disparities, scales, and shifts can be unstable because of scale ambiguity. Both subproblems are solved by Gauss–Newton with a block-structured Jacobian and Schur complement, and the disparity block’s Hessian is diagonal, which makes inversion cheap (Zhang et al., 2024).

The significance of DSPO is not restricted to local tracking. It is used for local bundle adjustment, loop-closure optimization, and online global bundle adjustment. In that sense, DSPO is not merely an auxiliary prior but part of the system’s global consistency strategy. A plausible implication is that GlORIE-SLAM treats monocular depth not as a one-off initialization cue but as a recurrent source of scale regularization throughout the lifetime of the map (Zhang et al., 2024).

5. Geometric deformation, loop closure, and scale consistency

A defining property of GlORIE-SLAM is that map correction is performed by geometric deformation of the point cloud. When bundle adjustment, DSPO, or loop closure refines keyframe poses and depths, each point triplet anchored to a keyframe ray is re-anchored. If an updated depth $\mathcal{E}$0 is available, the central point is moved to

$\mathcal{E}$1

The other two points along the ray are repositioned analogously relative to the central depth. If no updated depth is available, the original depth is rescaled as $\mathcal{E}$2, where $\mathcal{E}$3 is obtained by least-squares fitting the old depth map to the new one. Only the positions $\mathcal{E}$4 are updated; the features $\mathcal{E}$5 and $\mathcal{E}$6 remain fixed, and there is no re-training of features or grid backpropagation (Zhang et al., 2024).

Loop closure is detected through optical flow. For each active keyframe in the local window, the system computes mean flow magnitude to all past keyframes. A pair is accepted as a loop when the mean flow magnitude is below $\mathcal{E}$7, indicating sufficient visual overlap, and the temporal distance is above $\mathcal{E}$8, preventing redundant edges between nearby frames. Each accepted loop adds a unidirectional edge to the factor graph. Optimization is then restricted to the active keyframes and their connected loop nodes, limiting cost, after which the point cloud is deformed according to the updated poses and depths (Zhang et al., 2024).

Global consistency is reinforced by online global bundle adjustment. GlORIE-SLAM maintains a second graph over all keyframes, adds edges based on temporal and spatial distance as in GO-SLAM, and runs global BA every 20 keyframes. Before each global BA, it normalizes disparity and translation scales using the mean disparity $\mathcal{E}$9,

$\tilde{p}_{ij}$0

to improve numerical stability in the monocular setting. Global BA also uses DSPO, so disparity scale and shift as well as high-error disparities are globally refined rather than only locally corrected (Zhang et al., 2024).

Scale ambiguity is addressed through the interaction of three components: least-squares alignment of monocular depth to fused multi-view depth, repeated refinement of per-keyframe scale and shift in DSPO, and global optimization over the full keyframe graph. The system does not claim to remove monocular ambiguity through vision alone; rather, it uses the monocular prior as a frozen external cue whose scale is repeatedly re-estimated under geometric consistency constraints (Zhang et al., 2024).

6. Evaluation, limitations, and position in the literature

GlORIE-SLAM is evaluated on Replica, TUM-RGBD, and ScanNet. Reported metrics cover rendering quality through PSNR, SSIM, and LPIPS; reconstruction quality through Accuracy, Completion, Completion ratio, and Depth L1; and tracking through ATE RMSE. On Replica, it reports PSNR 31.04, SSIM 0.97, LPIPS 0.12, ATE RMSE approximately 0.35 cm, Depth L1 3.24 cm, Accuracy 2.96 cm, Completion 3.95 cm, and Completion ratio 83.72%. On ScanNet, its average rendering scores over six multi-room scenes are PSNR 22.45, SSIM 0.842, and LPIPS 0.304, while average ATE RMSE is reported as 7.5–7.6 cm. On TUM-RGBD, it reports the best average ATE RMSE at 2.1 cm among the listed RGB and RGB-D methods (Zhang et al., 2024).

Dataset	Reported highlights	Metrics
Replica	PSNR 31.04; SSIM 0.97; LPIPS 0.12; ATE RMSE $\tilde{p}_{ij}$1 0.35 cm	Rendering, tracking
Replica	Depth L1 3.24 cm; Accuracy 2.96 cm; Completion 3.95 cm; Completion ratio 83.72%	Reconstruction
ScanNet	PSNR 22.45; SSIM 0.842; LPIPS 0.304; ATE RMSE 7.5–7.6 cm	Rendering, tracking
TUM-RGBD	Best average ATE RMSE 2.1 cm among listed methods	Tracking

The ablations emphasize three effects. First, the deformable point cloud improves PSNR and reconstruction metrics; not deforming points leads to worse geometry and rendering. Second, including the monocular prior in proxy depth improves completeness, increasing completion ratio from 73% to 76.78%. Third, the DSPO layer improves both rendering and reconstruction, with PSNR increasing from 30.66 to 31.04, Depth L1 decreasing from 3.52 to 3.24 cm, Accuracy decreasing from 3.72 to 2.96 cm, and Completion ratio increasing from 83.4% to 83.72%. Visual comparisons further report that DBA alone yields inconsistent depth maps, a DBA-plus-monocular-prior baseline that regularizes all pixels equally remains noisy, and DSPO produces the most coherent and consistent depth (Zhang et al., 2024).

The implementation profile underscores the trade-off between global optimization and throughput. On Replica room0, peak GPU memory is reported as 15.22 GiB on an RTX 3090, comparable to GO-SLAM at 18.5 GiB and SplaTAM at 18.54 GiB, and higher than Point-SLAM at 7.11 GiB due to the larger number of points required in the RGB-only depth setting. Average FPS is reported as 0.23. The system is therefore described as not real-time in the strict sense, although it remains competitive in performance and memory usage within dense neural SLAM (Zhang et al., 2024).

The authors note three limitations. There is no explicit point pruning, so redundant or wrongly positioned neural points may remain due to depth noise even if many are corrected by deformation. Proxy-depth construction is relatively simple, relying on basic least squares and consistency checks rather than more advanced cues such as normal consistency. Runtime is slower than GO-SLAM. Proposed future directions include point pruning or refinement, more sophisticated fusion of monocular and keyframe depth, and hybrid tracking strategies that combine frame-to-frame and frame-to-model tracking (Zhang et al., 2024).

Within the literature, GlORIE-SLAM is positioned against grid-based systems, NeRF-style systems, RGB-D point-based systems, and Gaussian Splatting systems. Its distinct niche is a monocular dense SLAM formulation that uses a deformable neural point cloud rather than grids or Gaussians, integrates monocular depth priors through DSPO, and supports loop closure and online global BA without large-scale backpropagation. A later contrasting direction is GI-SLAM, which adopts a 3D Gaussian Splatting representation, introduces an IMU loss for frame-to-frame pose increments, and supports monocular, stereo, and RGB-D cameras with or without IMU. That comparison highlights a methodological divergence: GlORIE-SLAM centers global map deformation and depth-scale optimization in an RGB-only point-based system, whereas GI-SLAM centers IMU-enhanced tracking and differentiable Gaussian rendering in a 3DGS framework (Liu et al., 24 Mar 2025).