---
title: 'SGAD-SLAM: Depth-Adjusted RGB-D SLAM'
url: https://www.emergentmind.com/topics/sgad-slam
type: topic
---

# SGAD-SLAM: Depth-Adjusted RGB-D SLAM

SGAD-SLAM is a RGB-D SLAM (Simultaneous Localization and Mapping) system that introduces a novel formulation for scene representation and tracking based on pixel-aligned 3D Gaussian splatting with learnable depth adjustments. In contrast to traditional approaches using either unconstrained 3D Gaussians or over-constrained view-tied 3D Gaussians to represent radiance fields, SGAD-SLAM proposes per-pixel, depth-adjustable Gaussians to balance flexibility, accuracy, and computational efficiency in radiance field mapping and camera tracking [2603.21055].

## 1. System Architecture and Radiance Field Representation

SGAD-SLAM represents each RGB-D camera frame as a set of pixel-aligned 3D Gaussians. For pixel $j$ in frame $i$, a Gaussian $g_i^j = \{\mathbf{c}_i^j, r_i^j, \alpha_i^j, \delta_i^j\}$ is defined with the following attributes: 
- $\mathbf{c}_i^j \in \mathbb{R}^3$: per-pixel color (RGB radiance)
- $r_i^j > 0$: standard deviation ("radius")
- $\alpha_i^j > 0$: opacity (peak density)
- $\delta_i^j \in \mathbb{R}$: learnable depth offset along the viewing ray

The Gaussian center in world coordinates is
$$\mathbf{p}_i^j = \mathbf{t}_i + |D_i(j) + \delta_i^j|\,\mathbf{u}_i^j,$$
where $\mathbf{t}_i$ is the camera translation, $D_i(j)$ the observed depth at pixel $j$, and $\mathbf{u}_i^j$ the world-space ray direction. The absolute value ensures center positions remain physically valid.

The resulting density and radiance fields are:
- Density:
  $$\sigma_i^j(\mathbf{x}) = \alpha_i^j \exp\left(-\frac{\|\mathbf{x} - \mathbf{p}_i^j\|^2}{2(r_i^j)^2}\right), \quad \mathbf{x} \in \mathbb{R}^3$$
- View-independent Radiance:
  $$\mathbf{c}_i^j(\mathbf{x}, \mathbf{d}) = \mathbf{c}_i^j$$

Only the Gaussians of the current frame and a small window of neighbor frames are stored and optimized, which restricts GPU memory usage and computation.

## 2. Depth Adjustment Mechanism and Mapping Optimization

The pixel-aligned Gaussians in SGAD-SLAM include a depth offset $\delta_i^j$. During mapping, these offsets are optimized to adjust the center positions along each pixel’s viewing ray, allowing precise radiance field reconstruction without unconstrained 3D Gaussian motions.

The mapping loss,
$$
\mathcal{L}_{\mathrm{map}} = \sum_{k \in \{i, NN(i)\}} \left[ \rho \|V_k - V_k'\|_1 + \tau L_{\mathrm{SSIM}}(V_k, V_k') + \sigma \| U_k \odot (D_k - D_k') \|_1 \right],
$$
uses rendered vs. ground-truth RGB ($V$), a structural similarity measure (SSIM), and rendered vs. observed depth ($D$), with $U_k$ masking invalid depth. The differentiable splatting operator generates the rendered images for optimization. Gradient computations for $\delta_i^j$ propagate through the chain rule applied to the rendering pipeline.

## 3. Rendering Pipeline

SGAD-SLAM employs volumetric ray splatting for rendering. For a camera ray $\mathbf{r}(t) = \mathbf{o} + t\mathbf{d}$, the color is computed as:
$$
C(\mathbf{r}) = \sum_{i=1}^N T_i \big(1 - \exp(-\alpha_i \Delta t_i)\big) \mathbf{c}_i,
$$
where $T_i = \exp\left(-\sum_{j<i} \alpha_j \Delta t_j\right)$ is accumulated transmittance and $\Delta t_i = r_i^j$ is the "thickness" of the Gaussian. The depth image is rendered as the expected integration of depth values, using the same transmittance weighting.

This enables end-to-end differentiable rendering necessary for the joint optimization of radiance and depth parameters.

## 4. Frame-to-Map Tracking via Gaussian Geometry and GICP

For each frame, geometry extraction and tracking proceed as follows:
- Uniformly downsampled depth pixels are back-projected to 3D positions $\mathbf{q}_i^j$.
- Local geometry about each point is modeled using the covariance of $K_c$ (typically 10) nearest 3D points, forming a geometry Gaussian $\mathcal{N}(\mathbf{q}_i^j, C_i^j)$.
- A global map $T$ accumulates non-overlapping geometry Gaussians from previous frames.
- Frame alignment is posed as a Generalized ICP optimization:
  $$E_{\mathrm{GICP}}(p_i) = \sum_{a \in T_i, b \in T} (p_i \mathbf{q}_a - \mathbf{q}_b)^\top (C_a + C_b)^{-1} (p_i \mathbf{q}_a - \mathbf{q}_b)$$
  This is minimized with respect to the rigid transformation $(R_i, t_i)$ of the current frame pose, using alternating correspondence assignment and closed-form updates.

Residuals are also formulated for point-to-plane minimization within the Levenberg–Marquardt scheme.

## 5. System Design Choices and Scalability

SGAD-SLAM leverages several design principles to enhance scalability:
- Spherical Gaussians (1 radius) are used, avoiding the larger parameter count of ellipsoidal Gaussians.
- Only current and nearest-neighbor frame Gaussians are actively stored and optimized; no global dense map is held in GPU memory.
- Radii are scale-normalized across frames for robust field matching.
- The exclusion of local densification and rotation attributes increases computational throughput.

The result is a system with significantly lower memory and computation demands compared to methods using scene-wide Gaussian fields.

| Gaussian Attributes | SGAD-SLAM | Full Ellipsoid |
|----------------------|-----------|---------------|
| Radius               | 1         | 3             |
| Opacity              | 1         | 1             |
| Color                | 3         | 3             |
| Depth Offset         | 1         | 0             |
| Rotation             | 0         | 3             |
| Total                | 6         | 14            |

## 6. Experimental Evaluation

SGAD-SLAM has been evaluated on the Replica, TUM-RGBD, ScanNet, and ScanNet++ datasets using standard metrics: Absolute Trajectory Error (ATE RMSE, cm) for tracking and PSNR, SSIM, LPIPS for radiance field accuracy; Depth L1 and F1 for mesh reconstruction.

On Replica (8 scenes): ATE RMSE was 0.16 cm, matching GS-ICP and bettering NICE-SLAM (1.95 cm) and VTGS-SLAM (0.28 cm). Mapping quality: PSNR 44.87 dB, SSIM 0.998, LPIPS 0.021.

On TUM-RGBD (3 scenes): ATE 2.0 cm; PSNR 38.60 dB, SSIM 0.997, outperforming VTGS on radiance quality. On ScanNet: ATE 7.9 cm, PSNR 42.31 dB, SSIM 0.997.

In terms of efficiency on Replica: tracking time per frame is 0.01 s, mapping 0.89 s; overall 0.90 s. Compared to NICE-SLAM (2.21 s), GS-ICP (1.05 s), and SplaTAM (7.59 s). Scene-wide, 326 million Gaussians are used, with 0.816 million learnable Gaussians per frame.

## 7. Implementation and Optimization Details

Key configuration parameters include:
- Gaussians per frame $J = H \times W$ (e.g., 480$\times$640 $\approx$ 307K), downsampled by $R=1/4$ for tracking ($\sim$80K).
- Number of neighbors for covariance: $K_c=10$; neighbor frames $NN=3$.
- Mapping: Adam optimizer, learning rate $1\mathrm{e}{-2}$ (decreased to $1\mathrm{e}{-3}$ after 100 iterations), 200 mapping iterations per frame.
- Tracking: 10 iterations per frame.
- Hardware: NVIDIA A100; mapping one scene in $\sim$12 min using 8 GPUs (frame time $0.15$ s); single GPU frame rate $\sim$1.1 Hz, 8 GPUs at $\sim$6 Hz.

A succinct pseudocode for the main workflow is as follows:
```python
Initialize global map T = ∅
For each RGBD frame i = 1...I:
    if i == 1:  use odometry or constant-speed for p_i
    else: 
        p_i ← previous pose; optionally refine by rendering-based init
    Tracking:
        Build T_i = {Gaussians from D_i with (q,C)}
        Estimate p_i = argmin_p E_GICP(p; T_i, T)
    Update T by merging non-overlapping Gaussians from T_i with pose p_i
    Mapping:
        Initialize pixel-aligned Gaussians G_i on D_i
        for iter = 1...N_map_iters:
            Render {(V_k', D_k')} = splat(G_i, p_k), k ∈ {i, NN(i)}
            Compute mapping loss ℒ_map(G_i, δ_i)
            Backprop to update {c, r, α, δ} for G_i
End for
```

## 8. Context and Implications

SGAD-SLAM’s adoption of depth-adjustable, pixel-aligned 3D Gaussians introduces an effective trade-off between geometric flexibility (for high-quality radiance field estimation) and computational tractability (scalability, memory usage, and speed). The empirical results demonstrate competitive or superior accuracy in both tracking and mapping compared to previous SLAM systems employing deep implicit or explicit geometric models, such as NICE-SLAM, GS-ICP, VTGS, and SplaTAM. The system architecture supports efficient deployment for large-scale and real-time scenarios using commodity GPUs.

Potential implications include adaptation to relocalization, loop closure, or semantic mapping tasks, as well as extension to higher-level scene understanding, as the underlying representation naturally supports dense, differentiable, and photometrically meaningful radiance field construction [2603.21055].

Source: https://www.emergentmind.com/topics/sgad-slam