---
title: Triangle Splatting SLAM
url: https://www.emergentmind.com/topics/triangle-splatting-slam
type: topic
---

# Triangle Splatting SLAM

Triangle Splatting SLAM is a dense RGB-D simultaneous localization and mapping (SLAM) system that leverages differentiable triangles as explicit 3D map primitives. Its core innovation is the use of a dynamic “triangle soup” representation optimized online, providing both photorealistic rendering and explicit geometry amenable to downstream tasks such as simulation and mesh editing. Triangle Splatting SLAM employs online differentiable rendering of this triangle soup for both camera tracking and map optimization, and can extract a connected mesh on-the-fly via restricted Delaunay triangulation, supporting live mesh deformation and collision checking. Experimental results demonstrate state-of-the-art geometry accuracy and competitive camera-tracking performance on standard benchmarks [2605.31419].

## 1. System Pipeline and Map Representation

The system maintains a live triangle soup map $M=(V, F)$, where $V$ is the set of 3D vertices and $F$ the connectivity of the triangles. The pipeline operates in a single process, executing three tightly interleaved stages per RGB-D frame:

1. **Tracking:** Estimation of the 6-DOF camera pose $T_{CW}$ via minimization of a tracking energy.
2. **Keyframing:** Keyframe selection based on pose change and triangle visibility; addition of keyframes with new triangles back-projected from depth.
3. **Mapping:** Joint optimization of past keyframe poses and triangle parameters; densification and pruning of triangles; optional mesh extraction.

The pseudocode for the end-to-end, single-threaded pipeline is as follows:

```python
initialize M ← ∅ 
for each incoming RGB-D frame (Ī, D̄):
    # 1. Tracking
    T_CW ← argmin_T  E_track(V, F, T; Ī, D̄)
    # 2. Keyframing
    if is_new_keyframe(T_CW, V, F):
        push keyframe (Ī, D̄, T_CW, visible_triangles)
        spawn_new_triangles(D̄)
    # 3. Mapping
    for each keyframe k in selected_buffer:
        for iter=1…N_map_iters:
            compute E_map over V, F, T_keyframes
            ∇ ← backprop(E_map)
            update V, per-vertex colours c_i, opa o_i, and T_keyframes via Adam
    prune_triangles(V, F)
    densify_triangles(V, F)
    # optional: extract_mesh(V)
end
```

The mapping stage periodically extracts a mesh using restricted Delaunay triangulation, converting the “soup” into a connected surface suitable for simulation or editing.

## 2. Differentiable Triangle Splatting

Each triangle $F_m = (i,j,k)$ is stored with three world-space vertices $v_i = (x_i, y_i, z_i, c_i, o_i)$, where $c_i$ is color and $o_i$ opacity. Differentiable triangle splatting comprises:

- **Projection** into image space: $v_I = \pi(T_{CW}v_W)$.
- **Signed-Distance Field:** The image-space signed distance function $\phi(p)$ is computed relative to projected triangle edges, with a smooth per-pixel coverage function $I(p) = \text{ReLU}(\phi(p)/\phi(s))^\sigma$, where $s$ is the triangle incentre.
- **Alpha-composite Rendering:** Pixel color $C(p)$ is rendered using an alpha compositing stream over triangles,
  $$
  C(p) = \sum_{n=1}^N c_{F_n} o_{F_n} I_n(p) \prod_{i < n} (1 - o_{F_i} I_i(p))
  $$
- **Photometric Loss:** The photometric error over all pixels,
  $$
  L_\text{photo} = \sum_p |I(V, T)(p) - \bar{I}(p)|_1
  $$
- **Backpropagation:** Gradients are computed for vertex positions and appearance, leveraging analytic derivatives (Eq. 4 and window function Eq. 3), the pinhole model, and pose Jacobians in $se(3)$ (Eq. 7).

This differentiable pipeline enables gradient-based optimization of geometry and color parameters directly from image and depth supervision.

## 3. Camera Tracking with Photometric and Depth Alignment

Camera tracking solves for $T_{CW}$ per frame by minimizing a joint energy:
$$
E_\text{track} = E_\text{pho} + \lambda_\text{dep} E_\text{dep}
$$
where

- $E_\text{pho} = (1 - \lambda_\text{ssim}) \, \|I(V, T) - \bar{I}\|_1 + \lambda_\text{ssim} \, D\text{-SSIM}(I(V, T), \bar{I})$ (Eq. 11) is the combined photometric/structural loss,
- $E_\text{dep} = \|D(V, T) - \bar{D}\|_1$ (Eq. 12) aligns rendered and observed depths,
- $\lambda_\text{ssim}$ and $\lambda_\text{dep}$ are tunable hyperparameters.

Approximately 100 gradient-descent steps are performed per frame, using analytic pose Jacobians for efficiency.

## 4. Online Mapping, Densification, and Optimization

Mapping proceeds whenever a new keyframe is added. New triangles are back-projected from depth and assigned spatial support and normals via sensor data (Eq. 14). Optimization is performed by minimizing the mapping energy over vertices, colors, opacities, and past keyframe poses:

$$
E_\text{map} = E_\text{pho} + \lambda_\text{dep} E_\text{dep} + \lambda_\text{norm} E_\text{norm} + \lambda_\text{equi} E_\text{equi}
$$

where:

- $E_\text{norm} = \sum_p (1 - N_\text{render}(p)^\top N_\text{sensor}(p))$ (Eq. 15) penalizes normal misalignments,
- $E_\text{equi} = \frac{1}{|F|} \sum_{f \in F}\frac{1}{3}\sum_\text{angles} (\cos\theta - 0.5)^2$ (Eq. 16) encourages triangle equilateralness.

Optimization uses Adam per-parameter learning rates: positions $(\alpha_v \approx 5 \times 10^{-4})$, colors $(\alpha_c \approx 5 \times 10^{-4})$, and poses $(\alpha_T \approx 0.5\,\alpha_v)$. Densification (blur-split, Loop subdivision) and pruning (opacity and area-based) maintain map quality and efficiency.

## 5. On-the-Fly Mesh Extraction with Restricted Delaunay

To convert the triangle soup into a manifold mesh, restricted Delaunay triangulation is applied:

- Vertices with mean opacity $o_i > \varepsilon_o$ are selected.
- Delaunay tetrahedralisation is constructed in 3D. Only surface faces separating inside/outside are retained.
- Triangles exceeding the projected area threshold or fully occluded from all keyframes are pruned.

Incremental mesh updating is supported:

```python
function incremental_delaunay_update(new_V, old_tets):
    insert new_V into Delaunay structure (e.g. CGAL)
    remove old vertices flagged for pruning
    update restricted facets (alpha complex)
    return new triangle list
end
```

This allows efficient online mesh extraction and supports real-time mesh-based editing, deformation, and collision checking.

## 6. Implementation, Hyperparameters, and System Characteristics

The implementation utilizes a custom CUDA/C++ differentiable rasterizer for triangle splatting, with the SLAM loop managed in PyTorch. Hardware used includes an NVIDIA RTX 4090 GPU and AMD Ryzen 9 9950X CPU. Operational metrics are:

- **Frame time**: 430–1225 ms (0.8–2.3 FPS on TUM-RGBD).
- **Map sizes**: 24k–152k triangles; 4.4–16.4 MB checkpoint size; 0.5–1.25 GB GPU memory.
- **Hyperparameters (Replica benchmarks):** 100 tracking iterations per frame; learning rates for rotation $3 \times 10^{-3}$, translation $1 \times 10^{-3}$, mapping features and vertices $5 \times 10^{-4}$; loss weights $\lambda_\text{equi}=1.2$, $\lambda_\text{norm}=0.05$, $\lambda_\text{norm\_s}=0.15$, $\lambda_\text{dep}(\text{map})=0.05$; keyframing at 5-frame intervals, translation threshold 0.08 m, overlap 0.95; densification and pruning thresholds as specified.

## 7. Evaluation and Comparative Results

Triangle Splatting SLAM achieves:

- **Camera tracking (TUM-RGBD dataset; absolute trajectory error, cm)**:

| Method     | fr1/desk | fr2/xyz | fr3/office | Avg  |
|:-----------|---------:|--------:|-----------:|-----:|
| MonoGS-2D  |    1.58  |   1.20  |      1.83  | 1.54 |
| Ours       |    1.77  |   1.12  |      1.83  | 1.57 |

- **3D geometry (Replica; Chamfer distance in cm, L1 depth in cm):**

| Method                    | Chamfer Avg ↓ | Depth L1 Avg ↓ |
|:--------------------------|:-------------:|:--------------:|
| MonoGS-2D* + TSDF         |     1.36      |     0.74       |
| Ours + TSDF               |   **0.95**    |     0.68       |
| Ours + Delaunay (pruned)  |     1.14      |    —           |

- **Mesh extraction time (Replica; seconds, avg):**

| Method                   | Time [s] Avg ↓ |
|:-------------------------|---------------:|
| Ours + TSDF              |      33.44     |
| Ours + Delaunay          |      11.18     |
| Ours + Delaunay (pruned) |      15.66     |

The method provides live mapping with explicit, editable mesh geometry, supporting photorealistic novel-view rendering and mesh-based downstream tasks, while achieving state-of-the-art 3D geometric accuracy and camera-tracking comparable to established SLAM systems [2605.31419].

Source: https://www.emergentmind.com/topics/triangle-splatting-slam