---
title: 'TRGS-SLAM: Thermal Inertial 3DGS-SLAM'
url: https://www.emergentmind.com/topics/trgs-slam
type: topic
---

# TRGS-SLAM: Thermal Inertial 3DGS-SLAM

TRGS-SLAM is a 3D Gaussian Splatting (3DGS) based thermal inertial SLAM system for **blurry, rolling shutter, and noisy thermal images**, specifically targeting **uncooled microbolometer thermal cameras** under **motion blur, rolling shutter distortions, and fixed pattern noise (FPN)** [2603.20443]. It combines a **3DGS map**, a **continuous-time B-spline trajectory**, and **IMU fusion**, and introduces a **microbolometer-aware rendering model** together with **B-spline trajectory optimization with a two-stage IMU loss, view-diversity-based opacity resetting, and pose drift correction schemes**. The system is designed for the regime in which frame-to-frame thermal frontends and RGB-oriented assumptions break down, and it further supports **offline refinement** that yields thermal image restoration competitive with prior work that required ground truth poses [2603.20443].

## 1. Problem setting and system definition

TRGS-SLAM is motivated by the observation that thermal cameras are attractive for SLAM because they are **a passive, low-power solution to operating in darkness**, are **invariant to rapidly changing or high dynamic range illumination**, and **can see through fog, dust, and smoke**. The practical thermal sensors used in robotics, however, are typically **uncooled microbolometers**, and these exhibit an image formation process that is unfavorable for conventional SLAM: **motion blur** governed by a thermal response model, **rolling shutter**, **strong, slowly varying, and spatially structured fixed-pattern noise**, and additional distortions from **onboard filtering/AGC/NUC** [2603.20443].

The system addresses this by making both the map and the trajectory explicitly aware of the thermal sensor. Its core design is based on three claims stated throughout the paper: **continuous-time trajectory estimation** is needed to model blur and rolling shutter, **IMU fusion** is essential when the thermal image is unreliable, and **3D Gaussian Splatting** can serve as a dense, efficient map representation if the rendering model matches the thermal sensor [2603.20443].

The full pipeline has four main stages: **Initialization**, **Tracking**, **Mapping**, and **IMU initialization and offline refinement**. The full system jointly optimizes **trajectory spline control points, 3D Gaussian parameters, IMU biases, pixel-wise FPN spline, and global intensity spline** [2603.20443]. This places TRGS-SLAM within the broader family of dense 3DGS-SLAM methods, but with a sensor model and optimization structure specialized for thermal-inertial operation rather than RGB, RGB-D, LiDAR, or radar.

## 2. Thermal image formation and model-aware rendering

A defining feature of TRGS-SLAM is that it does not treat thermal frames as ordinary sharp images. Instead, it adopts the microbolometer model used in TRNeRF. For a raw thermal pixel \((x_p,y_p)\) at time \(t\),

\[
\tilde{n}'_{x_p, y_p}(t) = \tilde{m}'_{x_p, y_p}(t) + \tilde{o}'_{x_p, y_p}(t),
\]

where the blurred term is

\[
\tilde{m}'_{x_p, y_p}(t) = \frac{1}{\tau}\int_{-\infty}^{t} \exp\left(\frac{s-t}{\tau}\right)\tilde{p}'_{x_p, y_p}(s)\,ds,
\]

and \(\tilde{o}'\) is slowly varying FPN. Here \(\tau\) is the thermal time constant [2603.20443].

Rolling shutter is modeled explicitly by assigning different sampling times to different pixels:

\[
t'_{x_p, y_p, l} = t'_{0,0,l} + x_p \Delta t_{\text{pix}} + y_p w \Delta t_{\text{pix}}.
\]

This means the system reasons over a **continuous-time exposure/readout schedule**, not a single timestamp per frame [2603.20443].

The map consists of 3D Gaussians \(\mathcal{G}\), each with **mean \(\bar{\boldsymbol{\mu}}_g\), scale \(\bar{\mathbf{s}}_g\), rotation quaternion \(\mathbf{q}_g\), opacity \(\sigma_g\), and scalar intensity \(c_g\)**. Unlike RGB 3DGS, TRGS-SLAM uses **single-channel thermal intensity** and **no view-dependent color** [2603.20443]. Standard 3DGS rasterization is written as

\[
p = \sum_{g\in\mathcal{B}} c_g \alpha_g \prod_{j=1}^{g-1}(1-\alpha_j),
\]

but TRGS-SLAM replaces single-image rendering with a temporally sampled rendering process over the blur window:

\[
\mathbf{M}_l = \sum_{r=0}^{N_r-1} \mathbf{I}_{\mathcal{G}(t_r)} \odot \mathbf{W}_r.
\]

Here, \(t_r\) are evenly spaced sample times over \([t_{\max,l}-T_b, t_{\max,l}]\), and \(\mathbf{W}_r\) are **precomputed per-pixel weights** derived from the blur model, rolling shutter timing, and undistortion. The final rendered image is

\[
\mathbf{N}_l = \mathbf{M}_l + \mathbf{O}_{\text{pix}(t'_{0,0,l})} + o_{\text{global}(t'_{0,0,l})},
\]

where \(\mathbf{O}_{\text{pix}(t)}\) is a **pixel-wise FPN spline** and \(o_{\text{global}(t)}\) is a scalar image-wide intensity spline [2603.20443].

The corresponding image loss is

\[
\mathcal{L}_{\text{img},l}=\frac{1}{wh}\|\mathbf{N}_l-\tilde{\mathbf{N}}_l\|_1,
\]

with an FPN zero-mean regularizer

\[
\mathcal{L}_{\text{FPN}(t)}= \frac{\lambda_{\text{FPN}}}{wh}\left| \sum_{x_p=0}^{w-1}\sum_{y_p=0}^{h-1} [\mathbf{O}_{\text{pix}(t)}]_{y_p,x_p} \right|.
\]

The significance of this rendering model is methodological rather than merely implementation-level. TRGS-SLAM uses a **frame-to-model** strategy in which optimization is performed against a renderer that already accounts for blur, rolling shutter, and FPN. This suggests that the system’s robustness is tied not only to 3DGS as a scene representation, but to the fact that the rendering process is physically aligned with the thermal sensor.

## 3. Continuous-time trajectory representation and IMU coupling

TRGS-SLAM represents camera motion using two uniform B-splines: a position spline \(\mathbf{p}(t)\in\mathbb{R}^3\) and a rotation spline \(\mathbf{R}(t)\in\mathrm{SO}(3)\). For \(t\in[t_i,t_{i+1})\),

\[
\mathbf{p}(t) = \begin{bmatrix}\mathbf{p}_i & \mathbf{p}_{i+1} & \cdots & \mathbf{p}_{i+k-1}\end{bmatrix}\mathbf{M}^{(k)}\mathbf{u},
\]

\[
\mathbf{R}(t)=\mathbf{R}_i \prod_{j=1}^{k-1}\mathrm{Exp}\left(\lambda_j(t)\,\mathrm{Log}\left(\mathbf{R}_{i+j-1}^{-1}\mathbf{R}_{i+j}\right)\right),
\]

with

\[
\boldsymbol{\lambda}(t)=\mathbf{M}^{*(k)}\mathbf{u}, \qquad \mathbf{u}=\begin{bmatrix}1 & u & \cdots & u^{k-1}\end{bmatrix}^T,\quad u=\frac{t-t_i}{\Delta t_c}.
\]

Since \(k\ge 4\), the trajectory has continuous acceleration [2603.20443].

This continuous-time construction is necessary because different pixels in a rolling-shutter frame correspond to different times, motion blur integrates over a time interval, and IMU measurements can be matched directly to spline derivatives. In effect, the trajectory model is a structural prerequisite for the image model, not an auxiliary smoothing device.

IMU fusion is implemented through a **two-stage IMU loss**. Before IMU initialization, only the gyroscope is used:

\[
\mathbf{r}_{g,m} = \boldsymbol{\omega}_{\text{C}(t_m)} - (\tilde{\boldsymbol{\omega}}_{\text{I},m}^{\text{I}} - \mathbf{b}_g).
\]

After scale \(S\) and gravity direction \(\hat{\mathbf{g}}^{\text{W}}\) are initialized, both gyroscope and accelerometer residuals are used:

\[
\mathbf{r}_{g,m} = \boldsymbol{\omega}_{\text{I}(t_m)} - (\tilde{\boldsymbol{\omega}}_{\text{I},m}^{\text{I}} - \mathbf{b}_g),
\]

\[
\mathbf{r}_{a,m} = \mathbf{a}_{\text{I}(t_m)} - \left( \mathbf{R}_{\text{I}(t_m)}(\tilde{\mathbf{a}}_{\text{I},m}^{\text{I}}-\mathbf{b}_a) +G\hat{\mathbf{g}}^{\text{W}} \right).
\]

The IMU loss is

\[
\mathcal{L}_{\text{IMU},\mathcal{I}}= \sum_{m\in\mathcal{I}} \lambda_g\|\mathbf{r}_{g,m}\|_2+ \lambda_a\|\mathbf{r}_{a,m}\|_2,
\]

and the bias regularization is

\[
\mathcal{L}_{\text{bias}}= \lambda_{\text{bias}}\left(\|\mathbf{b}_g\|_2+\|\mathbf{b}_a\|_2\right).
\]

The rationale is explicit in the paper: before scale and gravity are known, early accelerometer constraints are not meaningful, but gyroscope information is immediately useful for stabilizing pose [2603.20443]. This staged inertial integration is one of the central reasons the method remains stable when thermal imagery is severely degraded.

## 4. Initialization, tracking, mapping, and map stabilization

Initialization begins with all trajectory control points set to **zero position and identity rotation**, FPN splines set to **zero**, and the map initialized as **a plane of \(N_{\text{init}}\) Gaussians one unit in front of the camera**. Pixels are randomly sampled from the first image, back-projected to depth \(1\), and used to initialize Gaussian means and intensities; opacities and scales are initialized heuristically. The initial map fitting minimizes

\[
\mathcal{L}_{\text{init}}= \frac{1}{wh}\|\mathbf{I}_{\mathcal{G}(t'_{0,0,0})}-\tilde{\mathbf{N}}_0\|_1.
\]

At this stage, blur and rolling shutter are **not yet modeled**; they are enabled only later after a warmup period for stability [2603.20443].

During tracking, the trajectory spline is extended to cover the new frame’s readout time. Early in the sequence this is done by duplicating the final control point. Later, future poses are predicted and the newly extended active control points are fitted to those predictions. Before IMU initialization, predictions use **constant velocity**; after IMU initialization, they use **IMU integration**. Tracking optimizes only the **position and rotation control points**, while Gaussians and FPN are frozen. The tracking loss is

\[
\mathcal{L}_{\text{track}}= \frac{1}{|\mathcal{I}_{\text{track}}|} \mathcal{L}_{\text{IMU},\mathcal{I}_{\text{track}}} +\mathcal{L}_{\text{img},l}.
\]

Mapping is triggered on keyframes and jointly updates **Gaussians, trajectory control points, IMU biases, and FPN splines**. The mapping formulation distinguishes a current keyframe window \(\mathcal{W}_K\) from randomly sampled past keyframes \(\mathcal{S}_K\), with \(\lambda_s > \lambda_w\), so that **past keyframes are weighted more strongly to avoid overwriting the map** [2603.20443]. This is presented as a key anti-drift mechanism.

A distinctive stabilization mechanism is **view-diversity-based opacity resetting**. For each Gaussian \(g\), the method accumulates a second-moment matrix of viewing directions,

\[
\mathbf{V}_g=\mathbf{V}_{g,\text{prev}}+\sum_{r\in\mathcal{R}_g}\hat{\mathbf{d}}_{r,g}\hat{\mathbf{d}}_{r,g}^{T}.
\]

The condition number of \(\mathbf{V}_g\) is then computed, opacity is reset for Gaussians whose condition number is above a threshold, and \(\mathbf{V}_g\) is reset to zero [2603.20443]. The criterion is geometric: Gaussians seen from only a narrow set of directions are poorly constrained and are more likely to be redundant or unstable. Compared with global opacity resets, this procedure is explicitly described as more selective and SLAM-friendly.

TRGS-SLAM also incorporates **pose drift correction / relocalization schemes**. If a current mapping step has low SSIM after fitting, the system interprets this as inconsistency with the map due to drift, **reverts to the pre-mapping state**, resumes tracking, and temporarily reduces the IMU residual weights \(\lambda_g,\lambda_a\) so that the optimizer can “jump” back to the correct pose [2603.20443]. After some time, it performs analytical IMU initialization to estimate **gravity direction \(\hat{\mathbf{g}}^{\text{W}}\), scale \(S\), gyro bias \(\mathbf{b}_g\), and accel bias \(\mathbf{b}_a\)**, and periodically re-runs IMU initialization to update scale and gravity.

## 5. Experimental evaluation and reported performance

Evaluation is conducted on the **TRNeRF dataset**, which contains **6 sequences: slow/medium/fast × indoor/outdoor**, recorded with **two FLIR ADK thermal cameras** and **400 Hz IMU**, with **pseudo-ground-truth poses from stereo SfM**. The sequence labels are **SO, MO, FO, SI, MI, FI** [2603.20443]. The implementation is built in **PyTorch**, uses **gsplat** for efficient rasterization, uses custom/on-manifold B-spline optimization with **PyPose/LieTorch/C++ routines**, and normalizes thermal images with sequence-specific **0.5 and 99.5 percentiles**.

The compared methods are **ORB-SLAM3, DROID-SLAM, DSM, DBA-VIO, MonoGS, DM-VIO, MASt3R-SLAM, and ROTIO**. The paper attributes their failures to several causes: feature-based methods fail under **low contrast, blur, and FPN**; frame-to-frame methods struggle with **severe degradation**; depth-network-based methods are unreliable on **thermal data**; and methods that do not model **rolling shutter** and **blur** cannot handle the fast sequences [2603.20443].

For RMSE ATE on the TRNeRF dataset, selected TRGS-SLAM results are:

- **SO**: **12.7 cm**
- **MO**: **5.5 cm**
- **FO**: **2.4 cm**
- **SI**: **6.4 cm**
- **MI**: **3.9 cm**
- **FI**: **4.2 cm**

The paper emphasizes that on the hardest sequences—**FO, MI, FI**—most baselines either **fail early** or **exceed 1 m error**, whereas TRGS-SLAM is the **only method that consistently succeeds** [2603.20443]. At the same time, it notes an important qualification: on the easiest sequence **SO**, TRGS-SLAM is **not the best**, because IMU initialization is difficult when motion is slow, leading to scale issues. This is a useful correction to the possible misconception that the method uniformly dominates across all regimes.

The ablation study identifies several components as particularly important. **Using gyro information early** is crucial; **modeling blur and rolling shutter** is necessary for fast sequences; **pixel-wise FPN estimation** is essential, especially indoors; **pose drift corrections / relocalization** matter; and **opacity resetting** helps stabilize and compact the map [2603.20443]. Additional findings include: **No gyro before IMU init: large degradation**; **No roll/blur modeling: fails on the hardest fast sequences**; **No pixel-wise FPN: many sequences fail**; **No keyframe weighting: can severely hurt MO**; **No opacity resetting: map grows large and performance degrades**.

The paper also reports thermal image restoration after offline refinement. Offline refinement **uses all frames, not just keyframes**, **increases raster samples per render**, **uses full-resolution images**, **increases blur integration interval**, **introduces gravity as a learned parameter**, **drops the IMU bias loss**, and **reintroduces global opacity resetting** [2603.20443]. Compared with **GS on the Move** and **TRNeRF**, the refined system achieves LPIPS values of **0.067** on **MO**, **0.098** on **FO**, **0.095** on **MI**, and **0.116** on **FI**, and the paper characterizes these results as close to TRNeRF while not requiring ground-truth poses.

Runtime and memory measurements are reported on **RTX A6000 + Ryzen 9 5950X**. Tracking runs at **5–7 Hz**, mapping takes **1.3–1.6 s per keyframe**, overall speed is **1–4 FPS**, and peak GPU memory is **423 MiB** [2603.20443]. The system is therefore **not real-time for fast motion**, but the paper attributes the low memory footprint to the facts that thermal Gaussians use **scalar intensity** and **view-diversity opacity resetting limits the map to about 20k–40k Gaussians**.

## 6. Position within 3DGS-SLAM research, limitations, and interpretation

Within 3DGS-SLAM research, TRGS-SLAM occupies a specialized but technically significant position. **DAGS-SLAM** addresses **RGB-D SLAM in dynamic environments** through **per-Gaussian temporal motion probability (MP)**, **YOLO instance priors**, **geometric cues**, and **uncertainty-aware semantic-on-demand scheduling** [2602.21644]. **Rad-GS** addresses **4D radar-camera SLAM** for **kilometer-scale outdoor environments**, using radar geometry and Doppler cues together with a **global octree management** strategy for 3D Gaussian maps [2511.16091]. **RTGS** focuses on the acceleration problem, providing an **algorithm-hardware co-design framework** that reduces redundancy for real-time 3DGS-SLAM on edge devices [2510.06644]. Against this background, TRGS-SLAM addresses a different failure mode: not scene dynamics, not radar-vision fusion, and not throughput on edge hardware, but the sensor-physics mismatch that arises with raw or minimally processed thermal data.

Several limitations are stated or strongly implied by the reported results [2603.20443]. First, performance depends on the availability of **raw or minimally processed microbolometer thermal camera** data; the paper notes that **onboard noise filters** can introduce **multi-view inconsistency** that the model cannot explain, and that this can **severely hurt performance**, especially indoors. Second, the system is accurate on highly degraded sequences but is **not always the best on easy sequences**, especially when **IMU initialization is difficult when motion is slow**. Third, the runtime profile—**5–7 Hz** tracking and **1–4 FPS** overall—shows that the method is not designed as a real-time fast-motion system in its current form [2603.20443].

A plausible implication is that TRGS-SLAM’s main contribution is methodological rather than merely modal. It shows that dense Gaussian SLAM can be extended beyond RGB-like assumptions if the image formation process, trajectory parameterization, and inertial coupling are redesigned together. In that sense, the system is a thermal-specific instance of a broader principle visible across recent 3DGS-SLAM work: performance improves when the Gaussian representation is coupled tightly to the dominant source of failure—dynamic content in DAGS-SLAM [2602.21644], multimodal sparsity and scale in Rad-GS [2511.16091], pipeline redundancy in RTGS [2510.06644], and thermal sensor degradation in TRGS-SLAM [2603.20443].

Source: https://www.emergentmind.com/topics/trgs-slam