DyPho-SLAM: Dynamic Photorealistic SLAM

• DyPho-SLAM is a dynamic visual SLAM system that uses 3D Gaussian Splatting to achieve photorealistic static mapping in moving scenes.
• It refines dynamic masks by combining semantic segmentation and optical flow, enhancing feature extraction and reducing camera tracking drift.
• The system demonstrates state-of-the-art performance with low Absolute Trajectory Error and maintains real-time operation on commodity hardware.

DyPho-SLAM is a real-time, resource-efficient visual SLAM system for dynamic environments that combines precise camera localization with photorealistic dense mapping through an explicit 3D Gaussian Splatting (3DGS) representation. It was introduced to address two coupled failure modes of dynamic-scene SLAM: camera tracking drift, caused by moving-object features violating static-scene assumptions, and fuzzy mapping or ghosting, caused by dynamic content being incorporated into dense reconstructions (Liu et al., 31 Aug 2025). The system takes an RGB-D stream with intrinsics $K$ and estimates camera poses $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$ together with a static photorealistic dense map represented by Gaussians $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$. Its defining strategy is to refine dynamic masks with prior image information and then recover pose-estimation constraints lost after dynamic removal through adaptive feature extraction.

1. Problem setting and design rationale

DyPho-SLAM is formulated for RGB-D visual SLAM in scenes containing moving objects, where standard static-world assumptions fail. In this regime, dynamic pixels induce incorrect data association, weakened optimization constraints, degeneracy in bundle adjustment, and consequent trajectory drift. The same disturbances also contaminate dense reconstruction, producing ghost artifacts and blurred renderings in novel views when dynamic observations are fused into the map (Liu et al., 31 Aug 2025).

The system adopts 3D Gaussian Splatting as its scene representation because 3DGS provides an explicit scene model with efficient rasterization, high-fidelity rendering, and differentiable optimization. In contrast to implicit radiance-field pipelines, the representation avoids the heavier optimization and memory footprint typical of NeRF-based methods while still supporting end-to-end optimization of camera poses and Gaussian parameters. This places DyPho-SLAM in the class of photorealistic SLAM systems that retain dense, renderable maps rather than sparse geometric reconstructions.

The design objective is therefore not merely dynamic-object filtering. DyPho-SLAM targets simultaneous improvement in localization and rendering quality under motion, while maintaining real-time performance on commodity hardware. Its central premise is that dynamic-scene robustness requires both better identification of static support and better use of the remaining support after masking.

2. System architecture and processing pipeline

DyPho-SLAM is organized as a five-stage pipeline that couples dynamic processing, feature-based tracking, and Gaussian-splatting mapping in a C++/CUDA implementation with LibTorch (Liu et al., 31 Aug 2025).

First, dynamic object detection combines two sources of evidence. Semantic segmentation with YOLO produces $M_{\mathrm{seg}}$, and optical flow under the brightness-constancy assumption produces $M_{\mathrm{flow}}$. These are merged into a raw mask,

$$M_{\mathrm{raw}} = M_{\mathrm{flow}} \oplus M_{\mathrm{seg}},$$

so that motion cues and semantic priors jointly determine candidate dynamic regions.

Second, the raw mask is refined using prior image information. DyPho-SLAM maintains a background depth model $B_i$ that fuses historical background estimates, rendered depth from the current Gaussian map at a predicted pose, and the current masked depth. This temporal fusion is intended to reduce single-frame misjudgment noise, especially missed masks and residual dynamic contamination. The refined output mask $M^i$ distinguishes static from dynamic pixels, with $M^i=1$ denoting static and $M^i=0$ denoting dynamic.

Third, camera tracking proceeds on dynamically filtered imagery. Feature extraction is performed after masking, but the system adaptively relaxes thresholds in masked regions to compensate for the loss of features. Feature matching is followed by motion-only bundle adjustment with robust loss and mask weighting; loop closure is applied on keyframes for global drift reduction.

Fourth, mapping updates the 3DGS representation incrementally. New Gaussians are created from unprojected keyframe observations and local densification around 2D feature regions. Differentiable rendering then supplies color and depth predictions for optimization of Gaussian parameters.

Fifth, rendering uses either depth-sorted alpha compositing or an order-independent approximation for fast photorealistic view synthesis. The latter is also used for efficient mask-based background modeling.

This modularity is significant because the system does not rely on a single dynamic-scene heuristic. Instead, mask generation, mask correction, pose estimation, mapping, and rendering are tightly coupled. A plausible implication is that DyPho-SLAM’s robustness derives as much from cross-module consistency as from any isolated component.

3. Mathematical formulation and optimization

The input sequence is written as

$$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$0

with known intrinsics $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$1. The target variables are the pose set $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$2 and Gaussian set $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$3. Each Gaussian carries a mean $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$4, covariance $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$5, opacity $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$6, and color $$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$7 modeled by spherical harmonics (Liu et al., 31 Aug 2025).

Mask refinement is driven by a constant-velocity pose prior,

$$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$8

and a temporally fused background depth update,

$$\mathcal{T}=\{\boldsymbol{T}_i=[\boldsymbol{R}_i|\boldsymbol{t}_i]\}$$9

Here $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$0 renders depth from the current Gaussian map at the predicted pose, while $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$1 and $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$2 control the trust placed in prior rendering and current masked depth. The final refined mask is computed by neighbor depth consistency,

$$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$3

where $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$4 denotes the neighbors of $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$5. This formulation operationalizes the claim that static support should be inferred temporally, not frame by frame.

After masking, DyPho-SLAM explicitly compensates for the reduced number of features. On a Gaussian-pyramid grayscale image $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$6, the adaptive threshold is

$$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$7

with $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$8 the image dimensions, $$\mathcal{G}=\{\boldsymbol{\mu}_j,\alpha_j,\boldsymbol{\Sigma}_j,\boldsymbol{c}_j\}$$9 the initial threshold, and $M_{\mathrm{seg}}$0 a control factor. As the support of the static mask decreases, the threshold is relaxed to preserve a stable feature bin size. This is central to DyPho-SLAM’s treatment of dynamic scenes: dynamic removal is not assumed to be free, and the system adds constraints back into the tracker in a controlled manner.

Pose estimation uses motion-only bundle adjustment with Huber loss and mask weighting: $M_{\mathrm{seg}}$1 Here $M_{\mathrm{seg}}$2 follows the refined mask, $M_{\mathrm{seg}}$3, and optimization is performed with Levenberg–Marquardt. Loop closure on keyframes is used to reduce global drift.

The 3DGS rendering model follows depth-sorted alpha compositing. For color,

$M_{\mathrm{seg}}$4

and depth is defined analogously using $M_{\mathrm{seg}}$5. DyPho-SLAM also uses an order-independent approximation for efficient background modeling. Map optimization minimizes a masked color-depth objective,

$M_{\mathrm{seg}}$6

with Gaussian parameters updated by SGD. The masking in this loss is essential: it formalizes the system’s commitment to reconstructing the static background photorealistically while suppressing dynamic contamination.

4. Empirical results and operating characteristics

DyPho-SLAM was evaluated on publicly dynamic RGB-D datasets, including TUM RGB-D dynamic sequences such as fr3/w/xyz, fr3/w/half, fr3/w/static, and fr3/s/half, as well as Bonn indoor handheld dynamic sequences. Camera tracking was evaluated with Absolute Trajectory Error RMSE and standard deviation; photorealistic reconstruction was assessed qualitatively through novel-view rendering; runtime was reported as tracking time, mapping time, and FPS (Liu et al., 31 Aug 2025).

On TUM dynamic scenes, the reported ATE values are 1.6 cm on fr3/w/xyz with Std. 0.8 cm, 2.6 cm on fr3/w/half with Std. 1.3 cm, 0.6 cm on fr3/w/static with Std. 0.3 cm, and 1.6 cm on fr3/s/half with Std. 0.7 cm, for an average of 1.6 cm with Std. 0.7 cm. The paper states that this is state-of-the-art ATE among dense SLAM methods on those dynamic sequences. It also reports a large margin over Photo-SLAM, whose average ATE is approximately 33.6 cm, and competitive performance relative to dynamic SLAM baselines including RoDyn-SLAM at approximately 4.1 cm average ATE, DGS-SLAM at approximately 3.0 cm, and GassiDy at approximately 2.6 cm.

Runtime on TUM fr3/w/xyz is reported as 61.7 ms for tracking and 3.2 ms for mapping, corresponding to 16.08 Hz, with overall operate time less than 1 min 30 s. Among dense methods in the comparison, only DyPho-SLAM and Photo-SLAM are reported to satisfy real-time constraints, with DyPho-SLAM adding dynamic-scene robustness at similar speed. The implementation platform is an Intel Core i7-13700F CPU at 5.20 GHz and an NVIDIA RTX 3060 GPU, using C++, CUDA, and LibTorch.

Ablation results isolate the two principal contributions. On TUM fr3/w/xyz and fr3/w/half, prior-image mask refinement alone yields ATEs of 3.12 cm and 3.83 cm, adaptive feature extraction alone yields 2.76 cm and 3.54 cm, and the full system yields 1.62 cm and 2.55 cm. The reported interpretation is that both modules significantly improve tracking accuracy: mask refinement reduces misjudgment noise, while adaptive features strengthen pose constraints after dynamic removal.

Qualitative comparisons show fewer ghosting artifacts and a clearer static background than SplaTAM and Photo-SLAM, particularly in scenes with moving people and diverse motion patterns. The reported visual effect is consistent with the system’s intended bias toward clean static-scene reconstruction rather than full dynamic-scene modeling.

5. Relation to dynamic photorealistic SLAM

DyPho-SLAM is positioned against several neighboring lines of work. Relative to static-scene SLAM systems such as ORB-SLAM3 and semantic or motion-masking systems such as DynaSLAM and DS-SLAM, it is distinguished by explicitly leveraging temporal continuity through a maintained background depth model and fused prior renderings, rather than relying only on single-frame segmentation or flow cues (Liu et al., 31 Aug 2025). Relative to NeRF-based methods such as NICE-SLAM, ESLAM, and RoDyn-SLAM, it adopts an explicit 3DGS map that renders quickly and supports photorealistic quality without the higher computational burden typically associated with implicit radiance fields. Relative to Gaussian-splatting SLAM systems such as SplaTAM, GS-SLAM, DGS-SLAM, and DG-SLAM, it is presented as materially reducing tracking drift and map noise under strong dynamics while retaining real-time performance.

Its closest baseline in the reported comparisons is Photo-SLAM, described as a Feature-GS baseline. DyPho-SLAM extends that hybrid feature-and-Gaussian design with refined masks and adaptive feature selection, producing lower ATE and cleaner maps in dynamic scenes. This comparison is important because it isolates the contribution of dynamic-scene handling from the underlying feature-assisted Gaussian-splatting paradigm.

A related 2025 development is D4DGS-SLAM, which approaches the same general problem from a different representational standpoint by replacing static 3DGS with 4D Gaussian Splatting and by incorporating a dynamics-aware InfoModule, LEAP, to estimate dynamics, visibility, and reliability of scene points (Sun et al., 7 Apr 2025). D4DGS-SLAM filters stable static points for tracking, applies dynamic-dependent isotropic regularization during mapping, reconstructs dynamic content without ghosting, and reports average ATEs of 5.1 cm on BONN and 3.6 cm on TartanAir-Shibuya, but operates at about 1.5 FPS on BONN and 0.9 FPS on TartanAir-Shibuya. By contrast, DyPho-SLAM retains a static-background 3DGS map and prioritizes real-time operation. A plausible implication is that dynamic photorealistic SLAM now exhibits two distinct design trajectories: static-background purification with real-time explicit 3DGS, and explicit spatio-temporal modeling with higher computational cost.

This comparison also clarifies a common misconception. Dynamic-scene robustness is not equivalent to dynamic-scene reconstruction. DyPho-SLAM is robust to motion primarily by removing dynamic contamination from tracking and mapping, whereas 4DGS-based systems such as D4DGS-SLAM aim to model moving content directly.

6. Limitations, assumptions, and practical configuration

DyPho-SLAM’s reported failure modes are concentrated in scenes with dense crowds, severe occlusions, or very fast motion, where mask quality and tracking stability can degrade (Liu et al., 31 Aug 2025). The approach depends on semantic segmentation and optical flow for raw-mask generation; out-of-distribution objects or extreme lighting can therefore induce mask errors. Although prior-image fusion reduces misjudgment noise, the method remains sensitive to the quality of the initial mask estimates.

A second limitation is representational. The mapping objective focuses on static background reconstruction and does not model moving-object reconstruction. Consequently, dynamic content may be absent from the final map. This is a deliberate trade-off rather than an incidental deficiency: the system seeks clean static photorealistic maps under motion, not full 4D scene recovery.

The implementation uses YOLO for segmentation, an ORB-SLAM3 tracking backbone, and 3DGS for mapping and rendering. Typical parameters are $M_{\mathrm{seg}}$7, $M_{\mathrm{seg}}$8, $M_{\mathrm{seg}}$9, $M_{\mathrm{flow}}$0, $M_{\mathrm{flow}}$1, and $M_{\mathrm{flow}}$2. The reported tuning guidance reflects the structure of the method. Increasing $M_{\mathrm{flow}}$3 is suggested when segmentation and flow are reliable, while increasing $M_{\mathrm{flow}}$4 is suggested when rendering is stable and the map is already clean. Lowering $M_{\mathrm{flow}}$5 and raising $M_{\mathrm{flow}}$6 makes static-pixel declaration more conservative in cluttered scenes. Raising $M_{\mathrm{flow}}$7 adds more features when the static mask covers fewer pixels, but the recommendation is to monitor bundle-adjustment convergence and runtime to avoid overloading computation. Balancing $M_{\mathrm{flow}}$8 depends on sensor noise, emphasizing depth in structured interiors and color in well-lit textured scenes.

These practical details underscore the system’s operating assumptions. Mask quality is critical, the retained map is predominantly static, and robustness emerges from careful interaction among mask refinement, adaptive feature support, robust pose optimization, and efficient Gaussian rendering. Within those assumptions, DyPho-SLAM defines a specific form of dynamic photorealistic SLAM: real-time localization and photorealistic static mapping in environments where motion is frequent enough to break conventional dense SLAM, but not so extreme that dynamic evidence ceases to be recoverable.