MonoEM-GS: Probabilistic Monocular SLAM

• MonoEM-GS is a monocular SLAM system that generates a probabilistic 3D map using a Gaussian mixture model fused via Expectation–Maximization and ICP alignment.
• It integrates noisy dense predictions from geometric foundation models like MapAnything to overcome limitations of traditional sparse keypoint methods.
• The system supports open-set semantic segmentation with fixed DINOv3 features, enabling efficient scene querying and robust mapping performance.

MonoEM-GS is a monocular SLAM system that turns a pure RGB video stream into a globally consistent, semantically rich 3D map represented as a mixture of Gaussians. It is designed to exploit feed-forward geometric foundation models, specifically MapAnything, while correcting their characteristic inconsistencies through an Expectation–Maximization formulation and ICP-based pose estimation. The method treats foundation-model outputs as noisy measurements that must be explained by a global probabilistic map, and augments the geometric map with high-dimensional features that support open-set, in-place semantic segmentation and other downstream queries directly on the reconstruction (Kruzhkov et al., 12 Apr 2026).

1. Problem setting and design rationale

MonoEM-GS is motivated by a limitation of both traditional monocular SLAM and recent learned-geometry pipelines. Traditional monocular SLAM, exemplified in the source material by ORB-SLAM2, relies on sparse keypoints, feature matching, and bundle adjustment. In the formulation described for MonoEM-GS, this framework struggles when scenes are texture-poor, when motion between frames is small or degenerate, and when only a single RGB camera is available without depth or LiDAR (Kruzhkov et al., 12 Apr 2026).

Recent geometric foundation models—including VGGT, DUSt3R, MASt3R, MUSt3R, Pi3-X, CUT3R, and MapAnything—change the reconstruction regime by predicting dense 3D point clouds and camera motion directly from one or several RGB frames, often in metric scale. MonoEM-GS is built around the observation that such predictions are powerful priors but are not directly suitable as a persistent SLAM map. The paper identifies three failure modes: view dependence and noise, inconsistent geometry under transformations, and local metric drift between frames. Naive fusion therefore accumulates inconsistent points and yields fuzzy, ambiguous structures rather than crisp surfaces (Kruzhkov et al., 12 Apr 2026).

The central design choice is to model the scene as a probabilistic Gaussian Mixture Model in 3D, update that model incrementally with Expectation–Maximization, and localize new observations by ICP-based alignment instead of trusting the foundation model’s poses. This suggests a shift in emphasis relative to learned-geometry SLAM methods such as VGGT-SLAM and MASt3R-SLAM: the primary object is the map itself, and incoming predictions are treated as measurements of that map rather than as geometry to be accumulated largely as-is (Kruzhkov et al., 12 Apr 2026).

2. System architecture and streaming workflow

MonoEM-GS is a monocular pipeline whose only sensor input is a stream of RGB images and known camera intrinsics. No depth images, LiDAR, stereo, or ground-truth depth sensors are used. At each time step, MapAnything processes the current frame together with a small buffer of past frames and a fixed anchor frame. From this batch, the system obtains dense 3D point clouds in metric scale, a pose for each input frame relative to the anchor, per-point color, per-point DINO-backbone features, and separately computed DINOv3 features; normals are then computed by MonoEM-GS itself (Kruzhkov et al., 12 Apr 2026).

The frame-to-frame workflow has four stages. First, model inference produces the dense geometric and feature predictions. Second, localization aligns the current predicted point cloud to the existing Gaussian map. Third, mapping applies an EM-like update to probabilistically associate points with nearby Gaussians and refine Gaussian parameters, followed by a rasterization-based refinement stage that merges additional correspondences in image space and adds Gaussians in genuinely new regions. Fourth, feature storage allows the map to be queried directly with text embeddings or other feature queries for open-set segmentation and retrieval (Kruzhkov et al., 12 Apr 2026).

A distinctive operational property is that MonoEM-GS performs online optimization without an inner-loop iterative optimization per frame of the kind used by many Gaussian Splatting SLAM systems. Each frame uses essentially a single EM step plus a rasterization step. Initialization is performed by filling a FIFO buffer with a fixed number of frames using an input stride, running MapAnything on these frames and the anchor, and clustering all predicted points into $K$ clusters via FAISS, with

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$

where $\lambda$ is a user-defined constant such as $128$ (Kruzhkov et al., 12 Apr 2026).

This architecture differs from RGB-only Gaussian SLAM systems such as MonoGS++, which also use only RGB inputs but delegate pose and sparse depth estimation to an external visual odometry module, specifically DPVO, and optimize the Gaussian map purely for appearance and mild regularization rather than for pose. MonoEM-GS instead couples monocular mapping to feed-forward geometric foundation-model predictions and re-estimates poses through ICP against the map (Li et al., 3 Apr 2025).

3. Gaussian mixture representation and incremental EM

The map in MonoEM-GS is a set of 3D Gaussians

$$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$

where $\boldsymbol{\mu}_k \in \mathbb{R}^3$ is the mean, $\mathbf{\Sigma}_k \in \mathbb{R}^{3 \times 3}$ is the covariance, $\mathbf{c}_k \in \mathbb{R}^3$ is RGB color, $\bar{\mathbf{n}}_k \in \mathbb{R}^3$ is a unit surface normal, and $\mathbf{f}_k \in \mathbb{R}^d$ is a DINOv3 feature vector. The scene is therefore treated as a Gaussian Mixture Model that explains noisy 3D measurements probabilistically rather than as a raw point aggregation (Kruzhkov et al., 12 Apr 2026).

For a current aligned frame, the observed data are 3D points

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$0

with associated normals, colors, and DINOv3 features. Latent variables $$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$1 assign each point to a Gaussian, and the E-step computes responsibilities $$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$2. MonoEM-GS evaluates, for each point and its $$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$3 nearest Gaussians, a Mahalanobis distance in 3D, a normal-consistency term, and a feature cosine similarity. These are combined into the score

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$4

from which responsibilities are computed by a softmax over the candidate Gaussians. Before that, an initial gating step filters points using normal and Mahalanobis thresholds, and a per-point quality score

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$5

further suppresses poorly explained points (Kruzhkov et al., 12 Apr 2026).

The M-step updates Gaussian means, covariances, normals, and colors in the standard weighted manner implied by the responsibilities, but MonoEM-GS does not replace parameters outright. Instead it uses an online running average

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$6

with

$$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$7

Here $$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$8 is an isotropy score that measures how symmetrically the assigned measurements support a Gaussian around its center in the plane orthogonal to the normal. Larger $$K = \frac{\text{total number of predicted points}}{\lambda \times \text{number of inferred frames}},$$9 means more symmetric support and therefore more confidence, so Gaussians are updated faster when observations come from both sides of the tangential support rather than from a one-sided configuration. This mechanism is explicitly intended to prevent overfitting to spurious geometry and to stabilize noisy, view-dependent foundation-model outputs (Kruzhkov et al., 12 Apr 2026).

A further implementation detail is that DINOv3 features are not updated after initialization. They are aggregated during initial clustering only, and then retained as stable semantic anchors for downstream tasks. This suggests an intentional separation between geometric adaptation, which is online and recurrent, and semantic representation, which remains fixed once assigned (Kruzhkov et al., 12 Apr 2026).

4. Pose estimation, rasterization-based refinement, and scale handling

MonoEM-GS does not trust the camera poses predicted by MapAnything directly. Instead, it re-estimates the pose of the current frame by aligning the current predicted point cloud to the existing map. The temporal mechanism relies on the fact that the last buffered RGB image appears in two consecutive inference batches, which provides correspondences between consecutive predictions. First, standard ICP is used between predictions to obtain a coarse alignment. Then a local submap is selected from the Gaussian map using a bounding box around already aligned points, and colored ICP refines the alignment of the current prediction to that local submap (Kruzhkov et al., 12 Apr 2026).

The refinement step minimizes a combined geometric and photometric objective of the form

$\lambda$0

where $\lambda$1 indexes the closest map Gaussian to point $\lambda$2, and $\lambda$3 are the rigid transform parameters. There is no explicit scale estimation in the ICP step. The scale is largely supplied by MapAnything’s metric-scale priors and is maintained through rigid alignment, anchor-based initialization, and the consistency induced by map-based data association and EM fusion (Kruzhkov et al., 12 Apr 2026).

Beyond the EM update, MonoEM-GS performs rasterization-based refinement using standard Gaussian Splatting via gsplat. Gaussians are projected into the image plane, composited along each ray with alpha-blending or volumetric compositing, and used to produce rendered color and depth images. Rendered depth functions as an expected depth signal for detecting new geometry and reasoning about foreground-background relationships (Kruzhkov et al., 12 Apr 2026).

This refinement stage addresses points that were not fused during the 3D EM update. Temporary Gaussians $\lambda$4 are built from unfused points, both the current temporary map and the local submap are rendered from the current pose, and depth-based reasoning distinguishes unknown regions from overlapping surfaces. Unknown regions—where predictions render depth but the map does not—yield new Gaussians. In overlapping regions, MonoEM-GS computes up to $\lambda$5 nearest map Gaussians per predicted Gaussian, checks a 2D Mahalanobis threshold and a depth ordering constraint, and merges using a weight

$\lambda$6

The update then takes the form

$\lambda$7

The combined effect of 3D EM and 2D rasterization-based refinement is to impose multi-view consistency over time while growing the map only where the current map cannot already explain the data (Kruzhkov et al., 12 Apr 2026).

5. Multi-modal Gaussian features and open-set map querying

Each Gaussian in MonoEM-GS stores a fixed DINOv3 feature vector. During initialization, features are aggregated from neighboring points using Mahalanobis kernel weights and normalized weights. However, the stored Gaussian feature is not the weighted average itself. Instead, the method chooses the neighbor feature that is closest to the weighted average in cosine similarity: $\lambda$8 This preserves a representative feature anchor while avoiding subsequent semantic drift during online geometric updates (Kruzhkov et al., 12 Apr 2026).

Open-set segmentation is performed directly on the map rather than on individual frames. Class names or textual descriptions are encoded with the same DINOv3 text encoder to obtain embeddings $\lambda$9, and each Gaussian is assigned the class with maximum cosine similarity

$128$0

The system is therefore open-set in the sense that it is not restricted to a fixed set of training classes; any text label representable by the DINOv3 encoder can be queried (Kruzhkov et al., 12 Apr 2026).

The same mechanism supports attribute queries such as “red chair” or “wooden table,” as well as object retrieval by thresholding feature-query similarity. Because the map is sparse—described in the source material as being on the order of thousands of Gaussians instead of millions of points—high-dimensional feature storage remains memory-efficient and querying is fast. A plausible implication is that MonoEM-GS treats semantic functionality as a first-class property of the reconstruction, not as a post-processing layer over dense per-frame predictions (Kruzhkov et al., 12 Apr 2026).

6. Empirical performance, comparative position, and limitations

MonoEM-GS is evaluated on 7-Scenes, TUM RGB-D, and Replica. The reported task categories are pose estimation using Absolute Trajectory Error (ATE RMSE) and relative scale error, mapping quality using accuracy, completion, F1-score at a $128$1 m threshold, and normal consistency, and semantic segmentation on Replica using mIoU, frequency-weighted mIoU, overall accuracy, and trajectory ATE. Baselines include VGGT-SLAM, VGGT-SLAM2, MASt3R-SLAM, MapAnything, and, for semantic evaluation on Replica, ConceptGraphs, RayFronts, OVO/ORB-SLAM2, and OMCL (Kruzhkov et al., 12 Apr 2026).

On 7-Scenes, MonoEM-GS reports accuracy $128$2 m, completion $128$3 m, F1-score $128$4, normal consistency $128$5, scale error $128$6, and ATE RMSE $128$7 m. On TUM RGB-D, it reports accuracy $128$8 m, completion $128$9 m, F1-score $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$0, normal consistency $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$1, scale error $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$2, and ATE RMSE $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$3 m. The paper characterizes these results as strong reconstruction performance, with best or tied-best results in several mapping metrics, while noting that ATE can remain worse than optimization-based baselines such as MASt3R-SLAM on some sequences (Kruzhkov et al., 12 Apr 2026).

On Replica semantics, MonoEM-GS reports mIoU $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$4, f-mIoU $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$5, accuracy $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$6, and ATE $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$7 cm. In the reported table, this yields second-best mIoU, best f-mIoU by a large margin, second-best overall accuracy, and better ATE than OMCL but worse ATE than OVO/ORB-SLAM2, which uses external localization and RGB-D. The interpretation given in the source is that the high f-mIoU indicates particularly strong performance on frequent classes while maintaining competitive semantics under monocular self-localization (Kruzhkov et al., 12 Apr 2026).

The following summary condenses the reported evaluation highlights.

Dataset / task	MonoEM-GS result	Reported characterization
7-Scenes F1-score	93.2%	Best
7-Scenes normal consistency	59.8%	Best
TUM RGB-D accuracy	0.06 m	Best
TUM RGB-D completion	0.15 m	Best
TUM RGB-D F1-score	86.8%	Best
Replica mIoU	31.5%	Second-best
Replica f-mIoU	63.2%	Best by a large margin

In the literature positioning supplied for the method, MonoEM-GS is contrasted with Gaussian Splatting SLAM systems such as Gaussian Splatting SLAM, Hi-SLAM2, SING3R-SLAM, Splat-SLAM, and SplaTAM, which primarily use 3D Gaussians for photorealistic novel-view synthesis, often run many optimization steps per frame, and frequently require additional geometric priors such as depth or LiDAR. MonoEM-GS differs in that it treats the map explicitly as a Gaussian Mixture Model, uses EM to fuse measurements, does not optimize for photorealism, and uses splatting chiefly for rasterization-based data association and visibility reasoning (Kruzhkov et al., 12 Apr 2026).

It is also distinct from MonoGS++, which is an RGB-only Gaussian SLAM system that offloads tracking to DPVO, initializes Gaussians from sparse VO patches, and introduces dynamic 3D Gaussian insertion, clarity-enhancing densification, and planar regularization, achieving a reported $$\mathcal{M} = \left\{(\boldsymbol{\mu}_k, \mathbf{\Sigma}_k, \mathbf{c}_k, \bar{\mathbf{n}}_k, \mathbf{f}_k)\right\}_{k=1}^{K},$$8 FPS improvement over MonoGS on Replica Room0. That contrast clarifies a methodological divide: MonoGS++ is appearance-optimized and VO-decoupled, whereas MonoEM-GS is map-centric, probabilistic, and driven by dense foundation-model geometry fused through EM (Li et al., 3 Apr 2025).

The limitations stated for MonoEM-GS are specific. It currently has no loop-closure mechanism, which limits it to small, room-scale scenes. It depends on the quality of MapAnything predictions, so severely incorrect geometry can be integrated into the map. Although more stable than raw feed-forward poses, it still shows less accurate poses than optimization-based baselines on some sequences. In addition, DINOv3 features are fixed after initialization rather than updated with additional views. Future directions suggested in the source include loop closure and global optimization, extension to larger-scale scenes, and joint optimization of semantic features with geometry (Kruzhkov et al., 12 Apr 2026).

Taken together, MonoEM-GS is positioned as a monocular SLAM pipeline whose defining triad is the use of foundation-model geometry as noisy priors, a Gaussian Splatting representation interpreted as a probabilistic GMM, and incremental EM fusion with ICP-based alignment, extended by multi-modal Gaussian features for open-set semantics. This suggests that its primary contribution lies not in replacing feed-forward geometric models, but in imposing a statistically coherent mapping backend that converts their inconsistent dense predictions into a globally consistent and semantically queryable 3D map (Kruzhkov et al., 12 Apr 2026).