Online Visual SLAM Frameworks

• Online Visual SLAM frameworks are real-time systems that jointly estimate camera pose and perform incremental map reconstruction from continuous visual inputs.
• They integrate feature-based, dense, and learning-based methods with multi-modal sensor fusion to adapt to dynamic scenes under computational and latency constraints.
• Practical implementations emphasize scalability, robust optimization, and memory management, achieving high accuracy as evidenced by metrics like ATE and RMSE across benchmarks.

Online Visual SLAM Frameworks comprise real-time systems that perform joint estimation of camera pose and map reconstruction from streaming visual input. They are distinguished by their ability to operate incrementally on temporally-ordered sensor data, continuously updating both localization and mapping in response to new information, often under stringent computational and latency constraints. Contemporary frameworks integrate diverse algorithmic paradigms—feature-based, direct, learning-based, or hybrid—and address practical demands including robustness, extensibility, multi-modal sensor fusion, and large-scale scalability. This article outlines foundational algorithms, representative architectures, sensor integration strategies, semantic and dynamic scene handling, and performs a comparative analysis of state-of-the-art online visual SLAM systems, referencing key advances in the literature.

1. Core Algorithmic Paradigms in Online Visual SLAM

Online visual SLAM frameworks are categorized by the mathematical and computational strategies employed in pose and map estimation:

• Feature-based Keyframe Graph SLAM: Systems such as ORB-SLAM3 (Campos et al., 2020), OpenVSLAM (Sumikura et al., 2019), OV$^{2}$SLAM (Ferrera et al., 2021), and RTAB-Map (Labbé et al., 2024) extract and track distinctive image features (e.g., ORB, BRISK) to build a pose-graph of keyframes. Bundle adjustment (BA) is performed locally and globally, leveraging robust cost minimization over reprojection errors. Loop detection uses bag-of-words or DBoW2/BoW variant for appearance-based retrieval, with subsequent geometric verification.
• Dense and Hybrid Mapping: LoopSmart (Zhang et al., 2018) fuses sparse front-end tracking with dense surfel-based model fusion, implementing CUDA-accelerated global point cloud registration to close loops in the dense surface domain. Other hybrids, such as Orbeez-SLAM (Chung et al., 2022), fuse ORB feature-based front ends with implicit learning-based dense mapping.
• Learning-Based and Neural Implicit SLAM: Online frameworks incorporating real-time learning include DK-SLAM (Qu et al., 2024), which uses a meta-learned deep keypoint extractor and online-adaptive binary BoW loop closure, and Orbeez-SLAM (Chung et al., 2022), integrating NeRF-style mapping trained incrementally and utilizing fast multi-resolution hash encoding for near-instant convergence. Neural Implicit Dense Semantic SLAM (Haghighi et al., 2023) leverages parallel feature-based tracking and online neural SDF mapping, supporting multi-map architectures for scalability.
• Visual-Inertial and Multi-Modal: Keyframe-based visual-inertial pipelines, e.g., ORB-SLAM3 (Campos et al., 2020), OKVIS2-X (Boche et al., 6 Oct 2025), and (Kasyanov et al., 2017), integrate tightly-coupled IMU preintegration into the state estimation, with joint non-linear batch optimization in a sliding window and pose graph. OKVIS2-X generalizes to fusion of stereo, depth, LiDAR, and GNSS via modular factor graphs and submapping.
• Online Semantic and Scene-Graph SLAM: Recent frameworks incorporate semantic labeling and scene understanding, either by integrating object detection as graph constraints (Hempel et al., 2022), or by building structural scene graphs, e.g., vS-Graphs (Tourani et al., 3 Mar 2025), which infer 3D rooms and corridors as optimizable graph nodes.

2. Systems Architecture and Data Flow

A common structure across online visual SLAM frameworks involves parallel asynchronous threads operating on streaming sensor inputs:

• Front-End Processing: Responsible for sensor data acquisition, image pre-processing, feature detection/tracking, and initial pose estimation via PnP, motion-only BA, or direct methods.
• Keyframe & Map Management: Dynamically selects keyframes based on coverage or tracking quality criteria and manages the creation, culling, and storage of map points and keyframes in memory-bounded graphs or hybrid memory systems (short-term, working, and long-term memory as in RTAB-Map (Labbé et al., 2024)).
• Local Mapping and Optimization: Performs trianguation/bundle adjustment over a local window; updates the pose graph with new edges arising from odometry, loop closures, and semantic constraints.
• Loop Closure and Global BA: Detects loop candidates by appearance-based or semantic techniques, verifies geometric consistency, adds cross-map constraints, and triggers global optimization routines to enforce global consistency and correct drift.
• Back-End Dense and Neural Mapping: Online implicit mapping modules, e.g., NeRF or SDF-based, are trained incrementally using tracked frame poses and RGB-D keyframes, often in parallel to front-end tracking (Haghighi et al., 2023, Chung et al., 2022). Occupancy submaps and dense volumetric fusion are used in multi-modal systems (Boche et al., 6 Oct 2025).
• Semantic and Scene-Structure Threads: Systems such as vS-Graphs (Tourani et al., 3 Mar 2025) or KM-ViPE (Nasser et al., 1 Dec 2025) maintain semantic segmentation, component/structural inference, and open-vocabulary labeling modules concurrently.

3. Sensor Modalities, Fusion, and Real-Time Adaptation

Modern online SLAM frameworks support a variety of sensor configurations:

• Monocular, Stereo, and RGB-D: Frameworks are generally modular to operate on monocular, stereo, or RGB-D data. Depth is integrated via geometric stereo or learned/MVS predictions.
• Inertial Measurements: Visual-inertial SLAM tightly couples IMU data via preintegration, fusing visual and inertial constraints in the estimation window (Kasyanov et al., 2017, Campos et al., 2020, Boche et al., 6 Oct 2025).
• Lidar and GNSS Integration: Multi-sensor systems extend to 2D/3D LiDAR and GNSS, building factor graphs with pose, scan, and global position constraints. OKVIS2-X employs frame-to-map and map-to-map alignment factors, federated fusion of depth uncertainties, and global GNSS anchors in the graph (Boche et al., 6 Oct 2025).
• Online Calibration: Several frameworks incorporate online geometric and extrinsic calibration (e.g., OpenGV 2.0 (Huang et al., 5 Mar 2025), OKVIS2-X (Boche et al., 6 Oct 2025)), supporting continuous adaptation in deployment.
• Real-Time Loop: End-to-end pipelines are designed for real-time operation, using multi-threading, memory management, and incremental optimization (e.g., RTAB-Map's memory bounding (Labbé et al., 2024), KM-ViPE's sliding window BA (Nasser et al., 1 Dec 2025)). Integration strategies maintain bounded update times even as the map grows.

4. Advanced Capabilities: Semantics, Dynamics, and Open-Vocabulary

Recent frameworks extend classical visual SLAM to richer spatial and semantic representations:

• Semantic Mapping and Data Association: Online semantic SLAM frameworks (e.g., (Hempel et al., 2022), vS-Graphs (Tourani et al., 3 Mar 2025)) generate object-level landmarks from CNN-based detections, perform uncertainty-aware association, and incorporate these as additional constraints in the graph, yielding direct improvements in pose estimation and supporting higher-level spatial reasoning tasks.
• Dynamic Scene Handling: VDO-SLAM (Zhang et al., 2020) explicitly models and estimates SE(3) motion for dynamic objects, leveraging instance segmentation and scene flow to separate and track independently moving rigid objects, with global factor graphs jointly optimizing camera and object trajectories.
• Learning-Based Open-Vocabulary Mapping: KM-ViPE (Nasser et al., 1 Dec 2025) fuses DINO-based high-level visual features with geometric tracking under adaptive robust kernels (ARK), performs language-vision alignment for open-vocabulary queries (via CLIP/Talk2DINO), and demonstrates competitive mIoU and mapping accuracy in dynamic monocular scenarios.
• Scene-Graph and Structural Reasoning: vS-Graphs (Tourani et al., 3 Mar 2025) builds hierarchically-structured graph representations inferring walls, rooms, and corridors as graph entities and linking these to traditional keyframes and map points, directly optimizing jointly over semantic structure and standard SLAM variables.

5. Quantitative Evaluation and Comparison

The efficacy of online visual SLAM frameworks is measured on benchmarks using standardized metrics such as Absolute Trajectory Error (ATE), Relative Pose Error (RPE), map completeness, and segmentation metrics. Representative results:

System	Primary Metric	Sensor	Remark/Value
ORB-SLAM3	ATE (EuRoC)	Mono/Stereo/VI	0.041 m (mono), 0.035 m (stereo-IMU) (Campos et al., 2020)
OKVIS2-X	ATE (EuRoC)	VI/depth/GNSS	3–5 cm (VI), 4.6 cm (VI+depth), 2.4 cm (VI+LiDAR) (Boche et al., 6 Oct 2025)
KM-ViPE	ATE (TUM-RGBD)	Mono	1.9 cm; mIoU (Replica): 3.8% (Nasser et al., 1 Dec 2025)
LoopSmart	RMSE (ICL-NUIM)	RGB-D	0.02–0.04 m; surface error ≈ 0.009 m (Zhang et al., 2018)
RTAB-Map	ATE (KITTI)	Stereo/LiDAR	0.2–5.3 m (ORB2-RTAB), 1.26% transl. error (Labbé et al., 2024)
DK-SLAM	r_rel (KITTI 00–10)	Mono	0.27°, avg t_rel 3.34% (Qu et al., 2024)
vS-Graphs	ATE reduction	RGB-D	3.38% avg over ORB-SLAM3, up to 9.58% (Tourani et al., 3 Mar 2025)
TAMBRIDGE	ATE, PSNR, SSIM	RGB-D	1–2 cm ATE; PSNR 20–23 dB, SSIM ≈0.85–0.90 (Jiang et al., 2024)
Orbeez-SLAM	PSNR, FPS	Mono	29.25 dB, 20–23 fps (Chung et al., 2022)

These results demonstrate that online frameworks can deliver state-of-the-art accuracy, real-time performance, and semantic awareness, even under challenging conditions, and highlight tradeoffs between sensor fusion, computational complexity, and mapping fidelity.

6. Implementation Strategies and Scalability

Key architectural and algorithmic strategies for achieving scalable, robust, and extensible online SLAM include:

• Multi-Threading and Asynchrony: Decoupled pipelines for tracking, mapping, loop closure, and semantic processing prevent bottlenecks and ensure real-time tracking even as global optimization or mapping tasks run in the background.
• Memory and Submapping: Long-term scalability is achieved via keyframe culling, dynamic memory management (e.g., STM/WM/LTM (Labbé et al., 2024)), submapping strategies (split by spatial overlap or keyframe count (Boche et al., 6 Oct 2025)), and marginalization.
• Sensor Abstraction and Modularity: Frameworks expose modular APIs for integrating arbitrary sensors (cameras, IMUs, LiDAR, GNSS), custom camera models, or new place-recognition backends (Sumikura et al., 2019, Labbé et al., 2024, Boche et al., 6 Oct 2025).
• Adaptive and Learning-Driven Components: Online adaptation of deep models (e.g., CNNs for depth (Loo et al., 2021), meta-learned keypoints (Qu et al., 2024)), and subcomponent plug-and-play for fusion (e.g., Fusion Bridge in TAMBRIDGE (Jiang et al., 2024)) enable fast adaptation and extensibility.
• Community and Open-Source Ecosystem: Most state-of-the-art frameworks are released open-source and integrate with ROS or other robotics middlewares, facilitating research and applied adoption.

7. Challenges, Extensions, and Research Outlook

Online visual SLAM frameworks continue to evolve in response to the demands for higher accuracy, richer map semantics, greater scale, and robust operation under dynamic, real-world conditions. Open problems include:

• Explicit dynamic object and deformable scene modeling under computational constraints;
• Integration of SLAM with large-scale foundation models for visual and language generalization (Nasser et al., 1 Dec 2025);
• Scalability in memory and computation for multi-session and urban-scale deployments;
• Incorporating learned priors, anisotropic and semantic 3D mapping, and unified multi-agent multi-modality;
• Automated viewpoint planning and active exploration (e.g., D-optimal online utility in ExplORB-SLAM (Placed et al., 2022));
• Continual learning and adaptation for long-term autonomy with resilience to catastrophic forgetting (Loo et al., 2021).

Ongoing developments suggest increasingly tight integration of geometric, semantic, and learned representations, with closed-loop mutual adaptation at all levels of the SLAM pipeline.