SL-SLAM: Deep-Learned Multi-Modal SLAM

• SL-SLAM is a system that fuses learned visual features, IMU, and radio data for scalable, robust localization and mapping.
• The architecture employs modern deep networks like SuperPoint and LightGlue to enhance traditional SLAM pipelines with superior feature extraction and matching.
• Cooperative multi-modal fusion in SL-SLAM improves estimation accuracy in diverse environments, enabling real-time mapping and adaptive beam management.

Simultaneous Localization and Mapping (SLAM) in modern sensing and communication systems is increasingly shaped by the integration of multi-modal data sources, deep learning, geometric modeling, and cooperative protocols. The term "SL-SLAM" encapsulates a class of systems and methodologies leveraging learned feature extraction, multi-modal inference, and cooperative strategies to deliver scalable, accurate, and robust localization and mapping—particularly within the context of visual-inertial robotics, mmWave/5G/6G ISAC, and hybrid environments. The following article provides a comprehensive review of the SL-SLAM paradigm across its major technical, algorithmic, and experimental axes.

1. Foundations and System Architectures

SL-SLAM systems inherit from classical SLAM pipelines—comprising front-end data association and motion estimation, and back-end optimization—but augment these with deep neural feature extractors, multi-modal fusion, and/or integrated communication-aware modules. In visual-inertial contexts such as "SL-SLAM: A robust visual-inertial SLAM based deep feature extraction and matching" (Xiao et al., 2024), the architecture replaces traditional hand-crafted feature detectors with deep convolutional networks. Specifically, SuperPoint serves as the universal keypoint detector and descriptor, while LightGlue, a transformer-based matcher, facilitates robust and adaptive feature matching. This dual-network design operates as a drop-in replacement for the entire ORB/BoW stack in ORB-SLAM3.

In radio-based and cooperative ISAC settings, as in "Cooperative Mapping, Localization, and Beam Management via Multi-Modal SLAM in ISAC Systems" (Que et al., 8 Jul 2025), the architecture is inherently multi-agent and multi-modal. Each user equipment (UE) executes a local SLAM instance, uploading partial maps and odometry to a base station which performs centralized map fusion and inference. The pipeline incorporates radio-based angle-of-arrival (AoA)/angle-of-departure (AoD) measurements, IMU/preintegrated inertial states, and optionally camera-based (YOLO) localization.

2. Deep Learning for Feature Extraction and Matching

SL-SLAM visual frameworks depart from classical point detectors (e.g., ORB) by employing learned, task-optimized networks. The SuperPoint feature extractor encodes each RGB or grayscale image via a VGG-style convolutional encoder, generating both a keypoint score tensor and per-pixel 256D descriptors. Adaptive detection thresholds are set dynamically by combining the image score statistics and history of match counts to regulate keypoint quality and density under variable conditions.

For feature matching, LightGlue applies a stack of transformer layers with mixed self- and cross-attention, coupled with an early-exit stop-mask classifier. This design yields match assignment matrices with confidence scores, permitting both soft and hard correspondence extraction with robust mutual consistency. These deep modules are not fine-tuned in SL-SLAM; instead, they leverage strong pre-training strategies such as homographic adaptation and equivariance-based losses, ensuring generalization across a range of environments (Xiao et al., 2024).

3. Integration into SLAM Pipelines: Visual, Inertial, and Hybrid Modes

SL-SLAM frameworks integrate these deep learning modules within the multi-threaded structure of established SLAM systems (e.g., ORB-SLAM3), applying them during tracking, local bundle adjustment, and loop closure.

• Tracking: Each new image undergoes SuperPoint detection/description and LightGlue matching to previous frames. Pose initialization may be visual (PnP/epipolar geometry) or IMU-propagated.
• Mapping/Bundle Adjustment: New keyframes are selected based on tracking quality and pose changes. Local mapping involves LightGlue matching across co-visible keyframes, with bundle adjustment jointly optimizing the windowed keyframe poses and landmarks under a robustified reprojection loss.
• Loop Closure: Place recognition is handled by a BoW database built from binarized SuperPoint descriptors, with geometric verification via LightGlue and global Sim(3) pose-graph optimization to enforce scale consistency.

Sensor configurations include monocular, stereo, monocular-inertial, and stereo-inertial, where visual-inertial fusion is realized via standard preintegration and joint sliding-window optimization, leveraging multi-state constraint Kalman filtering or factor-graph backends (Xiao et al., 2024).

4. Cooperative and Multi-Modal Mapping in ISAC Systems

For 5G/6G mmWave ISAC, SL-SLAM advances to a cooperative multi-user setting. The underlying Bayesian estimation framework jointly infers all UE trajectories and the global radio map by recursive prediction and measurement updates, as formalized by

$$p(\{x_{UE,j}(t)\}, m \mid \{z_j(1:t)\}, \{u_j(1:t)\})$$

where $m$ represents the set of virtual anchor (VA) locations and associated parameters populated from radio observations and IMU data across all UEs (Que et al., 8 Jul 2025).

The map construction algorithm proceeds in two stages:

1. Coarse Initialization: Each UE locally triangulates VAs and uploads feature position estimates (mean, covariance) to the BS, which fuses matches by minimum-Mahalanobis fusion, creating the global VA pool with confidence.
2. Refinement: UEs download the current global map, run local SLAM with these as legacy features, and the BS iteratively re-weights and merges legacy/new features using MMSE fusion.

This enables robust global radio map construction and supports highly dynamic, heterogeneous sensing conditions prevalent in real-world ISAC scenarios.

5. Multi-Modal Fusion and Localization

To further enhance UE pose estimates, SL-SLAM in ISAC systems fuses radio+IMU SLAM estimates (Gaussian posterior mean and covariance) with camera-based detections using an error-aware fusion scheme. YOLO-derived position hypotheses for tracked objects are associated to UEs by nearest-neighbor heuristics with covariance-aware rejection; the final fused position per user is computed as the mean of the two modality posteriors, weighted by their uncertainty (Que et al., 8 Jul 2025). If association fails, the fallback is the radio+IMU solution.

6. Sensing-Aided Beam Management and Communication Layer Integration

A distinguishing feature of ISAC-oriented SL-SLAM is the integration of SLAM outputs into beam management. By leveraging real-time UE positions and the global VA map, the BS synthesizes UE-specific path priors—predicted AoA/AoD, angular support, and path gain—for initial beam selection. The downlink beam search exploits these priors for high-probability initializations, with inter-user interference handled by ordering candidates to maximize spatial separation. Beam refinement targets a high-resolution codebook sector centered on the predicted direction, enabling efficient and adaptive MIMO communication (Que et al., 8 Jul 2025).

7. Experimental Results and Comparative Performance

Experimental campaigns on visual-inertial datasets, such as EuRoC MAV and TUM-VI, establish that SL-SLAM outperforms state-of-the-art baselines (VI-DSO, OKVIS, VINS-Mono, ORB-SLAM3) in absolute trajectory error (ATE), successfully winning the majority of test sequences across both monocular and stereo-inertial configurations. Drift is minimized even across long (>900m) sequences and in challenging visual environments (low-light, jitter, weak texture) (Xiao et al., 2024).

In ISAC and mmWave/5G/6G domains, SL-SLAM achieves up to 60% gains in radio map OSPA error, a 37.5% reduction in UE localization error, and spectral efficiency improvements of 36–149% relative to exhaustive legacy beam sweeping. Real and simulated results confirm resilience under heterogeneous user sensing conditions and multi-user cooperation regimes (Que et al., 8 Jul 2025).

8. Implementation and Efficiency Considerations

SL-SLAM frameworks achieve real-time operation despite deep feature computation overheads. ONNX Runtime is used for inference acceleration; total system throughput is maintained via parallelization across tracking, mapping, and loop closure threads. The system exhibits comparable or favorable per-frame runtimes relative to ORB-SLAM3, and deep components do not compromise online performance (Xiao et al., 2024).

In ISAC implementations, the radio+IMU and multi-modal fusion stages are computationally partitioned between UEs and the base station, supporting deployment in real-world heterogeneous sensing networks.

---

SL-SLAM thus denotes a class of SLAM systems characterized by the fusion of deep-learned visual features, multi-modal data integration (visual/IMU/radio), and cooperative mapping protocols, with demonstrated superiority in demanding robotic, mixed-modal, and wireless communications environments (Xiao et al., 2024, Que et al., 8 Jul 2025).