Mono-Hydra++: Real-Time Monocular Scene Graph Construction with Multi-Task Learning for 3D Indoor Mapping

Abstract: Autonomous agile robots need more than metric geometry: they must understand objects, rooms, places, and spatial relations for search, inspection, exploration, and human robot interaction. Conventional metric maps support localization and collision avoidance, but do not provide this semantic and relational structure. 3D scene graphs address this gap by connecting geometry with object level and room level understanding. Building such representations on agile platforms remains difficult because aerial and lightweight robots operate under strict payload, power, and compute limits, making RGB-D cameras and LiDAR sensors impractical for many onboard settings. We present Mono-Hydra++, a real time monocular RGB plus IMU pipeline for indoor metric semantic mapping and hierarchical 3D scene graph construction. The system combines M2H-MX, a DINOv3 based multi-task model for depth and semantics, with a deep feature visual inertial odometry front end, sparse predicted depth constraints in the VIO derived pose graph, semantic masking for dynamic regions, and pose aware temporal alignment before volumetric fusion in the Mono-Hydra backend. On the Go-SLAM ScanNet evaluation subset, Mono-Hydra++ achieves 1.6% lower average trajectory error than the strongest RGB-D baseline in our comparison, while using only monocular RGB plus IMU input. On calibrated 7-Scenes, it improves average ATE by 29.8% over the strongest competing calibrated baseline. We further validate Mono-Hydra++ in a real ITC building deployment using RealSense RGB plus IMU and demonstrate embedded feasibility by deploying the ONNX/TensorRT FP16 M2H-MX-L perception model at 25.53 FPS on a Jetson Orin NX 16GB. These results show that Mono-Hydra++ can provide real time metric semantic mapping and scene graph construction for resource constrained robotic platforms without relying on active depth sensors.

• The paper introduces Mono-Hydra++, which integrates multi-task dense perception, geometry-aware VIO coupling, and pose-aware temporal fusion for real-time 3D mapping.
• It achieves state-of-the-art metrics on NYUDv2 and Cityscapes while significantly reducing trajectory error and improving scene reconstruction quality.
• The framework demonstrates embedded real-time deployment feasibility on agile robotic platforms despite challenges in object-level semantic preservation.

Mono-Hydra++: Real-Time Monocular Scene Graph Construction for 3D Indoor Mapping

Motivation and Problem Statement

Mono-Hydra++ addresses the fundamental challenge of supporting high-level spatial reasoning on agile robotic platforms where power, payload, and compute constraints preclude the use of RGB-D or LiDAR sensors. Conventional SLAM systems provide metric geometry but lack explicit object-room-place relations needed for intelligent navigation, search, and HRI. The construction of 3D scene graphs from monocular RGB+IMU is exceptionally challenging due to unstable monocular depth estimation in low-texture indoor scenes, fragility of VIO under feature sparsity, and frame-wise prediction flicker that disrupts spatial consistency.

System Architecture and Contributions

Mono-Hydra++ integrates multiple system-level innovations for real-time monocular metric-semantic mapping and hierarchical scene graph generation:

• Multi-Task Dense Perception: The M2H-MX model, using a DINOv3 backbone with a Mamba-based decoder, jointly predicts depth and semantic segmentation. This multi-task learning approach leverages cross-task contextual signals and mitigates negative transfer at task boundaries. Adaptive binning (as in AdaBins) is used for depth prediction, enhancing resolution allocation.
• Geometry-Aware VIO Coupling: M2H-MX outputs inform the VIO pipeline via sparse predicted-depth constraints at robust keypoints (SuperPoint features), semantic masking for dynamic classes, and robustification to filter out unreliable cues. The adoption of an RVIO2-style square-root robocentric update enables computational efficiency and improved scale stability.
• Pose-Aware Temporal Fusion: Recent predictions are pose-aligned and temporally fused using VIO estimates, enforcing geometric consistency and reducing flicker in depth and semantic outputs before volumetric mesh fusion and scene graph construction.

The framework unifies multi-task perception, VIO-based geometric alignment, and temporally consistent mapping, showing clear improvements over prior Mono-Hydra and M2H variants.

Perception Module Analysis

The M2H-MX model is benchmarked on NYUDv2 and Cityscapes, attaining state-of-the-art results among comparable multi-task models:

• On NYUDv2, M2H-MX-L achieves a semantic mIoU of 65.60 and depth RMSE of 0.3800, outperforming previous baselines including TaskPrompter, MTMamba, and M2H (Udugama et al., 31 Mar 2026).
• On Cityscapes, M2H-MX-L reaches 82.28 mIoU (semantic) and 3.89 RMSE (disparity), with consistent generalization across indoor/outdoor dense prediction. The perception module’s improvements directly impact downstream SLAM and mapping performance.

Monocular RGB+IMU Mapping and Scene Graph Generation

Mono-Hydra++ demonstrates competitive monocular mapping performance:

• Trajectory Estimation: On ScanNet, Mono-Hydra++ achieves 1.6% lower average trajectory error (ATE) than the best RGB-D baseline (Go-SLAM) and outperforms all monocular baselines. On calibrated 7-Scenes, it improves average ATE by 29.8% over the best calibrated baseline.
• 3D Reconstruction: On 7-Scenes, Mono-Hydra++ attains the best ATE and competitive Chamfer distance, validating its reconstruction quality in both calibrated and uncalibrated settings (2605.17661).
• Semantic Mesh/Object-Level Quality: On ScanNet, Mono-Hydra++ (M2H-MX variant) achieves 44.96 global mesh mIoU, 42.59 Radius [email protected], and 33.81 Box [email protected], outperforming MTMamba++ and M2H in the same pipeline. Failure modes are concentrated at object boundaries and structurally ambiguous classes.

Robustness and Dynamic Scene Handling

Ablation studies on the uHumans2 dynamic-scene benchmark show that:

• Sparse predicted-depth factors and semantic masking reduce VIO drift and improve object-node detection in dynamic environments.
• Pose-warp temporal fusion (window K = 3) provides optimal trade-offs between trajectory stability and local scene graph consistency in longer, drift-prone sequences.
• Loop-closure candidates remain available, ensuring backend optimization even in highly dynamic sequences.

Real-Time Embedded Deployment

Mono-Hydra++ is shown to be viable for embedded deployment:

• ONNX/TensorRT FP16 inference with M2H-MX-L achieves 25.53 FPS on Jetson Orin NX 16GB at reduced resolution, with mean GPU compute of 39 ms per frame.
• On the ITC real-world mapping dataset, mean mapping errors of 0.08 m (M2H-MX-L) are sustained, validating practical onboard feasibility in long-corridor indoor scenarios.

Limitations and Practical Implications

Notwithstanding the strong trajectory and mesh quality, object-level semantic preservation remains a challenge: small and ambiguous objects are prone to boundary leakage and misclassification, negatively impacting scene graph node detection and semantic relations. Temporal fusion is sensitive to VIO drift; wide pose-warp windows risk introducing stale semantic evidence, especially in dynamic environments.

Mono-Hydra++ closes much of the performance gap with RGB-D/LiDAR-based pipelines without their sensor requirements, making it suitable for drones and agile robots.

Future Directions

Future directions include:

• Extending open-set and uncertainty-aware semantic modeling for handling unseen categories and improving object permanence.
• Integrating graph-level correction and language grounding to bridge missing object nodes and enable semantic querying.
• Optimizing downstream navigation and planning on compute-constrained platforms using scene graph representations.

Conclusion

Mono-Hydra++ presents a unified approach to real-time monocular metric-semantic mapping and hierarchical 3D scene graph construction with strong performance across standard benchmarks, dynamic environments, and real-world deployments. Its integration of multi-task dense perception, geometry-aware VIO coupling, and temporally consistent fusion establishes a new standard for lightweight, deployable spatial understanding on onboard robotic platforms where sensor-rich modalities are impractical (2605.17661).