Keyframe-based Dense Mapping with the Graph of View-Dependent Local Maps

Abstract: In this article, we propose a new keyframe-based mapping system. The proposed method updates local Normal Distribution Transform maps (NDT) using data from an RGB-D sensor. The cells of the NDT are stored in 2D view-dependent structures to better utilize the properties and uncertainty model of RGB-D cameras. This method naturally represents an object closer to the camera origin with higher precision. The local maps are stored in the pose graph which allows correcting global map after loop closure detection. We also propose a procedure that allows merging and filtering local maps to obtain a global map of the environment. Finally, we compare our method with Octomap and NDT-OM and provide example applications of the proposed mapping method.

• The paper introduces a novel hybrid mapping framework that integrates view-dependent local maps with keyframe-based pose graphs to model sensor uncertainty.
• The paper leverages image-plane NDT ellipsoids to adapt resolution dynamically based on object proximity, yielding higher fidelity than uniform voxel approaches.
• The paper demonstrates significant reductions in storage and computation while achieving lower RMSE and efficient loop closure corrections in dense mapping evaluations.

Keyframe-Based Dense Mapping with View-Dependent Local Map Graphs: Methodology and Evaluation

Motivation and Background

Environment mapping is foundational for autonomous robotics, with applications encompassing collision detection, path planning, and SLAM. Conventional dense mapping paradigms such as Octomap and Normal Distribution Transform Occupancy Map (NDT-OM) typically partition the environment into uniformly sized 3D voxels, a design choice that necessitates balancing map detail and computational resource constraints. These approaches also suffer from suboptimal integration of sensor uncertainty models, especially for RGB-D sensors where depth error is view-dependent. Moreover, in keyframe-based localization pipelines (e.g., ORB-SLAM2), global pose corrections following loop closures are ineffective for correcting already-fused dense map data because dense representations are monolithic.

The work "Keyframe-based Dense Mapping with the Graph of View-Dependent Local Maps" (2601.08520) addresses these critical obstacles by proposing a hybrid mapping strategy inspired by graph-based keyframe SLAM, applying local dense maps that are explicitly view-dependent in their representation and uncertainty modeling.

Method: View-Dependent Local Maps on a Pose Graph

The method introduces a dense mapping framework where local maps are constructed for each keyframe and stored as nodes in a pose graph. Each local map is a 2D grid defined in the image plane of the corresponding RGB-D keyframe. Instead of discretizing the global 3D space, the system discretizes the image into cells, and for each cell, it maintains a Normal Distribution Transform (NDT) ellipsoid that encodes the occupancy (mean and covariance of observed points) and color information.

Map Structure and Update

• Each keyframe node contains:
  • An RGB-D image pair
  • The camera pose
  • 2D grid cells, each storing a 3D NDT ellipsoid
• When integrating new observations, the system transforms sensor data to the local coordinate frame of the corresponding keyframe and updates the local 2D grid. Integration uses an incremental NDT update scheme with precise handling of means and covariances.
• Covisibility edges are maintained between keyframes based on normalized co-observed features, forming a pose graph similar to the ORB-SLAM2 co-visibility graph.
• Loop closures result in corrections of the global pose graph, and since local maps are maintained separately (not fused into a single occupancy grid), global corrections are feasible post-facto.

View-Dependent Representation

A defining property of this method is that the precision of the local representation varies as a function of viewpoint. Objects observed close to the camera are modeled with higher resolution (smaller ellipsoids in the grid), reflecting the greater certainty and sampling density of the RGB-D sensor. Distant objects are naturally represented with lower precision. This contrasts with uniform 3D voxel grids, where voxel size is constant and independent of observation geometry or sensor properties.

Global Map Generation

To construct a global map, local maps are iteratively merged. This nontrivial task involves:

• Transforming ellipsoid means and covariances into the global frame using the pose graph
• Filtering: Outlier and uncertain ellipsoids, especially those spanning object boundaries or manifesting excessive elongation due to depth ambiguity, are filtered using an explicit RGB-D uncertainty model
• Clustering: Overlapping ellipsoids from multiple local maps (typically from similar viewpoints/revisits) are mean-shift clustered in both image and 3D space for redundancy removal and global map compression
• Occlusion Filtering: If the same scene is observed from different distances/viewpoints, ellipsoids with lower uncertainty are retained, while redundant or occluded clusters are removed

Quantitative Evaluation

A series of experiments on TUM RGB-D and ICL-NUIM datasets quantitatively assess the method's storage efficiency, accuracy, and computational cost relative to Octomap and NDT-OM.

Key numerical findings:

• The proposed method achieves a significant reduction in both storage and computation by clustering and filtering ellipsoids post-merge: a typical global map's ellipsoid count is reduced by at least 2x compared to the combined size of all local maps.
• For the same environment, the global map input contains 892,232 ellipsoids, which is reduced to 358,258 after post-processing, while generating the map takes approximately 14.2 seconds on a standard laptop CPU.
• Update rates per frame for the merging procedure are on the order of 1–2 seconds for sequences with 100+ keyframes, rendering the approach suitable for online mapping on moderately powered compute platforms.
• When compared in a reconstruction task against Octomap, the proposed method yields far lower RMSE w.r.t. ground truth—e.g., 9.7–12.5 mm versus Octomap's 48.1–80.0 mm, even as the number of elements required for similar resolution is an order of magnitude lower. Octomap's RMSE improves only at costs of unsustainable memory growth.

The method supports higher fidelity representation of object geometry—especially for objects proximate to the sensor—compared to competing uniform-cell methods. This is shown qualitatively in global map reconstructions and quantitatively in error metrics.

Implications for Robotic Mapping

The key theoretical contribution is a principled integration of the measurement uncertainty of RGB-D sensors into the map representation, which is realized by storing and updating view-dependent NDT ellipsoids on the image grid. The result is a mapping strategy that is inherently adaptive: local regions of the environment are modeled at the maximal physically-justified fidelity as permitted by the viewpoint and sensor, and map representations remain correctable after global pose updates.

Practically, the proposed framework supports:

• High-precision local mapping for manipulation and navigation, where only a subset of the environment needs very fine-scale geometry
• Efficient map updates and corrections after loop closure, addressing one of the main drawbacks of dense SLAM modalities
• Compatibility with modular, keyframe-based SLAM and localization pipelines, enabling future coupling of sparse and dense mapping modalities within the same architecture

Since map storage is keyed to the number and distribution of keyframes rather than space, the storage requirements scale with trajectory complexity rather than purely environment size.

Future Research Directions

The work opens multiple avenues:

• Deeper integration with ORB-SLAM2 for dense-semi-dense fusion architectures, allowing the reuse of feature and pose graph data structures between localization and mapping
• Extensions to online joint optimization of global map and camera trajectory for further loop closure accuracy
• Adaptation to non-static/dynamic environments, in tandem with future RGB-D sensor uncertainty models that account for adverse observation conditions

Additionally, the architecture accommodates object- and region-specific semantic overlays and may facilitate active exploration via viewpoint selection that maximizes local map precision for objects of interest.

Conclusion

This paper presents a keyframe-centric, view-dependent dense mapping architecture that addresses fundamental limitations in existing 3D mapping with RGB-D sensors. By maintaining local maps as 2D NDT grids in the image plane of selected sensor viewpoints, the framework achieves variable-resolution, uncertainty-aware environment models that are both memory- and computation-efficient. These properties enable high-fidelity representation of close-proximity geometry and global updates consistent with graph-based SLAM. The quantitative and qualitative results highlight substantial improvements in mapping accuracy and efficiency compared to state-of-the-art voxel-based methods, laying the groundwork for scalable, real-time dense mapping in robotics and computer vision applications.