Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency (2401.09101v2)

Published 17 Jan 2024 in cs.RO and cs.CV

Abstract: Accurate and robust localization and mapping are essential components for most autonomous robots. In this paper, we propose a SLAM system for building globally consistent maps, called PIN-SLAM, that is based on an elastic and compact point-based implicit neural map representation. Taking range measurements as input, our approach alternates between incremental learning of the local implicit signed distance field and the pose estimation given the current local map using a correspondence-free, point-to-implicit model registration. Our implicit map is based on sparse optimizable neural points, which are inherently elastic and deformable with the global pose adjustment when closing a loop. Loops are also detected using the neural point features. Extensive experiments validate that PIN-SLAM is robust to various environments and versatile to different range sensors such as LiDAR and RGB-D cameras. PIN-SLAM achieves pose estimation accuracy better or on par with the state-of-the-art LiDAR odometry or SLAM systems and outperforms the recent neural implicit SLAM approaches while maintaining a more consistent, and highly compact implicit map that can be reconstructed as accurate and complete meshes. Finally, thanks to the voxel hashing for efficient neural points indexing and the fast implicit map-based registration without closest point association, PIN-SLAM can run at the sensor frame rate on a moderate GPU. Codes will be available at: https://github.com/PRBonn/PIN_SLAM.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (112)
  1. Particlenerf: Particle based encoding for online neural radiance fields. arXiv preprint, 2211.04041, 2022.
  2. Neural RGB-D Surface Reconstruction. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
  3. SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2019.
  4. J. Behley and C. Stachniss. Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments. In Proc. of Robotics: Science and Systems (RSS), 2018.
  5. P. Besl and N. McKay. A Method for Registration of 3D Shapes. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 14(2):239–256, 1992.
  6. Tensorf: Tensorial radiance fields. In Proc. of the Europ. Conf. on Computer Vision (ECCV), 2022.
  7. OverlapNet: Loop Closing for LiDAR-based SLAM. In Proc. of Robotics: Science and Systems (RSS), 2020.
  8. SuMa++: Efficient LiDAR-based Semantic SLAM. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2019.
  9. BundleFusion: Real-time Globally Consistent 3D Reconstruction using Online Surface Re-integration. ACM Trans. on Graphics (TOG), 36(3):1–18, 2017.
  10. F. Dellaert. Factor graphs and gtsam: A hands-on introduction. Georgia Institute of Technology, Tech. Rep, 2:4, 2012.
  11. CT-ICP Real-Time Elastic LiDAR Odometry with Loop Closure. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2022.
  12. Nerf-loam: Neural implicit representation for large-scale incremental lidar odometry and mapping. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2023.
  13. J. Deschaud. IMLS-SLAM: scan-to-model matching based on 3D data. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2018.
  14. Md-slam: Multi-cue direct slam. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2022.
  15. SegMap: 3D Segment Mapping using Data-Driven Descriptors. In Proc. of Robotics: Science and Systems (RSS), 2018.
  16. Neural points: Point cloud representation with neural fields for arbitrary upsampling. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
  17. The dynamic window approach to collision avoidance. IEEE Journal of Robotics and Automation, 4(1):23–33, 1997.
  18. Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2012.
  19. Improved Techniques for Grid Mapping with Rao-Blackwellized Particle Filters. IEEE Trans. on Robotics (TRO), 23(1):34–46, 2007.
  20. Implicit geometric regularization for learning shapes. In Proc. of the Intl. Conf. on Machine Learning (ICML), 2020.
  21. The hilti slam challenge dataset. IEEE Robotics and Automation Letters (RA-L), 7(3):7518–7525, 2022.
  22. Real-Time Loop Closure in 2D LIDAR SLAM. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2016.
  23. Virtual occupancy grid map for submap-based pose graph slam and planning in 3d environments. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2018.
  24. OctoMap: An Efficient Probabilistic 3D Mapping Framework Based on Octrees. Autonomous Robots, 34(3):189–206, 2013.
  25. Neural kernel surface reconstruction. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  26. Di-fusion: Online implicit 3d reconstruction with deep priors. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  27. Neural lidar fields for novel view synthesis. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2023.
  28. Loner: Lidar only neural representations for real-time slam. IEEE Robotics and Automation Letters (RA-L), 1(1):1 – 8, 2023.
  29. ESLAM: Efficient dense slam system based on hybrid representation of signed distance fields. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  30. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. on Graphics (TOG), 42(4), July 2023.
  31. Mulran: Multimodal range dataset for urban place recognition. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2020.
  32. Scan context++: Structural place recognition robust to rotation and lateral variations in urban environments. IEEE Trans. on Robotics (TRO), 38(3):1856–1874, 2022.
  33. G. Kim and A. Kim. Scan Context: Egocentric Spatial Descriptor for Place Recognition within 3D Point Cloud Map. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2018.
  34. D. Kingma and J. Ba. Adam: A Method for Stochastic Optimization. In Proc. of the Int. Conf. on Learning Representations (ICLR), 2015.
  35. A flexible and scalable slam system with full 3d motion estimation. In Proc. of the IEEE Intl. Sym. on Safety, Security, and Rescue Robotics (SSRR), 2011.
  36. A portable three-dimensional lidar-based system for long-term and wide-area people behavior measurement. Intl. Journal of Advanced Robotic Systems, 16(2):1–20, 2019.
  37. Voxelized gicp for fast and accurate 3d point cloud registration. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021.
  38. IR-MCL: Implicit Representation-Based Online Global Localization. IEEE Robotics and Automation Letters (RA-L), 8(3):1627–1634, 2023.
  39. Panoptic-phnet: Towards real-time and high-precision lidar panoptic segmentation via clustering pseudo heatmap. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
  40. Lo-net: Deep real-time lidar odometry. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
  41. Neuralangelo: High-fidelity neural surface reconstruction. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  42. J. Lin and F. Zhang. Loam_livox A Robust LiDAR Odemetry and Mapping LOAM Package for Livox LiDAR. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2019.
  43. Translo: A window-based masked point transformer framework for large-scale lidar odometry. In Proc. of the Conf. on Advancements of Artificial Intelligence (AAAI), 2023.
  44. Large-scale lidar consistent mapping using hierarchical lidar bundle adjustment. IEEE Robotics and Automation Letters (RA-L), 8(3):1523–1530, 2023.
  45. Efficient and consistent bundle adjustment on lidar point clouds. IEEE Trans. on Robotics (TRO), pages 1–21, 2023.
  46. W. Lorensen and H. Cline. Marching Cubes: a High Resolution 3D Surface Construction Algorithm. In Proc. of the Intl. Conf. on Computer Graphics and Interactive Techniques (SIGGRAPH), 1987.
  47. Occupancy networks: Learning 3d reconstruction in function space. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
  48. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Proc. of the Europ. Conf. on Computer Vision (ECCV), 2020.
  49. RangeNet++: Fast and Accurate LiDAR Semantic Segmentation. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2019.
  50. FastSLAM: A Factored Solution to the Simultaneous Localization and Mapping Problem. In Proc. of the Conf. on Advancements of Artificial Intelligence (AAAI), 2002.
  51. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. on Graphics, 41(4):102:1–102:15, 2022.
  52. KinectFusion: Real-Time Dense Surface Mapping and Tracking. In Proc. of the Intl. Symposium on Mixed and Augmented Reality (ISMAR), 2011.
  53. Real-time 3d reconstruction at scale using voxel hashing. ACM Trans. on Graphics (TOG), 32(6), 2013.
  54. Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2017.
  55. isdf: Real-time neural signed distance fields for robot perception. In Proc. of Robotics: Science and Systems (RSS), 2022.
  56. MULLS: Versatile LiDAR SLAM Via Multi-Metric Linear Least Square. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021.
  57. Voxfield: Non-projective signed distance fields for online planning and 3d reconstruction. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2022.
  58. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.
  59. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proc. of the Conf. on Neural Information Processing Systems (NeurIPS), 2019.
  60. Convolutional occupancy networks. In Proc. of the Europ. Conf. on Computer Vision (ECCV), 2020.
  61. Lins: A lidar-inertial state estimator for robust and efficient navigation. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2020.
  62. The newer college dataset: Handheld lidar, inertial and vision with ground truth. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2020.
  63. F. Ramos and L. Ott. Hilbert maps: Scalable continuous occupancy mapping with stochastic gradient descent. Intl. Journal of Robotics Research (IJRR), 35(14):1717–1730, 2016.
  64. Voxgraph: Globally consistent, volumetric mapping using signed distance function submaps. IEEE Robotics and Automation Letters (RA-L), 5(1):227–234, 2019.
  65. Locus 2.0: Robust and computationally efficient lidar odometry for real-time 3d mapping. IEEE Robotics and Automation Letters (RA-L), 2022.
  66. R.A. Rosu and S. Behnke. Permutosdf: Fast multi-view reconstruction with implicit surfaces using permutohedral lattices. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  67. Slamesh: Real-time lidar simultaneous localization and meshing. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2023.
  68. Gp-slam+: real-time 3d lidar slam based on improved regionalized gaussian process map reconstruction. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2020.
  69. S. Rusinkiewicz and M. Levoy. Efficient variants of the ICP algorithm. In Proc. of Int. Conf. on 3-D Digital Imaging and Modeling, 2001.
  70. Fast point feature histograms (fpfh) for 3d registration. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2009.
  71. Point-slam: Dense neural point cloud-based slam. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2023.
  72. Dynablox: Real-time detection of diverse dynamic objects in complex environments. IEEE Robotics and Automation Letters (RA-L), 8(10):6259 – 6266, 2023.
  73. Generalized-ICP. In Proc. of Robotics: Science and Systems (RSS), 2009.
  74. LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2020.
  75. T. Shan and B. Englot. LeGO-LOAM: Lightweight and Ground-Optimized Lidar Odometry and Mapping on Variable Terrain. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2018.
  76. Robust Double-Encoder Network for RGB-D Panoptic Segmentation. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2023.
  77. Information Gain-based Exploration Using Rao-Blackwellized Particle Filters. In Proc. of Robotics: Science and Systems (RSS), 2005.
  78. The replica dataset: A digital replica of indoor spaces. arXiv preprint, arXiv:1906.05797, 2019.
  79. imap: Implicit mapping and positioning in real-time. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2021.
  80. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  81. Mips-fusion: Multi-implicit-submaps for scalable and robust online neural rgb-d reconstruction. ACM Transactions on Graphics, 42(6), 2023.
  82. Robust Monte Carlo Localization for Mobile Robots. Artificial Intelligence, 128(1-2), 2001.
  83. A. Uy and G. Lee. PointNetVLAD: Deep point cloud based retrieval for large-scale place recognition. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018.
  84. Poisson Surface Reconstruction for LiDAR Odometry and Mapping. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021.
  85. VDBFusion: Flexible and Efficient TSDF Integration of Range Sensor Data. Sensors, 22(3):1296, 2022.
  86. KISS-ICP: In Defense of Point-to-Point ICP – Simple, Accurate, and Robust Registration If Done the Right Way. IEEE Robotics and Automation Letters (RA-L), 8(2):1029–1036, 2023.
  87. PyPose: A library for robot learning with physics-based optimization. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  88. Efficient 3d deep lidar odometry. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 45(5):5749–5765, 2022.
  89. PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
  90. F-LOAM: Fast LiDAR Odometry and Mapping. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2021.
  91. Co-slam: Joint coordinate and sparse parametric encodings for neural real-time slam. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  92. Go-surf: Neural feature grid optimization for fast, high-fidelity rgb-d surface reconstruction. In Proc. of the Intl. Conf. on 3D Vision (3DV), 2022.
  93. Elastic and Efficient LiDAR Reconstruction for Large-Scale Exploration Tasks. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021.
  94. ElasticFusion: Dense SLAM Without A Pose Graph. In Proc. of Robotics: Science and Systems (RSS), 2015.
  95. Locndf: Neural distance field mapping for robot localization. IEEE Robotics and Automation Letters (RA-L), 8(8):4999–5006, 2023.
  96. Log-gpis-mop: A unified representation for mapping, odometry, and planning. IEEE Trans. on Robotics (TRO), 39(5):4078–4094, 2023.
  97. Lio-ekf: High frequency lidar-inertial odometry using extended kalman filters. arXiv preprint, arXiv:2311.09887, 2023.
  98. Point-nerf: Point-based neural radiance fields. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2023.
  99. Fast-lio2: Fast direct lidar-inertial odometry. IEEE Trans. on Robotics (TRO), 38(4):2053–2073, 2022.
  100. Active neural mapping. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2023.
  101. Vox-fusion: Dense tracking and mapping with voxel-based neural implicit representation. In Proc. of the Intl. Symposium on Mixed and Augmented Reality (ISMAR), 2022.
  102. M2dgr: A multi-sensor and multi-scenario slam dataset for ground robots. IEEE Robotics and Automation Letters (RA-L), 7(2):2266–2273, 2022.
  103. Litamin2: Ultra light lidar-based slam using geometric approximation applied with kl-divergence. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2021.
  104. Nf-atlas: Multi-volume neural feature fields for large scale lidar mapping. IEEE Robotics and Automation Letters (RA-L), 8(9):5870–5877, 2023.
  105. Efficient and probabilistic adaptive voxel mapping for accurate online lidar odometry. IEEE Robotics and Automation Letters (RA-L), 7(3):8518–8525, 2022.
  106. Y. Yuan and A. Nüchter. An algorithm for the se(3)-transformation on neural implicit maps for remapping functions. IEEE Robotics and Automation Letters (RA-L), 7(3):7763–7770, 2022.
  107. On Degeneracy of Optimization-Based State Estimation Problems. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2016.
  108. J. Zhang and S. Singh. LOAM: Lidar Odometry and Mapping in Real-time. In Proc. of Robotics: Science and Systems (RSS), 2014.
  109. Z. Zhang and D. Scaramuzza. A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry. In Proc. of the IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS), 2018.
  110. In-place scene labelling and understanding with implicit scene representation. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision (ICCV), 2021.
  111. SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations. In Proc. of the IEEE Intl. Conf. on Robotics & Automation (ICRA), 2023.
  112. Nice-slam: Neural implicit scalable encoding for slam. In Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2022.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Yue Pan (47 papers)
  2. Xingguang Zhong (11 papers)
  3. Louis Wiesmann (8 papers)
  4. Thorbjörn Posewsky (2 papers)
  5. Jens Behley (50 papers)
  6. Cyrill Stachniss (98 papers)
Citations (16)
View on Semantic Scholar

Summary

Introduction to PIN-SLAM

Simultaneous Localization and Mapping (SLAM) is a fundamental challenge for autonomous robots, encompassing the twin tasks of understanding a robot's position within an environment while mapping the environment itself. A new system, PIN-SLAM, emerges from the SLAM domain, introducing an effective method to maintain a globally consistent map using a Point-based Implicit Neural (PIN) map representation. The PIN-SLAM system employs sparse neural points—key nodal points that, unlike grid-based counterparts, offer elasticity to adapt to pose corrections when identifying loops in the robot's journey, ensuring global consistency.

System Overview

PIN-SLAM is distinctive in its operation, alternating between mapping given pose estimations and odometry, the process of tracking the robot's position over time. Mapping involves constructing the local signed distance field, a mathematical model representing the proximity of the robot to the nearest surface. Odometry involves estimating each new scan's pose based on the current map. Notably, PIN-SLAM efficiently integrates loop closure detection and correction — a crucial aspect for maintaining a consistent map over large areas or longer periods.

Achievements and Capabilities

Experimentation shows that PIN-SLAM competes with the highest standards of LiDAR odometry and SLAM systems across a variety of datasets and environments. It completes mapping tasks with impressive accuracy and completeness and performs stably at standard sensor frame rates on moderate hardware. Moreover, its PIN map provides a significantly more compact data structure for storing environmental features compared to traditional explicit representations. Additionally, PIN-SLAM can conduct metric-semantic SLAM, linking measurable data from the environment with semantic information, like distinguishing between different types of objects and surfaces.

Future Horizons

There are areas where PIN-SLAM's capabilities can be expanded. For instance, integrating Inertial Measurement Units (IMUs) could refine the accuracy and robustness of the system. Furthermore, future iterations may focus on adaptively allocating neural points to optimize map quality and surface reconstruction. The journey to enhance SLAM systems continues as researchers aim to push boundaries, pinpointing novel data structures and integration methods to make autonomous navigation even more reliable and precise.

Summing Up

The PIN-SLAM system stands out for its approach to global consistency in mapping, a trait that is invaluable for autonomous robots working over extended periods or large spaces. With its point-based implicit neural maps and inherent robustness to environmental changes, PIN-SLAM represents a significant advancement in robot navigation technologies. As SLAM systems evolve, we move closer to robots that can traverse and understand the complexities of the physical world with increasing finesse.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown