Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps (2403.02751v2)

Published 5 Mar 2024 in cs.RO

Abstract: We present Splat-Nav, a real-time navigation pipeline designed to work with environment representations generated by Gaussian Splatting (GSplat), a popular emerging 3D scene representation from computer vision. Splat-Nav consists of two components: 1) Splat-Plan, a safe planning module, and 2) Splat-Loc, a robust pose estimation module. Splat-Plan builds a safe-by-construction polytope corridor through the map based on mathematically rigorous collision constraints and then constructs a B\'ezier curve trajectory through this corridor. Splat-Loc provides a robust state estimation module, leveraging the point-cloud representation inherent in GSplat scenes for global pose initialization, in the absence of prior knowledge, and recursive real-time pose localization, given only RGB images. The most compute-intensive procedures in our navigation pipeline, such as the computation of the B\'ezier trajectories and the pose optimization problem run primarily on the CPU, freeing up GPU resources for GPU-intensive tasks, such as online training of Gaussian Splats. We demonstrate the safety and robustness of our pipeline in both simulation and hardware experiments, where we show online re-planning at 5 Hz and pose estimation at about 25 Hz, an order of magnitude faster than Neural Radiance Field (NeRF)-based navigation methods, thereby enabling real-time navigation.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Vision-only robot navigation in a neural radiance world. IEEE Robotics and Automation Letters (RA-L), 7(2):4606–4613, 2022.
  2. Depth camera based indoor mobile robot localization and navigation. In 2012 IEEE International Conference on Robotics and Automation, pages 1697–1702. IEEE, 2012.
  3. Catnips: Collision avoidance through neural implicit probabilistic scenes. arXiv preprint arXiv:2302.12931, 2023.
  4. Object modelling by registration of multiple range images. Image and vision computing, 10(3):145–155, 1992.
  5. Lowe David. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60:91–110, 2004.
  6. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 224–236, 2018.
  7. Herbert Edelsbrunner. Surface Reconstruction by Wrapping Finite Sets in Space. In Boris Aronov, Saugata Basu, János Pach, and Micha Sharir, editors, Discrete and Computational Geometry: The Goodman-Pollack Festschrift, Algorithms and Combinatorics, pages 379–404. Springer, Berlin, Heidelberg, 2003. ISBN 978-3-642-55566-4.
  8. A. Elfes. Using occupancy grids for mobile robot perception and navigation. Computer, 22(6):46–57, June 1989. ISSN 1558-0814.
  9. Mobile robot localization using extended kalman filter. In 2020 3rd International Conference on Computer Applications & Information Security (ICCAIS), pages 1–5. IEEE, 2020.
  10. Particle filters for mobile robot localization. In Sequential Monte Carlo methods in practice, pages 401–428. Springer, 2001.
  11. A robust computational test for overlap of two arbitrary-dimensional ellipsoids in fault-detection of kalman filters. In 2012 15th International Conference on Information Fusion, pages 396–401, 2012.
  12. The robot localization problem. SIAM Journal on Computing, 26(4):1120–1138, 1997.
  13. Fiesta: Fast incremental euclidean distance fields for online motion planning of aerial robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4423–4430, 2019.
  14. A direct least-squares (dls) method for pnp. In 2011 International Conference on Computer Vision, pages 383–390. IEEE, 2011.
  15. Nick Alger (https://math.stackexchange.com/users/3060/nick alger). Detect if two ellipses intersect. Mathematics Stack Exchange, 2020. URL https://math.stackexchange.com/q/3678498. URL:https://math.stackexchange.com/q/3678498 (version: 2021-05-09).
  16. Differentiable particle filters: End-to-end learning with algorithmic priors. arXiv preprint arXiv:1805.11122, 2018.
  17. Anytime motion planning using the rrt. In 2011 IEEE international conference on robotics and automation, pages 1478–1483. IEEE, 2011.
  18. Particle filter networks with application to visual localization. In Conference on robot learning, pages 169–178. PMLR, 2018.
  19. Probabilistic roadmaps for path planning in high-dimensional configuration spaces. IEEE transactions on Robotics and Automation, 12(4):566–580, 1996.
  20. 3D Gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023. URL https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/.
  21. Slam-driven robotic mapping and registration of 3d point clouds. Automation in Construction, 89:38–48, 2018.
  22. SM LaValle. Planning algorithms. Cambridge University Press google schola, 2:3671–3678, 2006.
  23. Randomized kinodynamic planning. The international journal of robotics research, 20(5):378–400, 2001.
  24. Rapidly-exploring random trees: Progress and prospects. Algorithmic and computational robotics: new directions, 5:293–308, 2001.
  25. Epnp: Efficient perspective-n-point camera pose estimation. Int. J. Comput. Vis, 81(2):155–166, 2009.
  26. Lightglue: Local feature matching at light speed. arxiv 2023. arXiv preprint arXiv:2306.13643, 2023.
  27. Planning dynamically feasible trajectories for quadrotors using safe flight corridors in 3D complex environments. IEEE Robotics and Automation Letters (RA-L), 2(3):1688–1695, 2017. doi: 10.1109/LRA.2017.2663526.
  28. Loc-nerf: Monte carlo localization using neural radiance fields. In IEEE International Conference on Robotics and Automation (ICRA), pages 4018–4025. IEEE, 2023.
  29. Gaussian splatting slam. arXiv preprint arXiv:2312.06741, 2023.
  30. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), 2020.
  31. Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, July 2022.
  32. Voxblox: Incremental 3D Euclidean signed distance fields for on-board MAV planning. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.
  33. Level Set Methods and Dynamic Implicit Surfaces. Springer, New York, 2003. ISBN 978-0-387-22746-7.
  34. Colored point cloud registration revisited. In Proceedings of the IEEE international conference on computer vision, pages 143–152, 2017.
  35. Chomp: Gradient optimization techniques for efficient motion planning. In 2009 IEEE international conference on robotics and automation, pages 489–494. IEEE, 2009.
  36. NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields. arXiv preprint arXiv:2210.13641, 2022.
  37. Orb: An efficient alternative to sift or surf. In 2011 International conference on computer vision, pages 2564–2571. Ieee, 2011.
  38. Fast point feature histograms (fpfh) for 3d registration. In 2009 IEEE international conference on robotics and automation, pages 3212–3217. IEEE, 2009.
  39. Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research, 33(9):1251–1270, 2014.
  40. The Open Motion Planning Library. IEEE Robotics & Automation Magazine, 19(4):72–82, December 2012. doi: 10.1109/MRA.2012.2205651. https://ompl.kavrakilab.org.
  41. iMAP: Implicit mapping and positioning in real-time. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
  42. Localising faster: Efficient and precise lidar-based robot localisation in large-scale environments. In 2020 IEEE international conference on robotics and automation (ICRA), pages 4386–4392. IEEE, 2020.
  43. Nerfstudio: A framework for neural radiance field development. In SIGGRAPH, 2023.
  44. A consistently fast and globally optimal solution to the perspective-n-point problem. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16, pages 478–494. Springer, 2020.
  45. Enforcing safety for vision-based controllers via control barrier functions and neural radiance fields. In IEEE International Conference on Robotics and Automation (ICRA), pages 10511–10517. IEEE, 2023.
  46. A localization based on unscented kalman filter and particle filter localization algorithms. IEEE Access, 8:2233–2246, 2019.
  47. Shinji Umeyama. Least-squares estimation of transformation parameters between two point patterns. IEEE Transactions on Pattern Analysis & Machine Intelligence, 13(04):376–380, 1991.
  48. Evolutionary artificial potential fields and their application in real time robot path planning. In Proceedings of the 2000 congress on evolutionary computation. CEC00 (Cat. No. 00TH8512), volume 1, pages 256–263. IEEE, 2000.
  49. Reciprocal velocity obstacles for real-time multi-agent navigation. In 2008 IEEE international conference on robotics and automation, pages 1928–1935. Ieee, 2008.
  50. Perception-aware receding horizon trajectory planning for multicopters with visual-inertial odometry. IEEE Access, 10:87911–87922, 2022.
  51. GS-SLAM: Dense visual slam with 3d gaussian splatting. arXiv preprint arXiv:2311.11700, 2023.
  52. Inerf: Inverting neural radiance fields for pose estimation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1323–1330. IEEE, 2021.
  53. Gaussian-SLAM: Photo-realistic dense slam with gaussian splatting. arXiv preprint arXiv:2312.10070, 2023.
  54. An improved particle filter for mobile robot localization based on particle swarm optimization. Expert Systems with Applications, 135:181–193, 2019.
  55. Nice-SLAM: Neural implicit scalable encoding for slam. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
Citations (20)
View on Semantic Scholar

Summary

  • The paper introduces Splat-Nav, a comprehensive pipeline fusing safe trajectory planning with robust pose estimation in GSplat environments.
  • It develops Splat-Plan, which utilizes ellipsoidal collision detection to generate dynamically feasible, smooth paths at 5 Hz in real time.
  • It also presents Splat-Loc, a robust pose estimation module that achieves 20 Hz tracking by aligning RGB images with GSplat point cloud data.

Splat-Nav: Integrating Safe Planning and Robust Pose Estimation in Gaussian Splatting Maps

Introduction

Recent advances in 3D environment representation have led to significant developments in robot navigation and localization techniques. Gaussian Splatting (GSplat) has emerged as a powerful 3D scene representation technique, offering benefits over traditional methods such as meshes, occupancy grids, and point clouds, particularly in terms of rendering quality and computational efficiency. This paper introduces Splat-Nav, a comprehensive navigation pipeline designed specifically for GSplat environments, encompassing a real-time planning module, Splat-Plan, and a robust state estimation module, Splat-Loc. Built on the core advantages of the GSplat representation, Splat-Nav illustrates an innovative approach to achieving safe, efficient, and real-time navigation.

Splat-Plan: Safe Planning in GSplat Environments

Splat-Plan leverages the ellipsoidal representation in GSplats to formulate an efficient collision detection algorithm, allowing for the quick generation of safe polytopic corridors within the environment. This enables the computation of smooth trajectories that are guaranteed to be safe, based on rigorously derived collision constraints. The planning module can support both a complete trajectory planning approach and a local (receding horizon) trajectory planning approach, facilitating real-time replanning capabilities.

Key Contributions

  • Efficient Collision Detection: Utilizing ellipsoid-to-ellipsoid collision tests, Splat-Plan rapidly generates safe corridors through the environment.
  • Dynamic Feasibility: The derived trajectories are dynamically feasible, smooth, and adhere to the robot's physical constraints.
  • Real-time Operation: Demonstrated re-planning at 5 Hz, Splat-Plan significantly outpaces competing methods in computation speed without sacrificing safety or feasibility.

Splat-Loc: Robust Pose Estimation

Splat-Loc interprets the GSplat map as a point cloud for pose estimation, applying well-established point cloud alignment techniques for both global pose initialization and continuous pose tracking. This approach, unique to GSplat representations, allows for high-frequency (20 Hz) pose estimation using only RGB images, a substantial improvement over existing methods in terms of speed and reliability.

Innovations

  • Global Localization: Utilizes point cloud registration techniques to enable global pose alignment without prior knowledge of the robot's pose.
  • Continuous Pose Tracking: Leverages feature-based image-to-point cloud matching for accurate and real-time pose estimation.
  • Efficiency and Accuracy: Achieves higher accuracy and faster operation compared to NeRF-based and traditional pose estimation methods.

Empirical Validation

The efficacy of Splat-Nav is demonstrated through extensive experiments in both simulated and real-world environments. Splat-Plan consistently generates safe, non-conservative paths, outperforming RRT* and NeRF-based planners in terms of safety, path quality, and computation time. Splat-Loc offers superior pose estimation accuracy and speed, establishing its robustness and utility for real-time applications.

Implications and Future Directions

Splat-Nav represents a significant step forward in leveraging GSplat for robotic navigation, offering a scalable, efficient solution to the challenges of planning and localization within complex 3D environments. The successful integration of the Splat-Plan and Splat-Loc modules within Splat-Nav points towards the potential for further advancements in autonomous navigation systems. Future work may explore the extension of these techniques to a broader range of robotic platforms and environment types, reinforcing the versatile applicability of GSplat in robotics and artificial intelligence domains.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown