3QFP: Efficient neural implicit surface reconstruction using Tri-Quadtrees and Fourier feature Positional encoding (2401.07164v2)
Abstract: Neural implicit surface representations are currently receiving a lot of interest as a means to achieve high-fidelity surface reconstruction at a low memory cost, compared to traditional explicit representations.However, state-of-the-art methods still struggle with excessive memory usage and non-smooth surfaces. This is particularly problematic in large-scale applications with sparse inputs, as is common in robotics use cases. To address these issues, we first introduce a sparse structure, \emph{tri-quadtrees}, which represents the environment using learnable features stored in three planar quadtree projections. Secondly, we concatenate the learnable features with a Fourier feature positional encoding. The combined features are then decoded into signed distance values through a small multi-layer perceptron. We demonstrate that this approach facilitates smoother reconstruction with a higher completion ratio with fewer holes. Compared to two recent baselines, one implicit and one explicit, our approach requires only 10\%--50\% as much memory, while achieving competitive quality.
- “CT-ICP: Real-time elastic LiDAR odometry with loop closure” In 2022 International Conference on Robotics and Automation (ICRA), 2022, pp. 5580–5586 IEEE
- “KISS-ICP: In defense of point-to-point ICP–simple, accurate, and robust registration if done the right way” In IEEE Robotics and Automation Letters 8.2 IEEE, 2023, pp. 1029–1036
- Martin Magnusson, Achim J. Lilienthal and Tom Duckett “Scan Registration for Autonomous Mining Vehicles Using 3D-NDT” Wiley Periodicals, Inc., 2007, pp. 803–827 DOI: 10.1002/rob.20204
- “Appearance-based loop detection from 3D laser data using the normal distributions transform” In 2009 IEEE International Conference on Robotics and Automation, 2009, pp. 23–28 IEEE
- “Path planning in 3D environments using the normal distributions transform” In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2010, pp. 3263–3268 IEEE
- “Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning” In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 1366–1373 IEEE
- “Voxfield: Non-projective signed distance fields for online planning and 3d reconstruction” In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022, pp. 5331–5338 IEEE
- Daniel Ricao Canelhas, Todor Stoyanov and Achim J. Lilienthal “SDF Tracker: A parallel algorithm for on-line pose estimation and scene reconstruction from depth images” In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, pp. 3671–3676 URL: https://api.semanticscholar.org/CorpusID:13787371
- Daniel R. Canelhas, Todor Stoyanov and Achim J. Lilienthal “From Feature Detection in Truncated Signed Distance Fields to Sparse Stable Scene Graphs” In IEEE Robotics and Automation Letters 1.2, 2016, pp. 1148–1155 DOI: 10.1109/LRA.2016.2523555
- “OctoMap: An efficient probabilistic 3D mapping framework based on octrees” In Autonomous robots 34 Springer, 2013, pp. 189–206
- “UFOMap: An efficient probabilistic 3D mapping framework that embraces the unknown” In IEEE Robotics and Automation Letters 5.4 IEEE, 2020, pp. 6411–6418
- “Efficient Surfel-Based SLAM using 3D Laser Range Data in Urban Environments.” In Robotics: Science and Systems 2018, 2018, pp. 59
- “Poisson surface reconstruction for LiDAR odometry and mapping” In 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 5624–5630 IEEE
- “Deepsdf: Learning continuous signed distance functions for shape representation” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 165–174
- “Occupancy networks: Learning 3d reconstruction in function space” In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4460–4470
- “Neural geometric level of detail: Real-time rendering with implicit 3d shapes” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 11358–11367
- “Convolutional occupancy networks” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16, 2020, pp. 523–540 Springer
- “H2-Mapping: Real-time Dense Mapping Using Hierarchical Hybrid Representation” In arXiv preprint arXiv:2306.03207, 2023
- “NF-Atlas: Multi-Volume Neural Feature Fields for Large Scale LiDAR Mapping” In arXiv preprint arXiv:2304.04624, 2023
- “Shine-mapping: Large-scale 3d mapping using sparse hierarchical implicit neural representations” In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 8371–8377 IEEE
- Mohammad Mahdi Johari, Camilla Carta and François Fleuret “Eslam: Efficient dense slam system based on hybrid representation of signed distance fields” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 17408–17419
- “VDBFusion: Flexible and Efficient TSDF Integration of Range Sensor Data” In Sensors 22.3, 2022 DOI: 10.3390/s22031296
- “Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation” In 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2022, pp. 499–507 DOI: 10.1109/ISMAR55827.2022.00066
- “Vox-Surf: Voxel-Based Implicit Surface Representation” In IEEE Transactions on Visualization and Computer Graphics, 2022, pp. 1–12 DOI: 10.1109/TVCG.2022.3225844
- Jiawei Yang, Marco Pavone and Yue Wang “FreeNeRF: Improving Few-Shot Neural Rendering with Free Frequency Regularization” In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 8254–8263 URL: https://api.semanticscholar.org/CorpusID:257505220
- Hengyi Wang, Jingwen Wang and Lourdes Agapito “Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 13293–13302
- “Fourier features let networks learn high frequency functions in low dimensional domains” In Advances in Neural Information Processing Systems 33, 2020, pp. 7537–7547
- “Incremental visual-inertial 3d mesh generation with structural regularities” In 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 8220–8226 IEEE
- “PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017
- “Nerf: Representing scenes as neural radiance fields for view synthesis” In Communications of the ACM 65.1 ACM New York, NY, USA, 2021, pp. 99–106
- “iMAP: Implicit mapping and positioning in real-time” In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 6229–6238
- “isdf: Real-time neural signed distance fields for robot perception” In arXiv preprint arXiv:2204.02296, 2022
- “MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction” In Advances in Neural Information Processing Systems (NeurIPS), 2022
- Jingwen Wang, Tymoteusz Bleja and Lourdes Agapito “GO-Surf: Neural Feature Grid Optimization for Fast, High-Fidelity RGB-D Surface Reconstruction” In 2022 International Conference on 3D Vision (3DV), 2022, pp. 433–442 DOI: 10.1109/3DV57658.2022.00055
- “Instant neural graphics primitives with a multiresolution hash encoding” In ACM Transactions on Graphics (ToG) 41.4 ACM New York, NY, USA, 2022, pp. 1–15
- “Efficient Implicit Neural Reconstruction Using LiDAR” In arXiv preprint arXiv:2302.14363, 2023
- “Efficient geometry-aware 3D generative adversarial networks” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16123–16133
- “Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields” In arXiv preprint arXiv:2307.11335, 2023
- “Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes” In ACM Transactions on Graphics (TOG) 42.4 ACM New York, NY, USA, 2023, pp. 1–12
- Yiqun Wang, Ivan Skorokhodov and Peter Wonka “PET-NeuS: Positional Encoding Tri-Planes for Neural Surfaces” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 12598–12607
- “K-planes: Explicit radiance fields in space, time, and appearance” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 12479–12488
- William E. Lorensen and Harvey E. Cline “Marching Cubes: A High Resolution 3D Surface Construction Algorithm” In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’87 New York, NY, USA: Association for Computing Machinery, 1987, pp. 163–169 DOI: 10.1145/37401.37422
- “LocNDF: Neural Distance Field Mapping for Robot Localization”, 2023, pp. 4999–5006 DOI: 10.1109/LRA.2023.3291274
- “The newer college dataset: Handheld lidar, inertial and vision with ground truth” In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2020, pp. 4353–4360 IEEE
- “A Submap per Perspective - Selecting Subsets for SuPer Mapping that Afford Superior Localization Quality” In European Conference on Mobile Robots, 2019