BID-NeRF: RGB-D image pose estimation with inverted Neural Radiance Fields (2310.03563v1)
Abstract: We aim to improve the Inverted Neural Radiance Fields (iNeRF) algorithm which defines the image pose estimation problem as a NeRF based iterative linear optimization. NeRFs are novel neural space representation models that can synthesize photorealistic novel views of real-world scenes or objects. Our contributions are as follows: we extend the localization optimization objective with a depth-based loss function, we introduce a multi-image based loss function where a sequence of images with known relative poses are used without increasing the computational complexity, we omit hierarchical sampling during volumetric rendering, meaning only the coarse model is used for pose estimation, and we how that by extending the sampling interval convergence can be achieved even or higher initial pose estimate errors. With the proposed modifications the convergence speed is significantly improved, and the basin of convergence is substantially extended.
- Vision-only robot navigation in a neural radiance world. IEEE Robotics and Automation Letters, 7(2):4606–4613, 2022.
- Zip-nerf: Anti-aliased grid-based neural radiance fields. arXiv, 2023.
- ORB-SLAM3: An accurate open-source library for visual, visual–inertial, and multimap SLAM. IEEE Transactions on Robotics, 37(6):1874–1890, dec 2021.
- Depth-supervised nerf: Fewer views and faster training for free. CoRR, abs/2107.02791, 2021.
- Superpoint: Self-supervised interest point detection and description, 2018.
- Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence, mar 2018.
- Efficientnerf: Efficient neural radiance fields, 2022.
- Peter J. Huber. Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35(1):73 – 101, 1964.
- 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
- Adam: A method for stochastic optimization, 2017.
- Adanerf: Adaptive sampling for real-time rendering of neural radiance fields, 2022.
- Nerf-pose: A first-reconstruct-then-regress approach for weakly-supervised 6d object pose estimation. arXiv preprint arXiv:2203.04802, 2022.
- Parallel inversion of neural radiance fields for robust pose estimation, 2023.
- Loc-nerf: Monte carlo localization using neural radiance fields, 2022.
- On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics, 18(1):50 – 60, 1947.
- Progressively optimized local radiance fields for robust view synthesis. In CVPR, 2023.
- Nerf: Representing scenes as neural radiance fields for view synthesis. CoRR, abs/2003.08934, 2020.
- Instant neural graphics primitives with a multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1–102:15, July 2022.
- Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision, 2020.
- VINS-mono: A robust and versatile monocular visual-inertial state estimator. IEEE Transactions on Robotics, 34(4):1004–1020, aug 2018.
- Superglue: Learning feature matching with graph neural networks, 2020.
- Structure-from-motion revisited. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016.
- A micro lie theory for state estimation in robotics, 2018.
- Deep object pose estimation for semantic robotic grasping of household objects, 2018.
- Navinerf: Nerf-based 3d representation disentanglement by latent semantic navigation. arXiv preprint arXiv:2304.11342, 2023.
- Inerf: Inverting neural radiance fields for pose estimation, 2020.
- Plenoxels: Radiance fields without neural networks, 2021.
- Nerf++: Analyzing and improving neural radiance fields, 2020.