2000 character limit reached
3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surface
Published 13 Mar 2024 in cs.CV | (2403.08768v1)
Abstract: This paper introduces 3DFIRES, a novel system for scene-level 3D reconstruction from posed images. Designed to work with as few as one view, 3DFIRES reconstructs the complete geometry of unseen scenes, including hidden surfaces. With multiple view inputs, our method produces full reconstruction within all camera frustums. A key feature of our approach is the fusion of multi-view information at the feature level, enabling the production of coherent and comprehensive 3D reconstruction. We train our system on non-watertight scans from large-scale real scene dataset. We show it matches the efficacy of single-view reconstruction methods with only one input and surpasses existing techniques in both quantitative and qualitative measures for sparse-view 3D reconstruction.
- Planeformers: From sparse view planes to 3d reconstruction. In ECCV, 2022.
- Neural unsigned distance fields for implicit function learning. In NeurIPS, 2020.
- 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In ECCV, 2016.
- Panoptic 3d scene reconstruction from a single rgb image. NeurIPS, 2021.
- Sg-nn: Sparse generative neural networks for self-supervised scene completion of rgb-d scans. In CVPR, 2020.
- Monoslam: Real-time single camera slam. TPAMI, 2007.
- Depth-supervised nerf: Fewer views and faster training for free. In CVPR, 2022.
- Omnidata: A scalable pipeline for making multi-task mid-level vision datasets from 3d scans. In ICCV, 2021.
- A point set generation network for 3d object reconstruction from a single image. In CVPR, 2017.
- Justin Johnson Georgia Gkioxari, Nikhila Ravi. Learning 3d object shape and layout without 3d supervision. CVPR, 2022.
- Learning a predictable and generative vector representation for objects. In ECCV, 2016.
- Mesh r-cnn. In ICCV, 2019.
- Deep residual learning for image recognition. In CVPR, 2016.
- Efficient non-line-of-sight imaging from transient sinograms. In ECCV, 2020.
- Im2cad. In CVPR, 2017.
- Peek-a-boo: Occlusion reasoning in indoor scenes with plane representations. In CVPR, 2020.
- Planar surface reconstruction from sparse views. In ICCV, 2021.
- Learning a multi-view stereo machine. NeurIPS, 2017.
- 3d-relnet: Joint object and relational network for 3d prediction. In ICCV, 2019.
- Directed ray distance functions for 3d scene reconstruction. In ECCV, 2022.
- Learning to predict scene-level implicit 3d from posed rgbd data. In CVPR, 2023.
- Barf: Bundle-adjusting neural radiance fields. In ICCV, 2021.
- Occupancy networks: Learning 3d reconstruction in function space. In CVPR, 2019.
- Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
- Atlas: End-to-end 3d scene reconstruction from posed images. In ECCV, 2020.
- Total3dunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. In CVPR, 2020.
- Deepsdf: Learning continuous signed distance functions for shape representation. In CVPR, 2019.
- Wide baseline stereo matching. In ICCV, 1998.
- Associative3d: Volumetric reconstruction from sparse views. In ECCV, 2020.
- Vision transformers for dense prediction. ICCV, 2021.
- Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. TPAMI, 2022.
- Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In CVPR, 2020.
- Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representations. CVPR, 2022.
- A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. IJCV, 2002.
- Structure-from-motion revisited. In CVPR, 2016.
- Layered depth images. In Siggraph, 1998.
- Implicit neural representations with periodic activation functions. NeurIPS, 2020.
- LoFTR: Detector-free local feature matching with transformers. CVPR, 2021a.
- Neuralrecon: Real-time coherent 3d reconstruction from monocular video. In CVPR, 2021b.
- Nope-sac: Neural one-plane ransac for sparse-view planar 3d reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
- Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS, 2020.
- Factoring shape, pose, and layout from the 2d image of a 3d scene. In CVPR, 2018.
- Ibrnet: Learning multi-view image-based rendering. In CVPR, 2021.
- Multiview compressive coding for 3D reconstruction. CVPR, 2023.
- Gibson env: Real-world perception for embodied agents. In CVPR, 2018.
- Planarrecon: Real-time 3d plane detection and reconstruction from posed monocular videos. In CVPR, 2022.
- pixelNeRF: Neural radiance fields from one or few images. In CVPR, 2021.
- Taskonomy: Disentangling task transfer learning. In CVPR, 2018.
- Sparsefusion: Distilling view-conditioned diffusion for 3d reconstruction. In CVPR, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper.