Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dynamic Appearance Particle Neural Radiance Field (2310.07916v2)

Published 11 Oct 2023 in cs.CV

Abstract: Neural Radiance Fields (NeRFs) have shown great potential in modelling 3D scenes. Dynamic NeRFs extend this model by capturing time-varying elements, typically using deformation fields. The existing dynamic NeRFs employ a similar Eulerian representation for both light radiance and deformation fields. This leads to a close coupling of appearance and motion and lacks a physical interpretation. In this work, we propose Dynamic Appearance Particle Neural Radiance Field (DAP-NeRF), which introduces particle-based representation to model the motions of visual elements in a dynamic 3D scene. DAP-NeRF consists of superposition of a static field and a dynamic field. The dynamic field is quantised as a collection of {\em appearance particles}, which carries the visual information of a small dynamic element in the scene and is equipped with a motion model. All components, including the static field, the visual features and motion models of the particles, are learned from monocular videos without any prior geometric knowledge of the scene. We develop an efficient computational framework for the particle-based model. We also construct a new dataset to evaluate motion modelling. Experimental results show that DAP-NeRF is an effective technique to capture not only the appearance but also the physically meaningful motions in a 3D dynamic scene.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (36)
  1. ParticleNeRF: A Particle-Based Encoding for Online Neural Radiance Fields in Dynamic Scenes. CoRR, abs/2211.04041.
  2. Blender-Community. 2018. Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam.
  3. HexPlane: A Fast Representation for Dynamic Scenes. CVPR.
  4. Virtual Elastic Objects. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 15806–15816. IEEE.
  5. Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. In Jung, S. K.; Lee, J.; and Bargteil, A. W., eds., SIGGRAPH Asia 2022 Conference Papers, SA 2022, Daegu, Republic of Korea, December 6-9, 2022, 11:1–11:9. ACM.
  6. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. CoRR, abs/2301.10241.
  7. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. In CVPR.
  8. V4D: Voxel for 4D Novel View Synthesis. CoRR, abs/2205.14332.
  9. Dynamic View Synthesis from Dynamic Monocular Video. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, 5692–5701. IEEE.
  10. Smoothed particle hydrodynamics: theory and application to non-spherical stars. Monthly notices of the royal astronomical society, 181(3): 375–389.
  11. NeuroFluid: Fluid Dynamics Grounding with Particle-Driven Neural Radiance Fields. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvári, C.; Niu, G.; and Sabato, S., eds., International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, 7919–7929. PMLR.
  12. Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis. CoRR, abs/2206.07698.
  13. 3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 37(1): 1051–1059.
  14. Adam: A Method for Stochastic Optimization. In Bengio, Y.; and LeCun, Y., eds., 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  15. Dense RGB SLAM With Neural Implicit Maps. In Proceedings of the International Conference on Learning Representations.
  16. PAC-NeRF: Physics Augmented Continuum Neural Radiance Fields for Geometry-Agnostic System Identification. In ICLR. OpenReview.net.
  17. Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, 6498–6508. Computer Vision Foundation / IEEE.
  18. DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes. CoRR, abs/2205.15723.
  19. Neural volumes: learning dynamic renderable volumes from images. ACM Trans. Graph., 38(4): 65:1–65:14.
  20. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In Vedaldi, A.; Bischof, H.; Brox, T.; and Frahm, J., eds., Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part I, volume 12346 of Lecture Notes in Computer Science, 405–421. Springer.
  21. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph., 41(4): 102:1–102:15.
  22. Nerfies: Deformable Neural Radiance Fields. In ICCV, 5845–5854. IEEE.
  23. HyperNeRF: a higher-dimensional representation for topologically varying neural radiance fields. ACM Trans. Graph., 40(6): 238:1–238:12.
  24. Temporal Interpolation Is All You Need for Dynamic Neural Radiance Fields. CoRR, abs/2302.09311.
  25. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Wallach, H. M.; Larochelle, H.; Beygelzimer, A.; d’Alché-Buc, F.; Fox, E. B.; and Garnett, R., eds., Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, 8024–8035.
  26. D-NeRF: Neural Radiance Fields for Dynamic Scenes. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, 10318–10327. Computer Vision Foundation / IEEE.
  27. NeuPhysics: Editable Neural Geometry and Physics from Monocular Videos. In Conference on Neural Information Processing Systems (NeurIPS).
  28. NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields. arXiv preprint arXiv:2210.15947.
  29. A particle method for history-dependent materials. Computer Methods in Applied Mechanics and Engineering, 118: 179–196.
  30. Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 5449–5459. IEEE.
  31. Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, 12939–12950. IEEE.
  32. Template-free Articulated Neural Point Clouds for Reposable View Synthesis. CoRR, abs/2305.19065.
  33. Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-time. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 13514–13524. IEEE.
  34. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process., 13(4): 600–612.
  35. Space-Time Neural Irradiance Fields for Free-Viewpoint Video. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, 9421–9431. Computer Vision Foundation / IEEE.
  36. Point-NeRF: Point-based Neural Radiance Fields. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 5428–5438. IEEE.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now