Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fast High Dynamic Range Radiance Fields for Dynamic Scenes (2401.06052v1)

Published 11 Jan 2024 in cs.CV and cs.GR

Abstract: Neural Radiances Fields (NeRF) and their extensions have shown great success in representing 3D scenes and synthesizing novel-view images. However, most NeRF methods take in low-dynamic-range (LDR) images, which may lose details, especially with nonuniform illumination. Some previous NeRF methods attempt to introduce high-dynamic-range (HDR) techniques but mainly target static scenes. To extend HDR NeRF methods to wider applications, we propose a dynamic HDR NeRF framework, named HDR-HexPlane, which can learn 3D scenes from dynamic 2D images captured with various exposures. A learnable exposure mapping function is constructed to obtain adaptive exposure values for each image. Based on the monotonically increasing prior, a camera response function is designed for stable learning. With the proposed model, high-quality novel-view images at any time point can be rendered with any desired exposure. We further construct a dataset containing multiple dynamic scenes captured with diverse exposures for evaluation. All the datasets and code are available at \url{https://guanjunwu.github.io/HDR-HexPlane/}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. blender. https://www.blender.org/.
  2. Mixamo. https://www.mixamo.com/.
  3. Nope-nerf: Optimising neural radiance field with no pose prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  4. Hexplane: A fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  5. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, 2022.
  6. New stereo high dynamic range imaging method using generative adversarial networks. In 2019 IEEE International Conference on Image Processing (ICIP), 2019.
  7. Local-to-global registration for bundle-adjusting neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  8. Gaussian activated neural radiance fields for high fidelity reconstruction and pose estimation. In European Conference on Computer Vision, 2022.
  9. Recovering high dynamic range radiance maps from photographs. In ACM SIGGRAPH 2008 classes. 2008.
  10. Volume rendering. ACM Siggraph Computer Graphics, 1988.
  11. Neusample: Neural sample field for efficient view synthesis. arXiv preprint arXiv:2111.15552, 2021.
  12. Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia 2022 Conference Papers, 2022.
  13. Flownet: Learning optical flow with convolutional networks. arXiv preprint arXiv:1504.06852, 2015.
  14. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  15. You do not need additional priors or regularizers in retinex-based low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023a.
  16. Learning a simple low-light image enhancer from paired low-light instances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023b.
  17. Dynamic view synthesis from dynamic monocular video. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
  18. Artificial neural networks (the multilayer perceptron)—a review of applications in the atmospheric sciences. Atmospheric environment, 1998.
  19. Casual indoor hdr radiance capture from omnidirectional images. arXiv preprint arXiv:2208.07903, 2022.
  20. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020.
  21. Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
  22. Hdr-nerf: High dynamic range neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  23. Flownet 2.0: Evolution of optical flow estimation with deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
  24. D-tensorf: Tensorial radiance fields for dynamic scenes. arXiv preprint arXiv:2212.02375, 2022.
  25. Hdr-plenoxels: Self-calibrating high dynamic range radiance fields. In European Conference on Computer Vision, 2022.
  26. Deep high dynamic range imaging of dynamic scenes. ACM Trans. Graph., 2017.
  27. Adam: A method for stochastic optimization. Computer Science, 2014.
  28. Barf: Bundle-adjusting neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
  29. High-fidelity and real-time novel view synthesis for dynamic scenes. In SIGGRAPH Asia Conference Proceedings, 2023.
  30. Robust dynamic radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  31. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
  32. Matthieu Dupont. blendswap. www.blendswap.com, 2009.
  33. Progressively optimized local radiance fields for robust view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  34. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 2021.
  35. Nerf in the dark: High dynamic range view synthesis from noisy raw images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  36. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 2022.
  37. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 2019.
  38. Deep hdr reconstruction of dynamic scenes. In 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). IEEE, 2018.
  39. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
  40. Chen Quei-An. Nerf_pl: a pytorch-lightning implementation of nerf, 2020.
  41. Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. In Proceedings of the IEEE international conference on computer vision, 2017.
  42. Adop: Approximate differentiable one-pixel point rendering. ACM Transactions on Graphics (ToG), 2022.
  43. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.
  44. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphics, 2023.
  45. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  46. Block-nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  47. Mononerf: Learning a generalizable dynamic radiance field from monocular videos. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
  48. Mega-nerf: Scalable construction of large-scale nerfs for virtual fly-throughs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
  49. Glowgan: Unsupervised learning of hdr images from ldr images in the wild. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023a.
  50. Ultra-high-definition low-light image enhancement: A benchmark and transformer-based method. In Proceedings of the AAAI Conference on Artificial Intelligence, 2023b.
  51. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003. Ieee, 2003.
  52. Nerf–: Neural radiance fields without known camera parameters. arXiv preprint arXiv:2102.07064, 2021.
  53. Learning semantic-aware knowledge guidance for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  54. Low-light image enhancement via structure modeling and guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023a.
  55. 4k4d: Real-time 4d view synthesis at 4k resolution. arXiv preprint arXiv:2310.11448, 2023b.
  56. inerf: Inverting neural radiance fields for pose estimation. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.
  57. Generalizable neural voxels for fast human radiance fields. arXiv preprint arXiv:2303.15387, 2023.
  58. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. Fast High Dynamic Range Radiance Fields for Dynamic Scenes
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets