Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
157 tokens/sec
GPT-4o
43 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SPC-NeRF: Spatial Predictive Compression for Voxel Based Radiance Field (2402.16366v1)

Published 26 Feb 2024 in cs.CV and cs.MM

Abstract: Representing the Neural Radiance Field (NeRF) with the explicit voxel grid (EVG) is a promising direction for improving NeRFs. However, the EVG representation is not efficient for storage and transmission because of the terrific memory cost. Current methods for compressing EVG mainly inherit the methods designed for neural network compression, such as pruning and quantization, which do not take full advantage of the spatial correlation of voxels. Inspired by prosperous digital image compression techniques, this paper proposes SPC-NeRF, a novel framework applying spatial predictive coding in EVG compression. The proposed framework can remove spatial redundancy efficiently for better compression performance.Moreover, we model the bitrate and design a novel form of the loss function, where we can jointly optimize compression ratio and distortion to achieve higher coding efficiency. Extensive experiments demonstrate that our method can achieve 32% bit saving compared to the state-of-the-art method VQRF on multiple representative test datasets, with comparable training time.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Variational image compression with a scale hyperprior. arXiv preprint arXiv:1802.01436, 2018.
  2. Sine: Semantic-driven image-based nerf editing with prior-guided editing field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20919–20929, 2023.
  3. 3d scene compression through entropy penalized neural representation functions. In 2021 Picture Coding Symposium (PCS), pages 1–5. IEEE, 2021.
  4. Gisle Bjontegaard. Calculation of average psnr differences between rd-curves. ITU SG16 Doc. VCEG-M33, 2001.
  5. Inter-picture prediction in hevc. High Efficiency Video Coding (HEVC) Algorithms and Architectures, pages 113–140, 2014.
  6. Overview of the versatile video coding (vvc) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):3736–3764, 2021.
  7. Hexplane: A fast representation for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 130–141, 2023.
  8. Tensorf: Tensorial radiance fields. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXXII, pages 333–350. Springer, 2022.
  9. Pact: Parameterized clipping activation for quantized neural networks. arXiv preprint arXiv:1805.06085, 2018.
  10. Compressing explicit voxel grid representations: fast nerfs become also small. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1236–1245, 2023.
  11. Differential weight quantization for multi-model compression. IEEE Transactions on Multimedia, 2022.
  12. Fast dynamic radiance fields with time-aware neural voxels. In SIGGRAPH Asia 2022 Conference Papers, pages 1–9, 2022.
  13. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501–5510, 2022.
  14. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12479–12488, 2023.
  15. On quantizing implicit neural representations. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 341–350, 2023.
  16. Knowledge distillation: A survey. International Journal of Computer Vision, 129:1789–1819, 2021.
  17. A technical overview of av1. Proceedings of the IEEE, 109(9):1435–1462, 2021.
  18. Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision, pages 1389–1397, 2017.
  19. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1–13, 2017.
  20. Two-step progressive intra prediction for versatile video coding. In 2020 IEEE International Conference on Image Processing (ICIP), pages 1137–1141, 2020.
  21. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710, 2016.
  22. Compressing volumetric radiance fields to 1 mb. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4222–4231, 2023.
  23. Neural 3d video synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5521–5531, 2022.
  24. Holistic cnn compression via low-rank decomposition with knowledge transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(12):2889–2905, 2019.
  25. Neural sparse voxel fields. Advances in Neural Information Processing Systems, 33:15651–15663, 2020.
  26. Non-structured dnn weight pruning—is it beneficial in any platform? IEEE transactions on neural networks and learning systems, 33(9):4930–4944, 2021.
  27. Nelson Max. Optical models for direct volume rendering. IEEE Transactions on Visualization and Computer Graphics, 1(2):99–108, 1995.
  28. Nerf: Representing scenes as neural radiance fields for view synthesis. In Computer Vision – ECCV 2020, pages 405–421, Cham, 2020. Springer International Publishing.
  29. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4):1–15, 2022.
  30. Intra prediction and mode coding in vvc. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):3834–3847, 2021.
  31. D-nerf: Neural radiance fields for dynamic scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10318–10327, 2021.
  32. Masked wavelet representation for compact neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20680–20690, 2023.
  33. Light field compression via compact neural scene representation. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023.
  34. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5459–5469, 2022.
  35. Sparse low rank factorization for deep neural network compression. Neurocomputing, 398:185–196, 2020.
  36. Compressible-composable nerf via rank-residual decomposition. Advances in Neural Information Processing Systems, 35:14798–14809, 2022.
  37. Neural residual radiance fields for streamably free-viewpoint videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 76–87, 2023.
  38. The loco-i lossless image compression algorithm: principles and standardization into jpeg-ls. IEEE Transactions on Image Processing, 9(8):1309–1324, 2000.
  39. Source coding: Part i of fundamentals of source and video coding. Foundations and Trends® in Signal Processing, 4(1–2):151–179, 2011.
  40. Arithmetic coding for data compression. Communications of the ACM, 30(6):520–540, 1987.
  41. Hollownerf: Pruning hashgrid-based nerfs with trainable collision mitigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3480–3490, 2023.
  42. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1790–1799, 2020.
  43. Nerf-editing: geometry editing of neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18353–18364, 2022.
  44. Recent development of avs video coding standard: Avs3. In 2019 picture coding symposium (PCS), pages 1–5. IEEE, 2019.
  45. Tinynerf: Towards 100 x compression of voxel radiance fields. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3588–3596, 2023.
  46. A universal algorithm for sequential data compression. IEEE Transactions on information theory, 23(3):337–343, 1977.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now