Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Efficient Neural Representation of Volumetric Data using Coordinate-Based Networks (2401.08840v1)

Published 16 Jan 2024 in cs.CV and cs.GR

Abstract: In this paper, we propose an efficient approach for the compression and representation of volumetric data utilizing coordinate-based networks and multi-resolution hash encoding. Efficient compression of volumetric data is crucial for various applications, such as medical imaging and scientific simulations. Our approach enables effective compression by learning a mapping between spatial coordinates and intensity values. We compare different encoding schemes and demonstrate the superiority of multi-resolution hash encoding in terms of compression quality and training efficiency. Furthermore, we leverage optimization-based meta-learning, specifically using the Reptile algorithm, to learn weight initialization for neural representations tailored to volumetric data, enabling faster convergence during optimization. Additionally, we compare our approach with state-of-the-art methods to showcase improved image quality and compression ratios. These findings highlight the potential of coordinate-based networks and multi-resolution hash encoding for an efficient and accurate representation of volumetric data, paving the way for advancements in large-scale data visualization and other applications.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Bartz D.: Vcm, university of tübingen, germany. URL: http://www.volvis.org.
  2. A generative model for volume rendering. IEEE Transactions on Visualization and Computer Graphics 25, 4 (2019), 1636–1650.
  3. Tthresh: Tensor compression for multidimensional visual data. IEEE Transactions on Visualization and Computer Graphics 26, 9 (2020), 2891–2903. doi:10.1109/TVCG.2019.2904063.
  4. Ballester-Ripoll R., Pajarola R.: Lossy volume compression using tucker truncation and thresholding. The Visual Computer 32 (05 2015). doi:10.1007/s00371-015-1130-y.
  5. Center for simulation of accidental fires and explosions. URL: http://uintah.utah.edu/.
  6. Visualization and volumetric compression. In Medical Imaging (1991).
  7. Devkota S., Pattanaik S.: Deep learning based super-resolution for medical volume visualization with direct volume rendering. In Advances in Visual Computing (Cham, 2022), Bebis G., Li B., Yao A., Liu Y., Duan Y., Lau M., Khadka R., Crisan A., Chang R., (Eds.), Springer International Publishing, pp. 103–114.
  8. Model-agnostic meta-learning for fast adaptation of deep networks, 2017. arXiv:1703.03400.
  9. Fout N., Ma K.-L.: Transform coding for hardware-accelerated volume rendering. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1600–1607. doi:10.1109/TVCG.2007.70516.
  10. General electric. URL: http://www.volvis.org.
  11. Guthe S., Goesele M.: Variable length coding for gpu-based direct volume rendering. In Vision, Modeling and Visualization (2016), Hullin M., Stamminger M., Weinkauf T., (Eds.), The Eurographics Association. doi:10.2312/vmv.20161345.
  12. Formation of hard power laws in the energetic particle spectra resulting from relativistic magnetic reconnection. Phys. Rev. Lett. 113 (oct 2014), 155005. doi:10.1103/PhysRevLett.113.155005.
  13. Gürvit Ö.: Institute for neuroradiology, frankfurt. URL: http://www.volvis.org.
  14. Dnn-volvis: Interactive volume visualization supported by deep neural network. In 2019 IEEE Pacific Visualization Symposium (PacificVis) (2019), pp. 282–291. doi:10.1109/PacificVis.2019.00041.
  15. Han J., Wang C.: Tsr-tvd: Temporal super-resolution for time-varying data analysis and visualization. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 205–215. doi:10.1109/TVCG.2019.2934255.
  16. Han J., Wang C.: Coordnet: Data generation and visualization generation for time-varying volumes via a coordinate-based neural network. IEEE Transactions on Visualization and Computer Graphics (2022), 1–12. doi:10.1109/TVCG.2022.3197203.
  17. Han J., Wang C.: Ssr-tvd: Spatial super-resolution for time-varying data analysis and visualization. IEEE Transactions on Visualization and Computer Graphics 28, 6 (2022), 2445–2456. doi:10.1109/TVCG.2020.3032123.
  18. Insitunet: Deep image synthesis for parameter space exploration of ensemble simulations. IEEE Transactions on Visualization and Computer Graphics 26, 01 (jan 2020), 23–33. doi:10.1109/TVCG.2019.2934312.
  19. Ihm I., Park S.: Wavelet-based 3d compression scheme for interactive visualization of very large volume data. Computer Graphics Forum 18, 1 (1999), 3–15. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-8659.00298, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/1467-8659.00298, doi:https://doi.org/10.1111/1467-8659.00298.
  20. Local implicit grid representations for 3d scenes. CoRR abs/2003.08981 (2020). URL: https://arxiv.org/abs/2003.08981, arXiv:2003.08981.
  21. Neuralvdb: High-resolution sparse volume representation using hierarchical neural networks, 2022. arXiv:2208.04448.
  22. Kreeger K.: Viatronix inc. URL: http://www.volvis.org.
  23. Levoy M.: Volume rendering using the fourier projection-slice theorem. In Proceedings of the Conference on Graphics Interface ’92 (San Francisco, CA, USA, 1992), Morgan Kaufmann Publishers Inc., p. 61–69.
  24. Neural sparse voxel fields. CoRR abs/2007.11571 (2020). URL: https://arxiv.org/abs/2007.11571, arXiv:2007.11571.
  25. Levoy M., Hanrahan P.: Light field rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1996), SIGGRAPH ’96, Association for Computing Machinery, p. 31–42. URL: https://doi.org/10.1145/237170.237199, doi:10.1145/237170.237199.
  26. Lindstrom P.: Fixed-rate compressed floating-point arrays. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2674–2683. doi:10.1109/TVCG.2014.2346458.
  27. Compressive neural representations of volumetric scalar fields. Computer Graphics Forum 40, 3 (2021), 135–146. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.14295, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14295, doi:https://doi.org/10.1111/cgf.14295.
  28. Malzbender T.: Fourier volume rendering. ACM Trans. Graph. 12, 3 (jul 1993), 233–250. URL: https://doi.org/10.1145/169711.169705, doi:10.1145/169711.169705.
  29. Instant neural graphics primitives with a multiresolution hash encoding. CoRR abs/2201.05989 (2022). URL: https://arxiv.org/abs/2201.05989, arXiv:2201.05989.
  30. ACORN: adaptive coordinate networks for neural scene representation. CoRR abs/2105.02788 (2021). URL: https://arxiv.org/abs/2105.02788, arXiv:2105.02788.
  31. Mixed precision training, 2018. arXiv:1710.03740.
  32. Occupancy networks: Learning 3d reconstruction in function space, 2019. arXiv:1812.03828.
  33. Implicit surface representations as layers in neural networks. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), pp. 4742–4751. doi:10.1109/ICCV.2019.00484.
  34. Real-time neural radiance caching for path tracing. ACM Trans. Graph. 40, 4 (jul 2021). URL: https://doi.org/10.1145/3450626.3459812, doi:10.1145/3450626.3459812.
  35. Nerf: Representing scenes as neural radiance fields for view synthesis. CoRR abs/2003.08934 (2020). URL: https://arxiv.org/abs/2003.08934, arXiv:2003.08934.
  36. Muraki S.: Volume data and wavelet transforms. IEEE Computer Graphics and Applications 13, 4 (1993), 50–56. doi:10.1109/38.219451.
  37. On first-order meta-learning algorithms. CoRR abs/1803.02999 (2018). URL: http://arxiv.org/abs/1803.02999, arXiv:1803.02999.
  38. Ning P., Hesselink L.: Vector quantization for volume rendering. In Proceedings of the 1992 Workshop on Volume Visualization (New York, NY, USA, 1992), VVS ’92, Association for Computing Machinery, p. 69–74. URL: https://doi.org/10.1145/147130.147152, doi:10.1145/147130.147152.
  39. Open scivis dataset. URL: https://klacansky.com/open-scivis-datasets/.
  40. OSIRIX: Dicom image library. URL: https://www.osirix-viewer.com/resources/dicomimage-library/.
  41. Deepsdf: Learning continuous signed distance functions for shape representation. CoRR abs/1901.05103 (2019). URL: http://arxiv.org/abs/1901.05103, arXiv:1901.05103.
  42. Philips research. URL: https://www.philips.com/a-w/research/locations/hamburg.cs.
  43. Convolutional occupancy networks. In Computer Vision – ECCV 2020 (Cham, 2020), Vedaldi A., Bischof H., Brox T., Frahm J.-M., (Eds.), Springer International Publishing, pp. 523–540.
  44. Metasdf: Meta-learning signed distance functions. CoRR abs/2006.09662 (2020). URL: https://arxiv.org/abs/2006.09662, arXiv:2006.09662.
  45. Siemens medical solutions. URL: http://www.volvis.org.
  46. Tamresh – tensor approximation multiresolution hierarchy for interactive volume visualization. Computer Graphics Forum 32, 3pt2 (2013), 151–160. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.12102, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.12102, doi:https://doi.org/10.1111/cgf.12102.
  47. Implicit neural representations for image compression, 2022. arXiv:2112.04267.
  48. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction, 2022. arXiv:2111.11215.
  49. Schneider J., Westermann R.: Compression domain volume rendering. In IEEE Visualization, 2003. VIS 2003. (2003), pp. 293–300. doi:10.1109/VISUAL.2003.1250385.
  50. Real time, accurate, multi-featured rendering of bump mapped surfaces. Computer Graphics Forum 19, 3 (2000), 119–130. URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-8659.00404, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/1467-8659.00404, doi:https://doi.org/10.1111/1467-8659.00404.
  51. URL: http://www.volvis.org.
  52. Variable bitrate neural fields, 2022. arXiv:2206.07707.
  53. Optimized spatial hashing for collision detection of deformable objects. VMV’03: Proceedings of the Vision, Modeling, Visualization 3 (12 2003).
  54. Totsuka T., Levoy M.: Frequency domain volume rendering. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1993), SIGGRAPH ’93, Association for Computing Machinery, p. 271–278. URL: https://doi.org/10.1145/166117.166152, doi:10.1145/166117.166152.
  55. Neural geometric level of detail: Real-time rendering with implicit 3d shapes, 2021. arXiv:2101.10994.
  56. Learned initializations for optimizing coordinate-based neural representations. CoRR abs/2012.02189 (2020). URL: https://arxiv.org/abs/2012.02189, arXiv:2012.02189.
  57. Fourier features let networks learn high frequency functions in low dimensional domains. CoRR abs/2006.10739 (2020). URL: https://arxiv.org/abs/2006.10739, arXiv:2006.10739.
  58. Van Gelder A., Kim K.: Direct volume rendering with shading via three-dimensional textures. In Proceedings of 1996 Symposium on Volume Visualization (1996), pp. 23–30. doi:10.1109/SVV.1996.558039.
  59. Volume rendering. URL: https://www.cs.purdue.edu/homes/cs530/projects/project3.html.
  60. Volumetric isosurface rendering with deep learning-based super-resolution. IEEE Transactions on Visualization and Computer Graphics 27, 6 (2021), 3064–3078. doi:10.1109/TVCG.2019.2956697.
  61. Fast neural representations for direct volume rendering. CoRR abs/2112.01579 (2021). URL: https://arxiv.org/abs/2112.01579, arXiv:2112.01579.
  62. Wiki S.: Sampledata — slicer wiki,. [Online; accessed 5-June-2023]. URL: https://www.slicer.org/w/index.php?title=SampleData&oldid=62556.
  63. Plenoxels: Radiance fields without neural networks, 2021. arXiv:2112.05131.
  64. Yeo B.-L., Liu B.: Volume rendering of dct-based compressed 3d scalar data. IEEE Transactions on Visualization and Computer Graphics 1, 1 (1995), 29–43. doi:10.1109/2945.468390.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Sudarshan Devkota (3 papers)
  2. Sumanta Pattanaik (5 papers)
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets