Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Prediction-Traversal Approach for Compressing Scientific Data on Unstructured Meshes with Bounded Error (2312.06080v2)

Published 11 Dec 2023 in cs.GR

Abstract: We explore an error-bounded lossy compression approach for reducing scientific data associated with 2D/3D unstructured meshes. While existing lossy compressors offer a high compression ratio with bounded error for regular grid data, methodologies tailored for unstructured mesh data are lacking; for example, one can compress nodal data as 1D arrays, neglecting the spatial coherency of the mesh nodes. Inspired by the SZ compressor, which predicts and quantizes values in a multidimensional array, we dynamically reorganize nodal data into sequences. Each sequence starts with a seed cell; based on a predefined traversal order, the next cell is added to the sequence if the current cell can predict and quantize the nodal data in the next cell with the given error bound. As a result, one can efficiently compress the quantized nodal data in each sequence until all mesh nodes are traversed. This paper also introduces a suite of novel error metrics, namely continuous mean squared error (CMSE) and continuous peak signal-to-noise ratio (CPSNR), to assess compression results for unstructured mesh data. The continuous error metrics are defined by integrating the error function on all cells, providing objective statistics across nonuniformly distributed nodes/cells in the mesh. We evaluate our methods with several scientific simulations ranging from ocean-climate models and computational fluid dynamics simulations with both traditional and continuous error metrics. We demonstrated superior compression ratios and quality than existing lossy compressors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. Multilevel techniques for compression and reduction of scientific data—the unstructured case. SIAM Journal on Scientific Computing 42, 2 (2020), A1402–A1427.
  2. TTHRESH: Tensor compression for multidimensional visual data. IEEE Transactions on Visualization and Computer Graphics 26, 9 (2019), 2891–2903.
  3. Bachthaler S., Weiskopf D.: Continuous scatterplots. IEEE Transactions on Visualization and Computer Graphics 14, 6 (2008), 1428–1435.
  4. Simplification of tetrahedral meshes with accurate error evaluation. In Proceedings IEEE Visualization (2000), pp. 85–92.
  5. Chow M. M.: Optimized geometry compression for real-time rendering. 1997.
  6. Chiang Y.-J., Lu X.: Progressive simplification of tetrahedral meshes preserving all isosurface topologies. Computer Graphics Forum 22, 3 (2003), 493–504.
  7. A comparison of mesh simplification algorithms. Computers & Graphics 22, 1 (1998), 37–54.
  8. Integrating isosurface statistics and histograms. IEEE Transactions on Visualization and Computer Graphics 19, 2 (2012), 263–277.
  9. Triangulations: Structures for Algorithms and Applications. Springer, 2010.
  10. Optimal decomposition of polygonal models into triangle strips. In Proceedings of the 18th Annual Symposium on Computational Geometry (2002), pp. 254–263.
  11. Efficient triangular surface approximations using wavelets and quadtree data structures. IEEE Transactions on Visualization and Computer Graphics 2, 2 (1996), 130–143.
  12. Huffman D. A.: A method for the construction of minimum-redundancy codes. Proceedings of the IRE 40, 9 (1952), 1098–1101.
  13. STNet: An end-to-end generative framework for synthesizing spatiotemporal super-resolution volumes. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2022), 270–280.
  14. Fast and effective lossy compression algorithms for scientific datasets. In Euro-Par 2012 Parallel Processing (Berlin, Heidelberg, 2012), Kaklamanis C., Papatheodorou T., Spirakis P. G., (Eds.), Springer Berlin Heidelberg, pp. 843–856.
  15. Out-of-core compression and decompression of large n-dimensional scalar fields. Computer Graphics Forum 22(3) (2003), 343–348.
  16. Kamath C.: Compressing unstructured mesh data from simulations using machine learning. International Journal of Data Science and Analytics 9, 1 (2020), 113–130.
  17. An introduction to the E3SM special collection: Goals, science drivers, development, and analysis. Journal of Advances in Modeling Earth Systems 12, 11 (2020), e2019MS001821.
  18. Toward feature-preserving vector field compression. IEEE Transactions on Visualization and Computer Graphics 29, 12 (2023), 5434–5450.
  19. Error-controlled lossy compression optimized for high compression ratios of scientific datasets. In Proceedings of 2018 IEEE International Conference on Big Data (Big Data) (2018), pp. 438–447.
  20. Exploring autoencoder-based error-bounded compression for scientific data. In Proceedings of 2021 IEEE International Conference on Cluster Computing (CLUSTER) (2021), pp. 294–306.
  21. Lindstrom P., Isenburg M.: Fast and efficient compression of floating-point data. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 1245–1250.
  22. Lindstrom P.: Fixed-rate compressed floating-point arrays. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2674–2683.
  23. Compressive neural representations of volumetric scalar fields. Computer Graphics Forum 40, 3 (2021), 135–146.
  24. Compressing the incompressible with isabela: In-situ reduction of spatio-temporal data. In Euro-Par 2011 Parallel Processing: 17th International Conference, Euro-Par 2011, Bordeaux, France, August 29-September 2, 2011, Proceedings, Part I 17 (2011), Springer, pp. 366–379.
  25. Spatiotemporal wavelet compression for visualization of scientific simulation data. In Proceedings of 2017 IEEE International Conference on Cluster Computing (CLUSTER) (2017), pp. 216–227.
  26. Spatiotemporal wavelet compression for visualization of scientific simulation data. In Proceedings of 2017 IEEE International Conference on Cluster Computing, CLUSTER (2017), pp. 216–227.
  27. SZ3: A modular framework for composing prediction-based error-bounded lossy compressors. IEEE Transactions on Big Data 9, 2 (2022), 485–498.
  28. ACORN: Adaptive coordinate networks for neural scene representation. ACM Transactions on Graphics 40, 4 (2021), 58:1–58:13.
  29. Natarajan V., Edelsbrunner H.: Simplification of three-dimensional density maps. IEEE Transactions on Visualization and Computer Graphics 10, 5 (2004), 587–597.
  30. Technologies for 3D mesh compression: A survey. Journal of Visual Communication and Image Representation 16, 6 (2005), 688–733.
  31. Rossignac J., Borrel P.: Multi-resolution 3D approximations for rendering complex scenes. In Modeling in Computer Graphics: Methods and Applications. Springer, 1993, pp. 455–465.
  32. Rusinkiewicz S., Levoy M.: QSplat: A multiresolution point rendering system for large meshes. In Proceedings of the 27th Annual Conference on Computer graphics and Interactive Techniques (2000), pp. 343–352.
  33. Optimal compressed sensing and reconstruction of unstructured mesh datasets. Data Science and Engineering 3 (2018), 1–23.
  34. Si H.: A simple algorithm to triangulate a special class of 3D non-convex polyhedra without Steiner points. In Numerical Geometry, Grid Generation and Scientific Computing (2019), Springer, pp. 61–71.
  35. Comparing field data using Alpert multi-wavelets. Computational Mechanics 66 (2020), 893–910.
  36. Implicit neural representations with periodic activation functions. In Proceedings of Advances in Neural Information Processing Systems (2020).
  37. Topologically controlled lossy compression. In Proceedings of 2018 IEEE Pacific Visualization Symposium (PacificVis) (2018), pp. 46–55.
  38. Schneider J., Westermann R.: Compression domain volume rendering. In Proceedings of IEEE Visualization 2003 (2003), pp. 293–300.
  39. Significantly improving lossy compression for scientific data sets based on multidimensional prediction and error-controlled quantization. In Proceedings of 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (2017), IEEE, pp. 1129–1139.
  40. Tricoche X.: https://www.cs.purdue.edu/homes/cs530/projects/data/final/cfd.
  41. Simplification of unstructured tetrahedral meshes by point sampling. In Proceedings of the Fourth International Workshop on Volume Graphics (2005), IEEE, pp. 157–238.
  42. Deep hierarchical super resolution for scientific data. IEEE Transactions of Visualization and Computer Graphics 29, 12 (2023), 5483–5495.
  43. Fast neural representations for direct volume rendering. Comput. Graph. Forum 41, 6 (2022), 196–211.
  44. AMRIC: A novel in situ lossy compression framework for efficient I/O in adaptive mesh refinement applications. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2023, Denver, CO, USA, November 12-17, 2023 (2023), Arnold D., Badia R. M., Mohror K. M., (Eds.), ACM, pp. 44:1–44:15.
  45. Neural fields in visual computing and beyond. Computer Graphics Forum (2022).
  46. TopoSZ: Preserving topology in error-bounded lossy compression. IEEE Transactions on Computer Graphics and Visualization 30, 1 (2024), 1302–1312.
  47. Optimizing error-bounded lossy compression for scientific data by dynamic spline interpolation. In Proceedings of 2021 IEEE 37th International Conference on Data Engineering (ICDE) (2021), IEEE, pp. 1643–1654.
  48. Significantly improving lossy compression for HPC datasets with second-order prediction and parameter optimization. In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing (2020), pp. 89–100.
  49. A multi-branch decoder network approach to adaptive temporal data selection and reconstruction for big scientific simulation data. IEEE Trans. Big Data 8, 6 (2022), 1637–1649.
  50. ZSTD: http://www.zstd.net.
Citations (2)

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets