Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Dynamic Spatio-Temporal Summarization using Information Based Fusion (2310.01617v1)

Published 2 Oct 2023 in cs.CV, cs.IT, and math.IT

Abstract: In the era of burgeoning data generation, managing and storing large-scale time-varying datasets poses significant challenges. With the rise of supercomputing capabilities, the volume of data produced has soared, intensifying storage and I/O overheads. To address this issue, we propose a dynamic spatio-temporal data summarization technique that identifies informative features in key timesteps and fuses less informative ones. This approach minimizes storage requirements while preserving data dynamics. Unlike existing methods, our method retains both raw and summarized timesteps, ensuring a comprehensive view of information changes over time. We utilize information-theoretic measures to guide the fusion process, resulting in a visual representation that captures essential data patterns. We demonstrate the versatility of our technique across diverse datasets, encompassing particle-based flow simulations, security and surveillance applications, and biological cell interactions within the immune system. Our research significantly contributes to the realm of data management, introducing enhanced efficiency and deeper insights across diverse multidisciplinary domains. We provide a streamlined approach for handling massive datasets that can be applied to in situ analysis as well as post hoc analysis. This not only addresses the escalating challenges of data storage and I/O overheads but also unlocks the potential for informed decision-making. Our method empowers researchers and experts to explore essential temporal dynamics while minimizing storage requirements, thereby fostering a more effective and intuitive understanding of complex data behaviors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (81)
  1. An Image-Based Approach to Extreme Scale in Situ Visualization and Analysis. International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015-Janua, 424–434. doi:10.1109/SC.2014.40.
  2. Simultaneous classification of time-varying volume data based on the time histogram., in: EuroVis, pp. 1–8.
  3. Visualizing multivariate volume data from turbulent combustion simulations. Computing in Science and Engineering 9, 76–83. doi:10.1109/MCSE.2007.42.
  4. Vibe: A universal background subtraction algorithm for video sequences. IEEE Transactions on Image processing 20, 1709–1724.
  5. Feature analysis, tracking, and data reduction: An application to multiphase reactor simulation mfix-exa for in-situ use case. Computing in Science and Engineering 23, 75–82. doi:10.1109/MCSE.2020.3016927.
  6. An information-aware framework for exploring multivariate data sets. IEEE Transactions on Visualization and Computer Graphics 19, 2683–2692. doi:10.1109/TVCG.2013.133.
  7. Multimodal data fusion based on mutual information. IEEE Transactions on Visualization and Computer Graphics 18, 1574–1587. doi:10.1109/TVCG.2011.280.
  8. An information-theoretic observation channel for volume visualization. Computer Graphics Forum 32, 411–420. doi:10.1111/cgf.12128.
  9. Information theory-based automatic multimodal transfer function design. IEEE Journal of Biomedical and Health Informatics 17, 870–880. doi:10.1109/JBHI.2013.2263227.
  10. Cd8+ t cells orchestrate pdc-xcr1+ dendritic cell spatial and functional cooperativity to optimize priming. Immunity 46, 205–219.
  11. Isosurface similarity maps, in: Computer Graphics Forum, Wiley Online Library. pp. 773–782.
  12. How much information is associated with a particular stimulus? Network: Computation in Neural Systems 14, 177–187. doi:10.1088/0954-898X_14_2_301.
  13. Normalized measures of mutual information with general definitions of entropy for multimodal image registration. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 6204 LNCS, 258–268. doi:10.1007/978-3-642-14366-3_23.
  14. A Benchmarking Framework for Background Subtraction in RGBD Videos. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10590 LNCS, 219–229. doi:10.1007/978-3-319-70742-6_21.
  15. Use cases of lossy compression for floating-point data in scientific data sets. The International Journal of High Performance Computing Applications 33, 1201–1220.
  16. A review of data fusion techniques. The scientific world journal 2013.
  17. Comparison of spatiotemporal fusion models: A review. Remote Sensing 7, 1798–1835.
  18. Information theory tools for visualization. CRC Press.
  19. Data exploration at the exascale. Supercomputing frontiers and innovations 2, 5–13.
  20. Word association norms, mutual information, and lexicography. Computational linguistics 16, 22–29.
  21. Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience.
  22. How to measure the information gained from one symbol. Network: Computation in Neural Systems doi:10.1088/0954-898X_10_4_303.
  23. Fast error-bounded lossy hpc data compression with sz, in: 2016 ieee international parallel and distributed processing symposium (ipdps), IEEE. pp. 730–739.
  24. A practical guide to evaluating colocalization in biological microscopy. American Journal of Physiology-Cell Physiology 300, C723–C742.
  25. Multivariate Pointwise Information-Driven Data Sampling and Visualization. Entropy 21, 699. doi:10.3390/e21070699.
  26. In situ distribution guided analysis and visualization of transonic jet engine simulations. IEEE transactions on visualization and computer graphics 23, 811–820.
  27. Pointwise information guided visual analysis of time-varying multi-fields. SIGGRAPH Asia 2017 Symposium on Visualization, SA 2017 doi:10.1145/3139295.3139298.
  28. Distribution Driven Extraction and Tracking of Features for Time-varying Data Analysis. IEEE Transactions on Visualization and Computer Graphics 22, 837–846. doi:10.1109/TVCG.2015.2467436.
  29. In Situ Adaptive Spatio-Temporal Data Summarization. Proceedings - 2021 IEEE International Conference on Big Data, Big Data 2021 , 315–321doi:10.1109/BigData52589.2021.9671581.
  30. In situ feature analysis for large-scale multiphase flow simulations. Journal of Computational Science 63, 101773. URL: https://doi.org/10.1016/j.jocs.2022.101773, doi:10.1016/j.jocs.2022.101773.
  31. Homogeneity guided probabilistic data summaries for analysis and visualization of large-scale data sets. IEEE Pacific Visualization Symposium , 111–120doi:10.1109/PACIFICVIS.2017.8031585.
  32. Deepvideomvs: Multi-view stereo on video with recurrent spatio-temporal fusion, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15324–15333.
  33. Multi-view video summarization. IEEE Transactions on Multimedia 12, 717–729.
  34. Digital image processing. Addison-Wesley Longman Publishing Co., Inc.
  35. Entropy and information theory. Springer Science & Business Media.
  36. Information-based transfer functions for multimodal visualization. EG VCBM 2008 - Eurographics Workshop on Visual Computing for Biomedicine , 101–108.
  37. Medical image registration. Physics in medicine & biology 46, R1.
  38. A survey on visual content-based video indexing and retrieval. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 41, 797–819.
  39. Crisp boundary detection using pointwise mutual information. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8691 LNCS, 799–814. doi:10.1007/978-3-319-10578-9_52.
  40. Review of data fusion methods for real-time and multi-sensor traffic flow analysis. IEEE Access 9, 51258–51276.
  41. On information and sufficiency. The annals of mathematical statistics 22, 79–86.
  42. A review on image processing and image segmentation, in: 2016 International Conference on Data Mining and Advanced Computing (SAPIENCE), pp. 198–203. doi:10.1109/SAPIENCE.2016.7684170.
  43. Spatio-temporal fusion for remote sensing data: An overview and new benchmark. Science China Information Sciences 63, 1–17.
  44. Spatio-temporal data fusion for massive sea surface temperature data from modis and amsr-e instruments. Environmetrics 31, e2594.
  45. Multimodality image registration by maximization of mutual information. IEEE transactions on Medical Imaging 16, 187–198.
  46. T-cell priming by dendritic cells in lymph nodes occurs in three distinct phases. Nature 427, 154–159.
  47. Systems biology approaches for understanding cellular mechanisms of immunity in lymph nodes during infection. Journal of theoretical biology 287, 160–170.
  48. Inform: Efficient information-theoretic analysis of collective behaviors. Frontiers Robotics AI 5, 1–14. doi:10.3389/frobt.2018.00060.
  49. Rock regulates the intermittent mode of interstitial t cell migration in inflamed lungs. Nature communications 8, 1010.
  50. MFIX-Exa: A path toward exascale CFD-DEM simulations. International Journal of High Performance Computing Applications 36, 40–58. doi:10.1177/10943420211009293.
  51. Partitioning a large simulation as it runs. Technometrics 58, 329–340.
  52. Spatio-temporal data fusion for very large remote sensing datasets. Technometrics 56, 174–185.
  53. Decoding collective communications using information theory tools. Journal of the Royal Society Interface 17. doi:10.1098/rsif.2019.0563.
  54. Exascale computing and big data. Communications of the ACM 58, 56–68.
  55. RGBD2017, . Background learning for detection and tracking from rgbd videos. https://rgbd2017.na.icar.cnr.it/. Accessed: 2023-09-22.
  56. Two-photon microscopy of cells and tissue. Circulation research 95, 1154–1166.
  57. Automatic transfer functions based on informational divergence. IEEE Transactions on Visualization and Computer Graphics 17, 1932–1941. doi:10.1109/TVCG.2011.173.
  58. Information theory tools for computer graphics. Springer Nature.
  59. SBM-RGBD Dataset, . Sbm-rgbd dataset. https://rgbd2017.na.icar.cnr.it/SBM-RGBDdataset.html. Accessed: 2023-09-22.
  60. The visualization toolkit an object-oriented approach to 3D graphics. Prentice-Hall, Inc.
  61. A spatiotemporal data summarization approach for real-time operation of smart grid. IEEE Transactions on Big Data 6, 624–637.
  62. A Mathematical Theory of Communication. Bell System Technical Journal 27, 379–423. doi:10.1002/j.1538-7305.1948.tb01338.x.
  63. Introduction to the Theory of Computation. Third ed., Course Technology, Boston, MA.
  64. Quantifying the effect of registration error on spatio-temporal fusion. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13, 487–503.
  65. Information-Theoretic Exploration of Multivariate Time-Varying Image Databases. Computing in Science and Engineering 24, 61–70. doi:10.1109/MCSE.2022.3188291.
  66. Quantitative measurement of naïve T cell association with dendritic cells, FRCs, and blood vessels in lymph nodes. Frontiers in Immunology 9. doi:10.3389/fimmu.2018.01571.
  67. Salient time steps selection from large scale time-varying data sets with dynamic time warping, in: IEEE symposium on large data analysis and visualization (LDAV), IEEE. pp. 49–56.
  68. Fifty years of shannon theory. IEEE Transactions on Information Theory 44, 2057–2078. doi:10.1109/18.720531.
  69. A review of lossless and lossy image compression techniques. Int. Res. J. Eng. Technol.(IRJET) 3, 616–7.
  70. Importance-driven focus of attention. IEEE transactions on visualization and computer graphics 12, 933–940.
  71. Alignment by maximization of mutual information. International journal of computer vision 24, 137–154.
  72. Importance-driven time-varying data visualization. IEEE Transactions on Visualization and Computer Graphics 14, 1547–1554.
  73. Virtual image pair-based spatio-temporal fusion. Remote Sensing of Environment 249, 112009.
  74. Information guided data sampling and recovery using bitmap indexing, in: 2018 IEEE Pacific Visualization Symposium (PacificVis), IEEE. pp. 56–65.
  75. In-situ sampling of a large-scale particle simulation for interactive visualization and analysis, in: Computer Graphics Forum, Wiley Online Library. pp. 1151–1160.
  76. Spatially continuous and high-resolution land surface temperature product generation: A review of reconstruction and spatiotemporal fusion techniques. IEEE Geoscience and Remote Sensing Magazine 9, 112–137.
  77. A bayesian data fusion approach to spatio-temporal fusion of remotely sensed images. Remote Sensing 9, 1310.
  78. In situ generated probability distribution functions for interactive post hoc visualization and analysis, in: 2016 IEEE 6th Symposium on Large Data Analysis and Visualization (LDAV), IEEE. pp. 65–74.
  79. Probabilistic skimlets fusion for summarizing multiple consumer landmark videos. IEEE Transactions on Multimedia 17, 40–49.
  80. Video summarization via spatio-temporal deep architecture. Neurocomputing 332, 224–235.
  81. Key time steps selection for large-scale time-varying volume datasets using an information-theoretic storyboard, in: Computer Graphics Forum, Wiley Online Library. pp. 37–49.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now