Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Context-Aware Neural Video Compression on Solar Dynamics Observatory (2309.10784v1)

Published 19 Sep 2023 in eess.IV, astro-ph.SR, cs.CV, cs.IT, cs.LG, and math.IT

Abstract: NASA's Solar Dynamics Observatory (SDO) mission collects large data volumes of the Sun's daily activity. Data compression is crucial for space missions to reduce data storage and video bandwidth requirements by eliminating redundancies in the data. In this paper, we present a novel neural Transformer-based video compression approach specifically designed for the SDO images. Our primary objective is to efficiently exploit the temporal and spatial redundancies inherent in solar images to obtain a high compression ratio. Our proposed architecture benefits from a novel Transformer block called Fused Local-aware Window (FLaWin), which incorporates window-based self-attention modules and an efficient fused local-aware feed-forward (FLaFF) network. This architectural design allows us to simultaneously capture short-range and long-range information while facilitating the extraction of rich and diverse contextual representations. Moreover, this design choice results in reduced computational complexity. Experimental results demonstrate the significant contribution of the FLaWin Transformer block to the compression performance, outperforming conventional hand-engineered video codecs such as H.264 and H.265 in terms of rate-distortion trade-off.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. J. Schou, P. H. Scherrer, R. I. Bush, R. Wachter, S. Couvidat, M. C. Rabello-Soares, R. S. Bogart, J. Hoeksema, Y. Liu, T. Duvall et al., “Design and ground calibration of the helioseismic and magnetic imager (HMI) instrument on the solar dynamics observatory (SDO),” Solar Physics, 2012.
  2. C. E. Fischer, D. Müller, and I. De Moortel, “JPEG2000 image compression on solar EUV images,” Solar Physics, 2017.
  3. A. Zafari, A. Khoshkhahtinat, P. M. Mehta, N. M. Nasrabadi, B. J. Thompson, D. Da Silva, and M. S. Kirk, “Attention-based generative neural image compression on solar dynamics observatory,” in 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA).   IEEE, 2022, pp. 198–205.
  4. A. Zafari, A. Khoshkhahtinat, N. Nasrabadi, and P. Mehta, “Neural image compression on solar dynamics observatory,” The Third Triennial Earth-Sun Summit (TESS, vol. 54, no. 7, 2022.
  5. Y. Yang, S. Mandt, and L. Theis, “An introduction to neural data compression,” CoRR, 2022.
  6. S. Ma, X. Zhang, C. Jia, Z. Zhao, S. Wang, and S. Wang, “Image and video compression with neural networks: A review,” IEEE Transactions on Circuits and Systems for Video Technology, 2019.
  7. M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function,” Neural networks, 1993.
  8. V. K. Goyal, “Theoretical foundations of transform coding,” IEEE Signal Processing Magazine, vol. 18, no. 5, pp. 9–21, 2001.
  9. J. Ballé, P. A. Chou, D. Minnen, S. Singh, N. Johnston, E. Agustsson, S. J. Hwang, and G. Toderici, “Nonlinear transform coding,” IEEE Journal of Selected Topics in Signal Processing, vol. 15, no. 2, pp. 339–353, 2020.
  10. J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in ICLR, 2017.
  11. E. Agustsson, F. Mentzer, M. Tschannen, L. Cavigelli, R. Timofte, L. Benini, and L. V. Gool, “Soft-to-hard vector quantization for end-to-end learning compressible representations,” Advances in neural information processing systems, vol. 30, 2017.
  12. Y. Yang, R. Bamler, and S. Mandt, “Variational bayesian quantization,” in International Conference on Machine Learning.   PMLR, 2020, pp. 10 670–10 680.
  13. D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
  14. J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” arXiv preprint arXiv:1611.01704, 2016.
  15. J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in ICLR, 2018.
  16. D. Minnen, J. Ballé, and G. D. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” Advances in neural information processing systems, vol. 31, 2018.
  17. O. Rippel, S. Nair, C. Lew, S. Branson, A. G. Anderson, and L. Bourdev, “Learned video compression,” in ICCV, 2019.
  18. G. Lu, W. Ouyang, D. Xu, X. Zhang, C. Cai, and Z. Gao, “DVC: An end-to-end deep video compression framework,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11 006–11 015.
  19. A. Ranjan and M. J. Black, “Optical flow estimation using a spatial pyramid network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4161–4170.
  20. E. Agustsson, D. Minnen, N. Johnston, J. Balle, S. J. Hwang, and G. Toderici, “Scale-space flow for end-to-end optimized video compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8503–8512.
  21. Z. Hu, G. Lu, and D. Xu, “FVC: A new framework towards deep video compression in feature space,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1502–1511.
  22. Z. Guo, R. Feng, Z. Zhang, X. Jin, and Z. Chen, “Learning cross-scale prediction for efficient neural video compression,” arXiv e-prints, 2021.
  23. A. Habibian, T. v. Rozendaal, J. M. Tomczak, and T. S. Cohen, “Video compression with rate-distortion autoencoders,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 7033–7042.
  24. J. Pessoa, H. Aidos, P. Tomás, and M. A. Figueiredo, “End-to-end learning of video compression using spatio-temporal autoencoders,” in 2020 IEEE Workshop on Signal Processing Systems (SiPS).   IEEE, 2020, pp. 1–6.
  25. J. Lin, D. Liu, H. Li, and F. Wu, “M-LVC: Multiple frames prediction for learned video compression,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3546–3554.
  26. F. Mentzer, E. Agustsson, J. Ballé, D. Minnen, N. Johnston, and G. Toderici, “Neural video compression using gans for detail synthesis and propagation,” in European Conference on Computer Vision.   Springer, 2022, pp. 562–578.
  27. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
  28. F. Mentzer, G. Toderici, D. Minnen, S.-J. Hwang, S. Caelles, M. Lucic, and E. Agustsson, “VCT: A video compression transformer,” arXiv preprint arXiv:2206.07307, 2022.
  29. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  30. N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European conference on computer vision.   Springer, 2020, pp. 213–229.
  31. H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” in International conference on machine learning.   PMLR, 2021, pp. 10 347–10 357.
  32. Y. Wang, Z. Xu, X. Wang, C. Shen, B. Cheng, H. Shen, and H. Xia, “End-to-end video instance segmentation with transformers,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 8741–8750.
  33. A. Zafari, A. Khoshkhahtinat, P. Mehta, M. S. E. Saadabadi, M. Akyash, and N. M. Nasrabadi, “Frequency disentangled features in neural image compression,” in 2023 IEEE International Conference on Image Processing (ICIP).   IEEE, 2023, pp. 2815–2819.
  34. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2021.
  35. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  36. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
  37. A. Sarlak, A. Razi, X. Chen, and R. Amin, “Diversity maximized scheduling in roadside units for traffic monitoring applications,” in 2023 IEEE 48th Conference on Local Computer Networks (LCN).   IEEE, 2023, pp. 1–4.
  38. B. Adami, S. Tehranipoor, N. M. Nasrabadi, and N. Karimian, “A universal anti-spoofing approach for contactless fingerprint biometric systems,” in 2023 IEEE International Joint Conference on Biometrics (IJCB).   IEEE, 2023, pp. 1–8.
  39. Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in ICCV, 2021.
  40. Y. Yuan, R. Fu, L. Huang, W. Lin, C. Zhang, X. Chen, and J. Wang, “HRFormer:high-resolution transformer for dense prediction,” NeurIPS, 2021.
  41. “A guide to the mission and purpose of nasa’s solar dynamics observatory,” 2010. [Online]. Available: {https://sdo.gsfc.nasa.gov/assets/docs/SDO_Guide.pdf}
  42. R. Nematirad and A. Pahwa, “Solar radiation forecasting using artificial neural networks considering feature selection,” in 2022 IEEE Kansas Power and Energy Conference (KPEC).   IEEE, 2022, pp. 1–4.
  43. P. Bhuvela and A. Nasiri, “Design methodology for a medium voltage single stage llc resonant solar pv inverter.”
  44. H. Taghavi, A. El Shafei, and A. Nasiri, “Liquid cooling system for a high power, medium frequency, and medium voltage isolated power converter.”
  45. J. Schou, P. H. Scherrer, R. I. Bush, R. Wachter, S. Couvidat, M. C. Rabello-Soares, R. Bogart, J. Hoeksema, Y. Liu, T. Duvall et al., “Design and ground calibration of the Helioseismic and Magnetic Imager (HMI) instrument on the Solar Dynamics Observatory (SDO),” Solar Physics, vol. 275, pp. 229–259, 2012.
  46. J. R. Lemen, A. M. Title, D. J. Akin, P. F. Boerner, C. Chou, J. F. Drake, D. W. Duncan, C. G. Edwards, F. M. Friedlaender, G. F. Heyman et al., “The atmospheric imaging assembly (AIA) on the solar dynamics observatory (SDO),” Solar Physics, vol. 275, pp. 17–40, 2012.
  47. T. N. Woods, F. Eparvier, R. Hock, A. Jones, D. Woodraska, D. Judge, L. Didkovsky, J. Lean, J. Mariska, H. Warren et al., “Extreme Ultraviolet Variability Experiment (EVE) on the Solar Dynamics Observatory (SDO): Overview of science objectives, instrument design, data products, and model developments,” The solar dynamics observatory, pp. 115–143, 2012.
  48. R. Galvez, D. F. Fouhey, M. Jin, A. Szenicer, A. Muñoz-Jaramillo, M. C. Cheung, P. J. Wright, M. G. Bobra, Y. Liu, J. Mason et al., “A machine-learning data set prepared from the NASA solar dynamics observatory mission,” The Astrophysical Journal Supplement Series, 2019.
  49. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  50. A. Zafari, A. Khoshkhahtinat, P. M. Mehta, N. Nasrabadi, B. J. Thompson, D. da Silva, and M. Kirk, “Attention-based generative neural image compression on solar dynamics observatory,” in 103rd AMS Annual Meeting.   AMS, 2023.
  51. “Versatile Video Coding Reference Software,” Available at https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM, 2022.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now