Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Saliency-aware End-to-end Learned Variable-Bitrate 360-degree Image Compression (2402.08862v1)

Published 14 Feb 2024 in eess.IV

Abstract: Effective compression of 360$\circ$ images, also referred to as omnidirectional images (ODIs), is of high interest for various virtual reality (VR) and related applications. 2D image compression methods ignore the equator-biased nature of ODIs and fail to address oversampling near the poles, leading to inefficient compression when applied to ODI. We present a new learned saliency-aware 360$\circ$ image compression architecture that prioritizes bit allocation to more significant regions, considering the unique properties of ODIs. By assigning fewer bits to less important regions, significant data size reduction can be achieved while maintaining high visual quality in the significant regions. To the best of our knowledge, this is the first study that proposes an end-to-end variable-rate model to compress 360$\circ$ images leveraging saliency information. The results show significant bit-rate savings over the state-of-the-art learned and traditional ODI compression methods at similar perceptual visual quality.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. “State-of-the-art in 360° video/image processing: Perception, assessment and compression,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 1, pp. 5–26, 2020.
  2. “How do people explore virtual environments?,” IEEE Trans. on Visualization and Computer Graphics, 2017.
  3. B. W. Tatler, “The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions,” Journal of Vision, vol. 7, no. 14, pp. 4–4, 11 2007.
  4. “Look around you: Saliency maps for omnidirectional images in vr applications,” in 2017 Ninth Int. Conf. on Quality of Multimedia Experience (QoMEX), 2017, pp. 1–6.
  5. University of Nantes, “Salient360!: Visual attention modeling for 360 images grand challenge,” in Proc. IEEE Int. Conf. Multimedia Expo, 2017, pp. 35–42.
  6. M. Startsev and M. Dorr, “360-aware saliency estimation with conventional image saliency predictors,” Signal Processing: Image Communication, vol. 69, pp. 43–52, 2018.
  7. “A feature integrated saliency estimation model for omnidirectional immersive images,” Electronics, vol. 8, no. 12, 2019.
  8. P. Lebreton and A. Raake, “Gbvs360, bms360, prosal: Extending existing saliency prediction models from 2d to omnidirectional images,” Signal Processing: Image Communication, vol. 69, pp. 69–78, 2018.
  9. “Salnet360: Saliency maps for omni-directional images with cnn,” Signal Processing: Image Communication, vol. 69, pp. 26–34, 2018.
  10. “A multi-fov viewport-based visual saliency model using adaptive weighting losses for 360∘{}^{\circ}start_FLOATSUPERSCRIPT ∘ end_FLOATSUPERSCRIPT images,” IEEE Trans. on Multimedia, vol. 23, pp. 1811–1826, 2021.
  11. “Salgan360: Visual saliency prediction on 360 degree images with generative adversarial networks,” in 2018 IEEE Int. Conf. on Multimedia & Expo Workshops (ICMEW), 2018, pp. 01–04.
  12. “Mrgan360: Multi-stage recurrent generative adversarial network for 360 degree image saliency prediction,” ArXiv, vol. abs/2303.08525, 2023.
  13. “Atsal: An attention based architecture for saliency prediction in 360°videos,” in Int. Conf. on Pattern Recognition (ICPR). Springer, 2021, pp. 305–320.
  14. H. Hadizadeh and I. V. Bajić, “Saliency-aware video compression,” IEEE Trans. on Image Processing, vol. 23, no. 1, pp. 19–33, 2014.
  15. “Quality-oriented perceptual hevc based on the spatiotemporal saliency detection model,” Entropy, vol. 21, no. 2, 2019.
  16. “Saliency-driven rate-distortion optimization for 360-degree image coding,” Multimedia Tools and Applications, vol. 80, pp. 1–21, 03 2021.
  17. “Saliency-driven omnidirectional imaging adaptive coding: Modeling and assessment,” in 2017 IEEE 19th Int. Workshop on Multimedia Signal Processing (MMSP), 2017, pp. 1–6.
  18. “End-to-end optimized 360° image compression,” IEEE Trans. on Image Processing, vol. 31, pp. 6267–6281, 2022.
  19. “Oslo: On-the-sphere learning for omnidirectional images and its application to 360-degree image compression,” IEEE Trans. on Image Processing, vol. 31, pp. 5813–5827, 2022.
  20. “Sigvic: Spatial importance guided variable-rate image compression,” in ICASSP 2023 - 2023 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5.
  21. “Variable rate roi image compression optimized for visual quality,” in 2021 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), 2021, pp. 1936–1940.
  22. “Saliency driven perceptual image compression,” in 2021 IEEE Winter Conf. on Applications of Computer Vision (WACV), 2021, pp. 227–236.
  23. “Simple vs complex temporal recurrences for video saliency prediction,” in British Machine Vision Conf., 2019.
  24. K. Zhang and Z. Chen, “Video saliency prediction based on spatial-temporal two-stream network,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 29, no. 12, pp. 3544–3557, 2019.
  25. “Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding,” in Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 5718–5727.
  26. “Qvrf: A quantization-error-aware variable rate framework for learned image compression,” 2023 IEEE Int. Conf. on Image Processing (ICIP), pp. 1310–1314, 2023.
  27. “Weighted-to-spherically-uniform quality evaluation for omnidirectional video,” IEEE Signal Processing Letters, vol. 24, pp. 1408–1412, 2017.
  28. “Weighted-to-spherically-uniform ssim objective quality evaluation for panoramic video,” in 2018 14th IEEE Int. Conf. on Signal Processing (ICSP), 2018, pp. 54–57.
  29. “Salgan: Visual saliency prediction with generative adversarial networks,” in CVPR 2017 Scene Understanding Workshop (SUNw), Honolulu, Hawaii, USA, 2017.
  30. “Salbinet360: Saliency prediction on 360° images with local-global bifurcated deep network,” in 2020 IEEE Conf. on Virtual Reality and 3D User Interfaces (VR), 2020, pp. 92–100.
  31. “Variational image compression with a scale hyperprior,” in Int. Conf. on Learning Representations (ICLR), 2018.
  32. “Joint autoregressive and hierarchical priors for learned image compression,” in NeurIPS, 2018.
  33. “Learned image compression with discretized gaussian mixture likelihoods and attention modules,” in 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 7936–7945.
  34. “Versatile video coding reference software version 12.1 (vtm-12.1),” https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tags/VTM-12.1, 2021.
  35. G. Bjontegaard, “Calculation of average psnr differences between rd-curves,” Tech. Rep., VCEG-M33, Austin, TX, USA, April 2001.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now