Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Analysis of Neural Video Compression Networks for 360-Degree Video Coding (2402.10257v1)

Published 15 Feb 2024 in eess.IV

Abstract: With the increasing efforts of bringing high-quality virtual reality technologies into the market, efficient 360-degree video compression gains in importance. As such, the state-of-the-art H.266/VVC video coding standard integrates dedicated tools for 360-degree video, and considerable efforts have been put into designing 360-degree projection formats with improved compression efficiency. For the fast-evolving field of neural video compression networks (NVCs), the effects of different 360-degree projection formats on the overall compression performance have not yet been investigated. It is thus unclear, whether a resampling from the conventional equirectangular projection (ERP) to other projection formats yields similar gains for NVCs as for hybrid video codecs, and which formats perform best. In this paper, we analyze several generations of NVCs and an extensive set of 360-degree projection formats with respect to their compression performance for 360-degree video. Based on our analysis, we find that projection format resampling yields significant improvements in compression performance also for NVCs. The adjusted cubemap projection (ACP) and equatorial cylindrical projection (ECP) show to perform best and achieve rate savings of more than 55% compared to ERP based on WS-PSNR for the most recent NVC. Remarkably, the observed rate savings are higher than for H.266/VVC, emphasizing the importance of projection format resampling for NVCs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (33)
  1. “Overview of the H.264/AVC Video Coding Standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560–576, July 2003.
  2. “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.
  3. “Overview of the Versatile Video Coding (VVC) Standard and its Applications,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 10, pp. 3736–3764, Oct. 2021.
  4. I. I. Pearson, Map Projections: Theory and Applications, CRC Press, Boca Raton, Fla, 2nd edition edition, Mar. 1990.
  5. “Algorithm Descriptions of Projection Format Conversion and Video Quality Metrics in 360Lib Version 12, JVET-T2004-v2,” in Proc. 20th Meet. Jt. Video Experts Team, Oct. 2020, pp. 1–65.
  6. “Improved Motion Compensation for 360° Video Projected to Polytopes,” in Proc. IEEE Int. Conf. Multimed. Expo, July 2017, pp. 61–66.
  7. “Geometry-Corrected Deblocking Filter for 360° Video Coding using Cube Representation,” in Proc. Pict. Coding Symp., June 2018, pp. 66–70.
  8. “Video Coding Optimization for Virtual Reality 360-Degree Source,” IEEE J. Sel. Top. Signal Process., vol. 14, no. 1, pp. 118–129, Jan. 2020.
  9. “A Geodesic Translation Model for Spherical Video Compression,” IEEE Trans. Image Process., vol. 31, pp. 2136–2147, Feb. 2022.
  10. “Multi-Model Motion Prediction for 360-Degree Video Compression,” IEEE Access, vol. 11, pp. 117004–117017, Oct. 2023.
  11. “DVC: An End-To-End Deep Video Compression Framework,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., June 2019, pp. 10998–11007.
  12. “Scale-Space Flow for End-to-End Optimized Video Compression,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., June 2020, pp. 8500–8509.
  13. “Deep Contextual Video Compression,” in Proc. Adv. Neural Inf. Process. Syst., 2021, vol. 34, pp. 18114–18125.
  14. “Temporal Context Mining for Learned Video Compression,” IEEE Trans. Multimed., vol. 25, pp. 7311–7322, 2023.
  15. “Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression,” in Proc. 30th ACM Int. Conf. Multimed., New York, NY, USA, Oct. 2022, MM ’22, pp. 1503–1511.
  16. “Neural Video Compression With Diverse Contexts,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2023, pp. 22616–22626.
  17. “JVET Common Test Conditions and Evaluation Procedures for 360° Video, JVET-U2012,” in Proc. 21st Meet. Jt. Video Explor. Team, Jan. 2021, pp. 1–8.
  18. Minhua Zhou, “AHG8: A Study On Quality Impact of Line Re-Sampling Rate in EAP, JVET-G0051,” in Proc. 7th Meet. Jt. Video Explor. Team, July 2017, pp. 1–5.
  19. Minhua Zhou, “AHG8: A Study on Equi-Angular Cubemap Projection (EAC), JVET-G0056,” in Proc. 7th Meet. Jt. Video Explor. Team, July 2017, pp. 1–14.
  20. “CE13: Modified Cubemap Projection in JVET-J0019 (Test 5), JVET-K0131,” in Proc. 11th Meet. Jt. Video Explor. Team, July 2018, pp. 1–6.
  21. “AHG8: Adjusted Cubemap Projection for 360-Degree Video, JVET-F0025,” in Proc. 6th Meet. Jt. Video Explor. Team, Apr. 2017, pp. 1–6.
  22. “AHG6/AHG17: Generalized Cubemap Projection Syntax for 360-Degree Videos,” in Proc. 16th Meet. Jt. Video Experts Team, Nov. 2019, pp. 1–12.
  23. “AHG8: ECP with Padding for 360-Degree Video, JVET-G0074,” in Proc. 7th Meet. Jt. Video Explor. Team, July 2017, pp. 1–14.
  24. “AHG8: Rotated Sphere Projection for 360 Video, JVET-F0036,” in Proc. 6th Meet. Jt. Video Explor. Team, Apr. 2017, pp. 1–7.
  25. “AHG8: Icosahedral Projection for 360-Degree Video Content, JVET-D0028,” in Proc. 4th Meet. Jt. Video Explor. Team, Oct. 2016, pp. 1–5.
  26. JVET, “360Lib Software 360Lib-13.1,” https://vcgit.hhi.fraunhofer.de/jvet/360lib/-/tags/360Lib-13.1, Oct. 2021.
  27. Microsoft, “DCVC Github Repository,” https://github.com/microsoft/DCVC, Nov. 2023.
  28. “Video Enhancement with Task-Oriented Flow,” Int. J. Comput. Vis., vol. 127, no. 8, pp. 1106–1125, Aug. 2019.
  29. ITU-R, “Rec. ITU-R BT.709-6: Parameter Values for the HDTV Standards for Production and International Programme Exchange,” June 2015.
  30. JVET, “VVC Reference Software VTM-22.2,” https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/releases/VTM-22.2, Oct. 2023.
  31. “Weighted-to-Spherically-Uniform Quality Evaluation for Omnidirectional Video,” IEEE Signal Process. Lett., vol. 24, no. 9, pp. 1408–1412, Sept. 2017.
  32. ITU-T, “Working Practices Using Objective Metrics for Evaluation of Video Coding Efficiency Experiments,” July 2020.
  33. Gisle Bjøntegaard, “Calculation of Average PSNR Differences between RD-curves, VCEG-M33,” in Proc. 13th Meet. Video Coding Experts Group, Mar. 2001, pp. 1–5.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now