Papers
Topics
Authors
Recent
Search
2000 character limit reached

Video Decoding Energy Reduction Using Temporal-Domain Filtering

Published 12 Jun 2023 in eess.IV | (2306.06917v1)

Abstract: In this paper, we study decoding energy reduction opportunities using temporal-domain filtering and subsampling methods. In particular, we study spatiotemporal filtering using a contrast sensitivity function and temporal downscaling, i.e., frame rate reduction. We apply these concepts as a pre-filtering to the video before compression and evaluate the bitrate, the decoding energy, and the visual quality with a dedicated metric targeting temporally down-scaled sequences. We find that decoding energy savings yield 35% when halving the frame rate and that spatiotemporal filtering can lead to up to 5% of additional savings, depending on the content.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. [n. d.]. openHEVC: Real-Time Decoder for HEVC video decoding. https://github.com/OpenHEVC. accessed 2023-04.
  2. [n. d.]. x265: H.265 / HEVC video encoder application library. https://www.videolan.org/developers/x265.html. accessed 2023-04.
  3. Encoder design for H.264/AVC based on contrast sensitivity considering spatio-temporal direction dependency. In Proc. 15th IEEE International Conference on Image Processing (ICIP). 2124–2127.
  4. P. G. Barten. 1999. Contrast sensitivity of the human eye and its effects on image quality. SPIE press.
  5. G. Bjøntegaard. 2001. Calculation of average PSNR differences between RD curves. document. VCEG-M33, Austin, TX, USA.
  6. ITU Recommendation BT. 2019. 500-14, Methodologies for the Subjective Assessment of the Quality of Television Images. https://www.itu.int/rec/R-REC-BT.500. Geneva: International Telecommunication Union (2019). accessed 2023-04.
  7. A. Carroll and G. Heiser. 2013. The systems hacker’s guide to the galaxy - energy usage in a modern smartphone. In Proc. 4th Asia-Pacific Workshop on Systems (APSys). Singapore.
  8. OTED: Encoding Optimization Technique Targeting Energy-Efficient HEVC Decoding. In Proc. IEEE International Symposium on Circuits and Systems (ISCAS). 1–5.
  9. RAPL: Memory power estimation and capping. In ACM/IEEE International Symposium on Low-Power Electronics and Design (ISLPED). Washington, USA, 189–194.
  10. A. P. Ginsburg. 2003. Contrast sensitivity and functional vision. International ophthalmology clinics 43, 2 (2003), 5–15.
  11. Decoding-Energy Optimal Video Encoding For x265. In Proc. IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP). 1–6.
  12. Power Modeling for Video Streaming Applications on Mobile Devices. IEEE Access 8 (2020), 70234–70244.
  13. Decoding-Energy-Rate-Distortion Optimization for Video Coding. IEEE Transactions on Circuits and Systems for Video Technology 29, 1 (Jan. 2019), 172–181. Issue 1.
  14. Power-Efficient Video Streaming on Mobile Devices Using Optimal Spatial Scaling. In Proc. IEEE International Conference on Consumer Electronics (ICCE). Berlin, Germany.
  15. Beyond Bjøntegaard: Limits of Video Compression Performance Comparisons. In Proc. IEEE International Conference on Image Processing (ICIP) (2022-10).
  16. Modeling of Energy Consumption and Streaming Video QoE using a Crowdsourcing Dataset. In Proc. 14th International Conference on Quality of Multimedia Experience (QoMEX) (Lippstadt, Germany, September 2022). 1–6.
  17. Quality-driven Variable Frame-Rate for Green Video Coding in Broadcast Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 11 (2020), 4508–4522.
  18. D.H. Kelly. 1977. Visual contrast sensitivity. Optica Acta: International Journal of Optics 24, 2 (1977), 107–129.
  19. HEVC hardware vs software decoding: An objective energy consumption analysis and comparison. Journal of Systems Architecture 115 (2021), 102004.
  20. Energy Efficient Video Decoding for VVC Using a Greedy Strategy Based Design Space Exploration. IEEE Transactions on Circuits and Systems for Video Technology 32, 7 (2022), 4696–4709.
  21. Enhancing the contrast sensitivity function through action video game training. Nature neuroscience 12, 5 (2009), 549–551.
  22. A. Mackin and D. Bull. 2020. Characterizing the spatiotemporal envelope of the human visual system through the visibility of temporal aliasing artifacts. Journal of the Optical Society of America A 37, 7 (2020), 1116–1127.
  23. High frame rates and the visibility of motion artifacts. SMPTE Motion Imaging Journal 126, 5 (2017), 41–51.
  24. A study of high frame rate video formats. IEEE Transactions on Multimedia 21, 6 (2018), 1499–1512.
  25. ST-GREED: Space-Time Generalized Entropic Differences for Frame Rate Dependent Video Quality Prediction. IEEE Transactions on Image Processing 30 (2021), 7446–7457.
  26. A Decoding-Complexity and Rate-Controlled Video-Coding Algorithm for HEVC. Future Internet 12, 7 (2020), 120.
  27. E. Peli. 2001. Contrast sensitivity function and image discrimination. Journal of the Optical Society of America A 18, 2 (2001), 283–293.
  28. Modeling the HEVC Encoding Energy using the Encoder Processsing Time. In Proc. IEEE International Conference on Image Processing (ICIP). Bordeaux, France.
  29. J. G. Robson. 1966. Spatial and temporal contrast-sensitivity functions of the visual system. Journal of the Optical Society of America (Josa) 56, 8 (1966), 1141–1142.
  30. Sandvine. 2022. The Mobile Internet Phenomena Report. https://www.sandvine.com/global-internet-phenomena-report-2022. accessed 2023-04.
  31. The Shift Project. 2019. Climate Crisis: The Unsustainable Use of Online Video. https://theshiftproject.org/en/article/unsustainable-use-online-video/. accessed 2023-04.
  32. Carbon Trust. 2021. Carbon impact of video streaming. https://www.carbontrust.com/our-work-and-impact/guides-reports-and-tools/carbon-impact-of-video-streaming. accessed 2023-04.
  33. Efficient Mode Decision Schemes for HEVC Inter Prediction. IEEE Transactions on Circuits and Systems for Video Technology 24, 9 (Sep 2014), 1579–1593.
  34. Temporal contrast sensitivity and cortical magnification. Vision Research 22, 9 (1982), 1211–1217.
  35. A. B. Watson. 2013. High frame rates and human vision: A view through the window of visibility. SMPTE Motion Imaging Journal 122, 2 (2013), 18–32.
  36. A. B. Watson and A. J. Ahumada. 2005. A standard model for foveal detection of spatial contrast. Journal of Vision 5, 9 (10 2005), 6–6.
  37. A practical method of measuring the human temporal contrast sensitivity function. Biomedical optics express 1, 1 (2010), 47–58.
  38. Visual Contrast Sensitivity Guided Video Quality Assessment. In Proc. IEEE International Conference on Multimedia and Expo. 824–829.
  39. Reducing power consumption of HEVC codec with lossless reference frame recompression. In Poc. IEEE International Conference on Image Processing (ICIP). Paris, France, 2120–2124.
Citations (3)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up