Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Subjective Quality Assessment of Compressed Tone-Mapped High Dynamic Range Videos (2403.15061v1)

Published 22 Mar 2024 in eess.IV and cs.CV

Abstract: High Dynamic Range (HDR) videos are able to represent wider ranges of contrasts and colors than Standard Dynamic Range (SDR) videos, giving more vivid experiences. Due to this, HDR videos are expected to grow into the dominant video modality of the future. However, HDR videos are incompatible with existing SDR displays, which form the majority of affordable consumer displays on the market. Because of this, HDR videos must be processed by tone-mapping them to reduced bit-depths to service a broad swath of SDR-limited video consumers. Here, we analyze the impact of tone-mapping operators on the visual quality of streaming HDR videos. To this end, we built the first large-scale subjectively annotated open-source database of compressed tone-mapped HDR videos, containing 15,000 tone-mapped sequences derived from 40 unique HDR source contents. The videos in the database were labeled with more than 750,000 subjective quality annotations, collected from more than 1,600 unique human observers. We demonstrate the usefulness of the new subjective database by benchmarking objective models of visual quality on it. We envision that the new LIVE Tone-Mapped HDR (LIVE-TMHDR) database will enable significant progress on HDR video tone mapping and quality assessment in the future. To this end, we make the database freely available to the community at https://live.ece.utexas.edu/research/LIVE_TMHDR/index.html

Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. ITU-R, “ITU-R BT.709: Parameter values for the HDTV standards for production and international programme exchange,” 2011.
  2. IEC, “Multimedia systems and equipment - Colour measurement and management - Part 2-1: Colour management - Default RGB colour space - sRGB,” 1999.
  3. ITU-R, “ITU-R BT.2100: Image parameter values for high dynamic range television for use in production and international programme exchange,” 2018.
  4. SMPTE, “High dynamic range electro-optical transfer function of mastering reference displays,” SMPTE Standard, vol. 2084, no. 2014, p. 11, 2014.
  5. T. Borer and A. Cotton, “A Display-Independent High Dynamic Range Television System,” SMPTE Motion Imaging Journal, vol. 125, no. 4, pp. 50–56, 2016.
  6. CTA. Television technology consumer definitions. [Online]. Available: https://cdn.cta.tech/cta/media/media/membership/pdfs/videotechnology-consumer-definitions.pdf
  7. HDR10+ Technologies, LLC. (2019) Hdr10+ system whitepaper. [Online]. Available: https://hdr10plus.org/wp-content/uploads/2019/08/HDR10_WhitePaper.pdf
  8. C. Forrester, “SkyPerfect offers UHD-HDR by DTH.” [Online]. Available: https://advanced-television.com/2015/11/04/skyperfect-offers-uhd-hdr-by-dth/
  9. (2016). [Online]. Available: https://professional.dolby.com/siteassets/pdfs/dolby-vision-whitepaper_an-introduction-to-dolby-vision_0916.pdf
  10. ——. How To Choose an HDR Gaming Monitor. [Online]. Available: https://www.cnet.com/tech/computing/how-to-choose-an-hdr-gaming-monitor/
  11. J. P. Ebenezer, Z. Shang, Y. Wu, H. Wei, S. Sethuraman, and A. C. Bovik, “Making video quality assessment models robust to bit depth,” IEEE Signal Processing Letters, vol. 30, pp. 488–492, 2023.
  12. M. Čadík, M. Wimmer, L. Neumann, and A. Artusi, “Evaluation of HDR tone mapping methods using essential perceptual attributes,” Computers & Graphics, vol. 32, no. 3, pp. 330–349, 2008.
  13. J. Morovic and M. R. Luo, “The fundamentals of gamut mapping: A survey,” Journal of Imaging Science and Technology, vol. 45, no. 3, pp. 283–290, 2001.
  14. T. Wolff, H.-H. Ho, J. M. Foley, and S. K. Mitra, “H.264 coding artifacts and their relation to perceived annoyance,” in 2006 14th European Signal Processing Conference, 2006, pp. 1–5.
  15. F. Drago, W. L. Martens, K. Myszkowski, and H.-P. Seidel, “Perceptual evaluation of tone mapping operators,” in ACM SIGGRAPH 2003 Sketches & Applications, 2003, pp. 1–1.
  16. J. D. Carroll and J.-J. Chang, “Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition,” Psychometrika, vol. 35, no. 3, pp. 283–319, 1970.
  17. J. Kuang, H. Yamaguchi, G. M. Johnson, and M. D. Fairchild, “Testing HDR Image Rendering Algorithms,” in International Conference on Communications in Computing, 2004.
  18. P. Ledda, A. Chalmers, T. Troscianko, and H. Seetzen, “Evaluation of Tone Mapping Operators Using a High Dynamic Range Display,” ACM Trans. Graph., vol. 24, no. 3, p. 640–648, Jul 2005.
  19. L. L. Thurstone, “A law of comparative judgment.” Psychological review, vol. 34, no. 4, p. 273, 1927.
  20. F. Durand and J. Dorsey, “Fast Bilateral Filtering for the Display of High-Dynamic-Range Images,” in Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’02.   New York, NY, USA: Association for Computing Machinery, 2002, p. 257–266.
  21. E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, “Photographic Tone Reproduction for Digital Images,” in Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, ser. SIGGRAPH ’02.   Association for Computing Machinery, 2002, p. 267–276.
  22. G. Johnson and M. Fairchild, “Rendering HDR images,” 2003. [Online]. Available: https://scholarworks.rit.edu/other/151
  23. A. Yoshida, V. Blanz, K. Myszkowski, and H.-P. Seidel, “Perceptual evaluation of tone mapping operators with real-world scenes,” in Human Vision and Electronic Imaging X, vol. 5666, International Society for Optics and Photonics.   SPIE, 2005, pp. 192 – 203.
  24. L. Krasula, M. Narwaria, K. Fliegel, and P. Le Callet, “Preference of experience in image tone-mapping: Dataset and framework for objective measures comparison,” IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 1, pp. 64–74, 2017.
  25. G. Eilertsen, J. Unger, and R. K. Mantiuk, “Evaluation of Tone Mapping Operators for HDR Video,” in High Dynamic Range Video.   Academic Press, 2016, pp. 185–207.
  26. G. Eilertsen, R. K. Mantiuk, and J. Unger, “A comparative review of tone-mapping algorithms for high dynamic range video,” Computer Graphics Forum, vol. 36, no. 2, pp. 565–592, 2017.
  27. H. Yeganeh and Z. Wang, “Objective Quality Assessment of Tone-Mapped Images,” IEEE Transactions on Image Processing, vol. 22, no. 2, pp. 657–667, 2013.
  28. D. Kundu, D. Ghadiyaram, A. C. Bovik, and B. L. Evans, “Large-Scale Crowdsourced Study for Tone-Mapped HDR Pictures,” IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4725–4740, 2017.
  29. D. Kundu, D. Ghadiyaram, A. C. Bovik, and B. Evans, “No-Reference Quality Assessment of Tone-Mapped HDR Pictures,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2957–2971, 2017.
  30. A. Ak, A. Goswami, W. Hauser, P. Le Callet, and F. Dufaux, “RV-TMO: Large-Scale Dataset for Subjective Quality Assessment of Tone Mapped Images,” IEEE Transactions on Multimedia, pp. 1–12, 2022.
  31. Z. Shang, J. P. Ebenezer, A. C. Bovik, Y. Wu, H. Wei, and S. Sethuraman, “Subjective Assessment Of High Dynamic Range Videos Under Different Ambient Conditions,” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 786–790.
  32. The Consumer Digital Video Library. [Online]. Available: https://www.cdvl.org/
  33. L. Song, Y. Liu, X. Yang, G. Zhai, R. Xie, and W. Zhang, “The SJTU HDR Video Sequence Dataset,” in Proceedings of International Conference on Quality of Multimedia Experience (QoMEX 2016), Lisbon, Portugal, 2016, p. 2.
  34. Free Ultra-HD / HDR / HLG / Dolby Vision 4K video demos. [Online]. Available: https://4kmedia.org/
  35. Global internet phenomena report 2023. [Online]. Available: https://www.sandvine.com/global-internet-phenomena-report-2023
  36. SI/TI Tools. [Online]. Available: https://vqeg.org/VQEGSharedFiles/MeetingFiles/2021_12_UniversityOfSurry_online/VQEG_NORM_2021_204_SI-TI.pdf
  37. D. Hasler and S. Suesstrunk, “Measuring colourfulness in natural images,” Proceedings of SPIE - The International Society for Optical Engineering, vol. 5007, pp. 87–95, 06 2003.
  38. Z. Tu, Y. Wang, N. Birkbeck, B. Adsumilli, and A. C. Bovik, “UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content,” IEEE Transactions on Image Processing, vol. 30, pp. 4449–4464, 2021.
  39. F. Ebner and M. D. Fairchild, “Gamut mapping from below: Finding minimum perceptual distances for colors outside the gamut volume,” Color Research & Application, vol. 22, no. 6, pp. 402–413, 1997.
  40. N. Katoh and M. Ito, “Gamut mapping for computer generated images (II),” in Color and Imaging Conference, vol. 1996, no. 1.   Society for Imaging Science and Technology, 1996, pp. 126–128.
  41. J. Morovic and M. R. Luo, “Gamut mapping algorithms based on psychophysical experiment,” in Color and Imaging Conference, vol. 1997, no. 1.   Society for Imaging Science and Technology, 1997, pp. 44–49.
  42. G. J. Braun and M. D. Fairchild, “General-purpose gamut-mapping algorithms: Evaluation of contrast-preserving rescaling functions for color gamut mapping,” Journal of Imaging Science and Technology, vol. 44, no. 4, pp. 343–350, 2000.
  43. ITU-R, “ITU-R BT.2407: Colour gamut conversion from Recommendation ITU-R BT.2020 to Recommendation ITU-R BT.709,” 2017.
  44. J. Hable. Uncharted 2: HDR Lighting. [Online]. Available: https://www.gdcvault.com/play/1012351/Uncharted-2-HDR
  45. FFmpeg. [Online]. Available: https://ffmpeg.org/
  46. Q. Shan, T. DeRose, and J. Anderson, “Tone Mapping High Dynamic Range Videos Using Wavelets,” Pixar Technical Memo, 2012.
  47. E. Reinhard, T. Pouli, T. Kunkel, B. Long, A. Ballestad, and G. Damberg, “Calibrated Image Appearance Reproduction,” ACM Trans. Graph., vol. 31, no. 6, Nov 2012.
  48. G. Eilertsen, R. K. Mantiuk, and J. Unger, “Real-Time Noise-Aware Tone Mapping,” ACM Trans. Graph., vol. 34, no. 6, Nov 2015.
  49. M. Oskarsson, “Temporally Consistent Tone Mapping of Images and Video Using Optimal K-means Clustering,” Journal of Mathematical Imaging and Vision, vol. 57, no. 2, pp. 225–238, Feb 2017.
  50. A. Rana, P. Singh, G. Valenzise, F. Dufaux, N. Komodakis, and A. Smolic, “Deep Tone Mapping Operator for High Dynamic Range Images,” IEEE Transactions on Image Processing, vol. 29, pp. 1285–1298, 2020.
  51. J. Yang, Z. Liu, M. Lin, S. Yanushkevich, and O. Yadid-Pecht, “Deep Reformulated Laplacian Tone Mapping,” 2021.
  52. ITU-R, “ITU-R BT.2446: Methods for conversion of high dynamic range content to standard dynamic range content and vice-versa,” 2021.
  53. R. Boitard, K. Bouatouch, R. Cozot, D. Thoreau, and A. Gruson, “Temporal coherency for video tone mapping,” in Applications of Digital Image Processing XXXV, A. G. Tescher, Ed., vol. 8499, International Society for Optics and Photonics.   SPIE, 2012, p. 84990D.
  54. (2010) Report on the validation of video quality models for high definition video content. Video Quality Experts Group.
  55. Z. Shang, J. P. Ebenezer, Y. Wu, H. Wei, S. Sethuraman, and A. C. Bovik, “Study of the Subjective and Objective Quality of High Motion Live Streaming Videos,” IEEE Transactions on Image Processing, vol. 31, pp. 1027–1041, 2022.
  56. VideoLAN, “x264.” [Online]. Available: https://code.videolan.org/videolan/x264.git
  57. Can I use - HEVC/H.265 video format. [Online]. Available: https://caniuse.com/?search=H.265
  58. Can I use - MPEG-4/H.264 video format. [Online]. Available: https://caniuse.com/?search=H.264
  59. P. Teo and D. Heeger, “Perceptual image distortion,” in Proceedings of 1st International Conference on Image Processing, vol. 2, 1994, pp. 982–986 vol.2.
  60. J. M. Foley and G. M. Boynton, “New model of human luminance pattern vision mechanisms: analysis of the effects of pattern orientation, spatial phase and temporal frequency,” in Computational Vision Based on Neurobiology, T. B. Lawton, Ed., vol. 2054, International Society for Optics and Photonics.   SPIE, 1994, pp. 32 – 42. [Online]. Available: https://doi.org/10.1117/12.171150
  61. Amazon Mechanical Turk. [Online]. Available: https://www.mturk.com/
  62. D. Ghadiyaram and A. C. Bovik, “Massive Online Crowdsourced Study of Subjective and Objective Picture Quality,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 372–387, 2016.
  63. H. Lin, V. Hosu, and D. Saupe, “KADID-10k: A Large-scale Artificially Distorted IQA Database,” in 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX), 2019, pp. 1–3.
  64. Z. Sinno and A. C. Bovik, “Large-Scale Study of Perceptual Video Quality,” IEEE Transactions on Image Processing, vol. 28, no. 2, pp. 612–627, 2019.
  65. Z. Ying, M. Mandal, D. Ghadiyaram, and A. C. Bovik, “Patch-VQ: ’Patching Up’ the Video Quality Problem,” CoRR, vol. abs/2011.13544, 2020. [Online]. Available: https://arxiv.org/abs/2011.13544
  66. F. Götz-Hahn, V. Hosu, H. Lin, and D. Saupe, “KonVid-150k: A Dataset for No-Reference Video Quality Assessment of Videos in-the-Wild,” in IEEE Access 9.   IEEE, 2021, pp. 72 139–72 160.
  67. Z. Li, C. G. Bampis, L. Janowski, and I. Katsavounidis, “A simple model for subject behavior in subjective experiments,” Electronic Imaging, vol. 2020, no. 11, pp. 131–1, 2020.
  68. H. Ziaei Nafchi, A. Shahkolaei, R. Farrahi Moghaddam, and M. Cheriet, “FSITM: A Feature Similarity Index For Tone-Mapped Images,” IEEE Signal Processing Letters, vol. 22, no. 8, pp. 1026–1029, 2015.
  69. L. Krasula, K. Fliegel, and P. Le Callet, “FFTMI: Features Fusion for Natural Tone-Mapped Images Quality Evaluation,” IEEE Transactions on Multimedia, vol. 22, no. 8, pp. 2038–2047, 2020.
  70. H. Yeganeh, S. Wang, K. Zeng, M. Eisapour, and Z. Wang, “Objective quality assessment of tone-mapped videos,” in 2016 IEEE International Conference on Image Processing (ICIP), 2016, pp. 899–903.
  71. A. K. Venkataramanan, C. Stejerean, I. Katsavounidis, and A. C. Bovik, “One Transform To Compute Them All: Efficient Fusion-Based Full-Reference Video Quality Assessment,” 2023.
  72. A. K. Venkataramanan, C. Stejerean, I. Katsavounidis, H. Tmar, and A. C. Bovik, “Cut-FUNQUE: Objective quality assessment of compressed and tone mapped high dynamic range videos,” Manuscript Under Preparation, vol. 1, 2024.
  73. A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-Reference Image Quality Assessment in the Spatial Domain,” IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4695–4708, 2012.
  74. A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013.
  75. A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: From natural scene statistics to perceptual quality,” IEEE Transactions on Image Processing, vol. 20, no. 12, pp. 3350–3364, 2011.
  76. K. Gu, S. Wang, G. Zhai, S. Ma, X. Yang, W. Lin, W. Zhang, and W. Gao, “Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure,” IEEE Transactions on Multimedia, vol. 18, no. 3, pp. 432–443, 2016.
  77. C. S. Ravuri, R. Sureddi, S. V. R. Dendi, S. Raman, and S. S. Channappayya, “Deep no-reference tone mapped image quality assessment,” in 2019 53rd Asilomar Conference on Signals, Systems, and Computers.   IEEE, 2019, pp. 1906–1910.
  78. Q. He, D. Li, T. Jiang, and M. Jiang, “Quality Assessment for Tone-Mapped HDR Images Using Multi-Scale and Multi-Layer Information,” in 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018, pp. 1–6.
  79. A. K. Venkataramanan, C. Stejerean, and A. C. Bovik, “FUNQUE: Fusion of Unified Quality Evaluators,” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 2147–2151.
  80. M. Delacre, D. Lakens, and C. Leys, “Why psychologists should by default use welch’s t-test instead of student’s t-test,” International Review of Social Psychology, vol. 30, no. 1, pp. 92–101, 2017.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now