Encoder-Quantization-Motion-based Video Quality Metrics (2404.06620v1)
Abstract: In an adaptive bitrate streaming application, the efficiency of video compression and the encoded video quality depend on both the video codec and the quality metric used to perform encoding optimization. The development of such a quality metric need large scale subjective datasets. In this work we merge several datasets into one to support the creation of a metric tailored for video compression and scaling. We proposed a set of HEVC lightweight features to boost performance of the metrics. Our metrics can be computed from tightly coupled encoding process with 4% compute overhead or from the decoding process in real-time. The proposed method can achieve better correlation than VMAF and P.1204.3. It can extrapolate to different dynamic ranges, and is suitable for real-time video quality metrics delivery in the bitstream. The performance is verified by in-distribution and cross-dataset tests. This work paves the way for adaptive client-side heuristics, real-time segment optimization, dynamic bitrate capping, and quality-dependent post-processing neural network switching, etc.
- Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
- Z. Wang, E. Simoncelli, and A. Bovik, “Multiscale structural similarity for image quality assessment,” in The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, vol. 2, 2003, pp. 1398–1402 Vol.2.
- A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013.
- Z. Li, A. Aaron, I. Katsavounidis, A. Moorthy, and M. Manohara, “Toward a practical perceptual video quality metric,” The Netflix Tech Blog, vol. 6, no. 2, 2016.
- A. Raake, S. Borer, S. M. Satti, J. Gustafsson, R. R. R. Rao, S. Medagli, P. List, S. Göring, D. Lindero, W. Robitza, G. Heikkilä, S. Broom, C. Schmidmer, B. Feiten, U. Wüstenhagen, T. Wittmann, M. Obermann, and R. Bitto, “Multi-model standard for bitstream-, pixel-based and hybrid video quality assessment of uhd/4k: Itu-t p.1204,” IEEE Access, vol. 8, pp. 193 020–193 049, 2020.
- Z. Shang, J. P. Ebenezer, A. C. Bovik, Y. Wu, H. Wei, and S. Sethuraman, “Subjective assessment of high dynamic range videos under different ambient conditions,” 2022.
- Z. Shang, J. Ebenezer, Y. Chen, Y. Wu, H. Wei, S. Sethuraman, and A. Bovik, “A subjective and objective study of adaptive quantization of hdr videos,” Submitted to IEEE Transactions on Circuits and Systems for Video Technology, 2024.
- Z. Shang, Y. Chen, Y. Wu, H. Wei, and S. Sethuraman, “Subjective and objective video quality assessment of high dynamic range sports content,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) Workshops, January 2023, pp. 556–564.
- J. P. Ebenezer, Z. Shang, Y. Chen, Y. Wu, H. Wei, S. Sethuraman, and A. C. Bovik, “Hdr or sdr? a subjective and objective study of scaled and compressed videos,” 2023.
- J. P. Ebenezer, Y. Chen, Y. Wu, H. Wei, and S. Sethuraman, “Subjective and objective quality assessment of high-motion sports videos at low-bitrates,” in 2022 IEEE International Conference on Image Processing (ICIP), 2022, pp. 521–525.
- R. R. Ramachandra Rao, S. Göring, W. Robitza, B. Feiten, and A. Raake, “Avt-vqdb-uhd-1: A large scale video quality database for uhd-1,” in 2019 IEEE International Symposium on Multimedia (ISM), 2019, pp. 17–177.
- T. Installations and L. Line, “Itu-tp. 910,” Subjective video quality assessment methods for multimedia applications, Recommendation ITU-T, 2023.
- Z. Li, C. G. Bampis, L. Krasula, L. Janowski, and I. Katsavounidis, “A simple model for subject behavior in subjective experiments,” 2020. [Online]. Available: https://arxiv.org/abs/2004.02067
- P. Pérez, “The transmission rating scale and its relation to subjective scores,” in 2023 15th International Conference on Quality of Multimedia Experience (QoMEX), 2023, pp. 31–36.
- M. Pérez-Ortiz, A. Mikhailiuk, E. Zerman, V. Hulusic, G. Valenzise, and R. K. Mantiuk, “From pairwise comparisons and rating to a unified quality scale,” IEEE Transactions on Image Processing, vol. 29, pp. 1139–1151, 2020.
- Y. Pitrey, U. Engelke, M. Barkowsky, R. Pépion, and P. Le Callet, “Aligning subjective tests using a low cost common set,” in Euro ITV, Lisbonne, Portugal, Jun. 2011, p. irccyn contribution. [Online]. Available: https://hal.science/hal-00608310
- S. Li, F. Zhang, L. Ma, and K. N. Ngan, “Image quality assessment by separately evaluating detail losses and additive impairments,” IEEE Transactions on Multimedia, vol. 13, no. 5, pp. 935–949, 2011.