2000 character limit reached
VMAF Re-implementation on PyTorch: Some Experimental Results (2310.15578v4)
Published 24 Oct 2023 in cs.LG and cs.CV
Abstract: Based on the standard VMAF implementation we propose an implementation of VMAF using PyTorch framework. For this implementation comparisons with the standard (libvmaf) show the discrepancy $\lesssim 10{-2}$ in VMAF units. We investigate gradients computation when using VMAF as an objective function and demonstrate that training using this function does not result in ill-behaving gradients. The implementation is then used to train a preprocessing filter. It is demonstrated that its performance is superior to the unsharp masking filter. The resulting filter is also easy for implementation and can be applied in video processing tasks for video copression improvement. This is confirmed by the results of numerical experiments.
- Toward a practical perceptual video quality metric. [Online]. Available: https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652
- M. Siniukov, A. Antsiferova, D. Kulikov, and D. Vatolin, “Hacking VMAF and VMAF NEG: vulnerability to different preprocessing methods,” 2021.
- [Online]. Available: https://github.com/Netflix/vmaf
- H. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 3646–6565, 2006.
- R. Soundararajan and A. C. Bovik, “Survey of information theory in visual quality assessment,” Signal, Image, and Video Processing, no. 7, pp. 391–401, 2013.
- [Online]. Available: https://live.ece.utexas.edu/research/Quality/VIF.htm
- S. Li, F. Zhang, L. Ma, and K. Ngan, “Image quality assessment by separately evaluating detail losses and additive impairments,” IEEE Transactions on Multimedia, vol. 13, pp. 935–949, 10 2011.
- On VMAF’s property in the presence of image enhancement operations. [Online]. Available: https://docs.google.com/document/d/1dJczEhXO0MZjBSNyKmd3ARiCTdFVMNPBykH4_HMPoyY/edit#heading=h.oaikhnw46pw5
- Toward a better quality metric for the video community. [Online]. Available: https://netflixtechblog.com/toward-a-better-quality-metric-for-the-video-community-7ed94e752a30
- [Online]. Available: http://ufldl.stanford.edu/tutorial/supervised/DebuggingGradientChecking/
- D. Ramsook, A. Kokaram, N. O’Connor, N. Birkbeck, Y. Su, and B. Adsumilli, “A differentiable estimator of VMAF for video,” in 2021 Picture Coding Symposium (PCS), 2021, pp. 1–5.
- A. K. Venkataramanan, C. Stejerean, I. Katsavounidis, and A. C. Bovik, “One transform to compute them all: Efficient fusion-based full-reference video quality assessment,” 2023.
- M. Sedighizad and B. Seyfe, “Gradient of the mutual information in stochastic systems: A functional approach,” IEEE Signal Processing Letters, vol. 26, no. 10, p. 99–112, Oct. 2019.
- L.-H. Chen, C. G. Bampis, Z. Li, A. Norkin, and A. C. Bovik, “ProxIQA: A proxy approach to perceptual optimization of learned image compression,” IEEE Transactions on Image Processing, vol. 30, pp. 360–373, 2021.