Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning with Noisy Low-Cost MOS for Image Quality Assessment via Dual-Bias Calibration (2311.15846v1)

Published 27 Nov 2023 in cs.CV and eess.IV

Abstract: Learning based image quality assessment (IQA) models have obtained impressive performance with the help of reliable subjective quality labels, where mean opinion score (MOS) is the most popular choice. However, in view of the subjective bias of individual annotators, the labor-abundant MOS (LA-MOS) typically requires a large collection of opinion scores from multiple annotators for each image, which significantly increases the learning cost. In this paper, we aim to learn robust IQA models from low-cost MOS (LC-MOS), which only requires very few opinion scores or even a single opinion score for each image. More specifically, we consider the LC-MOS as the noisy observation of LA-MOS and enforce the IQA model learned from LC-MOS to approach the unbiased estimation of LA-MOS. In this way, we represent the subjective bias between LC-MOS and LA-MOS, and the model bias between IQA predictions learned from LC-MOS and LA-MOS (i.e., dual-bias) as two latent variables with unknown parameters. By means of the expectation-maximization based alternating optimization, we can jointly estimate the parameters of the dual-bias, which suppresses the misleading of LC-MOS via a gated dual-bias calibration (GDBC) module. To the best of our knowledge, this is the first exploration of robust IQA model learning from noisy low-cost labels. Theoretical analysis and extensive experiments on four popular IQA datasets show that the proposed method is robust toward different bias rates and annotation numbers and significantly outperforms the other learning based IQA models when only LC-MOS is available. Furthermore, we also achieve comparable performance with respect to the other models learned with LA-MOS.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. G. Zhai and X. Min, “Perceptual image quality assessment: a survey,” Science China Information Sciences, vol. 63, no. 11, pp. 1–52, 2020.
  2. Y. Deng, C. C. Loy, and X. Tang, “Image aesthetic assessment: An experimental survey,” IEEE Signal Processing Magazine, vol. 34, no. 4, pp. 80–106, 2017.
  3. H. Lin, V. Hosu, and D. Saupe, “Kadid-10k: A large-scale artificially distorted iqa database,” in 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).   IEEE, 2019, pp. 1–3.
  4. D. Kundu, D. Ghadiyaram, A. C. Bovik, and B. L. Evans, “Large-scale crowdsourced study for tone-mapped hdr pictures,” IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4725–4740, 2017.
  5. A. Zaric, N. Tatalovic, N. Brajkovic, H. Hlevnjak, M. Loncaric, E. Dumic, and S. Grgic, “Vcl@ fer image quality assessment database,” in Proceedings ELMAR-2011.   IEEE, 2011, pp. 105–110.
  6. T. Song, L. Li, J. Wu, Y. Yang, Y. Li, Y. Guo, and G. Shi, “Knowledge-guided blind image quality assessment with few training samples,” IEEE Transactions on Multimedia, 2022.
  7. L. Wang, Q. Wu, K. N. Ngan, H. Li, F. Meng, and L. Xu, “Blind tone-mapped image quality assessment and enhancement via disentangled representation learning,” in 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).   IEEE, 2020, pp. 1096–1102.
  8. ITU-R BT.500, “Methodologies for the subjective assessment of the quality of television images,” https://www.itu.int/rec/R-REC-BT.500.
  9. ITU-T P.910, “Subjective video quality assessment methods for multimedia applications,” https://www.itu.int/rec/T-REC-P.910.
  10. ITU-T P.913, “Methods for the subjective assessment of video quality audio quality and audiovisual quality of internet video and distribution quality television in any environment,” https://www.itu.int/rec/T-REC-P.913.
  11. K. Ma, W. Liu, K. Zhang, Z. Duanmu, Z. Wang, and W. Zuo, “End-to-end blind image quality assessment using deep neural networks,” IEEE Transactions on Image Processing, vol. 27, no. 3, pp. 1202–1213, 2017.
  12. W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, “Blind image quality assessment using a deep bilinear convolutional neural network,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 1, pp. 36–47, 2018.
  13. H. Talebi and P. Milanfar, “Nima: Neural image assessment,” IEEE transactions on image processing, vol. 27, no. 8, pp. 3998–4011, 2018.
  14. S. Su, Q. Yan, Y. Zhu, C. Zhang, X. Ge, J. Sun, and Y. Zhang, “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3667–3676.
  15. W. Zhang, K. Ma, G. Zhai, and X. Yang, “Uncertainty-aware blind image quality assessment in the laboratory and wild,” IEEE Transactions on Image Processing, vol. 30, pp. 3474–3486, 2021.
  16. S. Sun, T. Yu, J. Xu, W. Zhou, and Z. Chen, “Graphiqa: Learning distortion graph representations for blind image quality assessment,” IEEE Transactions on Multimedia, 2022.
  17. H. Zhu, L. Li, J. Wu, W. Dong, and G. Shi, “Metaiqa: Deep meta-learning for no-reference image quality assessment,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 14 143–14 152.
  18. J. Ma, J. Wu, L. Li, W. Dong, X. Xie, G. Shi, and W. Lin, “Blind image quality assessment with active inference,” IEEE Transactions on Image Processing, vol. 30, pp. 3650–3663, 2021.
  19. W. Zhang, D. Li, C. Ma, G. Zhai, X. Yang, and K. Ma, “Continual learning for blind image quality assessment,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
  20. W. Zhang, G. Zhai, Y. Wei, X. Yang, and K. Ma, “Blind image quality assessment via vision-language correspondence: A multitask learning perspective,” arXiv preprint arXiv:2303.14968, 2023.
  21. W. Zhang, D. Li, X. Min, G. Zhai, G. Guo, X. Yang, and K. Ma, “Perceptual attacks of no-reference image quality models with human-in-the-loop,” arXiv preprint arXiv:2210.00933, 2022.
  22. H. Duan, G. Zhai, X. Min, Y. Zhu, Y. Fang, and X. Yang, “Perceptual quality assessment of omnidirectional images,” in 2018 IEEE international symposium on circuits and systems (ISCAS).   IEEE, 2018, pp. 1–5.
  23. W. Sun, X. Min, G. Zhai, K. Gu, H. Duan, and S. Ma, “Mc360iqa: A multi-channel cnn for blind 360-degree image quality assessment,” IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 1, pp. 64–77, 2019.
  24. V. K. Adhikarla, M. Vinkler, D. Sumin, R. Mantiuk, K. Myszkowski, H.-P. Seidel, and P. Didyk, “Towards a quality metric for dense light fields,” in Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017.
  25. X. Min, G. Zhai, K. Gu, X. Yang, and X. Guan, “Objective quality evaluation of dehazed images,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 8, pp. 2879–2892, 2018.
  26. X. Min, G. Zhai, K. Gu, Y. Zhu, J. Zhou, G. Guo, X. Yang, X. Guan, and W. Zhang, “Quality evaluation of image dehazing methods using synthetic hazy images,” IEEE Transactions on Multimedia, vol. 21, no. 9, pp. 2319–2333, 2019.
  27. Y. Fang, H. Zhu, Y. Zeng, K. Ma, and Z. Wang, “Perceptual quality assessment of smartphone photography,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3677–3686.
  28. D. Arpit, S. Jastrzebski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio et al., “A closer look at memorization in deep networks,” in International conference on machine learning.   PMLR, 2017, pp. 233–242.
  29. A. Ghosh, H. Kumar, and P. Sastry, “Robust loss functions under label noise for deep neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 31, no. 1, 2017.
  30. D. Tanaka, D. Ikami, T. Yamasaki, and K. Aizawa, “Joint optimization framework for learning with noisy labels,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5552–5560.
  31. J. Goldberger and E. Ben-Reuven, “Training deep neural-networks using a noise adaptation layer,” 2016.
  32. X. Xia, T. Liu, B. Han, M. Gong, J. Yu, G. Niu, and M. Sugiyama, “Sample selection with uncertainty of losses for learning with noisy labels,” arXiv preprint arXiv:2106.00445, 2021.
  33. C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning (still) requires rethinking generalization,” Communications of the ACM, vol. 64, no. 3, pp. 107–115, 2021.
  34. Z. Zhang and M. Sabuncu, “Generalized cross entropy loss for training deep neural networks with noisy labels,” Advances in neural information processing systems, vol. 31, 2018.
  35. X. Zhou, X. Liu, C. Wang, D. Zhai, J. Jiang, and X. Ji, “Learning with noisy labels via sparse regularization,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 72–81.
  36. S. Hahn and H. Choi, “Self-knowledge distillation in natural language processing,” in Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), 2019, pp. 423–430.
  37. A. Gotmare, N. S. Keskar, C. Xiong, and R. Socher, “A closer look at deep learning heuristics: Learning rate restarts, warmup and distillation,” in International Conference on Learning Representations, 2019.
  38. Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey, “Symmetric cross entropy for robust learning with noisy labels,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 322–330.
  39. H. Zhuang and J. Young, “Leveraging in-batch annotation bias for crowdsourced active learning,” in Proceedings of the Eighth ACM International Conference on Web Search and Data Mining, 2015, pp. 243–252.
  40. C. Hube, B. Fetahu, and U. Gadiraju, “Understanding and mitigating worker biases in the crowdsourced collection of subjective judgments,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019, pp. 1–12.
  41. G. Li, J. Wang, Y. Zheng, and M. J. Franklin, “Crowdsourced data management: A survey,” IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 9, pp. 2296–2319, 2016.
  42. T. Virtanen, M. Nuutinen, M. Vaahteranoksa, P. Oittinen, and J. Häkkinen, “Cid2013: A database for evaluating no-reference image quality assessment algorithms,” IEEE Transactions on Image Processing, vol. 24, no. 1, pp. 390–402, 2014.
  43. Z. Li, C. G. Bampis, L. Krasula, L. Janowski, and I. Katsavounidis, “A simple model for subject behavior in subjective experiments,” arXiv preprint arXiv:2004.02067, 2020.
  44. T. Moon, “The expectation-maximization algorithm,” IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 47–60, 1996.
  45. L. Yang, H. Li, F. Meng, Q. Wu, and K. N. Ngan, “Task-specific loss for robust instance segmentation with noisy class labels,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 1, pp. 213–227, 2023.
  46. N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti et al., “Image database tid2013: Peculiarities, results and perspectives,” Signal processing: Image communication, vol. 30, pp. 57–77, 2015.
  47. D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 372–387, 2015.
  48. V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, “Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment,” IEEE Transactions on Image Processing, vol. 29, pp. 4041–4056, 2020.
  49. Q. Wu, H. Li, F. Meng, and K. N. Ngan, “A perceptually weighted rank correlation indicator for objective image quality assessment,” IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2499–2513, 2018.
  50. Q. Wu, H. Li, K. N. Ngan, and K. Ma, “Blind image quality assessment using local consistency aware retriever and uncertainty aware evaluator,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 9, pp. 2078–2089, 2018.
  51. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  52. J. Benesty, J. Chen, Y. Huang, and I. Cohen, “Pearson correlation coefficient,” in Noise reduction in speech processing.   Springer, 2009, pp. 1–4.
  53. J. H. Zar, “Spearman rank correlation,” Encyclopedia of biostatistics, vol. 7, 2005.
  54. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  55. I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983, 2016.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now