Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Diffusion Model Based Visual Compensation Guidance and Visual Difference Analysis for No-Reference Image Quality Assessment (2402.14401v1)

Published 22 Feb 2024 in cs.CV, cs.LG, and eess.IV

Abstract: Existing free-energy guided No-Reference Image Quality Assessment (NR-IQA) methods still suffer from finding a balance between learning feature information at the pixel level of the image and capturing high-level feature information and the efficient utilization of the obtained high-level feature information remains a challenge. As a novel class of state-of-the-art (SOTA) generative model, the diffusion model exhibits the capability to model intricate relationships, enabling a comprehensive understanding of images and possessing a better learning of both high-level and low-level visual features. In view of these, we pioneer the exploration of the diffusion model into the domain of NR-IQA. Firstly, we devise a new diffusion restoration network that leverages the produced enhanced image and noise-containing images, incorporating nonlinear features obtained during the denoising process of the diffusion model, as high-level visual information. Secondly, two visual evaluation branches are designed to comprehensively analyze the obtained high-level feature information. These include the visual compensation guidance branch, grounded in the transformer architecture and noise embedding strategy, and the visual difference analysis branch, built on the ResNet architecture and the residual transposed attention block. Extensive experiments are conducted on seven public NR-IQA datasets, and the results demonstrate that the proposed model outperforms SOTA methods for NR-IQA.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Z. Pan, F. Yuan, J. Lei, Y. Fang, X. Shao, and S. Kwong, “Vcrnet: Visual compensation restoration network for no-reference image quality assessment,” IEEE Trans. Image Process., vol. 31, pp. 1613–1627, Jan. 2022.
  2. H. R. Sheikh, M. F. Sabir, and A. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Trans. Image Process., vol. 15, no. 11, pp. 3440–3451, Oct. 2006.
  3. D. Ghadiyaram and A. C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” IEEE Trans. Image Process., vol. 25, no. 1, pp. 372–387, Nov. 2015.
  4. V. Hosu, H. Lin, T. Sziranyi, and D. Saupe, “Koniq-10k: An ecologically valid database for deep learning of blind image quality assessment,” IEEE Trans. Image Process., vol. 29, pp. 4041–4056, Jan. 2020.
  5. Y. Fang, J. Yan, L. Li, J. Wu, and W. Lin, “No reference quality assessment for screen content images with both local and global feature representation,” IEEE Trans. Image Process., vol. 27, no. 4, pp. 1600–1610, Dec. 2017.
  6. Y. Fang, K. Ma, Z. Wang, W. Lin, Z. Fang, and G. Zhai, “No-reference quality assessment of contrast-distorted images based on natural scene statistics,” IEEE Signal Process. Letters, vol. 22, no. 7, pp. 838–842, Nov. 2014.
  7. Q. Li, W. Lin, J. Xu, and Y. Fang, “Blind image quality assessment using statistical structural and luminance features,” IEEE Trans. Multimedia, vol. 18, no. 12, pp. 2457–2469, Aug. 2016.
  8. A. K. Moorthy and A. C. Bovik, “Blind image quality assessment: From natural scene statistics to perceptual quality,” IEEE Trans. Image Process., vol. 20, no. 12, pp. 3350–3364, Apr. 2011.
  9. A. Mittal, A. K. Moorthy, and A. C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695–4708, Aug. 2012.
  10. C. Li, A. C. Bovik, and X. Wu, “Blind image quality assessment using a general regression neural network,” IEEE Trans. Neural Netw., vol. 22, no. 5, pp. 793–799, Apr. 2011.
  11. P. Ye, J. Kumar, L. Kang, and D. Doermann, “Unsupervised feature learning framework for no-reference image quality assessment,” in 2012 IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 1098–1105.
  12. J. Xu, P. Ye, Q. Li, H. Du, Y. Liu, and D. Doermann, “Blind image quality assessment based on high order statistics aggregation,” IEEE Trans. Image Process., vol. 25, no. 9, pp. 4444–4457, Jun. 2016.
  13. L. Xu, J. Li, W. Lin, Y. Zhang, L. Ma, Y. Fang, and Y. Yan, “Multi-task rank learning for image quality assessment,” IEEE Trans. Circuits Syst. Video Technol., vol. 27, no. 9, pp. 1833–1843, Mar. 2016.
  14. K. Gu, D. Tao, J.-F. Qiao, and W. Lin, “Learning a no-reference quality assessment model of enhanced images with big data,” IEEE Trans. Neural Netw. Trans. Neural Netw. Syst., vol. 29, no. 4, pp. 1301–1313, Mar. 2017.
  15. L. Kang, P. Ye, Y. Li, and D. Doermann, “Convolutional neural networks for no-reference image quality assessment,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jan. 2014, pp. 1733–1740.
  16. W. Zhang, K. Ma, J. Yan, D. Deng, and Z. Wang, “Blind image quality assessment using a deep bilinear convolutional neural network,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 1, pp. 36–47, Jan. 2018.
  17. S. Su, Q. Yan, Y. Zhu, C. Zhang, X. Ge, J. Sun, and Y. Zhang, “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jan. 2020, pp. 3667–3676.
  18. H. Chen, X. Chai, F. Shao, X. Wang, Q. Jiang, M. Chao, and Y.-S. Ho, “Perceptual quality assessment of cartoon images,” IEEE Trans. Multimedia, vol. 25, pp. 140–153, Oct. 2021.
  19. W. Zhou, J. Xu, Q. Jiang, and Z. Chen, “No-reference quality assessment for 360-degree images by analysis of multifrequency information and local-global naturalness,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 4, pp. 1778–1791, May. 2021.
  20. J. Xu, W. Zhou, H. Li, F. Li, and Q. Jiang, “Quality assessment of multi-exposure image fusion by synthesizing local and global intermediate references,” Displays, vol. 74, p. 102188, Sep. 2022.
  21. Q. Jiang, F. Shao, W. Lin, K. Gu, G. Jiang, and H. Sun, “Optimizing multistage discriminative dictionaries for blind image quality assessment,” IEEE Trans. Multimedia, vol. 20, no. 8, pp. 2035–2048, Oct. 2017.
  22. H. Ren, D. Chen, and Y. Wang, “Ran4iqa: Restorative adversarial nets for no-reference image quality assessment,” in Proc. AAAI Conf. artificial Intell., vol. 32, no. 1, Aug. 2018.
  23. K.-Y. Lin and G. Wang, “Hallucinated-iqa: No-reference image quality assessment via adversarial trans. neural netw.ing,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Apr. 2018, pp. 732–741.
  24. S. Sun, T. Yu, J. Xu, W. Zhou, and Z. Chen, “Graphiqa: Learning distortion graph representations for blind image quality assessment,” IEEE Trans. Multimedia, Jan 2022.
  25. J. Ke, Q. Wang, Y. Wang, P. Milanfar, and F. Yang, “Musiq: Multi-scale image quality transformer,” in Proc. IEEE/CVF International Conf. Comput. Vis. (ICCV), Aug. 2021, pp. 5148–5157.
  26. S. A. Golestaneh, S. Dadsetan, and K. M. Kitani, “No-reference image quality assessment via transformers, relative ranking, and self-consistency,” in Proc. IEEE/CVF Winter Conf. Applications of Comput. Vis., Jan. 2022, pp. 1220–1230.
  27. S. Yang, T. Wu, S. Shi, S. Lao, Y. Gong, M. Cao, J. Wang, and Y. Yang, “Maniqa: Multi-dimension attention network for no-reference image quality assessment,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Apr. 2022, pp. 1191–1200.
  28. K. Zhao, K. Yuan, M. Sun, M. Li, and X. Wen, “Quality-aware pre-trained models for blind image quality assessment,” Mar. 2023.
  29. P. C. Madhusudana, N. Birkbeck, Y. Wang, B. Adsumilli, and A. C. Bovik, “Image quality assessment using contrastive learning,” IEEE Trans. Image Process., vol. 31, pp. 4149–4161, Jun. 2022.
  30. S. Roy, S. Mitra, S. Biswas, and R. Soundararajan, “Test time adaptation for blind image quality assessment,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), Aug. 2023, pp. 16 742–16 751.
  31. J. Wang, K. C. Chan, and C. C. Loy, “Exploring clip for assessing the look and feel of images,” in Proc. AAAI Conf. Artificial Intell., vol. 37, no. 2, Jun. 2023, pp. 2555–2563.
  32. W. Zhang, G. Zhai, Y. Wei, X. Yang, and K. Ma, “Blind image quality assessment via vision-language correspondence: A multitask learning perspective,” in Proc. IEEE/CVF Conf. Comput. Vis. and Pattern Recognit. (CVPR), Jan. 2023, pp. 14 071–14 081.
  33. Ž. Babnik, P. Peer, and V. Štruc, “Diffiqa: Face image quality assessment using denoising diffusion probabilistic models,” arXiv preprint arXiv:2305.05768, May. 2023.
  34. J. Kim, H. Zeng, D. Ghadiyaram, S. Lee, L. Zhang, and A. C. Bovik, “Deep convolutional neural models for picture-quality prediction: Challenges and solutions to data-driven image quality assessment,” IEEE Signal Process. Mag., vol. 34, no. 6, pp. 130–141, Nov. 2017.
  35. S. Bosse, D. Maniry, K.-R. Müller, T. Wiegand, and W. Samek, “Deep neural networks for no-reference and full-reference image quality assessment,” IEEE Trans. Image Process., vol. 27, no. 1, pp. 206–219, Oct. 2017.
  36. S. Bianco, L. Celona, P. Napoletano, and R. Schettini, “On the use of deep learning for blind image quality assessment,” Signal, Image and Video Process., vol. 12, pp. 355–362, Aug. 2018.
  37. Q. Yan, D. Gong, and Y. Zhang, “Two-stream convolutional networks for blind image quality assessment,” IEEE Trans. Image Process., vol. 28, no. 5, pp. 2200–2211, Nov. 2018.
  38. L.-M. Po, M. Liu, W. Y. Yuen, Y. Li, X. Xu, C. Zhou, P. H. Wong, K. W. Lau, and H.-T. Luk, “A novel patch variance biased convolutional neural network for no-reference image quality assessment,” IEEE Trans. Circuits Syst. Video Technol., vol. 29, no. 4, pp. 1223–1229, Jan. 2019.
  39. X. Liu, J. Van De Weijer, and A. D. Bagdanov, “Rankiqa: Learning from rankings for no-reference image quality assessment,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Jul. 2017, pp. 1040–1049.
  40. K. Ma, W. Liu, K. Zhang, Z. Duanmu, Z. Wang, and W. Zuo, “End-to-end blind image quality assessment using deep neural networks,” IEEE Trans. Image Process., vol. 27, no. 3, pp. 1202–1213, Nov. 2017.
  41. X. Chen, H. Fan, R. Girshick, and K. He, “Improved baselines with momentum contrastive learning,” arXiv preprint arXiv:2003.04297, Mar. 2020.
  42. C. Chen, J. Mo, J. Hou, H. Wu, L. Liao, W. Sun, Q. Yan, and W. Lin, “Topiq: A top-down approach from semantics to distortions for image quality assessment,” arXiv preprint arXiv:2308.03060, Aug. 2023.
  43. J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Process. Syst., H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33.   Curran Associates, Inc., 2020, pp. 6840–6851. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/4c5bcfec8584af0d967f1ab10179ca4b-Paper.pdf
  44. A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in Proc. 38th International Conf. Machine Learning, ser. Proceedings of Machine Learning Research, M. Meila and T. Zhang, Eds., vol. 139.   PMLR, 18–24 Jul Apr. 2021, pp. 8162–8171. [Online]. Available: https://proceedings.mlr.press/v139/nichol21a.html
  45. H. Li, Y. Yang, M. Chang, S. Chen, H. Feng, Z. Xu, Q. Li, and Y. Chen, “Srdiff: Single image super-resolution with diffusion probabilistic models,” Neurocomputing, vol. 479, pp. 47–59, Mar. 2022.
  46. C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet, and M. Norouzi, “Image super-resolution via iterative refinement,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4713–4726, Sep. 2022.
  47. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Apr. 2022, pp. 10 684–10 695.
  48. A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, Jun. 2020.
  49. E. C. Larson and D. M. Chandler, “Most apparent distortion: full-reference image quality assessment and the role of strategy,” Journal of electronic imaging, vol. 19, no. 1, pp. 011 006–011 006, Jan. 2010.
  50. N. Ponomarenko, L. Jin, O. Ieremeiev, V. Lukin, K. Egiazarian, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti et al., “Image database tid2013: Peculiarities, results and perspectives,” Signal Process.: Image Commun., vol. 30, pp. 57–77, Jan. 2015.
  51. H. Lin, V. Hosu, and D. Saupe, “Kadid-10k: A large-scale artificially distorted iqa database,” in 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX).   IEEE, Jun. 2019, pp. 1–3.
  52. J. Gu, H. Cai, H. Chen, X. Ye, J. Ren, and C. Dong, “Image quality assessment for perceptual image restoration: A new dataset, benchmark and metric,” arXiv preprint arXiv:2011.15002, Nov. 2020.
  53. A. Saha, S. Mishra, and A. C. Bovik, “Re-iqa: Unsupervised learning for image quality assessment in the wild,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Jan. 2023, pp. 5846–5855.
  54. G. Qin, R. Hu, Y. Liu, X. Zheng, H. Liu, X. Li, and Y. Zhang, “Data-efficient image quality assessment with attention-panel decoder,” arXiv preprint arXiv:2304.04952, Apr. 2023.
  55. S. Shi, Q. Bai, M. Cao, W. Xia, J. Wang, Y. Chen, and Y. Yang, “Region-adaptive deformable network for image quality assessment,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Apr. 2021, pp. 324–333.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Zhaoyang Wang (33 papers)
  2. Bo Hu (110 papers)
  3. Mingyang Zhang (56 papers)
  4. Jie Li (553 papers)
  5. Leida Li (26 papers)
  6. Maoguo Gong (36 papers)
  7. Xinbo Gao (194 papers)
Citations (1)
View on Semantic Scholar