Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deep Bi-directional Attention Network for Image Super-Resolution Quality Assessment (2403.10406v2)

Published 15 Mar 2024 in cs.MM

Abstract: There has emerged a growing interest in exploring efficient quality assessment algorithms for image super-resolution (SR). However, employing deep learning techniques, especially dual-branch algorithms, to automatically evaluate the visual quality of SR images remains challenging. Existing SR image quality assessment (IQA) metrics based on two-stream networks lack interactions between branches. To address this, we propose a novel full-reference IQA (FR-IQA) method for SR images. Specifically, producing SR images and evaluating how close the SR images are to the corresponding HR references are separate processes. Based on this consideration, we construct a deep Bi-directional Attention Network (BiAtten-Net) that dynamically deepens visual attention to distortions in both processes, which aligns well with the human visual system (HVS). Experiments on public SR quality databases demonstrate the superiority of our proposed BiAtten-Net over state-of-the-art quality assessment methods. In addition, the visualization results and ablation study show the effectiveness of bi-directional attention.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160, 1981.
  2. “New edge-directed interpolation,” IEEE TIP, vol. 10, no. 10, pp. 1521–1527, 2001.
  3. “An edge-guided image interpolation algorithm via directional filtering and data fusion,” IEEE TIP, vol. 15, no. 8, pp. 2226–2238, 2006.
  4. Image Interpolation and Resampling, p. 393–420, Academic Press, Inc., 2000.
  5. “Single-image super-resolution based on rational fractal interpolation,” IEEE TIP, vol. 27, no. 8, pp. 3782–3797, 2018.
  6. “Image super-resolution via sparse representation,” IEEE TIP, vol. 19, no. 11, pp. 2861–2873, 2010.
  7. “Image super-resolution as sparse representation of raw image patches,” in CVPR, 2008, pp. 1–8.
  8. “Deep neural networks for no-reference and full-reference image quality assessment,” IEEE TIP, vol. 27, no. 1, pp. 206–219, 2018.
  9. “The unreasonable effectiveness of deep features as a perceptual metric,” in CVPR, 2018.
  10. “MetaIQA: Deep meta-learning for no-reference image quality assessment,” in CVPR, 2020.
  11. “Blindly assess image quality in the wild guided by a self-adaptive hyper network,” in CVPR, 2020.
  12. “Image super-resolution using deep convolutional networks,” IEEE PAMI, vol. 38, no. 2, pp. 295–307, 2016.
  13. “Image super-resolution using dense skip connections,” in ICCV, 2017.
  14. “Photo-realistic single image super-resolution using a generative adversarial network,” in CVPR, 2017, pp. 105–114.
  15. “ESRGAN: Enhanced super-resolution generative adversarial networks,” in ECCV, 2019.
  16. “Image quality assessment: from error visibility to structural similarity,” IEEE TIP, vol. 13, no. 4, pp. 600–612, 2004.
  17. “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE TIP, vol. 14, no. 12, pp. 2117–2128, 2005.
  18. “Gradient magnitude similarity deviation: A highly efficient perceptual image quality index,” IEEE TIP, vol. 23, no. 2, pp. 684–695, 2014.
  19. “A deep learning based no-reference image quality assessment model for single-image super-resolution,” in ICASSP, 2018, pp. 1223–1227.
  20. “Attentions help cnns see better: Attention-based hybrid image quality assessment network,” in CVPR Workshops, 2022.
  21. “Madnet: A fast and lightweight network for single-image super resolution,” IEEE Transactions on Cybernetics, vol. 51, no. 3, pp. 1443–1453, 2021.
  22. “Self-attention negative feedback network for real-time image super-resolution,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 8, pp. 6179–6186, 2022.
  23. “Complex wavelet structural similarity: A new image similarity index,” IEEE TIP, vol. 18, no. 11, pp. 2385–2401, 2009.
  24. “Multiscale structural similarity for image quality assessment,” in ACSSC, 2003.
  25. “Fsim: A feature similarity index for image quality assessment,” IEEE TIP, vol. 20, no. 8, pp. 2378–2386, 2011.
  26. “Visual quality assessment for super-resolved images: Database and method,” IEEE TIP, vol. 28, no. 7, pp. 3528–3541, 2019.
  27. “Image super-resolution quality assessment: Structural fidelity versus statistical naturalness,” in QoMEX, 2021.
  28. “Quality assessment of image super-resolution: Balancing deterministic and statistical fidelity,” in ACM Multimedia, 2022.
  29. “Blind quality assessment for image superresolution using deep two-stream convolutional networks,” Information Sciences, vol. 528, pp. 205–218, 2020.
  30. “Super-resolution image visual quality assessment based on structure–texture features,” Signal Processing: Image Communication, vol. 117, pp. 117025, 2023.
  31. “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE TIP, vol. 15, no. 11, pp. 3440–3451, 2006.
  32. “Color image database TID2013: Peculiarities and preliminary results,” in EUVIP, 2013.
  33. “PIPAL: A large-scale image quality assessment dataset for perceptual image restoration,” in ECCV, 2020.
  34. “Learning-based quality assessment for image super-resolution,” IEEE TMM, vol. 24, pp. 3570–3581, 2021.
  35. “Image quality assessment: Unifying structure and texture similarity,” IEEE TPAMI, vol. 44, no. 5, pp. 2567–2581, 2022.
  36. “Blind image quality assessment using a deep bilinear convolutional neural network,” IEEE TCSVT, vol. 30, no. 1, pp. 36–47, 2020.
  37. “A saliency dispersion measure for improving saliency-based image quality metrics,” IEEE TCSVT, vol. 28, no. 6, pp. 1462–1466, 2018.
  38. “Saliency-based deep convolutional neural network for no-reference image quality assessment,” Multimedia Tools and Applications, vol. 77, pp. 14859–14872, 2018.
  39. “Joint channel-spatial attention network for super-resolution image quality assessment,” Appl. Intell, vol. 52, pp. 17118–17132, 2022.
  40. “TADSRNet: A triple-attention dual-scale residual network for super-resolution image quality assessment,” Appl. Intell, vol. 53, pp. 26708–26724, 2023.
  41. “SGDNet: An end-to-end saliency-guided deep neural network for no-reference image quality assessment,” in ACM Multimedia, 2019.
  42. “Boosting no-reference super-resolution image quality assessment with knowledge distillation and extension,” in ICASSP, 2023.
  43. “A no-reference deep learning quality assessment method for super-resolution images based on frequency maps,” in ISCAS, 2022.
  44. “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929, 2020.
  45. “Swin transformer: Hierarchical vision transformer using shifted windows,” in ICCV, 2021, pp. 9992–10002.
  46. “Deep residual learning for image recognition,” in CVPR, 2016.
  47. “Learning a no-reference quality metric for single-image super-resolution,” CVIU, vol. 158, pp. 1–16, 2017.
  48. “No-reference image quality assessment in the spatial domain,” IEEE TIP, vol. 21, no. 12, pp. 4695–4708, 2012.
  49. “Making a “completely blind” image quality analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013.
  50. “A highly efficient method for blind image quality assessment,” in ICIP, 2015.
  51. “A reduced-reference quality assessment metric for super-resolution reconstructed images with information gain and texture similarity,” Signal Processing: Image Communication, vol. 79, pp. 32–39, 2019.
  52. “Encoding distortions for multi-task full-reference image quality assessment,” in ICME, 2019.
  53. “Image quality assessment using kernel sparse coding,” IEEE TMM, vol. 23, pp. 1592–1604, 2021.
  54. “Super-resolution quality assessment: Subjective evaluation database and quality index based on perceptual structure measurement,” in ICME, 2018.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yixiao Li (14 papers)
  2. Xiaoyuan Yang (16 papers)
  3. Jun Fu (28 papers)
  4. Guanghui Yue (9 papers)
  5. Wei Zhou (311 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now