Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adapting Pretrained Networks for Image Quality Assessment on High Dynamic Range Displays (2405.00670v1)

Published 1 May 2024 in cs.CV and eess.IV

Abstract: Conventional image quality metrics (IQMs), such as PSNR and SSIM, are designed for perceptually uniform gamma-encoded pixel values and cannot be directly applied to perceptually non-uniform linear high-dynamic-range (HDR) colors. Similarly, most of the available datasets consist of standard-dynamic-range (SDR) images collected in standard and possibly uncontrolled viewing conditions. Popular pre-trained neural networks are likewise intended for SDR inputs, restricting their direct application to HDR content. On the other hand, training HDR models from scratch is challenging due to limited available HDR data. In this work, we explore more effective approaches for training deep learning-based models for image quality assessment (IQA) on HDR data. We leverage networks pre-trained on SDR data (source domain) and re-target these models to HDR (target domain) with additional fine-tuning and domain adaptation. We validate our methods on the available HDR IQA datasets, demonstrating that models trained with our combined recipe outperform previous baselines, converge much quicker, and reliably generalize to HDR inputs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (60)
  1. Deep learning. Nature, 521:436–44, 05 2015.
  2. DeepFL-IQA: Weak supervision for deep IQA feature learning. arXiv preprint arXiv:2001.08113, 2020.
  3. Consolidated dataset and metrics for high-dynamic-range image quality. IEEE Transactions on Multimedia, PP:1–1, 04 2021.
  4. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Transactions on Image Processing, 27(1):206–219, 2018.
  5. PieAPP: Perceptual image-error assessment through pairwise preference. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1808–1817, Los Alamitos, CA, USA, jun 2018. IEEE Computer Society.
  6. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NeurIPS’17, page 6000–6010, Red Hook, NY, USA, 2017. Curran Associates Inc.
  7. Perceptual image quality assessment with transformers. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 433–442, 2021.
  8. MUSIQ: Multi-scale image quality transformer. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 5128–5137, Los Alamitos, CA, USA, oct 2021. IEEE Computer Society.
  9. A. Chubarau and J. Clark. VTAMIQ: Transformers for attention modulated image quality assessment. CoRR, abs/2110.01655, 2021.
  10. Attentions help CNNs see better: Attention-based hybrid image quality assessment network. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1139–1148, Los Alamitos, CA, USA, jun 2022. IEEE Computer Society.
  11. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Trans. Graph., 30(4):40:1–40:14, July 2011.
  12. HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content, 2023.
  13. H.-P. Seidel T. O. Aydın, R. Mantiuk. Extending quality metrics to full luminance range images. In Human Vision and Electronic Imaging, pages 68060B–10. Spie, 2008.
  14. R. K. Mantiuk and M. Azimi. PU21: A novel perceptually uniform encoding for adapting existing quality metrics for hdr. In 2021 Picture Coding Symposium (PCS), pages 1–5, 2021.
  15. Perceptual signal coding for more efficient usage of bit codes. In The 2012 Annual Technical Conference & Exhibition, pages 1–9, 2012.
  16. Blind high dynamic range image quality assessment using deep learning. In 2017 IEEE International Conference on Image Processing (ICIP), pages 765–769, 2017.
  17. Y. Bengio. Deep learning of representations for unsupervised and transfer learning. In Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning Workshop - Volume 27, UTLW’11, page 17–37. JMLR.org, 2011.
  18. Image quality assessment based on a degradation model. IEEE Transactions on Image Processing, 9(4):636–650, April 2000.
  19. D. M. Chandler; S. S. Hemami. VSNR: A wavelet-based visual signal-to-noise ratio for natural images. IEEE Transactions on Image Processing, 16(9):2284–2298, Sep. 2007.
  20. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, April 2004.
  21. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, 2003, volume 2, pages 1398–1402 Vol.2, Nov 2003.
  22. Z. Wang and Q. Li. Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing, 20(5):1185–1198, May 2011.
  23. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing, 14(12):2117–2128, Dec 2005.
  24. Image information and visual quality. IEEE Transactions on Image Processing, 15(2):430–444, Feb 2006.
  25. Predicting visible differences in high dynamic range images: model and its calibration. In Human Vision and Electronic Imaging X, volume 5666, pages 204 – 214. International Society for Optics and Photonics, SPIE, 2005.
  26. FSIM: A feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 20(8):2378–2386, Aug 2011.
  27. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index. IEEE Transactions on Image Processing, 23(2):684–695, 2014.
  28. Mean deviation similarity index: Efficient and reliable full-reference image quality evaluator. IEEE Access, 4:5579–5590, 2016.
  29. A Haar wavelet-based perceptual similarity index for image quality assessment. Signal Processing: Image Communication, 61:33–43, 2018.
  30. Image quality assessment using human visual dog model fused with random forest. IEEE Transactions on Image Processing, 24(11):3282–3292, 2015.
  31. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Transactions on Image Processing, 23(10):4270–4281, Oct 2014.
  32. Understanding low- and high-level contributions to fixation prediction. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  33. Image quality assessment by comparing CNN features between images. Journal of Imaging Science and Technology, 60:604101–6041010, 11 2016.
  34. The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 586–595, June 2018.
  35. RankIQA: Learning from rankings for no-reference image quality assessment. In The IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  36. A survey on deep transfer learning. In Artificial Neural Networks and Machine Learning – ICANN 2018, pages 270–279, Cham, 2018. Springer International Publishing.
  37. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  38. J. Kim and S. Lee. Deep learning of human visual sensitivity in image quality assessment framework. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1969–1977, 2017.
  39. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
  40. R. Wanat and R. K. Mantiuk. Simulating and compensating changes in appearance between day and night vision. ACM Trans. Graph., 33(4):147:1–147:12, July 2014.
  41. R. S. Berns. Methods for characterizing CRT displays. Displays, 16(4):173 – 182, 1996. To Achieve WYSIWYG Colour.
  42. T. Borer and A. Cotton. A display-independent high dynamic range television system. SMPTE Motion Imaging Journal, 125(4):50–56, 2016.
  43. High Dynamic Range Imaging, pages 1–42. American Cancer Society, 2015.
  44. Solving challenges and improving the performance of automotive displays. Information Display, 35(1):13–27, 2019.
  45. Perceptual image quality assessment for various viewing conditions and display systems. Electronic Imaging, Image Quality and System Performance XVII, pp. 67-1-67-9(9), 2020.
  46. H. Daumé III. Frustratingly easy domain adaptation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256–263, Prague, Czech Republic, June 2007. Association for Computational Linguistics.
  47. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, page 513–520, Madison, WI, USA, 2011. Omnipress.
  48. M. Wang and W. Deng. Deep visual domain adaptation: A survey. Neurocomputing, 312:135–153, 2018.
  49. B. Sun and K. Saenko. Deep CORAL: Correlation alignment for deep domain adaptation. In Computer Vision – ECCV 2016 Workshops, pages 443–450, Cham, 2016. Springer International Publishing.
  50. LIVE image quality assessment database release 2. http://live.ece.utexas.edu/research/quality, January 2005.
  51. Image database TID2013: Peculiarities, results and perspectives. Signal Processing: Image Communication, 30:57–77, 2015.
  52. Subjective quality assessment database of HDR images compressed with JPEG XT. In 2015 Seventh International Workshop on Quality of Multimedia Experience (QoMEX), pages 1–6, 2015.
  53. Tone mapping-based high-dynamic-range image compression: study of optimization criterion and perceptual quality. Optical Engineering, 52(10):102008, 2013.
  54. KADID-10k: A large-scale artificially distorted IQA database. In 2019 Tenth International Conference on Quality of Multimedia Experience (QoMEX), pages 1–3. IEEE, 2019.
  55. Comparison of single image HDR reconstruction methods — the caveats of quality assessment. In ACM SIGGRAPH 2022 Conference Proceedings, SIGGRAPH ’22, New York, NY, USA, 2022. Association for Computing Machinery.
  56. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition, 2015.
  57. A. et. al. Paszke. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2019. Curran Associates Inc.
  58. I. Loshchilov and F. Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019.
  59. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing, 15(11):3440–3451, Nov 2006.
  60. HDR-VQM: An objective quality measure for high dynamic range video. Signal Processing: Image Communication, 35:46–60, 2015.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now