Augmenting Perceptual Super-Resolution via Image Quality Predictors (2504.18524v1)

Abstract: Super-resolution (SR), a classical inverse problem in computer vision, is inherently ill-posed, inducing a distribution of plausible solutions for every input. However, the desired result is not simply the expectation of this distribution, which is the blurry image obtained by minimizing pixelwise error, but rather the sample with the highest image quality. A variety of techniques, from perceptual metrics to adversarial losses, are employed to this end. In this work, we explore an alternative: utilizing powerful non-reference image quality assessment (NR-IQA) models in the SR context. We begin with a comprehensive analysis of NR-IQA metrics on human-derived SR data, identifying both the accuracy (human alignment) and complementarity of different metrics. Then, we explore two methods of applying NR-IQA models to SR learning: (i) altering data sampling, by building on an existing multi-ground-truth SR framework, and (ii) directly optimizing a differentiable quality score. Our results demonstrate a more human-centric perception-distortion tradeoff, focusing less on non-perceptual pixel-wise distortion, instead improving the balance between perceptual fidelity and human-tuned NR-IQA measures.

• The paper analyzes Non-Reference Image Quality Assessment (NR-IQA) metrics, identifying those like MUSIQ that align with human judgment, enabling automated enhancement of perceptual super-resolution without human annotation.
• It proposes integrating NR-IQA into the super-resolution learning process through data sampling, utilizing a multi-ground-truth framework weighted by NR-IQA scores for refined training data selection.
• The research explores direct optimization by using NR-IQA scores as a differentiable objective function, integrating them into the loss function to improve perceptual quality while minimizing non-perceptual distortions.

Overview of "Augmenting Perceptual Super-Resolution via Image Quality Predictors"

The paper "Augmenting Perceptual Super-Resolution via Image Quality Predictors" presents advancements in the domain of single-image super-resolution (SISR), specifically focusing on enhancing perceptual quality through the application of non-reference image quality assessment (NR-IQA) models. As SISR is an inherently ill-posed problem, previous approaches have attempted to balance the perception-distortion tradeoff, often relying on human annotations to achieve higher perceptual fidelity at the expense of pixel-level accuracy.

Key Contributions

1. Analysis of NR-IQA Metrics: The authors conduct a thorough evaluation of existing NR-IQA models against human preference datasets. They identify which metrics are most aligned with human judgment, enabling the replacement of manual annotation with automated, metric-driven processes. Metrics such as MUSIQ are highlighted for their ability to improve SR image quality without human involvement.
2. Data Sampling: The paper proposes two methods for integrating NR-IQA into the SR learning process. Firstly, it alters data sampling by leveraging a multi-ground-truth framework weighted by NR-IQA scores. This allows for a more refined selection of training data based on perceptual quality rather than arbitrary or uniform selection.
3. Direct Optimization: Secondly, the paper explores direct optimization using NR-IQA scores as a differentiable objective function. By integrating NR-IQA into the loss function of SR models, the approach improves perceptual image quality in a human-centric manner, minimizing non-perceptual pixel space distortions.

Numerical Results

The paper underscores the efficacy of NR-IQA models through comparisons to state-of-the-art techniques, showcasing superior performance in perceptual metrics without causing increased distortion artifacts. The use of metrics like MUSIQ and complementary models such as NIMA and Q-Align provides a solid foundation for training SR models to enhance aesthetic quality effectively.

Implications and Future Directions

The adoption of NR-IQA models in SISR training opens the door to scalable and domain-independent perceptual quality assessment. This advancement mitigates the need for extensive human annotation, making SR technology more accessible and adaptable across varied applications. Moreover, it presents a robust framework for further research in adaptive training approaches using AI-driven metrics.

Future work may explore enhancements in NR-IQA models for more sophisticated discrimination or the development of SR-specific quality metrics to better tackle domain-specific challenges, such as the super-resolution of text or medical images that have stringent perceptual fidelity requirements.

Conclusion

The paper clearly demonstrates that automated perceptual quality assessment can significantly advance super-resolution technology by shifting focus from traditional distortion-based metrics to more perceptually aligned objectives. Through methodological innovations and careful metric selection, the research contributes valuable insights to the field, positively impacting both theoretical research and practical applications in AI-driven image processing.