PSCR: Patches Sampling-based Contrastive Regression for AIGC Image Quality Assessment (2312.05897v1)

Abstract: In recent years, Artificial Intelligence Generated Content (AIGC) has gained widespread attention beyond the computer science community. Due to various issues arising from continuous creation of AI-generated images (AIGI), AIGC image quality assessment (AIGCIQA), which aims to evaluate the quality of AIGIs from human perception perspectives, has emerged as a novel topic in the field of computer vision. However, most existing AIGCIQA methods directly regress predicted scores from a single generated image, overlooking the inherent differences among AIGIs and scores. Additionally, operations like resizing and cropping may cause global geometric distortions and information loss, thus limiting the performance of models. To address these issues, we propose a patches sampling-based contrastive regression (PSCR) framework. We suggest introducing a contrastive regression framework to leverage differences among various generated images for learning a better representation space. In this space, differences and score rankings among images can be measured by their relative scores. By selecting exemplar AIGIs as references, we also overcome the limitations of previous models that could not utilize reference images on the no-reference image databases. To avoid geometric distortions and information loss in image inputs, we further propose a patches sampling strategy. To demonstrate the effectiveness of our proposed PSCR framework, we conduct extensive experiments on three mainstream AIGCIQA databases including AGIQA-1K, AGIQA-3K and AIGCIQA2023. The results show significant improvements in model performance with the introduction of our proposed PSCR framework. Code will be available at \url{https://github.com/jiquan123/PSCR}.

Introduction

Artificial Intelligence Generated Content (AIGC) is transforming numerous domains by enabling the creation of dynamic content such as images, text, audio, and video. A critical component within this domain is assessing the quality of AI-generated images (AIGIs) from a human perception standpoint. AIGC Image Quality Assessment (AIGCIQA) is an evolving area within computer vision intending to solve this. The traditional methods followed to evaluate AIGIs often suffer from limitations like not considering the variations in images or dealing with information loss due to preprocessing steps like resizing and cropping. To combat these challenges, a new approach called Patches Sampling-based Contrastive Regression (PSCR) for image quality assessment is proposed.

Methodology

The suggested framework, PSCR, revolutionizes the traditional ways of assessing AIGI quality by addressing two primary concerns - the lack of comparative analysis between different images and the loss of image quality due to geometric distortion from pre-processing. Introducing a contrastive regression framework changes how the model learns by focusing on the differences among AIGIs and predicting scores based on these distinctions. Moreover, to prevent information loss, the method incorporates a patches sampling strategy. This strategy employs a sliding window to capture overlapping portions of images, retaining maximum detail and avoiding geometric distortions.

Experimental Validation

The validation of the PSCR framework was conducted on three leading AIGCIQA databases, demonstrating significant improvements over baseline models and previous approaches. It delivers a robust comparative across metrics like the Spearman rank correlation coefficient (SRCC) and Pearson linear correlation coefficient (PLCC). The gains achieved indicate the method's effectiveness in better accommodating the nuances of AIGI quality assessment by preserving detail and capturing a fine-grained understanding through direct contrasts across different AI-generated images.

Conclusion and Future Work

The PSCR stands out as a comprehensive framework tackling AIGCIQA with a nuanced understanding and comparative analysis of image differences. Its integration into current methodologies offers substantial performance improvements, showcased across different AIGCIQA datasets. There are areas for future exploration, such as minimizing information redundancy in the patches sampling process and refining the balance between preserving detail and avoiding global information loss. The code is made available, thus encouraging further research and development in this promising direction for AIGC image quality assessment.