AU-IQA: Benchmark for AI-Enhanced UGC

• AU-IQA is a benchmark dataset and evaluation suite that defines rigorous subjective (MOS) and objective standards for assessing AI-enhanced user-generated content.
• It systematically augments 4,800 images using diverse enhancement methods like super-resolution, low-light enhancement, and denoising to represent realistic AI-enhanced outputs.
• Empirical results show that traditional UGC models often outperform AIGC-specific methods, revealing key challenges and future directions for robust perceptual quality assessment.

AI-based image enhancement has become integral to modern user-generated content (UGC), improving resolution, noise characteristics, and visibility in low-light scenes. However, established image quality assessment (IQA) protocols lack dedicated models and data for the perceptual evaluation of AI-enhanced UGC (AI-UGC)—images that blend the authentic capture characteristics of user devices with sophisticated AI restoration. AU-IQA is a benchmark dataset and evaluation suite specifically tailored to this hybrid domain, defining rigorous standards for subjective and objective assessment, and benchmarking a variety of traditional and large multimodal models on perceptual fidelity in AI-UGC. The dataset and corresponding analyses demonstrate both the limitations of conventional approaches and the future directions necessary for robust, perceptually aligned IQA in this emerging modality (Wang et al., 7 Aug 2025).

1. Motivation and Scope

The proliferation of AI-based image enhancement—including super-resolution (SR), low-light enhancement (LLE), and denoising—has outpaced the development of metrics that accurately characterize perceptual improvements and failures in AI-UGC. Conventional IQA approaches, such as PSNR and SSIM, measure pixel-level agreement and structural similarity but fail to reflect subjective perception in the presence of modern AI artifacts—such as hallucinated details or resynthesized textures—prevalent in AI-UGC. Existing IQA research focuses either on traditional, device-captured UGC or on pure AI-generated content (AIGC), with each exhibiting unique statistical and perceptual signatures. No prior resource systematically addresses the intersectional case of AI-UGC. AU-IQA was designed to (i) canonically define AI-UGC within an evaluation framework, (ii) supply a large-scale, human-annotated image benchmark, and (iii) provide a platform for evaluating and comparing IQA models—including large vision-language transformers—on this complex hybrid domain (Wang et al., 7 Aug 2025).

2. Dataset Generation and Design Principles

AU-IQA comprises 4,800 images created via structured augmentation of 400 high-resolution originals from KonIQ-10k. For each source, three degradations were synthetically introduced: downsampling for low resolution, brightness reduction for low light, and additive Gaussian noise. Each degraded image was then enhanced using four contemporary AI models per degradation type:

• Super-resolution: DiffBIR, OSEDiff, PASD, SUPIR
• Low-light enhancement: GLARE, LightenDiffusion, NeRCo, QuadPrior
• Denoising: DiffBIR, MaskedDenoising, PASD, SUPIR

All outputs were resized to a standard resolution, resulting in a balanced, representative set of SR, LLE, and denoised AI-UGC, spanning a broad range of enhancement quality. This systematic approach ensures both diversity and realism in the resulting corpus.

Degradation	Enhancement Models	Images per Type
Super-resolution	DiffBIR, OSEDiff, PASD, SUPIR	1,600
Low-light enhancement	GLARE, LightenDiffusion, NeRCo, QuadPrior	1,600
Denoising	DiffBIR, MaskedDenoising, PASD, SUPIR	1,600

3. Subjective Quality Annotation Protocol

AU-IQA applies a standardized Mean Opinion Score (MOS) methodology. Five trained annotators independently score each image on a 1–5 scale, subtracting points for visible resolution loss, brightness errors, noise, and generic distortion, with heightened penalties for resolution and structural distortions. Discrepancies trigger a relabeling cycle to ensure judgment consistency. The MOS for image $j$ is computed as:

$MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$

where $s_{i,j} \in [1,5]$ is annotator $i$’s score for image $j$, $N=5$. Inter-annotator agreement is quantified as

$$\sigma_{j} = \sqrt{\frac{1}{N}\sum_{i=1}^{N}(s_{i,j}-MOS_{j})^{2}}$$

and confidence intervals are calculated using the $t$-distribution:

$$MOS_{j} \pm t_{\alpha/2,\,N-1} \frac{\sigma_{j}}{\sqrt{N}}$$

Typical inter-annotator standard deviations $\sigma$ less than 0.3 indicate strong consistency. This MOS protocol provides a robust ground truth for subsequent benchmarking (Wang et al., 7 Aug 2025).

4. Objective Metrics and Model Benchmarking

AU-IQA establishes as baselines the following objective measures:

• PSNR: $MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$0, where $MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$1
• SSIM:

$MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$2

• LPIPS:

$MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$3

with $MOS_{j} = \frac{1}{N} \sum_{i=1}^{N} s_{i,j}$4 the normalized layer activations in a deep network.

AU-IQA further benchmarks three IQA model families:

1. Traditional UGC models: TOPIQ, LIQE, ARNIQA, HyperIQA (all no-reference)
2. AIGC-specific: MA-AGIQA (side-by-side quality rating)
3. Large multimodal models (LMMs): InternVL2, Qwen2-VL, LLaVA-v1.6, together with LMM-finetuned quality predictors such as Q-Align, Q-Eval-Score, DEQA

Vision-language LMM backbones were fine-tuned on KADID-10K (UGC) and AGIQA-3K (AIGC). Models were evaluated using Pearson (PLCC) and Spearman (SRCC) correlations to MOS. No new assessment metric was introduced, but deficiencies of existing measures were critically analyzed in the AI-UGC context.

5. Empirical Results and Qualitative Analysis

On AU-IQA, models originally trained for UGC outperform both AIGC-specialized and generic perceptual metrics on the AI-UGC domain. Key results include:

• DEQA attains the highest PLCC/SRCC on super-resolution (SR: 0.703/0.733) and low-light enhancement (LLE: 0.726/0.707).
• Q-Align exhibits strong performance in denoising (PLCC/SRCC ≈ 0.38/0.37).
• HyperIQA (traditional no-reference) leads specifically in pure denoising (0.684/0.697).
• MA-AGIQA and standard LPIPS/PSNR/SSIM exhibit poor alignment with subjective MOS (usually <0.3).

Evaluation reveals that super-resolution is comparatively tractable for UGC methods (PLCC ≈ 0.6–0.7), low-light enhancement is most challenging (PLCC often <0.5), and denoising is of intermediate difficulty. Subset-wise correlation is more stable using the entire 4,800 sample test set, reinforcing the importance of large-scale benchmarking for reliable model comparison (Wang et al., 7 Aug 2025).

6. Design Insights and Forward-Looking Recommendations

Analysis of AU-IQA highlights that AI-UGC is perceptually closer to traditional UGC than AIGC, resulting in superior transfer of no-reference UGC models. Different enhancement types introduce distinctive artifact profiles; no single metric or model achieves uniformly strong performance across all types. To address current limitations, the following strategies are recommended:

• Integrate enhancement-type-sensitive features, such as resolution awareness for SR or contrast/histogram-based cues for low-light.
• Expand datasets to incorporate additional enhancement modes—e.g., deblurring, color grading, style transfer, and real-world AI-UGC uploads.
• Fuse no-reference statistical features with referenceless perceptual signatures to capture the complex interactions between AI hallucinations and real-world noise.

A plausible implication is that future IQA for AI-UGC will require model architectures and loss functions jointly optimized for the compound, type-conditioned artifact distributions unique to each enhancement pipeline.

7. Distribution, Reproducibility, and Usage Policy

AU-IQA is openly available at https://github.com/WNNGGU/AU-IQA-Dataset under the CC BY 4.0 license. The dataset comprises the full set of 4,800 images, each annotated with MOS and associated metadata. Standard practice is to use the unified test set or conduct experiments stratified by enhancement type. An 80/20 train-validation split (3,840/960 images) is provided, with explicit fold definitions. This facilitates both fair benchmarking of existing IQA models and the development of new learning-based predictors specialized for the AI-UGC domain (Wang et al., 7 Aug 2025).