ViDA-UGC-Bench: UGC Distortion Analysis

• ViDA-UGC-Bench is a benchmark designed for explainable distortion analysis in UGC images, emphasizing fine-grained distortion grounding, low-level perception, and reasoning-based quality description.
• It derives a curated subset from the larger ViDA-UGC dataset using a multi-stage pipeline with human revision and GPT-4o guidance, ensuring 6,149 QA pairs over 476 images.
• The benchmark challenges current models by evidencing a 29% drop in low-level perception scores versus similar tests, highlighting the unique complexities of UGC distortion assessment.

Searching arXiv for ViDA-UGC-Bench and closely related UGC quality benchmarks to ground the article in current papers. First, I’ll locate the primary ViDA-UGC paper and then a few adjacent UGC quality benchmark papers for contextualization. ViDA-UGC-Bench is a benchmark for detailed visual distortion assessment on user-generated content images, introduced together with the larger ViDA-UGC dataset in “ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images” (Liao et al., 18 Aug 2025). It is designed for explainable image quality assessment rather than scalar-only scoring, and it targets three abilities that the paper treats as central to UGC quality analysis: fine-grained distortion grounding, detailed low-level perception, and reasoning quality description. The benchmark is a professionally revised evaluation subset of ViDA-UGC, derived from a distortion-oriented pipeline that combines human annotation, GPT-4o-based data generation, and a Chain-of-Thought assessment framework; the paper presents it as the first UGC distortion assessment benchmark (Liao et al., 18 Aug 2025).

1. Definition and intended scope

ViDA-UGC-Bench is explicitly framed around UGC image distortions rather than generic low-level vision. The motivation is that existing explainable IQA resources, especially Q-Bench in the paper’s comparison, primarily evaluate broad low-level perception and description, but do not sufficiently test distortion-centric analysis for UGC images. The authors argue that UGC assessment cannot be treated as interchangeable with AIGC-oriented evaluation, because UGC images are dominated by real capture and processing artifacts such as noise, blur, and compression, and therefore require distortion-specific reasoning rather than general commentary (Liao et al., 18 Aug 2025).

Within that framing, the benchmark measures three complementary capabilities. The first is fine-grained distortion grounding, meaning localization of distortions. The second is detailed low-level perception, meaning recognition and interpretation of distortion-relevant attributes. The third is reasoning quality description, meaning longer-form quality analysis that is complete, precise, and logically structured. The benchmark is therefore not merely a multiple-choice IQA test and not merely a captioning benchmark; it is a composite evaluation suite for explainable, distortion-oriented UGC image analysis (Liao et al., 18 Aug 2025).

The paper also positions ViDA-UGC-Bench as a response to a broader shift from unexplainable image quality scoring toward explainable IQA with practical use in quality control and optimization guidance. In that sense, ViDA-UGC-Bench functions as an evaluation instrument for MLLMs that are expected not only to score image quality, but also to localize, diagnose, and explain quality defects in UGC imagery (Liao et al., 18 Aug 2025).

2. Relation to the broader ViDA-UGC dataset

ViDA-UGC-Bench is derived from the larger ViDA-UGC instruction-tuning dataset. In the main paper, ViDA-UGC contains 11,534 images, 36K distortion bounding boxes, and 534K instruction tuning data, and is organized into three sub-datasets: ViDA-Grounding, ViDA-Perception, and ViDA-Description (Liao et al., 18 Aug 2025).

The benchmark is the manually revised evaluation subset of that larger framework. Its role is to provide a smaller but more trustworthy test bed, with additional professional checking intended to reduce GPT bias and improve evaluation reliability. The main text states that the benchmark was formed by carefully selecting data from the three ViDA-UGC components and then revising the selected items with a professional image-processing research team (Liao et al., 18 Aug 2025).

Benchmark component	Source subset	Count
Overall quality analyses	ViDA-Description	476
Multi-choice questions	ViDA-Perception	2,567
Grounding data	ViDA-Grounding	3,106

These components sum to 6,149 QA pairs over 476 images, matching the benchmark statistics reported in the abstract and benchmark section. The benchmark is also stated to cover all ten common UGC distortions, although the accessible main text does not enumerate the ten distortion names explicitly (Liao et al., 18 Aug 2025).

This derivation matters methodologically. ViDA-UGC serves as the large-scale supervision source for instruction tuning, while ViDA-UGC-Bench serves as the curated evaluation target. The distinction is important because much of ViDA-UGC is generated through GPT-4o under human-guided constraints, whereas the benchmark receives additional manual revision specifically for evaluation fidelity (Liao et al., 18 Aug 2025).

3. Task structure and representational format

The benchmark is organized around three task families.

Distortion grounding evaluates spatial localization of distortions. In the broader dataset design, grounding includes three sub-tasks: distortion grounding, referring grounding, and region perception. Distortion grounding asks for bounding boxes given a distortion type or query; referring grounding asks for the box corresponding to a target distortion description; region perception asks for distortions within a specified region of interest. ViDA-UGC-Bench includes 3,106 grounding data drawn from this part of the framework (Liao et al., 18 Aug 2025).

Low-level perception evaluates distortion-aware image understanding through multiple-choice questions. The benchmarked perception dimensions reported in the results table are Type, Position, Severity, Significance, and Overall. The underlying generation process uses distortion-box attributes including type, position, severity, impact, and significance, and the broader perception setup also mentions low-level features such as lighting and composition. The benchmark includes 2,567 multi-choice questions for this task family (Liao et al., 18 Aug 2025).

Quality description evaluates richer explainable IQA. The benchmark includes 476 overall quality analyses from ViDA-Description. In the reported evaluation table, ViDA-UGC-Bench quality-description performance is summarized by completeness, precision, reasoning, and overall. The benchmark therefore treats descriptive quality analysis not as free-form captioning alone, but as structured explanation with explicit reasoning quality as a measured dimension (Liao et al., 18 Aug 2025).

The quality-description component is closely tied to the paper’s distortion-oriented CoT framework. For overall quality analysis, the framework proceeds through five verbal steps: obtain a general impression of the image content; find and analyze the distortions to gather rich low-level information; further identify the key distortions; analyze overall quality based on the previous analysis; and rate the image quality. The paper also states that grounding is interleaved into description using the format [distortion](bounding box), which explicitly binds explanatory text to localized evidence (Liao et al., 18 Aug 2025).

4. Annotation pipeline and benchmark curation

The benchmark inherits a multi-stage construction pipeline from ViDA-UGC. In Step 1, images are sampled from multiple UGC datasets and annotated by five human subjects. The process collects image quality scores and distortion bounding boxes. If the difference between the maximum and minimum quality scores exceeds 1, the scores are invalidated and re-annotated; otherwise, MOS is computed as the average of the five scores. Bounding boxes are cleaned by splitting them into global and local types with Area Ratio threshold = 0.7. If global boxes exist, the largest global box is kept and the others are discarded; otherwise, Non-Maximum Suppression is applied to local boxes. The resulting dataset has on average 3.6 bounding boxes per image and one MOS score per image (Liao et al., 18 Aug 2025).

In Step 2, GPT-4o is prompted to generate low-level distortion information. The system uses a set-of-mark strategy in which distortion regions are visually marked, and GPT-4o receives the image, the marked boxes, textual distortion definitions, and IQA-related instructions. The output is a distortion-oriented triplet consisting of image, distortion boxes, and distortion attributes. The five explicit distortion-box attributes are type, position, severity, impact, and significance (Liao et al., 18 Aug 2025).

In Step 3, GPT-4o is used again under the proposed CoT assessment framework to generate quality descriptions. The prompt integrates MOS, rating criteria, and the distortion triplets to produce both overall quality analysis and individual distortion assessment. In Step 4, those descriptions are converted into benchmarkable conversations and question formats, including VQA/MCQ for low-level perception and chat-style grounding data (Liao et al., 18 Aug 2025).

ViDA-UGC-Bench adds a benchmark-specific revision stage on top of that pipeline. The paper states that a professional team consisting of image-processing researchers revised the selected question-answer pairs and checked the accuracy of the low-level information to minimize GPT-4o bias. The curation policy is illustrated by accepted and rejected examples: a rejected item in Figure 1 is described as too simplistic because the answer could be guessed without consulting the image, whereas accepted items are more tightly linked to distortion understanding. The main text does not provide inter-annotator agreement values, exact revision counts, or a detailed adjudication protocol (Liao et al., 18 Aug 2025).

5. Evaluation setup and reported results

The benchmark evaluates three explainable IQA abilities with task-specific metrics, but the main paper does not provide explicit formulas for these metrics or for benchmark splits. For perception, the reported outputs are accuracy percentages over dimensions such as Type, Position, Severity, Significance, and Overall. For quality description, the reported outputs are completeness, precision, reasoning, and overall. For grounding, the reported outputs are Response Rate, Acc$_{0.5}$, and mIoU. The paper states that detailed evaluation settings are in supplementary material, so the main text does not specify train/validation/test splits for ViDA-UGC-Bench (Liao et al., 18 Aug 2025).

The evaluated baselines are Qwen-VL-Chat, Qwen2VL-7B-Instruct, InternVL2.5-8B, InternVL3-8B, and GPT-4o in zero-shot form. The paper also compares untuned baselines, models tuned on Q-Instruct, and models tuned on ViDA-UGC. For quality description, it distinguishes a Q-Instruct-style prompt—“Describe and evaluate the quality of the image. Think step by step.”—from the proposed distortion-oriented CoT prompt (Liao et al., 18 Aug 2025).

On low-level perception, the benchmark is reported to be substantially harder than Q-Bench: baseline models show an average 29% performance decline on ViDA-UGC-Bench relative to Q-Bench. GPT-4o zero-shot achieves 55.20% overall on ViDA-UGC-Bench perception, with 46.38% on Type, 60.28% on Position, 53.57% on Severity, and 59.58% on Significance. ViDA-UGC-tuned models improve markedly: Qwen-VL-Chat-ViDA reaches 63.34%, Qwen2-VL-7B-ViDA reaches 71.45%, InternVL2.5-8B-ViDA reaches 74.80%, and InternVL3-8B-ViDA reaches 73.00% overall. The paper emphasizes that Qwen2-VL-7B-ViDA, InternVL2.5-8B-ViDA, and InternVL3-8B-ViDA all exceed GPT-4o zero-shot on this perception benchmark (Liao et al., 18 Aug 2025).

The comparison with Q-Instruct tuning is one of the paper’s strongest empirical claims. For Qwen2-VL-7B, the overall perception score changes from 47.53% in the baseline to 44.14% with Q-Instruct tuning, but rises to 71.45% with ViDA-UGC tuning. For InternVL3-8B, the corresponding values are 47.37%, 39.77%, and 73.00%. The authors interpret this as evidence that ViDA-UGC supervision is better aligned with detailed UGC distortion assessment than generic low-level instruction tuning (Liao et al., 18 Aug 2025).

On quality description, the distortion-oriented CoT prompt improves performance even without fine-tuning. For example, Qwen2-VL-7B improves from 4.21 under the ordinary prompt to 4.54 under the proposed CoT prompt. ViDA-UGC tuning plus the CoT framework yields higher overall scores: Qwen-VL-Chat-ViDA† reaches 5.36, Qwen2-VL-7B-ViDA† reaches 5.78, InternVL2.5-8B-ViDA† reaches 5.72, and InternVL3-8B-ViDA† reaches 5.87. The paper summarizes this pattern by stating that the CoT framework consistently improves description metrics and that ViDA-UGC fine-tuning further strengthens reasoning-based quality analysis (Liao et al., 18 Aug 2025).

On grounding, the reported results are specifically for referring grounding. ViDA-UGC tuning improves both localization accuracy and reliability of response generation. Qwen-VL-Chat moves from Response Rate 1 / Acc$_{0.5}$ 32.4 / mIoU 37.3 to 1 / 41.3 / 43.4 with ViDA tuning. Qwen2-VL-7B moves from 0.79 / 24.9 / 29.8 to 0.99 / 42.1 / 45.2. InternVL2.5-8B moves from 0.98 / 29.0 / 37.0 to 1 / 43.3 / 46.4, and InternVL3-8B moves from 0.95 / 25.8 / 33.5 to 1 / 44.2 / 47.0. The paper highlights the contrast with high-level referring-expression benchmarks such as RefCOCO and RefCOCO+, on which the same base models reportedly approach 90 Acc$_{0.5}$; the drop on ViDA-UGC-Bench is presented as evidence that low-level distortion grounding is a distinct and harder problem (Liao et al., 18 Aug 2025).

6. Significance, neighboring benchmarks, and limitations

Within the UGC quality-assessment literature, ViDA-UGC-Bench occupies a specific niche: it is an image benchmark centered on explainable, distortion-centric IQA. This differentiates it from video-quality resources such as the blind UGC-VQA benchmarking framework built on KoNViD-1k, LIVE-VQC, and YouTube-UGC (Tu et al., 2020); from FineVD, which provides 6104 UGC videos with six-dimensional MOS and descriptive supervision for fine-grained video quality assessment (Duan et al., 2024); and from BVI-UGC, which focuses on UGC transcoding and non-pristine references in a full-reference/no-reference video setting (Qi et al., 2024). ViDA-UGC-Bench is therefore best understood not as a general UGC benchmark, but as a benchmark for detailed distortion analysis in UGC images (Liao et al., 18 Aug 2025).

It is also distinct from model-centered UGC video work such as PriorFormer, which addresses blind UGC-VQA using a prior-augmented perceptual vision transformer with content and distortion priors over sampled frames (Pei et al., 2024). A plausible implication is that ViDA-UGC-Bench and those video benchmarks address related quality-assessment questions at different representational levels: ViDA-UGC-Bench emphasizes grounded explanation and reasoning over static UGC images, while the video literature emphasizes correlation with MOS and temporal quality modeling.

Several limitations are explicit or strongly implied in the benchmark presentation. The paper acknowledges that GPT-generated data inevitably exhibit biases, which is the main reason for professional post-revision. It also states that the proposed CoT framework remains constrained by the pretrained low-level knowledge of the underlying MLLMs and can suffer from error propagation and reasoning inefficiency. In addition, the main paper does not provide explicit formulas for benchmark metrics, loss functions, or scoring functions, and it defers detailed evaluation settings to supplementary material, leaving split details unspecified in the main text. The benchmark covers all ten common UGC distortions, but the accessible main text does not list their names. Finally, the reliance on professional revision by image-processing researchers improves fidelity but implies substantial annotation cost and limited scalability (Liao et al., 18 Aug 2025).

Taken together, ViDA-UGC-Bench represents a move from opaque scalar IQA toward distortion-aware, grounded, and reasoning-based evaluation for UGC images. Its central contribution is not merely a harder test set, but a benchmark structure that operationalizes three linked abilities—where the distortion is, what the distortion is, and how that distortion affects perceived image quality—within a single UGC-specific evaluation framework (Liao et al., 18 Aug 2025).