---
title: 'ViDA-UGC-Bench: UGC Distortion Analysis'
url: https://www.emergentmind.com/topics/vida-ugc-bench
type: topic
---

# ViDA-UGC-Bench: UGC Distortion Analysis

Searching arXiv for ViDA-UGC-Bench and closely related UGC quality benchmarks to ground the article in current papers.
First, I’ll locate the primary ViDA-UGC paper and then a few adjacent UGC quality benchmark papers for contextualization.
ViDA-UGC-Bench is a benchmark for detailed visual distortion assessment on user-generated content images, introduced together with the larger ViDA-UGC dataset in “ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images” [2508.12605]. It is designed for explainable image quality assessment rather than scalar-only scoring, and it targets three abilities that the paper treats as central to UGC quality analysis: fine-grained distortion grounding, detailed low-level perception, and reasoning quality description. The benchmark is a professionally revised evaluation subset of ViDA-UGC, derived from a distortion-oriented pipeline that combines human annotation, GPT-4o-based data generation, and a Chain-of-Thought assessment framework; the paper presents it as the first UGC distortion assessment benchmark [2508.12605].

## 1. Definition and intended scope

ViDA-UGC-Bench is explicitly framed around UGC image distortions rather than generic low-level vision. The motivation is that existing explainable IQA resources, especially Q-Bench in the paper’s comparison, primarily evaluate broad low-level perception and description, but do not sufficiently test distortion-centric analysis for UGC images. The authors argue that UGC assessment cannot be treated as interchangeable with AIGC-oriented evaluation, because UGC images are dominated by real capture and processing artifacts such as noise, blur, and compression, and therefore require distortion-specific reasoning rather than general commentary [2508.12605].

Within that framing, the benchmark measures three complementary capabilities. The first is **fine-grained distortion grounding**, meaning localization of distortions. The second is **detailed low-level perception**, meaning recognition and interpretation of distortion-relevant attributes. The third is **reasoning quality description**, meaning longer-form quality analysis that is complete, precise, and logically structured. The benchmark is therefore not merely a multiple-choice IQA test and not merely a captioning benchmark; it is a composite evaluation suite for explainable, distortion-oriented UGC image analysis [2508.12605].

The paper also positions ViDA-UGC-Bench as a response to a broader shift from unexplainable image quality scoring toward explainable IQA with practical use in quality control and optimization guidance. In that sense, ViDA-UGC-Bench functions as an evaluation instrument for MLLMs that are expected not only to score image quality, but also to localize, diagnose, and explain quality defects in UGC imagery [2508.12605].

## 2. Relation to the broader ViDA-UGC dataset

ViDA-UGC-Bench is derived from the larger ViDA-UGC instruction-tuning dataset. In the main paper, ViDA-UGC contains **11,534 images**, **36K distortion bounding boxes**, and **534K instruction tuning data**, and is organized into three sub-datasets: **ViDA-Grounding**, **ViDA-Perception**, and **ViDA-Description** [2508.12605].

The benchmark is the manually revised evaluation subset of that larger framework. Its role is to provide a smaller but more trustworthy test bed, with additional professional checking intended to reduce GPT bias and improve evaluation reliability. The main text states that the benchmark was formed by carefully selecting data from the three ViDA-UGC components and then revising the selected items with a professional image-processing research team [2508.12605].

| Benchmark component | Source subset | Count |
|---|---|---:|
| Overall quality analyses | ViDA-Description | 476 |
| Multi-choice questions | ViDA-Perception | 2,567 |
| Grounding data | ViDA-Grounding | 3,106 |

These components sum to **6,149 QA pairs** over **476 images**, matching the benchmark statistics reported in the abstract and benchmark section. The benchmark is also stated to cover **all ten common UGC distortions**, although the accessible main text does not enumerate the ten distortion names explicitly [2508.12605].

This derivation matters methodologically. ViDA-UGC serves as the large-scale supervision source for instruction tuning, while ViDA-UGC-Bench serves as the curated evaluation target. The distinction is important because much of ViDA-UGC is generated through GPT-4o under human-guided constraints, whereas the benchmark receives additional manual revision specifically for evaluation fidelity [2508.12605].

## 3. Task structure and representational format

The benchmark is organized around three task families.

**Distortion grounding** evaluates spatial localization of distortions. In the broader dataset design, grounding includes three sub-tasks: **distortion grounding**, **referring grounding**, and **region perception**. Distortion grounding asks for bounding boxes given a distortion type or query; referring grounding asks for the box corresponding to a target distortion description; region perception asks for distortions within a specified region of interest. ViDA-UGC-Bench includes **3,106 grounding data** drawn from this part of the framework [2508.12605].

**Low-level perception** evaluates distortion-aware image understanding through multiple-choice questions. The benchmarked perception dimensions reported in the results table are **Type**, **Position**, **Severity**, **Significance**, and **Overall**. The underlying generation process uses distortion-box attributes including **type**, **position**, **severity**, **impact**, and **significance**, and the broader perception setup also mentions low-level features such as **lighting** and **composition**. The benchmark includes **2,567 multi-choice questions** for this task family [2508.12605].

**Quality description** evaluates richer explainable IQA. The benchmark includes **476 overall quality analyses** from ViDA-Description. In the reported evaluation table, ViDA-UGC-Bench quality-description performance is summarized by **completeness**, **precision**, **reasoning**, and **overall**. The benchmark therefore treats descriptive quality analysis not as free-form captioning alone, but as structured explanation with explicit reasoning quality as a measured dimension [2508.12605].

The quality-description component is closely tied to the paper’s distortion-oriented CoT framework. For overall quality analysis, the framework proceeds through five verbal steps: obtain a general impression of the image content; find and analyze the distortions to gather rich low-level information; further identify the key distortions; analyze overall quality based on the previous analysis; and rate the image quality. The paper also states that grounding is interleaved into description using the format ``[distortion](bounding box)``, which explicitly binds explanatory text to localized evidence [2508.12605].

## 4. Annotation pipeline and benchmark curation

The benchmark inherits a multi-stage construction pipeline from ViDA-UGC. In **Step 1**, images are sampled from multiple UGC datasets and annotated by **five human subjects**. The process collects image quality scores and distortion bounding boxes. If the difference between the maximum and minimum quality scores exceeds **1**, the scores are invalidated and re-annotated; otherwise, MOS is computed as the average of the five scores. Bounding boxes are cleaned by splitting them into **global** and **local** types with **Area Ratio threshold = 0.7**. If global boxes exist, the largest global box is kept and the others are discarded; otherwise, **Non-Maximum Suppression** is applied to local boxes. The resulting dataset has on average **3.6 bounding boxes per image** and **one MOS score per image** [2508.12605].

In **Step 2**, GPT-4o is prompted to generate low-level distortion information. The system uses a set-of-mark strategy in which distortion regions are visually marked, and GPT-4o receives the image, the marked boxes, textual distortion definitions, and IQA-related instructions. The output is a distortion-oriented triplet consisting of image, distortion boxes, and distortion attributes. The five explicit distortion-box attributes are **type**, **position**, **severity**, **impact**, and **significance** [2508.12605].

In **Step 3**, GPT-4o is used again under the proposed CoT assessment framework to generate quality descriptions. The prompt integrates **MOS**, **rating criteria**, and the distortion triplets to produce both **overall quality analysis** and **individual distortion assessment**. In **Step 4**, those descriptions are converted into benchmarkable conversations and question formats, including VQA/MCQ for low-level perception and chat-style grounding data [2508.12605].

ViDA-UGC-Bench adds a benchmark-specific revision stage on top of that pipeline. The paper states that a **professional team consisting of image-processing researchers** revised the selected question-answer pairs and checked the accuracy of the low-level information to minimize GPT-4o bias. The curation policy is illustrated by accepted and rejected examples: a rejected item in Figure 4 is described as too simplistic because the answer could be guessed without consulting the image, whereas accepted items are more tightly linked to distortion understanding. The main text does not provide inter-annotator agreement values, exact revision counts, or a detailed adjudication protocol [2508.12605].

## 5. Evaluation setup and reported results

The benchmark evaluates three explainable IQA abilities with task-specific metrics, but the main paper does not provide explicit formulas for these metrics or for benchmark splits. For perception, the reported outputs are **accuracy percentages** over dimensions such as Type, Position, Severity, Significance, and Overall. For quality description, the reported outputs are **completeness**, **precision**, **reasoning**, and **overall**. For grounding, the reported outputs are **Response Rate**, **Acc$_{0.5}$**, and **mIoU**. The paper states that detailed evaluation settings are in supplementary material, so the main text does not specify train/validation/test splits for ViDA-UGC-Bench [2508.12605].

The evaluated baselines are **Qwen-VL-Chat**, **Qwen2VL-7B-Instruct**, **InternVL2.5-8B**, **InternVL3-8B**, and **GPT-4o** in zero-shot form. The paper also compares untuned baselines, models tuned on **Q-Instruct**, and models tuned on **ViDA-UGC**. For quality description, it distinguishes a Q-Instruct-style prompt—“**Describe and evaluate the quality of the image. Think step by step.**”—from the proposed distortion-oriented CoT prompt [2508.12605].

On **low-level perception**, the benchmark is reported to be substantially harder than Q-Bench: baseline models show an average **29% performance decline** on ViDA-UGC-Bench relative to Q-Bench. **GPT-4o zero-shot** achieves **55.20%** overall on ViDA-UGC-Bench perception, with **46.38%** on Type, **60.28%** on Position, **53.57%** on Severity, and **59.58%** on Significance. ViDA-UGC-tuned models improve markedly: **Qwen-VL-Chat-ViDA** reaches **63.34%**, **Qwen2-VL-7B-ViDA** reaches **71.45%**, **InternVL2.5-8B-ViDA** reaches **74.80%**, and **InternVL3-8B-ViDA** reaches **73.00%** overall. The paper emphasizes that **Qwen2-VL-7B-ViDA**, **InternVL2.5-8B-ViDA**, and **InternVL3-8B-ViDA** all exceed GPT-4o zero-shot on this perception benchmark [2508.12605].

The comparison with Q-Instruct tuning is one of the paper’s strongest empirical claims. For **Qwen2-VL-7B**, the overall perception score changes from **47.53%** in the baseline to **44.14%** with Q-Instruct tuning, but rises to **71.45%** with ViDA-UGC tuning. For **InternVL3-8B**, the corresponding values are **47.37%**, **39.77%**, and **73.00%**. The authors interpret this as evidence that ViDA-UGC supervision is better aligned with detailed UGC distortion assessment than generic low-level instruction tuning [2508.12605].

On **quality description**, the distortion-oriented CoT prompt improves performance even without fine-tuning. For example, **Qwen2-VL-7B** improves from **4.21** under the ordinary prompt to **4.54** under the proposed CoT prompt. ViDA-UGC tuning plus the CoT framework yields higher overall scores: **Qwen-VL-Chat-ViDA†** reaches **5.36**, **Qwen2-VL-7B-ViDA†** reaches **5.78**, **InternVL2.5-8B-ViDA†** reaches **5.72**, and **InternVL3-8B-ViDA†** reaches **5.87**. The paper summarizes this pattern by stating that the CoT framework consistently improves description metrics and that ViDA-UGC fine-tuning further strengthens reasoning-based quality analysis [2508.12605].

On **grounding**, the reported results are specifically for **referring grounding**. ViDA-UGC tuning improves both localization accuracy and reliability of response generation. **Qwen-VL-Chat** moves from **Response Rate 1 / Acc$_{0.5}$ 32.4 / mIoU 37.3** to **1 / 41.3 / 43.4** with ViDA tuning. **Qwen2-VL-7B** moves from **0.79 / 24.9 / 29.8** to **0.99 / 42.1 / 45.2**. **InternVL2.5-8B** moves from **0.98 / 29.0 / 37.0** to **1 / 43.3 / 46.4**, and **InternVL3-8B** moves from **0.95 / 25.8 / 33.5** to **1 / 44.2 / 47.0**. The paper highlights the contrast with high-level referring-expression benchmarks such as RefCOCO and RefCOCO+, on which the same base models reportedly approach **90 Acc$_{0.5}$**; the drop on ViDA-UGC-Bench is presented as evidence that low-level distortion grounding is a distinct and harder problem [2508.12605].

## 6. Significance, neighboring benchmarks, and limitations

Within the UGC quality-assessment literature, ViDA-UGC-Bench occupies a specific niche: it is an **image** benchmark centered on **explainable, distortion-centric IQA**. This differentiates it from video-quality resources such as the blind UGC-VQA benchmarking framework built on KoNViD-1k, LIVE-VQC, and YouTube-UGC [2005.14354]; from FineVD, which provides **6104 UGC videos** with six-dimensional MOS and descriptive supervision for fine-grained **video** quality assessment [2412.19238]; and from BVI-UGC, which focuses on UGC transcoding and non-pristine references in a full-reference/no-reference video setting [2408.07171]. ViDA-UGC-Bench is therefore best understood not as a general UGC benchmark, but as a benchmark for detailed distortion analysis in UGC **images** [2508.12605].

It is also distinct from model-centered UGC video work such as PriorFormer, which addresses blind UGC-VQA using a prior-augmented perceptual vision transformer with content and distortion priors over sampled frames [2406.16297]. A plausible implication is that ViDA-UGC-Bench and those video benchmarks address related quality-assessment questions at different representational levels: ViDA-UGC-Bench emphasizes grounded explanation and reasoning over static UGC images, while the video literature emphasizes correlation with MOS and temporal quality modeling.

Several limitations are explicit or strongly implied in the benchmark presentation. The paper acknowledges that GPT-generated data inevitably exhibit biases, which is the main reason for professional post-revision. It also states that the proposed CoT framework remains constrained by the pretrained low-level knowledge of the underlying MLLMs and can suffer from error propagation and reasoning inefficiency. In addition, the main paper does not provide explicit formulas for benchmark metrics, loss functions, or scoring functions, and it defers detailed evaluation settings to supplementary material, leaving split details unspecified in the main text. The benchmark covers all ten common UGC distortions, but the accessible main text does not list their names. Finally, the reliance on professional revision by image-processing researchers improves fidelity but implies substantial annotation cost and limited scalability [2508.12605].

Taken together, ViDA-UGC-Bench represents a move from opaque scalar IQA toward distortion-aware, grounded, and reasoning-based evaluation for UGC images. Its central contribution is not merely a harder test set, but a benchmark structure that operationalizes three linked abilities—where the distortion is, what the distortion is, and how that distortion affects perceived image quality—within a single UGC-specific evaluation framework [2508.12605].

Source: https://www.emergentmind.com/topics/vida-ugc-bench