AgenticIQA-200K: Structured IQA Instruction Dataset

• The paper introduces AgenticIQA-200K as a 200K-example instruction corpus that supervises multi-stage IQA reasoning through planner, executor, and summarizer roles.
• It combines synthetic and authentic image sources with GPT-4o-generated queries and structured task decompositions to enable distortion detection and tool routing.
• Experimental results indicate improved distortion analysis and overall performance when integrated with a shared VLM backbone and Q-Instruct-200K.

AgenticIQA-200K is a large-scale instruction dataset for training image quality assessment agents within the AgenticIQA framework. It is defined as a corpus that aligns vision-LLMs with modular IQA reasoning through image-query pairs paired with structured task decompositions and response traces, thereby supporting “plan–execute–summarize learning” rather than conventional scalar-only quality regression. In the framework described by the authors, AgenticIQA-200K is the training and alignment resource, whereas AgenticIQA-Eval is the separate benchmark used to assess planning, execution, and summarization capabilities (Zhu et al., 30 Sep 2025).

1. Definition and intended function

AgenticIQA-200K was introduced because prior IQA resources largely support either scalar score regression for traditional full-reference or no-reference IQA models, or single-step language-based IQA responses for VLMs, but not the multi-stage, adaptive, interpretable, query-aware behavior required by AgenticIQA. The dataset is therefore designed to supervise four explicit subtasks—distortion detection, distortion analysis, tool selection, and tool execution—under a broader planner–executor–summarizer architecture (Zhu et al., 30 Sep 2025).

Each sample consists of an image-query pair accompanied by a structured task decomposition and a corresponding response trace. The immediate training objective is to teach a VLM how to parse user intent, determine whether full-reference or no-reference reasoning is needed, distinguish global from local scope, identify or infer target distortions, choose appropriate IQA tools, and summarize the resulting evidence into scores and explanations. This suggests that the dataset is best understood as policy-style supervision for modular IQA reasoning rather than as a benchmark of human opinion scores.

A common misunderstanding is to treat AgenticIQA-200K as the framework’s evaluation set. The paper is explicit that the evaluation resource is AgenticIQA-Eval, not AgenticIQA-200K. Another possible misunderstanding is to read the suffix “200K” as a context-window claim; in the paper it refers to the total of 200K instruction-response pairs.

2. Data sources and construction pipeline

The dataset is built from two pre-existing IQA-oriented image pools. DQ-495K contributes 55,620 synthetically distorted images with carefully documented artificial distortions, and Q-Pathway contributes 10,797 authentically distorted images representing real-world degradations. The appendix further states that the selected images are paired with comprehensive perceptual reasoning annotations, which seed the subsequent synthetic instruction generation process (Zhu et al., 30 Sep 2025).

The construction pipeline proceeds in two major generative stages. First, using the image-associated perceptual reasoning descriptions, GPT-4o converts the source annotations into IQA-oriented question-answer pairs. The paper states that this yields approximately 140K question-response pairs, divided into 60K What/How/Which questions, 40K Yes/No questions, and 40K Extended questions. The appendix prompt instructs GPT-4o to generate three multi-choice questions beginning with What / How / Why, two Yes/No questions, and two Extended open-ended questions, while enforcing diversity across global versus local perception, descriptive versus analytic versus causal versus comparative versus judgmental tasks, tool-use questions, and both distortion-centric and tool-centric questions. It also requires that at least one question involve a specific distortion and at least one reference an IQA tool.

Second, the image-query pairs are augmented with structured task decompositions and response traces aligned to the AgenticIQA pipeline. GPT-4o is guided by programmatically designed prompting schemas to decompose a query into a sequential task plan, select relevant IQA tools, and generate execution traces aligned with the subtasks. The authors state that a filtering process removes “apparent incorrect and irrelevant responses generated by GPT-4o,” but they do not specify the number of discarded examples, the exact heuristics, or whether human annotators verified the generated instructions.

The final corpus is category-balanced at the role level:

Subset	Size	Role
Planner	50K	Task decomposition and routing
Executor	100K	Distortion reasoning and tool-related actions
Summarizer	50K	Evidence integration and final answer generation

The paper does not report an official train/validation/test split for AgenticIQA-200K itself, which is consistent with its use as a training corpus rather than as a held-out benchmark.

3. Supervision schema and structured outputs

The supervision format is JSON-oriented and role-specific. For planner instructions, the target includes fields such as task type or query type, reference type or reference mode, required object names or query scope, required distortions, required tools, distortion source, and a boolean plan over distortion_detection, tool_selection, distortion_analysis, and tool_execute. This directly supervises the planning policy.

Executor supervision is more heterogeneous. Distortion detection returns a JSON object that maps global or object-level scope to a distortion list. The valid distortion categories are constrained to Blurs, Color distortions, Compression, Noise, Brightness change, Sharpness, and Contrast. Distortion analysis returns, per object or globally, tuples of distortion type, severity, and explanation. The paper formalizes this as

$$A_i = \left\{ \left(d_i, l_i, r_i\right) \mid d_i \in D_k \right\} = \mathcal{E}_{da}(x, t_{da}, D_k, O_k),$$

where $d_i$ is the distortion type, $l_i$ is the severity label, and $r_i$ is the textual reasoning. Tool selection is formalized as

$T_i = \mathcal{E}_{ts}(d_i, t_{ts}, \mathcal{T}),$

and supervises distortion-aware routing to specific IQA tools (Zhu et al., 30 Sep 2025).

The summarizer target returns quality_reasoning and final_answer. Its inputs include distortion analysis, IQA tool scores in $[1,5]$, reference type, an optional prior answer, and an optional image. This stage teaches the model to integrate modular evidence into interpretable quality judgments.

Two caveats are explicit in the paper. First, tool execution is less directly represented as generated text than planning or distortion reasoning, because actual tool execution relies on external IQA models rather than on natural-language traces alone. Second, the severity taxonomy is inconsistent across prompts: one instruction version uses none, slight, moderate, severe, extreme, while another uses none, mild, moderate, heavy, severe. That inconsistency is present in the paper and matters for strict reproducibility.

4. Position within the AgenticIQA framework

AgenticIQA uses a shared VLM backbone for the Planner, Executor, and Summarizer roles. The paper states that Qwen2.5-VL-7B is fine-tuned on AgenticIQA-200K to produce Qwen2.5-VL*, thereby imparting “task-aligned agentic reasoning.” More precisely, the reported training uses AgenticIQA-200K together with Q-Instruct-200K for full-parameter fine-tuning under next-token prediction loss. The backbone is specified as Qwen2.5-VL with a CLIP-ViT-L14 vision encoder and a Qwen2.5-7B language decoder. The reported setup uses 2 epochs, batch size 512, learning rate $1\mathrm{e}{-5}$, a cosine learning rate schedule, and 4 $\times$ NVIDIA A100 80GB GPUs (Zhu et al., 30 Sep 2025).

The paper does not specify the mixture ratio between AgenticIQA-200K and Q-Instruct-200K, whether tasks are sampled uniformly, the prompt packing strategy, maximum token length, image resolution during training, or the exact preprocessing pipeline. It also does not provide a dedicated ablation varying the size of AgenticIQA-200K, removing planner/executor/summarizer subsets, or changing the source-data composition. Accordingly, the empirical effect of AgenticIQA-200K is shown only in combination with the broader AgenticIQA system and the additional Q-Instruct-200K corpus.

The measured gains are nevertheless role-specific and informative. On AgenticIQA-Eval, overall performance improves from 73.20% for Qwen2.5-VL to 75.40% for Qwen2.5-VL*. Planner performance rises from 74.40% to 76.80%; Executor / Distortion improves from 55.60% to 63.20%; Executor / Tool declines slightly from 78.00% to 76.40%; and Summarizer improves from 84.80% to 85.20%. The strongest gain is therefore on distortion-related executor capability, which aligns closely with the dataset’s explicit supervision of distortion inference and analysis. On LLVisionQA, the improvement is small overall, from 75.22% to 75.25%, although the paper claims stronger gains on contextual perceptual reasoning categories. For quality scoring, AgenticIQA with Qwen2.5-VL* reaches SRCC 0.7780 on TID2013, 0.7771 on BID, and 0.7165 on AGIQA-3K, but the paper also notes that these outcomes are not an isolated effect of the dataset, since scoring depends on the full agentic pipeline and tool integration.

5. Relation to earlier IQA datasets and task formulations

AgenticIQA-200K differs substantially from earlier IQA datasets that were built primarily for direct score prediction. A useful comparison point is AIGCIQA2023, which is a manually rated benchmark for AI-generated images rather than an instruction-tuning corpus. AIGCIQA2023 contains exactly 2,400 AI-generated images, generated from 100 text prompts using six text-to-image models—Glide, Lafite, DALLE, Stable-diffusion, Unidiffuser, and Controlnet—and provides separate MOS labels for quality, authenticity, and correspondence (Wang et al., 2023).

The contrast is methodological as well as scalar. AIGCIQA2023 collects human ratings from 28 graduate students under an ITU-R BT.500-14-style subjective protocol and evaluates no-reference IQA metrics against those MOS targets. AgenticIQA-200K, by contrast, is automatically generated using GPT-4o on top of pre-existing IQA image pools and is intended to supervise modular agent behavior rather than to serve as a human-judgment benchmark. AIGCIQA2023 asks models, in effect, to predict human preferences on three dimensions; AgenticIQA-200K asks a VLM to plan, analyze distortions, route to tools, and summarize evidence.

The relation between the two is still conceptually important. AIGCIQA2023 argues that AI-generated image assessment should be multi-dimensional—quality, authenticity, and correspondence—and shows that existing IQA methods are “far away from satisfactory,” with correspondence being the hardest target (Wang et al., 2023). A plausible implication is that AgenticIQA-200K extends the field in a different direction: instead of enlarging a MOS benchmark, it operationalizes IQA as a structured decision process in which query intent, distortion semantics, reference mode, and tool appropriateness become first-class supervised targets.

6. Limitations, release status, and unresolved details

Several limitations are explicit. AgenticIQA-200K is automatically generated using GPT-4o, so annotation quality depends on the reliability of that model. The authors report filtering out “apparent incorrect and irrelevant responses,” but they do not state how many examples were discarded, whether human annotators checked the outputs, how deduplication was handled, or how the final 140K intermediate QA pool maps exactly onto the 200K role-specific instructions. The paper also omits the number of unique images in the final release, the average number of instructions per image, the final split between full-reference and no-reference cases, the synthetic versus authentic ratio after balancing, the per-distortion histogram, and the split between local and global scope.

The dataset’s supervision of tool execution is indirect. The paper is explicit that actual runtime tool execution uses external IQA models; the dataset most directly supervises planning, distortion inference, severity-and-explanation reasoning, tool choice, and final summarization. It therefore should not be mistaken for a conventional regression dataset with raw numeric tool outputs attached to every sample.

Release information is also only partially specified. The paper provides a project page and code link at https://agenticiqa.github.io/. It explicitly states that AgenticIQA-Eval will be released with standardized evaluation scripts. For AgenticIQA-200K itself, release through the project page is strongly implied, but the text does not explicitly provide a dataset download URL or a formal statement of immediate public release.

In summary, AgenticIQA-200K is a 200K-example instruction corpus for agentic image quality assessment, derived from 55,620 synthetic images from DQ-495K and 10,797 authentic images from Q-Pathway, with an intermediate GPT-4o-generated pool of about 140K question-response pairs and a final balance of 50K planner, 100K executor, and 50K summarizer instances (Zhu et al., 30 Sep 2025). Its central contribution is not a new MOS benchmark, but a structured supervision scheme for training VLMs to behave as IQA agents through planning, distortion-centric analysis, tool-aware routing, and evidence-grounded summarization.