CoTIR-Bench: Chain-of-Thought Restoration Benchmark

• CoTIR-Bench is a comprehensive dataset featuring explicit chain-of-thought traces for image restoration tasks.
• It provides paired degraded and ground-truth images along with rich intermediate reasoning to support model training.
• The benchmark covers both single and composite degradation types, enabling consistent evaluation of restoration quality and reasoning coherence.

CoTIR-Bench (Chain-of-Thought Image Restoration Benchmark) is a large-scale, richly supervised dataset designed for the training, validation, and benchmarking of universal image restoration models that leverage explicit chain-of-thought (CoT) reasoning. Introduced in "Universal Image Restoration via Internalized Chain-of-Thought Reasoning," CoTIR-Bench provides end-to-end paired data with comprehensive intermediate reasoning traces, supporting both classic and composite image degradations at scale (Guo et al., 16 Jun 2026).

1. Scope, Purpose, and Design Objectives

CoTIR-Bench was constructed to address the limitations of traditional and multi-round restoration benchmarks, particularly with respect to weak modeling of interactions between multiple degradations and the computational costs of chaining specialized models. The benchmark aims to:

• Enable large-scale fine-tuning of generative restoration models using rich, intermediate reasoning supervision.
• Span both single degradation types (e.g., deblurring, denoising) and composite synthetic or real-world degradations.
• Provide a unified, standardized test set for consistent comparison across restoration methods regarding perceptual quality, fidelity, and reasoning coherence.

2. Dataset Composition and Degradation Types

CoTIR-Bench comprises approximately 5.20 million training image pairs, a 100,000-sample validation split, and a 2,000-image representative test split uniformly drawn from diverse degradation categories. The benchmark encompasses the following degradation types:

• Single degradations: Low-Light Enhancement (LLE), Light Adjustment (LA), Dehazing (DH, NHDH), Deraining (DR, DRD), Desnowing (DS), Reflection Removal (RR), Deshadowing (DSH), Deblurring (DB), Super-Resolution (SR), Denoising (DN), Underwater Enhancement (UE), Deflaring (DF), Dewatermarking (DW), DSLR Conversion (DSLR), Coloring (C), JPEG Artifact Repair (JPEG).
• Composite degradations: Randomly synthesized multi-degradation scenarios (e.g., LLE + DN, DR + JPEG, DH + SR + DB), reflecting practical restoration challenges.

Split	Number of Samples	Coverage
Training	≈5,200,000	Single/composite
Validation	100,000	Hyperparameter tuning
Testing	2,000	Uniform by category

The degradation distribution in the training split includes Deraining (DR, DRD) at 17%, Deblurring (DB) at 12%, Dehazing (DH, NHDH) at 11%, Denoising (DN) at 10%, with all other types between 2–8%, and composite mixtures accounting for 25%. Severity levels for each degradation are uniformly sampled from 1–5 ($\text{mean} \approx 3.0$, $\sigma \approx 1.2$).

3. Construction Methodology and Annotation Process

The dataset synthesizes its training pairs by re-processing over 60 publicly available restoration datasets, cropping/resizing all images to $512 \times 512$ pixels.

For each $(x, y, d_1, ..., d_k)$ sample (with $x$ as degraded input, $y$ as ground-truth, and $d_i$ as text tokens indicating degradations), structured CoT traces are generated as follows:

Automated CoT Trace Generation:

1. Instruction Preparation: A "precise prompt" ($\varphi_\mathrm{pre}$) specifies the degradations and instructs on restoration, feature description, and planning.
2. Chain-of-Thought Generation: The triplet $C = \{c_s, c_d, c_p\}$ is output by Qwen2.5-VL, where $c_s$ describes sharp features of $\sigma \approx 1.2$0, $\sigma \approx 1.2$1 identifies and localizes degradations in $\sigma \approx 1.2$2, and $\sigma \approx 1.2$3 describes an ordered restoration plan.
3. Prompt Augmentation: A "vague prompt" ($\sigma \approx 1.2$4) is generated via templates or GPT-5 Codex for diversity.
4. Filtering: CLIP-based consistency checks (score threshold 0.25) enforce sample relevance.

The dataset synthesis logic is:

$(x, y, d_1, ..., d_k)$1

4. Data Format, Annotation, and Tokenization

Each sample is provided as:

• images/ID_x.png: degraded image (PNG, uint8)
• images/ID_y.png: ground-truth clean image (PNG, uint8)
• meta/ID.json: UTF-8 JSON metadata, containing:
  • "id"
  • "degradation_types" (e.g., ["DR", "JPEG"])
  • "severity" (per-type, 1–5)
  • "prompt_precise" and "prompt_vague" (text)
  • "cot" with fields "c_s", "c_d", "c_p"

Tokenization for all text fields (prompts and chain-of-thought substeps) uses T5's SentencePiece vocabulary (32,000 subword units). During model training, substeps are processed by the CoT Adapter, projecting them into three separate embedding heads.

Mathematical encoding of CoT: For each triplet $\sigma \approx 1.2$5 stored with tolerance bounds $\sigma \approx 1.2$6, the restoration model predicts $\sigma \approx 1.2$7 and enforces

$\sigma \approx 1.2$8

Trace lengths: Mean tokens per sub-step: $\sigma \approx 1.2$9: $512 \times 512$0 ($512 \times 512$1), $512 \times 512$2: $512 \times 512$3 ($512 \times 512$4), $512 \times 512$5: $512 \times 512$6 ($512 \times 512$7). 90% of traces contain 5–30 tokens per sub-step, maximum observed is 60.

5. Evaluation Protocols and Benchmarking

CoTIR-Bench defines a rigorous benchmarking procedure, supporting both full-reference and no-reference evaluation metrics.

Full-Reference Metrics:

• PSNR (dB)
• SSIM
• LPIPS (lower is better)

No-Reference Metrics:

• CLIP-IQA+
• Q-Align (PSNR-like alignment in CLIP space)
• LIQE
• MACLIP

Benchmarking Procedure:

• Test set of 2,000 images, covering all degradation categories.
• All metrics reported on the entire set.
• CPU+GPU runtime measured on 512×512 inputs and averaged.
• Baselines include: PromptIR, OneRestore, InstructIR, DA-CLIP, HYPIR, LucidFlux, AutoDIR, DiffUIR, UniRestore, AgenticIR, Q-Agent (either with public checkpoints or retrained on CoTIR-Bench).

Submissions to the leaderboard are made by uploading restored images that match test filenames; a blind re-evaluation script outputs a JSON with all metric scores. The code repository provides all scripts and documentation.

6. Statistical Distribution and Data Characteristics

• Degradation Distribution: Deraining and derain-drop 17%, deblurring 12%, dehazing/non-heterogeneous dehazing 11%, denoising 10%; all other classes 2–8%; 25% of samples have composite mixtures (two or more degradations).
• Severity Levels: Uniform sampling from 1–5 for every degradation; average severity is approximately 3.0 (standard deviation 1.2).
• CoT Trace Lengths: Tokens per sub-step (mean ± standard deviation) — $512 \times 512$8: 12 ± 5; $512 \times 512$9: 15 ± 6; $(x, y, d_1, ..., d_k)$0: 18 ± 7.
• File Organization: Image data in PNG format, metadata in UTF-8 JSON; split files for train, val, and test enumerate all sample IDs.

7. Access, Licensing, and Usage Terms

CoTIR-Bench is distributed under a CC BY-NC-SA 4.0 license, limiting use to non-commercial, share-alike applications. The dataset and codebase are available at https://github.com/gy65896/CoTIR. Split files (train.txt, val.txt, test.txt) support reproducibility and controlled evaluation.

The recommended citation is:

Guo et al., “Universal Image Restoration via Internalized Chain-of-Thought Reasoning,” IEEE TPAMI, 2025.

Researchers are encouraged to use CoTIR-Bench as a standard for training, validating, and benchmarking universal CoT-driven restoration models under consistent and richly supervised settings (Guo et al., 16 Jun 2026).