TID2013: Tampere Image Quality Benchmark

• TID2013 is a benchmark dataset consisting of 25 undistorted images and 3000 distorted images created via 24 controlled distortion types at 5 levels, enabling diverse IQA evaluations.
• It provides subjective quality scores obtained from pairwise comparisons, making it ideal for full-reference, reduced-reference, and no-reference quality assessment protocols.
• The dataset’s design supports detailed analysis of pooling strategies and contrast distortions, while highlighting challenges in generalizing controlled lab experiments to in-the-wild image quality.

TID2013, or the Tampere Image Database 2013, is a full-reference image quality assessment database built around 25 reference images, 24 distortion types, and 5 distortion levels per type, yielding 3000 distorted images with subjective quality annotations. In contemporary literature it functions simultaneously as a benchmark for full-reference, reduced-reference, and no-reference IQA, as a stress test for pooling and regression protocols, as a contrast-distortion benchmark, and as a controlled probe of whether learned visual representations align with human perceptual judgments (Talebi et al., 2017, Hernández-Cámara et al., 13 Aug 2025).

1. Canonical dataset design

The structural description of TID2013 is stable across the cited literature. It contains 25 original, undistorted reference images, and each reference is corrupted by 24 distortion types at 5 intensity levels, producing $25 \times 24 \times 5 = 3000$ distorted images (Talebi et al., 2017, Hernández-Cámara et al., 13 Aug 2025).

Component	Value	Notes
Reference images	25	original, undistorted
Distortion types	24	controlled synthetic degradations
Levels per distortion	5	per reference image
Distorted images	3000	$25 \times 24 \times 5$

The subjective annotation protocol is described in detail in work on NIMA. Observers see two distorted images at a time together with the reference image, choose which distorted image looks better, and assign the chosen image 1 point and the other 0. Each distorted image participates in 9 random pairwise comparisons, so image scores range from 0 to 9, and 985 experiments are carried out to obtain overall mean scores. The dataset provides mean opinion scores and standard deviations, but not the full rating histogram (Talebi et al., 2017).

In later work, these subjective scores are treated as the operational ground truth for human perception. One formulation writes the score for image $k$ as

$\text{MOS}_k = \frac{1}{M}\sum_{j=1}^{M} s_{j,k},$

where $s_{j,k}$ is the score assigned by observer $j$ to image $k$, and $M$ is the number of observers. In the perceptual-alignment setting, the MOS is used to quantify “how much humans see the difference” between a reference image and its distorted counterpart (Hernández-Cámara et al., 13 Aug 2025).

2. Distortion coverage and important subsets

A notable property of TID2013 is the breadth of its distortion taxonomy. One universal NR-IQA study explicitly enumerates the 24 distortion types as: additive Gaussian noise, additive white noise in color components, spatially correlated noise, masked noise, high frequency noise, impulse noise, quantization noise, Gaussian blur, image denoising, JPEG, JPEG2000, JPEG transmission errors, JP2K transmission errors, non eccentricity pattern noise, local block-wise distortions, mean shift, contrast change, change of color saturation, multiplicative Gaussian noise, comfort noise, lossy compression of noisy images, image color quantization with dither, chromatic aberrations, and sparse sampling and reconstruction (Ou et al., 2019).

This distortion inventory is broader than the distortion sets used in LIVE and CSIQ, and several papers exploit that breadth explicitly. In the pooling literature, TID2013 is the database in which differences between spatial pooling strategies are most statistically pronounced, and in universal NR-IQA it is treated as a challenging multi-distortion benchmark rather than a narrow synthetic test set (Temel et al., 2018, Ou et al., 2019).

A particularly important reuse pattern is the contrast-focused subset. In the MDM study, TID2013 contributes 125 images with global contrast change and 125 images with mean shift distortion, for a total of 250 contrast-distorted images. That paper emphasizes that TID2013 is the only dataset it uses that has more than one type of contrast distortion and known labels for each distortion type (Nafchi et al., 2018). A later pseudo-reference method uses the same two distortion families—contrast change and mean-shift—again obtaining a 250-image evaluation subset from all 25 references and all 5 levels (Mahmoudpour et al., 6 Oct 2025).

3. Evaluation paradigms built on TID2013

Although TID2013 is a full-reference database by design, the literature uses it under several distinct evaluation paradigms. In full-reference work, a local quality or distortion map is first computed between reference and distorted images—examples in the pooling study are squared error, SSIM, and PerSIM—and a spatial pooling rule is then applied to obtain a scalar score. In that setting, monotonic logistic regression is used for fair comparison in terms of linearity, and performance is reported with PLCC and SROCC (Temel et al., 2018).

In no-reference work, the reference image is not used at test time even though TID2013 was originally built around reference-based human experiments. NIMA is a representative example: it trains CNNs to predict image quality from the distorted image alone, using an 80%/20% train/test split, $256 \times 256$ rescaling, random $224 \times 224$ crops, and random horizontal flipping. Because TID2013 provides only the mean and standard deviation of opinion scores rather than full rating histograms, NIMA approximates the target score distributions through maximum entropy optimization and trains with an EMD-based loss over the score range $25 \times 24 \times 5$0 to $25 \times 24 \times 5$1 (Talebi et al., 2017).

Ranking-based pipelines treat TID2013 differently again. RankIQA uses large synthetically generated ranking datasets for Siamese pre-training, then extracts one branch and fine-tunes it on TID2013 MOS. That paper follows the HOSA protocol, splitting TID2013 by reference images so that 80% of the 25 references are used for training and 20% for testing, with all distorted versions of a given reference kept in the same split; at test time, 30 random sub-images per distorted image are averaged to obtain the final score (Liu et al., 2017). CLRIQA adopts a random 80%/20% split, 224-by-224 crops, and 60 random sub-images at test time, combining list-wise ranking pre-training with fine-tuning on original MOS labels (Ou et al., 2019).

TID2013 is also used for zero-shot representational analysis rather than IQA regression. In a ViT perceptual-alignment study, each original image and distorted image are passed through a pretrained ViT encoder, Euclidean distance between the feature vectors is computed for each reference–distorted pair, and Spearman rank correlation between those distances and the corresponding MOS values over all 3000 pairs is used as the perceptual alignment score. No task-specific retraining or regression head is added, so the protocol is explicitly zero-shot with respect to TID2013 (Hernández-Cámara et al., 13 Aug 2025).

4. Central role in no-reference IQA and deep learning

TID2013 has been used to support several different deep NR-IQA design philosophies. In NIMA, the database serves as a technical-quality training and evaluation source for distribution prediction rather than simple MOS regression. On TID2013, NIMA(VGG16) reports LCC $25 \times 24 \times 5$2 and SRCC $25 \times 24 \times 5$3 on mean scores, LCC $25 \times 24 \times 5$4 and SRCC $25 \times 24 \times 5$5 on score standard deviation, and EMD $25 \times 24 \times 5$6 for full distributions, while simpler NIMA backbones perform less strongly (Talebi et al., 2017).

RankIQA uses TID2013 mainly for fine-tuning and evaluation after ranking-based pre-training on synthetic distortions. Under the HOSA protocol, its VGG-16-based RankIQA+FT reaches an “ALL” SROCC of $25 \times 24 \times 5$7, compared with $25 \times 24 \times 5$8 for HOSA, which the paper describes as an improvement of over 5 percentage points in SROCC. The same study stresses that a VGG-16 baseline trained only on TID2013 underperforms both the ranking-pretrained model and the fine-tuned model, using TID2013 as evidence that deep NR-IQA benefits from auxiliary ranking supervision when labeled IQA data are limited (Liu et al., 2017).

CLRIQA extends the ranking idea to controllable list-wise ranking. On TID2013 it reports SROCC $25 \times 24 \times 5$9 and PLCC $k$0 after pre-training plus fine-tuning, and states that CLRIQA+FT is best in 18 out of 24 distortion types. The distortion-wise analysis is especially important for TID2013 because the paper highlights difficult categories such as JPEG transmission errors, non eccentricity pattern noise, mean shift, contrast change, change of color saturation, and comfort noise, on which many competing methods perform weakly (Ou et al., 2019).

Contrast-specific work uses a narrower subset of TID2013 but often extracts more task-specific conclusions. The MDM paper uses only the 250 contrast-distorted images and maps a three-dimensional feature vector—two Minkowski-based features and entropy—to MOS via SVR. On that subset it reports PCC $k$1 and SRC $k$2, and it further uses TID2013 as the only labeled source for contrast-distortion classification, reaching classification accuracy $k$3 at the 80%/20% train/test split (Nafchi et al., 2018). By contrast, the PRICCE method uses TID2013 only for evaluation, not training: its classifier is trained on Waterloo-generated data, and on the 250-image TID2013 contrast subset its MS-SSIM-based final configuration reports SROCC $k$4, KROCC $k$5, PLCC $k$6, and RMSE $k$7 (Mahmoudpour et al., 6 Oct 2025).

These results are not directly commensurate. They depend on different subsets, different split protocols, different target formulations, and in some cases different tasks altogether. This suggests that TID2013 is less a single leaderboard than a common experimental substrate on which distinct IQA problems can be posed.

5. Probe of perceptual alignment and pooling behavior

TID2013 is not only a scoring benchmark; it is also a probe of how models organize perceptual information. In the ViT alignment study, the database is used to compare zero-shot feature-space distances with human MOS. The reported pattern is counterintuitive from a pure recognition perspective: larger ViTs exhibit lower perceptual alignment; increasing dataset diversity has minimal impact when the total number of images seen is fixed; exposing models to the same images more times reduces alignment; stronger data augmentation consistently decreases alignment; and stronger regularization also reduces alignment, especially in heavily trained models. The relation between TID2013 alignment and ImageNet-1k classification accuracy forms an inverted U, with alignment initially increasing and then decreasing as accuracy improves (Hernández-Cámara et al., 13 Aug 2025).

A second recurring use is to study how local quality maps should be pooled. The comparative pooling study evaluates mean, min/max, percentile pooling, 5-number summary pooling, Minkowski pooling, quality/distortion-weighted pooling, information-weighted pooling, and the proposed weighted percentile pooling on TID2013. The paper reports that TID2013 yields the highest statistical significance total among the databases considered, with a database sum of 116, precisely because its wide range of distortion types allows pooling strategies to stand out. For TID2013 overall, weighted percentile pooling with PerSIM gives the best linearity, while Minkowski-based pooling with PerSIM gives the best ranking (Temel et al., 2018).

These two lines of work correct common oversimplifications. One oversimplification is that stronger discriminative training should automatically make learned features more human-like; the ViT results do not support that conclusion on TID2013. Another is that pooling is a minor implementation detail once a local IQA feature map is fixed; on TID2013, pooling can materially change PLCC and SROCC, especially for structural and perceptual similarity maps.

6. Scope, limitations, and continuing relevance

The literature also identifies clear boundaries to what TID2013 measures. The reference set is small by modern deep-learning standards, and one NIMA study explicitly calls it a relatively small dataset. Its images are predominantly traditional Kodak scenes with artificially applied distortions rather than large-scale in-the-wild imagery, and the absence of full rating histograms means that methods aiming to predict score distributions must reconstruct them indirectly from mean and standard deviation (Talebi et al., 2017).

The distortions are controlled and diverse, but they are still distortion-oriented. One perceptual-alignment study notes that TID2013 is centered on well-controlled low-level or mid-level degradations such as noise, compression artifacts, blur, and color distortions; it therefore emphasizes low-level perceptual similarity rather than high-level semantic alignment. The same study also notes limitations associated with any MOS-based dataset: observer variability, scale interpretation, and the fact that the benchmark reflects still-image perception under laboratory conditions rather than dynamic or task-dependent visual judgment (Hernández-Cámara et al., 13 Aug 2025).

Cross-dataset generalization results reinforce those limits. NIMA reports that models trained solely on TID2013 do not generalize well to AVA, with LCC around $k$8 and SRCC around $k$9, while AVA-trained models transfer better to TID2013 and LIVE (Talebi et al., 2017). A plausible implication is that TID2013 is strongest as a controlled benchmark for distortion-based quality and perceptual alignment, not as a complete surrogate for general visual preference or semantic image understanding.

Its continuing relevance follows from that specialization. Across the cited literature, TID2013 remains one of the classical IQA datasets alongside LIVE, CSIQ, KADID, and related benchmarks, but it is distinguished by the combination of 24 distortion types and 5 levels per distortion. That combination makes it valuable for controlled comparison of IQA models, pooling strategies, contrast-specific metrics, ranking-based pre-training schemes, and zero-shot perceptual alignment analyses, while also making clear that broader evaluation requires complementary datasets (Hernández-Cámara et al., 13 Aug 2025).