AVT-VQDB-UHD-1-NVC: 4K Video Quality Dataset

• AVT-VQDB-UHD-1-NVC is a rigorously constructed video quality assessment dataset featuring 4K/UHD sequences encoded by both traditional (AV1, VVC) and neural codecs (DCVC-FM, DCVC-RT).
• The dataset comprises 216 processed video sequences, methodically varied through resolutions and quantizer settings to target specific PSNR values across diverse content.
• It employs a robust evaluation framework with subjective assessments from 26 observers and objective metrics such as VMAF and PSNR to benchmark codec performance.

AVT-VQDB-UHD-1-NVC is a systematically designed video quality assessment dataset that targets both traditional and neural video codecs under realistic, high-resolution (4K/UHD-1) encoding conditions. As presented in "Evaluating Video Quality Metrics for Neural and Traditional Codecs using 4K/UHD-1 Videos" (Herb et al., 2 Nov 2025), it provides a rigorous ground truth for the evaluation of full-reference, hybrid, and no-reference video quality metrics, supporting direct comparison between state-of-the-art neural and conventional compression methods. The dataset, released following an open-science policy, enables further research into metric reliability, subjective-objective correlation, and cross-codec analysis for ultra-high-definition video.

1. Dataset Composition and Structure

AVT-VQDB-UHD-1-NVC comprises six source video sequences, each with a native resolution of 3840×2160 (4K/UHD-1), YUV 4:2:0, 8-bit precision, and 60 frames per second. The source clips, with durations of 8–10 seconds, were selected to reflect diverse content, though the specific content types are not enumerated. Four codecs were used:

• Traditional codecs: AV1 (AOM v3.12.0) and VVC (vvencFFapp v1.13.1)
• Neural codecs: DCVC-FM (commit b67129d) and DCVC-RT (commit 9b7acf7)

Each source video was encoded by all codecs at four output resolutions (2160p, 1080p, 720p, and 360p), employing nine quantizer parameter (QP) settings per codec, targeted to span three PSNR values at each resolution. This yields a comprehensive matrix of content, codec, resolution, and quality settings, resulting in 216 processed video sequences (PVS):

Codec	360p QP	720p QP	1080p QP	2160p QP
AV1	54	48 / 61	36 / 55 / 63	31 / 50 / 61
VVC	34	32 / 41	27 / 36 / 45	25 / 34 / 42
DCVC-FM	38	46 / 25	59 / 37 / 18	63 / 43 / 26
DCVC-RT	34	42 / 17	58 / 32 / 10	63 / 39 / 19

Each combination is crafted to ensure PSNR targets (35, 38, 41, and 44 dB, as applicable) are robustly sampled across resolutions and codec algorithms.

2. Subjective Quality Assessment Protocol

Subjective quality ratings were sourced from 30 paid participants (students and staff), all of whom had normal or corrected-to-normal vision confirmed via a FrACT10 test. Following the ITU-T P.910 recommendation, outlier rejection was performed by discarding subjects with a Pearson correlation $r < 0.75$ against the mean opinion score (MOS), resulting in 26 valid observers.

The assessment was executed in a controlled laboratory using an ASUS XG43UQ 43-inch UHD display, with a fixed viewing distance of 1.5× the screen height. Test stimuli were presented in randomized order using mpv with avrateNG. The rating followed a single-stimulus, 5-point Absolute Category Rating (ACR) protocol, requiring participants to rate each PVS individually. Each viewing session was approximately 45 minutes in duration, covering all 216 PVS.

3. Objective Quality Metrics and Quantitative Methodology

A broad evaluation of objective video quality models was undertaken, classifying them as full-reference (FR), hybrid, and no-reference (NR):

• FR Metrics: PSNR, SSIM, MS-SSIM, VMAF (standard and “no-enhancement-gain” neg), LPIPS, CVQA-FR
• Hybrid Metric: AVQBits∣H0∣f, which incorporates codec metadata (bitrate, output resolution, frame rate) and a spatial gradient uniformity feature (H0|f), using a linear model to estimate MOS.
• NR Metrics: MUSIQ, CVQA-NR, FasterVQA, Dover (technical branch), Q-Align

Metrics were evaluated for their predictive alignment with MOS across the 216 PVS. Statistical quantifiers included the Pearson correlation coefficient (PCC) and the Spearman rank-order correlation coefficient (SRCC), with closed-form definitions as supplied in the dataset materials:

• $MSE = (1/N) \sum_{i=1}^N (x_i - y_i)^2$
• $$PSNR = 10\, \log_{10} \left( \frac{MAX_I^2}{MSE} \right)$$
• $$r = \frac{\sum_{i=1}^N (x_i - \bar x)(y_i - \bar y)}{\sqrt{\sum (x_i - \bar x)^2 \cdot \sum (y_i - \bar y)^2}}$$
• $\rho = 1 - \frac{6 \sum d_i^2}{N(N^2-1)}$

VMAF (neg) computes without the “enhancement gain” heuristic to improve MOS alignment.

4. Comparative Results and Analysis

The benchmarking revealed the following global (all sequences/codecs) objective-to-subjective correlations:

Metric	PCC	SRCC
VMAF (neg)	0.889	0.909
VMAF	0.886	0.907
AVQBits∣H0∣f	0.887	0.861
FasterVQA (NR)	0.802	0.803
PSNR	0.750	0.768
SSIM	0.705	0.851
MS-SSIM	0.695	0.774
CVQA-FR	0.814	0.840
MUSIQ (NR)	0.664	0.683
Dover (NR)	0.634	0.629
CVQA-NR (NR)	0.469	0.491
Q-Align (NR)	0.245	0.263
LPIPS	0.646	0.716

For within-sequence (same-source) ranking:

• SRCC: PSNR (0.953, highest), VMAF (0.940), MS-SSIM (0.937), SSIM (0.936)
• PCC: MS-SSIM (0.978, highest), VMAF (0.970), VMAF (neg) (0.971), SSIM (0.964)

Importantly, VMAF remains the leading full-reference metric in both cross- and within-sequence prediction. AVQBits∣H0∣f achieves almost identical PCC by leveraging encoding parameters. FasterVQA demonstrates superior performance among tested NR models but still lags behind FR metrics. PSNR, while not the highest in global correlation, is optimal for within-sequence comparative tasks.

5. Codec-Agnosticism and Metric Reliability

The methodology included an explicit analysis of metric reliability across traditional (AV1, VVC) and neural (DCVC-FM, DCVC-RT) codecs using the $\Delta NvT$ metric, defined as $([M_N–M_T] – [MOS_N–MOS_T])$ for neural ($N$) minus traditional ($T$) codecs. The bias of most quality metrics between these codec classes remained within ±0.6 on the 5-point MOS scale, with no evidence of systematic over- or under-estimation of neural video codec (NVC) quality by modern metrics, excepting minor outliers for AVQBits∣H0∣f and CVQA-NR at extremely low bitrates. No significant degradation in metric performance was observed when applying these models to DCVC-FM or DCVC-RT compared to AV1 and VVC.

This suggests that, for video codecs compressing 4K content under comparable conditions, state-of-the-art objective quality metrics remain statistically consistent and reliable predictors of MOS regardless of underlying codec architecture.

6. Dataset Accessibility and Utility

The entirety of the AVT-VQDB-UHD-1-NVC dataset—the six original source videos, all 216 encoded outputs, MOS ground truth, and metric scores—has been publicly released under an open-science license and is available at https://github.com/Telecommunication-Telemedia-Assessment/AVT-VQDB-UHD-1-NVC. Researchers and industry practitioners are expressly encouraged to utilize and extend the dataset for further validation of video quality assessment models, cross-codec analysis, and development of novel VQA methodologies for UHD content.

A plausible implication is that this dataset will become a foundational benchmark for the rigorous validation of both traditional and neural-based video quality evaluation models, particularly in the context of emerging high-resolution, high-frame-rate video applications.