AV2 Common Test Conditions Overview

• AV2 Common Test Conditions are a rigorous evaluation framework for the AV2 video coding standard, defining fixed datasets, QP ladders, and comprehensive metrics.
• The framework specifies multiple configurations including All-Intra, Random-Access, Low-Delay, and Adaptive-Streaming to benchmark compression performance across diverse content types.
• It integrates objective quality measures, BD-rate calculations, and convex-hull adaptive streaming analysis to quantify efficiency gains over the AV1 baseline.

The AV2 Common Test Conditions (CTC) are the quality and performance evaluation methodology defined for the AV2 video coding standard by the Alliance for Open Media (AOMedia). AV2 is intended to supersede AV1 and to deliver substantial compression efficiency gains across diverse media applications. The CTC specifies a reproducible benchmark framework spanning dataset selection, encoder operation, decoding, objective quality measurement, rate–distortion (RD) curve construction, Bjøntegaard-Delta Rate (BD-rate) computation, and complexity measurement. It also introduces new evaluation methods and content, including convex-hull-based adaptive streaming, user-generated content (UGC), and extended chroma formats (Lei et al., 15 May 2026).

1. Scope, purpose, and standardization role

The AV2 CTC prescribes a multi-stage, automated pipeline so that every new tool or release is measured under identical, reproducible conditions. Its function is not limited to reporting aggregate coding gains; it also constrains the evaluation environment by fixing the dataset, the operational configurations, the QP ladders, the resampling procedure for adaptive streaming, and the metric suite. In this form, the CTC acts as the normative performance envelope against which AV2 tools and releases are compared (Lei et al., 15 May 2026).

Four mandated video configurations are defined for moving-picture coding: All-Intra (AI), Random-Access (RA), Low-Delay (LD), and Adaptive-Streaming (AS). A separate Still Image configuration is also specified. The anchor for the reported compression results is the AV1 baseline identified as research-alt-v1_r4.0. The methodology additionally includes complexity measurement, with encoder and decoder wall-clock times measured relative to the AV1 anchor and checked against targets of at most $5\times$ for encoding and at most $2\times$ for decoding.

A plausible implication is that the CTC is designed less as an ad hoc benchmarking recipe than as a release-governance mechanism: by requiring the same evaluation pathway for all candidate tools, it supports comparability across successive AV2 versions and across independent laboratories.

2. End-to-end evaluation workflow

The workflow begins with source preparation. The official dataset of public test sequences is selected from Classes A–G and ECF in native color space, bit-depth, and chroma subsampling, and down-scaled versions are generated where adaptive streaming requires them (Lei et al., 15 May 2026).

Encoding is then performed with the AV2 encoder and the anchor AV1 encoder under the mandated configurations. Each configuration uses fixed six-point QP ladders, and non-normative techniques such as look-ahead, two-pass, and content-adaptive QP are disabled. This restriction is central to the methodology because it limits implementation-specific optimization paths that could otherwise confound codec-level comparisons.

After encoding, each bitstream is decoded into raw frames. For AS, decoded lower-resolution frames are upsampled back to the source resolution using a Lanczos-$\alpha=5$ filter with 14-bit precision in HDRTools. Objective metrics are then computed against the original source. The reported suite comprises PSNR-Y, PSNR-U, PSNR-V, weighted PSNR-YUV, PSNR-HVS, SSIM, MS-SSIM, CIEDE2000, VMAF, and CAMBI.

RD-curve construction differs by configuration. For AI, RA, and LD, each set of six $(\text{Rate}, \text{Quality})$ points is fitted with a piecewise cubic Hermite interpolating polynomial (PCHIP) in the log-rate domain. For AS, the six measured points per resolution are augmented by bilinear interpolation with seven intermediate QPs in log-rate space; all resolutions are then pooled, and the convex hull, interpreted as the Pareto frontier, is extracted across resolutions.

Coding gain computation follows by numerically integrating the difference between the test and anchor RD curves over their common quality range. Both per-plane BD-rate and weighted PSNR-YUV BD-rate are reported. The workflow concludes with complexity measurement, although the paper does not detail that stage further.

3. Content classes and coverage of operating regimes

The AV2 CTC dataset is divided into six major classes, each chosen to stress distinct coding conditions. All videos are publicly available at https://media.xiph.org/video/aomctc/test_set and are trimmed to a common frame count per class (Lei et al., 15 May 2026).

Class A contains natural camera-captured video at 270p–4K, 8/10-bit, 4:2:0. Class B contains synthetic screen content, including animation and game screen sharing, at 1080p, 8/10-bit, 4:2:0. Class E contains UGC up to 4K, 8-bit, 4:2:0, explicitly including diverse quality and pre-existing artefacts. Class F contains still images up to 8K, 8-bit, 4:2:0. Class G contains HDR material in BT.2100/PQ at 2160p, 10-bit, 4:2:0. Class ECF extends the test set to chroma-rich operating points, with six sub-classes covering 4:2:2, 4:4:4, SDR/HDR, RGB, and YCoCg-RE.

This partitioning is methodologically significant because it broadens evaluation beyond conventional camera-captured 4:2:0 SDR sequences. The inclusion of UGC and extended chroma formats indicates that the CTC is intended to cover deployment scenarios in which source impairments, chroma fidelity, and non-natural-image statistics materially affect codec behavior. A plausible implication is that aggregate efficiency claims under the CTC should be interpreted as cross-regime summaries rather than as results dominated by a single content family.

4. Test configurations, QP ladders, and adaptive-streaming convex hulls

Every configuration uses six QP ($qindex$) points to reduce interpolation bias. The exact ladders are fixed by configuration (Lei et al., 15 May 2026).

Configuration	QP ladder
Still Image	60, 85, 110, 135, 160, 185
All-Intra (AI)	85, 110, 135, 160, 185, 210
Random-Access (RA)	110, 135, 160, 185, 210, 235
Low-Delay (LD)	110, 135, 160, 185, 210, 235
Adaptive-Stream (AS)	110, 135, 160, 185, 210, 235

The AS configuration adds a multi-resolution construction absent from the single-resolution AI, RA, and LD tests. Each Class A 4K sequence is further down-sampled to five lower resolutions: $2560\times1440$, $1920\times1080$, $1280\times720$, $960\times540$, and $640\times360$. Each lower-resolution version is encoded at the six AS QPs in RA mode, then upsampled back to 4K, after which the multi-resolution convex hull is constructed.

The paper describes this convex-hull procedure in four steps: raw RD points at 4K, log-bitrate interpolation, merging points across resolutions, and extracting the Pareto-optimal envelope. This makes AS qualitatively different from a simple fixed-resolution random-access test. A plausible implication is that AS results characterize ladder-level rate–quality efficiency under resolution switching, rather than merely the coding efficiency of a single operating point.

5. Objective metrics and BD-rate formalism

The CTC reports seven core metrics, with PSNR variants occupying a central role. For mean squared error and PSNR, the paper gives (Lei et al., 15 May 2026)

$2\times$0

where $2\times$1 is the peak sample value, such as $2\times$2 for 8-bit video.

Weighted PSNR-YUV is defined as

$2\times$3

with $2\times$4 for 4:2:0, and with other weights used for 4:2:2 or 4:4:4. The remaining reported measures are PSNR-HVS, described as a perceptual extension of PSNR incorporating contrast sensitivity filters; SSIM and MS-SSIM; CIEDE2000, defined in L$2\times$5a$2\times$6b$2\times$7 space; VMAF, described as a fusion-based quality index combining multiple elementary metrics via machine learning; and CAMBI, used for flat-region artefact detection.

The CTC’s coding-gain statistic is the BD-rate $2\times$8, which measures average bitrate savings at the same quality level over a common interval $2\times$9. If $\alpha=5$0 and $\alpha=5$1 are the anchor and test log-rate curves fitted by PCHIP, then

$\alpha=5$2

To report a single PSNR-YUV BD-rate, the CTC uses

$\alpha=5$3

For each configuration and class, BD-rates are computed separately for low, medium, and high quality sub-ranges, namely QP1–4, QP2–5, and QP3–6, to verify consistency across bitrates. A plausible implication is that the framework attempts to distinguish broad average efficiency gains from gains concentrated in only one portion of the operating range.

6. Reported AV2 v13.0 gains, reproducibility, and trajectory

The paper reports overall BD-rate savings of AV2 v13.0 relative to the AV1 anchor, averaged over all 4:2:0 test sequences (Lei et al., 15 May 2026).

Configuration	PSNR-YUV (\%)	VMAF (\%)
All-Intra (AI)	22.32	23.58
Random-Access (RA)	29.81	33.79
Low-Delay (LD)	26.05	27.28
Adaptive-Stream (AS)	31.34	35.77
Still Image (Class F)	13.81	15.12

The headline random-access result is a BD-rate reduction of $\alpha=5$4 for PSNR-YUV and $\alpha=5$5 for VMAF. Adaptive streaming shows $\alpha=5$6 PSNR-YUV and $\alpha=5$7 VMAF savings. The paper further notes that screen content in Class B2 sees up to approximately $\alpha=5$8 PSNR-YUV gain in AI and approximately $\alpha=5$9 in RA; HDR in Class G yields approximately $(\text{Rate}, \text{Quality})$0 PSNR-YUV gain in RA; and extended chroma content in ECF consistently shows more than $(\text{Rate}, \text{Quality})$1 PSNR-YUV savings in RA across 4:2:2 and 4:4:4. For still-image coding, Class F achieves approximately $(\text{Rate}, \text{Quality})$2 VMAF BD-rate, with the paper stating that still-image coding benefits from inter-coding improvements.

Reproducibility is treated as a formal property of the framework. By fixing the exact dataset, the six-point QP ladders, the codec configuration flags, the tiling and threading parameters, the resampling filters, and metric versions such as libvmaf v3.1.0, HDRTools, and convexhull_framework, the CTC is said to guarantee that every laboratory can reproduce the reported results bit-for-bit. The paper identifies this transparency as critical for tracking AV2’s efficiency from v1.0 in 2021 through v13.0 in 2025, at which point RA and AS routinely save approximately $(\text{Rate}, \text{Quality})$3–$(\text{Rate}, \text{Quality})$4 bitrate over AV1 for the same perceptual quality.

Ongoing work is described as focusing on complexity–quality trade-offs, expansion beyond 4:2:0, and incorporation of next-generation perceptual metrics as they mature. Taken together, these elements position the AV2 CTC as a rigorously defined and publicly transparent benchmark framework whose methodological distinctiveness lies in combining diverse content classes, multiple operational configurations, fixed six-point QP ladders, a broad objective-metric suite, and convex-hull analysis for adaptive-streaming use cases.