RealisVideo-4K Benchmark for 4K Super-Resolution

• RealisVideo-4K is a 4K video super-resolution benchmark that provides 1,000 high-definition video-text pairs curated for high-frequency detail recovery.
• It employs a two-order degradation pipeline to generate low-resolution inputs, enabling full-reference evaluation with metrics like PSNR, SSIM, LPIPS, and DISTS.
• The benchmark addresses limitations of 720P datasets by emphasizing detailed texture restoration and temporal consistency for ultra-high-resolution video.

Searching arXiv for the primary paper and closely related VSR benchmark context. RealisVideo-4K is a 4K video super-resolution benchmark introduced in “RealisVSR: Detail-enhanced Diffusion for Real-World 4K Video Super-Resolution” (Zhao et al., 25 Jul 2025). It is described as the first public, open-access 4K VSR benchmark containing 1,000 high-definition video-text pairs, with 4K defined as $2160\times3840$ pixels in the supplementary material. The benchmark was created to address a specific limitation in existing public VSR benchmarks—REDS, SPMCS, UDM10, YouTube-HQ, Vid4, and VideoLQ—which are predominantly 720P and lack ultra-fine high-frequency details. In the RealisVSR formulation, RealisVideo-4K provides detail-rich 4K ground-truth videos explicitly collected for evaluating high-frequency texture restoration and temporal fidelity, and it enables full-reference assessment at ultra-high resolution (Zhao et al., 25 Jul 2025).

1. Motivation and benchmark scope

RealisVideo-4K was motivated by three conditions identified in the VSR literature: existing benchmarks are mainly 720P, high-frequency detail recovery under complex real-world degradations remains limited, and current evaluation practice often relies on no-reference metrics that can be distorted by artifacts. The benchmark is therefore positioned around detail-rich 4K ground truth rather than around low-resolution-only collections or native 720P sources (Zhao et al., 25 Jul 2025).

The dataset is specifically curated to contain fine textures and structures that better stress high-frequency restoration. The paper gives examples such as hair strands and fabric patterns, and qualitative examples cite intricate textures on skin, eyebrows, glasses, necklaces, and poolside clutter. The stated target is “diverse scenes” with rich details, but the paper does not provide distributions over indoor versus outdoor scenes, motion types, or lighting conditions.

A central distinction of RealisVideo-4K is that it emphasizes full-reference evaluation at 4K. This directly addresses the paper’s criticism that no-reference metrics can be artifact-inflated. The benchmark therefore functions both as a dataset and as an evaluation protocol for ultra-high-resolution VSR, especially for detail enhancement and temporal fidelity under a controlled low-resolution generation process (Zhao et al., 25 Jul 2025).

2. Composition, curation, and annotations

RealisVideo-4K contains 1,000 videos, each paired with text annotations. The paper describes these as “video-text pairs.” The text is generated automatically by QWen2.5-VL and then filtered in two stages: a keyword-based filter selects detail-rich candidates using terms such as “close-up” and “detailed,” and manual filtering removes low-quality or wrongly annotated items (Zhao et al., 25 Jul 2025).

The benchmark includes a 140-video test set, randomly selected from the 1,000 videos, and this same test selection is used for evaluation on both the 4K benchmark and the derived 720P split. A companion downsampled 720P version, RealisVideo-720P, is derived from the 4K sources to preserve subpixel-level details typically lost in native 720P datasets. The paper presents this derived split as a stronger 720P benchmark for detail restoration.

Several dataset properties remain unspecified. The paper does not report total frame count, typical clip length, frame rate, bit depth, color space, compression or coding format, capture devices, or capture platforms. It also does not define a detailed caption schema, the number of captions per video, or alignment metadata between text and temporal segments. These omissions are part of the benchmark’s documented metadata gaps rather than an implicit omission by later summaries (Zhao et al., 25 Jul 2025).

A further technical point concerns degradation generation. The ground-truth videos are native 4K, while low-resolution inputs are constructed via a two-order video degradation pipeline, described as following RealBasicVSR, to form LR–HR pairs. The precise parameter settings of that degradation pipeline are not provided in the main text.

3. Evaluation protocol and metrics

The benchmark protocol centers on super-resolution from synthetically degraded low-resolution inputs back to the original 4K resolution. The paper does not state a fixed scaling factor such as $\times 4$; instead, the two-order degradation pipeline generates LR inputs that are restored to the original 4K HR target (Zhao et al., 25 Jul 2025).

A fairness rule is specified for cross-method comparison at 4K. For methods that cannot run at native 4K because of GPU constraints, outputs are generated at the largest feasible resolution and then upscaled via bicubic interpolation to 4K for evaluation. This provision is part of the benchmark’s effort to maintain comparability across methods with different inference limits.

The principal metrics reported on RealisVideo-4K are full-reference metrics: PSNR, SSIM, LPIPS, and DISTS. DOVER is also shown in the RealisVideo-4K results table, while MUSIQ and NIQE are reported for VideoLQ rather than for RealisVideo-4K in the main table. CLIP-IQA is mentioned as a metric used for real datasets in general, but it is not listed for RealisVideo-4K in the main results table. Warping error $E_{\text{warp}}$ is used for other datasets such as SPMCS, UDM10, REDS, and YouTube-HQ, but it is not reported for RealisVideo-4K.

The paper gives the standard definitions of PSNR and SSIM:

$$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$

where $x$ is the reference, $y$ the restored result, $N$ the number of pixels, and $L$ the dynamic range.

$$\mathrm{SSIM}(x,y)= \frac{(2\mu_x\mu_y+C_1)(2\sigma_{xy}+C_2)} {(\mu_x^2+\mu_y^2+C_1)(\sigma_x^2+\sigma_y^2+C_2)}$$

For LPIPS, DISTS, DOVER, NIQE, MUSIQ, and $E_{\text{warp}}$, the paper does not reproduce formulas and instead treats them as standard literature metrics (Zhao et al., 25 Jul 2025).

4. Relation to RealisVSR training and model design

RealisVideo-4K is not only an evaluation benchmark; it is also tied to the training regime of RealisVSR. The paper states that the model leverages Wan2.1’s spatio-temporal guidance and “requires only 5–25% of the training data volume compared to existing approaches.” The supplementary comparison gives the following data volumes: RealisVSR uses 50K clips, UAV uses approximately 372K clips, MGLD-VSR uses 300K clips, and STAR uses 200K clips. Within the paper’s own framing, this situates RealisVSR at 13–25% of several baselines’ data sizes, consistent with the abstract-level “5–25%” claim (Zhao et al., 25 Jul 2025).

The reported training pre-processing pipeline starts from 4K source videos, crops them to 480P patches using a predefined resolution bucket, and uses 49 frames per training clip. LR–HR pairs are then constructed with the same two-order video degradation pipeline. The supplementary material states that models are trained on RealisVideo-4K using 50K clips, but it does not resolve the discrepancy between 1,000 full videos and 50K training clips; whether the 50K are short segments from those 1,000 videos or include additional 4K sources is not explicitly clarified.

The benchmark is also linked to two technical components of RealisVSR. The first is the Consistency Preserved ControlNet (CPC) integrated with Wan2.1. CPC modifies ControlNet conditioning by discarding the noisy latent $\times 4$0 at the ControlNet input and using only the condition branch $\times 4$1:

$\times 4$2

Residual features are fused into the main network according to

$\times 4$3

where $\times 4$4 is a learnable scaling factor. The paper associates this design with improved temporal consistency and reduced artifacts at 4K.

The second component is the High-Frequency Rectified Diffusion Loss, or HR-Loss. The diffusion process is written as

$\times 4$5

with $\times 4$6 sampled from the real video distribution and $\times 4$7. The rectified flow loss is

$\times 4$8

The wavelet-based term is

$\times 4$9

where $E_{\text{warp}}$0, Haar kernels use low-pass $E_{\text{warp}}$1 and high-pass $E_{\text{warp}}$2, and the weights are $E_{\text{warp}}$3. The HOG-based term is

$E_{\text{warp}}$4

where orientations are discretized into 9 bins over $E_{\text{warp}}$5 with $E_{\text{warp}}$6 per bin and magnitudes are normalized via L2-Hys. The final loss is

$E_{\text{warp}}$7

The paper states that this loss accentuates high-frequency components while preserving structural coherence, leading to better texture recovery at 4K (Zhao et al., 25 Jul 2025).

5. Position within the VSR dataset landscape

RealisVideo-4K is presented against a set of existing benchmarks whose principal limitation is resolution and detail density. In the paper’s comparison, REDS, SPMCS, UDM10, YouTube-HQ, and Vid4 are 720P datasets with generally fewer high-frequency details, while VideoLQ is a real-world low-resolution dataset without ground truth and is therefore evaluated with no-reference metrics. RealisVideo-4K, by contrast, is a 4K $E_{\text{warp}}$8 dataset with very high detail, designed to stress high-frequency recovery and to support full-reference evaluation at 4K (Zhao et al., 25 Jul 2025).

The benchmark also occupies a different position from other 4K-oriented resources. MVSR4$E_{\text{warp}}$9 provides realistic, aligned real-world LR–HR video pairs captured with dual-lens smartphone hardware, but its HR resolution is 1080p rather than native 4K (Wang et al., 2022). The AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content introduced a 4K test set of 19 high-quality 4K YCbCr 4:2:0 sequences for validating efficient UHD approaches, especially 540p$$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$04K and 360p$$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$11080p on AV1-compressed inputs (Conde et al., 2024). RealisVideo-4K differs from both: it is a detail-rich 4K VSR benchmark with 1,000 video-text pairs and a consistent LR–HR construction pipeline for full-reference evaluation (Zhao et al., 25 Jul 2025).

Dataset or benchmark	Resolution emphasis	Pairing or supervision regime
RealisVideo-4K	4K $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$2, very high detail	Native 4K HR; LR synthesized by a two-order degradation pipeline
RealisVideo-720P	Downsampled from 4K sources	Intended to preserve subpixel-level details beyond native 720P datasets
MVSR4$$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$3	1080p HR, real-world $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$4	Aligned dual-lens smartphone LR–HR pairs
AIM 2024 4K test set	4K YCbCr 4:2:0	AV1-compressed evaluation set for efficient VSR

A frequent misconception is that RealisVideo-4K is a benchmark of native degraded low-resolution video. The paper does not describe such a setup. Its 4K HR videos are native, but the LR inputs used for evaluation are synthesized through a known two-order degradation pipeline. Another possible misconception is that the presence of “video-text pairs” implies a fully specified multimodal annotation schema; the paper states only that captions are derived by QWen2.5-VL and filtered, without providing a detailed schema or alignment metadata (Zhao et al., 25 Jul 2025).

6. Experimental results, significance, and limitations

On RealisVideo-4K, the main comparative table reports all metrics at 4K. RealisVSR achieves the best full-reference results among the listed methods: PSNR $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$5, SSIM $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$6, LPIPS $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$7, and DISTS $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$8. For comparison, RealViformer reports PSNR $$\mathrm{PSNR} = 10\cdot \log_{10}\!\left(\frac{L^2}{\mathrm{MSE}}\right), \qquad \mathrm{MSE}=\frac{1}{N}\sum (x_i-y_i)^2$$9, SSIM $x$0, LPIPS $x$1, and DISTS $x$2; Upscale-A-Video reports PSNR $x$3 and SSIM $x$4; MGLD-VSR reports PSNR $x$5; SeedVR reports PSNR $x$6; and STAR reports PSNR $x$7. On DOVER, RealisVSR is second-best at $x$8, behind SeedVR at $x$9 (Zhao et al., 25 Jul 2025).

The paper’s qualitative analysis states that RealisVSR restores intricate textures such as eyebrows, skin detail, necklaces, glasses, poolside clutter, and pet shadows, while removing degradations more effectively than Upscale-A-Video, RealViformer, VEnhancer, STAR, and SeedVR, with strong temporal coherence. No user study is reported.

The 720P counterpart also shows quantitative gains. On RealisVideo-720P, RealisVSR attains PSNR $y$0, SSIM $y$1, LPIPS $y$2, DISTS $y$3, and DOVER $y$4. The downsampled 720P benchmark is presented as preserving more subpixel details than native 720P datasets and therefore as a more demanding 720P testbed for detail restoration (Zhao et al., 25 Jul 2025).

The benchmark’s limitations are explicitly documented. Metadata such as frame rate, bit depth, color space, compression formats, and capture devices or platforms are not reported. The annotation pipeline includes automatically generated captions, so annotation noise remains possible even after manual filtering. Training and validation splits for the benchmark are not specified beyond the 140-video test set. Finally, although the benchmark is motivated by real-world degradations, its evaluation LR inputs are synthesized rather than paired with true degraded LR captures. The paper does not specify planned expansions.

Taken together, RealisVideo-4K fills a stated gap in 4K VSR benchmarking by providing detail-rich 4K ground truth, a consistent LR–HR construction pipeline, and full-reference evaluation at ultra-high resolution. Within the RealisVSR paper, its significance lies in supporting research on high-frequency detail enhancement and temporal consistency, which the authors identify as central goals for modern diffusion-based VSR systems (Zhao et al., 25 Jul 2025).