---
title: 'HDRTV4K-Scene: HDR Video Reconstruction Benchmark'
url: https://www.emergentmind.com/topics/hdrtv4k-scene
type: topic
---

# HDRTV4K-Scene: HDR Video Reconstruction Benchmark

HDRTV4K-Scene is a scene-segmented benchmark for HDR video reconstruction obtained by reorganizing the publicly accessible HDRTV4K training set into coherent sequences with scene-level train/test splits. It was introduced in the context of "Wavelet-Domain Masked Image Modeling for Color-Consistent HDR Video Reconstruction" to support temporally aware SDR-to-HDRTV reconstruction, particularly for models that require scene identities, adjacent-frame fusion, and memory across frames. In this formulation, the dataset is not a new raw capture corpus but a reorganized subset of HDRTV4K in which frames are grouped by scene consistency, enabling evaluation of color fidelity, temporal coherence, and long-range consistency in ways that frame-collection organization does not readily permit [2602.07393].

## 1. Origin and Motivation

HDRTV4K-Scene derives from HDRTV4K, a dataset introduced for practical SDR-to-HDRTV up-conversion. HDRTV4K was described as a label HDRTV dataset for supervised learning, with **3878** frame pairs at **HD–UHD** resolution and **16bit lossless .tif RGB** storage, manually chosen from **>220 different videos clips** [2303.13031]. In the later WMNet work, HDRTV4K is characterized as containing paired LDR/HDR videos from HDRTV1K, user-generated content, and natural scenes, but not being organized by coherent scenes; frames from different shots are mixed, and the official test split is not publicly available [2602.07393].

That organizational issue is central to the creation of HDRTV4K-Scene. Frame-level collections are adequate for per-frame learning, but they are problematic when the objective is to model scene-level temporal dependencies, evaluate temporal coherence, or maintain scene-specific memory. The reorganized benchmark therefore addresses two explicit requirements: learning from contiguous frame sequences and performing scene-disjoint evaluation for temporally consistent HDR reconstruction.

This restructuring also places HDRTV4K-Scene within a broader transition in SDRTV-to-HDRTV research. Earlier work emphasized frame-based conversion, including physically informed global color mapping and local enhancement on HDRTV1K [2108.07978], its later refinement in HDRTVNet++ [2309.04084], and joint super-resolution plus inverse tone-mapping in Deep SR-ITM [1904.11176]. Multi-frame sequence design had already appeared in HDRTVMF1K, where scene segmentation was used to form 10-frame 4K HDR10 sequences for DSLNet [2211.02297]. HDRTV4K-Scene differs in being specifically a scene-level reorganization of HDRTV4K for temporally aware HDR video reconstruction.

## 2. Dataset Definition and Composition

HDRTV4K-Scene is defined by grouping frames from the publicly accessible HDRTV4K training set according to scene consistency and enforcing that each scene contain at least **8 frames** [2602.07393]. After filtering and grouping, the dataset contains **100** scenes and **1,092** paired LDR/HDR frames. These scenes are then split randomly at the scene level into training and testing subsets.

| Split or property | Value |
|---|---|
| Total scenes | 100 |
| Total frame pairs | 1,092 |
| Training set | 80 scenes, 881 LDR–HDR frame pairs |
| Test set | 20 scenes, 211 LDR–HDR frame pairs |
| Minimum scene length | 8 frames |

A scene in HDRTV4K-Scene is described as a contiguous sequence of frames that share consistent content and conditions, such as the same shot or camera setting, and are long enough to support temporal modeling. The split is scene-disjoint rather than frame-disjoint, which is critical because WMNet’s Dynamic Memory Module keeps scene-specific memory keyed by scene name.

The benchmark inherits the signal-level properties of HDRTV4K. Original frames are available at **4K (2160×3840)** and **Full HD (1080×1920)**, and both resolutions are used. On the LDR side, the video data follows the **YouTube compression algorithm**, consistent with HDRTV1K, so the inputs are SDR BT.709 with YouTube-like compression. On the HDR side, the references follow the HDRTV4K standard, namely HDRTV in **BT.2020, PQ, 10–12 bit**, although the exact bit depth is inherited rather than restated. Each pair therefore consists of a compressed LDR frame and a corresponding HDRTV reference frame. Frame rate and camera metadata are not specified.

## 3. Pre-processing and Evaluation Protocol

Training and evaluation are patch-based for efficiency. From the original **2160×3840** or **1080×1920** frames, training data are pre-cropped into **128×128** patches with a **step size of 240** to accelerate data loading. Patches are sampled from **8 adjacent frames** per batch. No extra tone mapping or exposure normalization is applied; the paired LDR/HDR data are used directly. For HDR-VDP3 evaluation, color encoding is explicitly set to `"rgb-bt.2020"` [2602.07393].

The benchmark protocol is scene-based throughout. WMNet is trained on the **80-scene** training subset and tested on the **20-scene** test subset. Per-frame metrics are aggregated over all **211** test-frame pairs, while the temporal metric is averaged over adjacent frame pairs. No explicit validation split is described, and hyperparameters are chosen via ablation on the HDRTV4K-Scene test set.

The evaluation suite comprises seven metrics designed to span signal fidelity, structure, color accuracy, perceptual quality, and temporal stability:

- **PSNR**: reported as the average over all test frames.
- **SSIM**: used both as an evaluation metric and as a training loss component.
- **SR-SIM**: structural similarity via spectral residual saliency.
- **\(\Delta E_{ITP}\)**: HDRTV-specific color difference in ITP space; lower is better.
- **HDR-VDP3**: computed with `"side-by-side"` comparison, `"rgb-bt.2020"` color encoding, **50 pixels/degree**, and `"led-lcd-wcg"` display type.
- **LPIPS**: learned perceptual similarity; lower is better.
- **\(E_{warp}\)**: a temporal consistency measure based on optical flow from RAFT; lower values indicate better temporal coherence and less flicker.

This protocol makes HDRTV4K-Scene a benchmark for both spatial and temporal aspects of SDR-to-HDRTV reconstruction rather than only per-frame restoration.

## 4. Long-Sequence Extension and Relation to Neighboring Benchmarks

Because many HDRTV4K-Scene sequences contain only **8–15 frames**, the WMNet study introduces **HDRTV4K-LongScene**, a longer-sequence subset collected following Chen et al. (HDRTVNet, ICCV 2021). It contains **10 new scenes**, each with at least **26 frames**, and is used specifically to evaluate long-range temporal consistency and memory behavior [2602.07393].

This extension clarifies the scope of HDRTV4K-Scene itself. The base benchmark is sufficient for scene-aware temporal training and testing, but its relatively short sequences constrain evaluation of very long-term memory. A plausible implication is that HDRTV4K-Scene is best understood as a scene-consistent benchmark for short-to-moderate temporal context, while HDRTV4K-LongScene serves as its auxiliary long-context stress test.

Within the wider HDRTV literature, related datasets address adjacent but distinct needs. HDRTV1K was introduced as a paired **4K** HDR10 image benchmark with **1235** training and **117** testing images, emphasizing display-referred SDRTV-to-HDRTV conversion at the same spatial resolution [2108.07978; 2309.04084]. HDRTVMF1K instead uses explicit scene segmentation to construct multi-frame **10-frame** 4K HDR10 sequences, yielding **18,790** training sets and **74** test sequences for DSLNet [2211.02297]. HDRTV4K, from which HDRTV4K-Scene is derived, was originally organized as a frame-based HDRTV training dataset rather than a temporal benchmark [2303.13031].

This suggests that HDRTV4K-Scene occupies a specific methodological niche: it adapts an HDRTV4K-style HDR supervision source to the needs of temporally consistent HDR video reconstruction without redefining the underlying SDR/HDR pairing standard.

## 5. Use in WMNet and Reported Results

HDRTV4K-Scene is the primary training and testing benchmark for WMNet, a network that combines Wavelet domain Masked Image Modeling, Temporal Mixture of Experts, and a Dynamic Memory Module [2602.07393]. Training follows a two-phase strategy.

In **Phase I**, W-MIM performs self-reconstruction pre-training on LDR video clips. Masking is applied in the wavelet domain on Haar DWT coefficients so that color and detail information are selectively masked. The model reconstructs the original LDR frames using a per-pixel L1 loss, and a curriculum gradually increases the masking ratio in the **low-frequency** band from **0** to **0.5**. In **Phase II**, the encoder is initialized from the pre-trained weights and fine-tuned for SDR-to-HDRTV reconstruction. The full network includes the pretrained encoder, T-MoE between ResBlock groups, DMM with **scene-specific memory** keyed by scene name and default memory length **\(l=2\)**, and a decoder that outputs HDR frames. The HDR reconstruction loss combines L1 and SSIM with **\(\lambda=1\)**.

The implementation details reported for HDRTV4K-Scene are specific: **15 residual blocks** in the encoder, **channel size 64**, batches of **8 consecutive frames** with **16 patches**, initial learning rate **\(2\times10^{-4}\)**, MultiStepLR halving every **20k** iterations for a total of **100k** iterations, and Adam with **\(\beta_1=0.9\)** and **\(\beta_2=0.99\)**.

On the HDRTV4K-Scene test set, WMNet achieves the following reported results:

| Metric | WMNet |
|---|---|
| PSNR | 36.23 dB |
| SSIM | 0.9630 |
| SR-SIM | 0.9987 |
| \(\Delta E_{ITP}\) | 10.78 |
| HDR-VDP3 | 8.173 |
| LPIPS | 0.0641 |
| \(E_{warp}\) | 0.01392 |

The paper states that these results are state-of-the-art on HDRTV4K-Scene across fidelity, structure, color, perceptual, and temporal metrics. The comparison is made against traditional or LUT-based methods such as HuoPhyEO, KovaleskiEO, Ada-3DLUT, and CSRNet, as well as deep SDR-to-HDRTV methods including HDRTVNet, FMNet, and HDRTVNet++.

Ablation experiments on the same benchmark attribute gains to each principal module. The baseline without W-MIM, T-MoE, or DMM reports **PSNR 35.15**, **SSIM 0.9612**, and **\(\Delta E_{ITP}\) 12.11**. Adding W-MIM yields **36.00 / 0.9627 / 11.05**; adding T-MoE yields **36.17 / 0.9629 / 10.85**; and adding DMM produces the full WMNet result of **36.23 / 0.9630 / 10.78**. In this sense, HDRTV4K-Scene functions not only as a benchmark but also as the experimental substrate for studying the effects of pretraining, temporal fusion, and scene-specific memory.

## 6. Significance, Constraints, and Open Issues

HDRTV4K-Scene matters because it attaches scene identities and boundaries to a corpus previously organized primarily for frame-based learning. This enables scene-specific memory, fair scene-disjoint train/test splitting, and evaluation of temporal coherence rather than only per-frame PSNR or SSIM [2602.07393]. It is also described as establishing a new benchmark for HDR video reconstruction.

The benchmark is especially relevant to research that depends on temporal context. WMNet uses scene names to index its Dynamic Memory Module, so cross-scene leakage would be problematic if train and test scenes overlapped. Scene-level splitting therefore has methodological significance beyond dataset hygiene: it is bound to the correctness of the model’s memory mechanism.

Several limitations are explicit or directly inferable from the source description. Many scenes are short, which motivated the introduction of HDRTV4K-LongScene. HDRTV4K-Scene is a filtered subset of HDRTV4K, so some rare or difficult content may be excluded by the requirement that each scene contain at least **8 frames**. Scene identifiers are used internally, but semantic metadata such as indoor/outdoor or day/night are not described. The paper also does not explicitly state whether the scene annotations or splits for HDRTV4K-Scene and HDRTV4K-LongScene are released, even though WMNet code is released. A plausible implication is that reproducibility depends not only on model code but also on whether the exact scene partition is made available.

In the larger research trajectory, HDRTV4K-Scene can be read as an intermediate point between frame-oriented HDRTV datasets and explicitly temporal 4K HDR video datasets. HDRTV4K emphasized strong HDR/WCG training targets and degradation modeling for practical SDR-to-HDRTV up-conversion [2303.13031]; HDRTV1K and HDRTVNet++ emphasized formation-aware display-referred translation [2108.07978; 2309.04084]; HDRTVMF1K emphasized multi-frame 4K HDR10 scene sequences for feature fusion [2211.02297]. HDRTV4K-Scene specializes that lineage toward scene-consistent HDR video reconstruction with explicit attention to color fidelity and temporal stability.

As a benchmark concept, HDRTV4K-Scene therefore denotes more than a renamed subset. It encodes a particular experimental assumption: that SDR-to-HDRTV reconstruction for video should be trained and evaluated on coherent scenes rather than shuffled frames, because color consistency, flicker suppression, and long-range temporal behavior are scene-level properties.

Source: https://www.emergentmind.com/topics/hdrtv4k-scene