HDRTV4K-Scene: HDR Video Reconstruction Benchmark

• HDRTV4K-Scene is a scene-segmented benchmark that organizes HDRTV4K into coherent sequences, facilitating evaluation of temporal dependencies in HDR video reconstruction.
• It groups frames by scene consistency with a minimum of 8 frames per scene, enabling scene-disjoint train/test splits and scene-specific memory use.
• The benchmark employs patch-based sampling and seven key metrics to assess fidelity, structure, color accuracy, perceptual quality, and temporal stability, with WMNet achieving state-of-the-art results.

HDRTV4K-Scene is a scene-segmented benchmark for HDR video reconstruction obtained by reorganizing the publicly accessible HDRTV4K training set into coherent sequences with scene-level train/test splits. It was introduced in the context of "Wavelet-Domain Masked Image Modeling for Color-Consistent HDR Video Reconstruction" to support temporally aware SDR-to-HDRTV reconstruction, particularly for models that require scene identities, adjacent-frame fusion, and memory across frames. In this formulation, the dataset is not a new raw capture corpus but a reorganized subset of HDRTV4K in which frames are grouped by scene consistency, enabling evaluation of color fidelity, temporal coherence, and long-range consistency in ways that frame-collection organization does not readily permit (Zhang et al., 7 Feb 2026).

1. Origin and Motivation

HDRTV4K-Scene derives from HDRTV4K, a dataset introduced for practical SDR-to-HDRTV up-conversion. HDRTV4K was described as a label HDRTV dataset for supervised learning, with 3878 frame pairs at HD–UHD resolution and 16bit lossless .tif RGB storage, manually chosen from >220 different videos clips (Guo et al., 2023). In the later WMNet work, HDRTV4K is characterized as containing paired LDR/HDR videos from HDRTV1K, user-generated content, and natural scenes, but not being organized by coherent scenes; frames from different shots are mixed, and the official test split is not publicly available (Zhang et al., 7 Feb 2026).

That organizational issue is central to the creation of HDRTV4K-Scene. Frame-level collections are adequate for per-frame learning, but they are problematic when the objective is to model scene-level temporal dependencies, evaluate temporal coherence, or maintain scene-specific memory. The reorganized benchmark therefore addresses two explicit requirements: learning from contiguous frame sequences and performing scene-disjoint evaluation for temporally consistent HDR reconstruction.

This restructuring also places HDRTV4K-Scene within a broader transition in SDRTV-to-HDRTV research. Earlier work emphasized frame-based conversion, including physically informed global color mapping and local enhancement on HDRTV1K (Chen et al., 2021), its later refinement in HDRTVNet++ (Chen et al., 2023), and joint super-resolution plus inverse tone-mapping in Deep SR-ITM (Kim et al., 2019). Multi-frame sequence design had already appeared in HDRTVMF1K, where scene segmentation was used to form 10-frame 4K HDR10 sequences for DSLNet (Xu et al., 2022). HDRTV4K-Scene differs in being specifically a scene-level reorganization of HDRTV4K for temporally aware HDR video reconstruction.

2. Dataset Definition and Composition

HDRTV4K-Scene is defined by grouping frames from the publicly accessible HDRTV4K training set according to scene consistency and enforcing that each scene contain at least 8 frames (Zhang et al., 7 Feb 2026). After filtering and grouping, the dataset contains 100 scenes and 1,092 paired LDR/HDR frames. These scenes are then split randomly at the scene level into training and testing subsets.

Split or property	Value
Total scenes	100
Total frame pairs	1,092
Training set	80 scenes, 881 LDR–HDR frame pairs
Test set	20 scenes, 211 LDR–HDR frame pairs
Minimum scene length	8 frames

A scene in HDRTV4K-Scene is described as a contiguous sequence of frames that share consistent content and conditions, such as the same shot or camera setting, and are long enough to support temporal modeling. The split is scene-disjoint rather than frame-disjoint, which is critical because WMNet’s Dynamic Memory Module keeps scene-specific memory keyed by scene name.

The benchmark inherits the signal-level properties of HDRTV4K. Original frames are available at 4K (2160×3840) and Full HD (1080×1920), and both resolutions are used. On the LDR side, the video data follows the YouTube compression algorithm, consistent with HDRTV1K, so the inputs are SDR BT.709 with YouTube-like compression. On the HDR side, the references follow the HDRTV4K standard, namely HDRTV in BT.2020, PQ, 10–12 bit, although the exact bit depth is inherited rather than restated. Each pair therefore consists of a compressed LDR frame and a corresponding HDRTV reference frame. Frame rate and camera metadata are not specified.

3. Pre-processing and Evaluation Protocol

Training and evaluation are patch-based for efficiency. From the original 2160×3840 or 1080×1920 frames, training data are pre-cropped into 128×128 patches with a step size of 240 to accelerate data loading. Patches are sampled from 8 adjacent frames per batch. No extra tone mapping or exposure normalization is applied; the paired LDR/HDR data are used directly. For HDR-VDP3 evaluation, color encoding is explicitly set to "rgb-bt.2020" (Zhang et al., 7 Feb 2026).

The benchmark protocol is scene-based throughout. WMNet is trained on the 80-scene training subset and tested on the 20-scene test subset. Per-frame metrics are aggregated over all 211 test-frame pairs, while the temporal metric is averaged over adjacent frame pairs. No explicit validation split is described, and hyperparameters are chosen via ablation on the HDRTV4K-Scene test set.

The evaluation suite comprises seven metrics designed to span signal fidelity, structure, color accuracy, perceptual quality, and temporal stability:

• PSNR: reported as the average over all test frames.
• SSIM: used both as an evaluation metric and as a training loss component.
• SR-SIM: structural similarity via spectral residual saliency.
• $\Delta E_{ITP}$: HDRTV-specific color difference in ITP space; lower is better.
• HDR-VDP3: computed with "side-by-side" comparison, "rgb-bt.2020" color encoding, 50 pixels/degree, and "led-lcd-wcg" display type.
• LPIPS: learned perceptual similarity; lower is better.
• $E_{warp}$: a temporal consistency measure based on optical flow from RAFT; lower values indicate better temporal coherence and less flicker.

This protocol makes HDRTV4K-Scene a benchmark for both spatial and temporal aspects of SDR-to-HDRTV reconstruction rather than only per-frame restoration.

4. Long-Sequence Extension and Relation to Neighboring Benchmarks

Because many HDRTV4K-Scene sequences contain only 8–15 frames, the WMNet study introduces HDRTV4K-LongScene, a longer-sequence subset collected following Chen et al. (HDRTVNet, ICCV 2021). It contains 10 new scenes, each with at least 26 frames, and is used specifically to evaluate long-range temporal consistency and memory behavior (Zhang et al., 7 Feb 2026).

This extension clarifies the scope of HDRTV4K-Scene itself. The base benchmark is sufficient for scene-aware temporal training and testing, but its relatively short sequences constrain evaluation of very long-term memory. A plausible implication is that HDRTV4K-Scene is best understood as a scene-consistent benchmark for short-to-moderate temporal context, while HDRTV4K-LongScene serves as its auxiliary long-context stress test.

Within the wider HDRTV literature, related datasets address adjacent but distinct needs. HDRTV1K was introduced as a paired 4K HDR10 image benchmark with 1235 training and 117 testing images, emphasizing display-referred SDRTV-to-HDRTV conversion at the same spatial resolution (Chen et al., 2021, Chen et al., 2023). HDRTVMF1K instead uses explicit scene segmentation to construct multi-frame 10-frame 4K HDR10 sequences, yielding 18,790 training sets and 74 test sequences for DSLNet (Xu et al., 2022). HDRTV4K, from which HDRTV4K-Scene is derived, was originally organized as a frame-based HDRTV training dataset rather than a temporal benchmark (Guo et al., 2023).

This suggests that HDRTV4K-Scene occupies a specific methodological niche: it adapts an HDRTV4K-style HDR supervision source to the needs of temporally consistent HDR video reconstruction without redefining the underlying SDR/HDR pairing standard.

5. Use in WMNet and Reported Results

HDRTV4K-Scene is the primary training and testing benchmark for WMNet, a network that combines Wavelet domain Masked Image Modeling, Temporal Mixture of Experts, and a Dynamic Memory Module (Zhang et al., 7 Feb 2026). Training follows a two-phase strategy.

In Phase I, W-MIM performs self-reconstruction pre-training on LDR video clips. Masking is applied in the wavelet domain on Haar DWT coefficients so that color and detail information are selectively masked. The model reconstructs the original LDR frames using a per-pixel L1 loss, and a curriculum gradually increases the masking ratio in the low-frequency band from 0 to 0.5. In Phase II, the encoder is initialized from the pre-trained weights and fine-tuned for SDR-to-HDRTV reconstruction. The full network includes the pretrained encoder, T-MoE between ResBlock groups, DMM with scene-specific memory keyed by scene name and default memory length $l=2$, and a decoder that outputs HDR frames. The HDR reconstruction loss combines L1 and SSIM with $\lambda=1$.

The implementation details reported for HDRTV4K-Scene are specific: 15 residual blocks in the encoder, channel size 64, batches of 8 consecutive frames with 16 patches, initial learning rate $2\times10^{-4}$, MultiStepLR halving every 20k iterations for a total of 100k iterations, and Adam with $\beta_1=0.9$ and $\beta_2=0.99$.

On the HDRTV4K-Scene test set, WMNet achieves the following reported results:

Metric	WMNet
PSNR	36.23 dB
SSIM	0.9630
SR-SIM	0.9987
$\Delta E_{ITP}$	10.78
HDR-VDP3	8.173
LPIPS	0.0641
$E_{warp}$	0.01392

The paper states that these results are state-of-the-art on HDRTV4K-Scene across fidelity, structure, color, perceptual, and temporal metrics. The comparison is made against traditional or LUT-based methods such as HuoPhyEO, KovaleskiEO, Ada-3DLUT, and CSRNet, as well as deep SDR-to-HDRTV methods including HDRTVNet, FMNet, and HDRTVNet++.

Ablation experiments on the same benchmark attribute gains to each principal module. The baseline without W-MIM, T-MoE, or DMM reports PSNR 35.15, SSIM 0.9612, and $\Delta E_{ITP}$ 12.11. Adding W-MIM yields 36.00 / 0.9627 / 11.05; adding T-MoE yields 36.17 / 0.9629 / 10.85; and adding DMM produces the full WMNet result of 36.23 / 0.9630 / 10.78. In this sense, HDRTV4K-Scene functions not only as a benchmark but also as the experimental substrate for studying the effects of pretraining, temporal fusion, and scene-specific memory.

6. Significance, Constraints, and Open Issues

HDRTV4K-Scene matters because it attaches scene identities and boundaries to a corpus previously organized primarily for frame-based learning. This enables scene-specific memory, fair scene-disjoint train/test splitting, and evaluation of temporal coherence rather than only per-frame PSNR or SSIM (Zhang et al., 7 Feb 2026). It is also described as establishing a new benchmark for HDR video reconstruction.

The benchmark is especially relevant to research that depends on temporal context. WMNet uses scene names to index its Dynamic Memory Module, so cross-scene leakage would be problematic if train and test scenes overlapped. Scene-level splitting therefore has methodological significance beyond dataset hygiene: it is bound to the correctness of the model’s memory mechanism.

Several limitations are explicit or directly inferable from the source description. Many scenes are short, which motivated the introduction of HDRTV4K-LongScene. HDRTV4K-Scene is a filtered subset of HDRTV4K, so some rare or difficult content may be excluded by the requirement that each scene contain at least 8 frames. Scene identifiers are used internally, but semantic metadata such as indoor/outdoor or day/night are not described. The paper also does not explicitly state whether the scene annotations or splits for HDRTV4K-Scene and HDRTV4K-LongScene are released, even though WMNet code is released. A plausible implication is that reproducibility depends not only on model code but also on whether the exact scene partition is made available.

In the larger research trajectory, HDRTV4K-Scene can be read as an intermediate point between frame-oriented HDRTV datasets and explicitly temporal 4K HDR video datasets. HDRTV4K emphasized strong HDR/WCG training targets and degradation modeling for practical SDR-to-HDRTV up-conversion (Guo et al., 2023); HDRTV1K and HDRTVNet++ emphasized formation-aware display-referred translation (Chen et al., 2021, Chen et al., 2023); HDRTVMF1K emphasized multi-frame 4K HDR10 scene sequences for feature fusion (Xu et al., 2022). HDRTV4K-Scene specializes that lineage toward scene-consistent HDR video reconstruction with explicit attention to color fidelity and temporal stability.

As a benchmark concept, HDRTV4K-Scene therefore denotes more than a renamed subset. It encodes a particular experimental assumption: that SDR-to-HDRTV reconstruction for video should be trained and evaluated on coherent scenes rather than shuffled frames, because color consistency, flicker suppression, and long-range temporal behavior are scene-level properties.