DiffHDR: Video Diffusion HDR Reconstruction

• DiffHDR is a video-native HDR conversion method that treats LDR-to-HDR transformation as generative radiance inpainting using a pretrained video diffusion model.
• It employs a bounded Log-Gamma representation along with exposure-aware masks and context-focused attention to ensure temporal coherence and controllable editing of bright and dark regions.
• The approach demonstrates superior perceptual performance over baselines, though it faces challenges with clipped regions and potential domain gaps from its synthetic HDR training dataset.

DiffHDR most specifically denotes the HDR video method introduced in “DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models,” which converts a single 8-bit low-dynamic-range (LDR) video into a temporally coherent high-dynamic-range (HDR) video by treating LDR-to-HDR conversion as generative radiance inpainting in the latent space of a pretrained video diffusion model (Yu et al., 7 Apr 2026). The framework is built around a video-native generative prior, a bounded Log-Gamma representation for HDR radiance, exposure-aware conditioning, and a synthetic HDR video training pipeline designed to compensate for the scarcity of paired HDR video data. In the same broader literature, the name should not be conflated with “Single-Shot HDR Recovery via a Video Diffusion Prior,” whose method is not called DiffHDR (Talegaonkar et al., 12 May 2026).

1. Scope, nomenclature, and problem setting

In the HDR imaging literature, DiffHDR addresses the conversion of ordinary 8-bit sRGB LDR video into HDR video suitable for re-exposure, grading, tone mapping, and HDR display (Yu et al., 7 Apr 2026). Its task differs from multi-exposure HDR fusion, which assumes access to bracketed captures, and from single-image inverse tone mapping, which predicts HDR from one image without explicit temporal modeling. The motivating observation is that most digital videos are stored in 8-bit LDR form, where highlight clipping, shadow crushing, and quantization eliminate much of the original scene radiance. This prevents accurate luminance reproduction on HDR displays and limits meaningful post-production re-exposure.

The method frames these losses as a one-to-many inverse problem. Overexposed regions collapse distinct high radiances to saturated values; underexposed regions suppress structure into near-black noise; quantization further destroys continuity, especially in dark regions. DiffHDR therefore rejects deterministic regression as the primary formulation and instead treats the problem as generative radiance inpainting: visible, non-clipped regions provide partial evidence, while missing or unreliable radiance is synthesized by a pretrained video diffusion prior under explicit exposure-aware conditioning (Yu et al., 7 Apr 2026).

The term “DiffHDR” is also ambiguous across neighboring literatures. It names an unrelated diffusion network for historical document repair (Yang et al., 2024), and it should also be distinguished from the single-image HDR method of “Single-Shot HDR Recovery via a Video Diffusion Prior,” which reformulates single-shot HDR as conditional video generation of an exposure bracket rather than using the name DiffHDR (Talegaonkar et al., 12 May 2026). This suggests that, within HDR imaging, DiffHDR is best understood as a video-native radiance-inpainting framework rather than a generic label for all diffusion-based HDR methods.

2. Generative radiance inpainting and Log-Gamma representation

A central design problem is compatibility between HDR radiance and pretrained LDR video latents. DiffHDR does not finetune the video VAE for HDR. Instead, it maps linear HDR radiance into a bounded Log-Gamma color space so that the transformed data remain compatible with the latent statistics expected by the pretrained Wan-2.1-VAE (Yu et al., 7 Apr 2026). The paper reports that direct HDR encoding fails badly, whereas the Log-Gamma mapping preserves VAE compatibility without VAE finetuning and outperforms direct linear encoding, pure log mapping, and a log-only variant without gamma compression.

The input pipeline begins with inverse sRGB linearization and Rec.709 luminance computation. Overexposed and underexposed regions are detected by thresholding luminance at $\tau_{high}=0.95$ and $\tau_{low}=0.05$, respectively. To stabilize mask conditioning over time, DiffHDR uses exponential smoothing:

$$\tilde{M}_t = \alpha M_t + (1-\alpha)\tilde{M}_{t-1},$$

with $\alpha=0.7$ (Yu et al., 7 Apr 2026). The masks identify regions likely to require hallucinated radiance and also support region-specific prompting.

The latent diffusion stage is formulated with rectified flow rather than DDPM notation. Given an HDR latent $\mathbf{x}_1$, Gaussian noise $\mathbf{x}_0 \sim \mathcal{N}(0,\mathbf{I})$, and timestep $t \in [0,1]$, the interpolated latent is

$\mathbf{x}_t = t\mathbf{x}_1 + (1-t)\mathbf{x}_0,$

and the training objective is

$$\mathcal{L} = \mathbb{E}_{\mathbf{x}_0,\mathbf{x}_1,t}\left\|u_\Theta(\mathbf{x}_t,t,\mathbf{c}) - (\mathbf{x}_1-\mathbf{x}_0)\right\|_2^2,$$

where $\mathbf{c}$ contains the LDR input, exposure masks, and optional text or reference-image conditions (Yu et al., 7 Apr 2026). This formulation makes DiffHDR a conditional latent video generation system whose target is HDR radiance expressed in a bounded representation rather than raw linear HDR pixels.

3. Architecture and conditioning mechanisms

DiffHDR is built on VACE with Wan-2.1 components: the pretrained backbone is Wan-2.1-VACE-14B, and the latent autoencoder is Wan-2.1-VAE (Yu et al., 7 Apr 2026). VACE supplies a Video Condition Unit that unifies context video, masks, text, and reference-image inputs. The encoded LDR video is sent through the context branch, while the main latent branch starts from Gaussian noise and is denoised into an HDR latent. Because the model is video-native, temporal coherence is learned jointly in latent space rather than being enforced only by post hoc temporal losses.

The VAE has a spatiotemporal compression ratio of $\tau_{low}=0.05$0, and the VAE is kept frozen. The paper explicitly reports that VAE finetuning oversmooths high-frequency detail, while BF16 VAE decoding causes banding, so FP32 is used for the VAE and BF16 only for DiT finetuning (Yu et al., 7 Apr 2026). To preserve pretrained priors, DiffHDR freezes backbone parameters and inserts rank-32 LoRA modules into attention and feed-forward layers of the DiT blocks.

Conditioning extends beyond simple context-video input. Text prompts are represented in a structured form separating bright and dark failure modes:

$\tau_{low}=0.05$1

Reference images are VAE-encoded and concatenated along the temporal dimension. At inference, DiffHDR adds Context-Focused Attention, which routes different text embeddings into different spatial regions by combining base, overexposed, and underexposed cross-attention outputs with the exposure masks (Yu et al., 7 Apr 2026). This yields region-specific semantic control without retraining. A key implication is that DiffHDR is simultaneously a reconstruction system and a controllable HDR editing framework.

4. Synthetic HDR video data and optimization

Because paired HDR video data are scarce, DiffHDR constructs a synthetic training corpus from static HDR panoramas. The source is 16K HDRIs from Polyhaven; about 800 HDRIs are rendered in Blender to produce approximately 5400 HDR video sequences, each with 81 frames, in linear color space (Rec.709) (Yu et al., 7 Apr 2026). The camera motion patterns are deliberately designed to stress bright and dark regions: highlight-focused zoom-in/out, shadow-focused zoom-in/out, and camera rotations covering 360° via three 120° segments.

LDR supervision is synthesized from the rendered HDR videos using a camera/exposure simulation pipeline. A random exposure shift $\tau_{low}=0.05$2 stops scales linear intensity by $\tau_{low}=0.05$3. Heteroscedastic Gaussian sensor noise follows CBDNet:

$\tau_{low}=0.05$4

with $\tau_{low}=0.05$5, $\tau_{low}=0.05$6, and temporal correlation introduced through an AR(1) process with $\tau_{low}=0.05$7 (Yu et al., 7 Apr 2026). Final LDR videos are obtained by converting to sRGB, clipping to $\tau_{low}=0.05$8, and quantizing to 8-bit precision. The result is a paired HDR/LDR dataset containing clipping, noise, and quantization artifacts while preserving temporal consistency.

Text conditioning for training is produced automatically using Qwen3-VL after Reinhard tone mapping. Captions follow the same structured overexposed/underexposed format. Optimization uses AdamW with constant learning rate $\tau_{low}=0.05$9 for 10,000 steps on 8 NVIDIA A100 GPUs, at spatiotemporal resolution $$\tilde{M}_t = \alpha M_t + (1-\alpha)\tilde{M}_{t-1},$$0 (Yu et al., 7 Apr 2026). The paper does not report auxiliary HDR-specific perceptual or adversarial losses beyond the rectified flow objective.

5. Empirical performance, ablations, and observed behavior

DiffHDR is evaluated on SI-HDR, the Cinematic Video dataset, 50 held-out Polyhaven synthetic videos, 50 in-the-wild videos from Pexels, and 10 videos generated by Veo2 (Yu et al., 7 Apr 2026). On SI-HDR it reports the best PU21-PIQE and FID among compared methods, with HDR-VDP3 of 6.98, PU21-PIQE of 19.37, and FID of 18.68. SingleHDR attains a slightly higher HDR-VDP3 of 7.37, but DiffHDR leads on the perceptual metrics emphasized by the paper. On Cinematic and Polyhaven synthetic videos, DiffHDR is best on all reported FovVideoVDP, DOVER, MUSIQ, and CLIPIQA scores. On the in-the-wild and Veo2 sets, it is likewise best across all reported non-reference metrics (Yu et al., 7 Apr 2026).

The qualitative case centers on three effects. First, highlights: saturated skies, bright windows, and the sun retain broader highlight radiance after re-exposure. Second, shadows: the method both denoises dark regions and restores structure. Third, temporal stability: prior frame-based baselines such as LEDiff and SingleHDR are reported to flicker in high-intensity regions, whereas DiffHDR is more stable because the prior is video-native (Yu et al., 7 Apr 2026).

The ablation study identifies Log-Gamma mapping as the main representational enabler. In VAE reconstruction, the full Log-Gamma design reaches PSNR 32.86, SSIM 0.86, and LPIPS 0.15, compared with 22.16/0.74/0.28 for direct linear HDR, 14.61/0.74/0.57 for pure log, and 25.38/0.75/0.34 for the log-only variant without gamma (Yu et al., 7 Apr 2026). Removing either data augmentation or mask guidance slightly degrades FovVideoVDP, DOVER, MUSIQ, and CLIPIQA. Context-focused prompting is assessed qualitatively and is reported to improve recovery of targeted bright and dark regions.

Notably, the paper does not provide detailed inference timing or FLOP counts. A plausible implication is that the 14B video backbone remains computationally heavy even though it substantially improves temporal coherence and controllability.

6. Position within diffusion-based HDR research and limitations

DiffHDR occupies a specific position within the diffusion HDR literature. Relative to “Diffusion-Promoted HDR Video Reconstruction” (HDR-V-Diff), which combines a single-frame HDR latent diffusion prior with an explicit temporal alignment and fusion branch, DiffHDR uses a video-native generative model and frames the task directly as latent radiance inpainting (Guan et al., 2024). Relative to single-image systems such as LEDiff, Bracket Diffusion, X2HDR, GMODiff, and LatentHDR, DiffHDR is distinguished by operating on videos, using a pretrained video diffusion prior, and targeting temporal coherence rather than only image fidelity (Wang et al., 2024, Bemana et al., 2024, Wu et al., 4 Feb 2026, Hu et al., 18 Dec 2025, Fekri et al., 11 May 2026). Relative to the single-image “Single-Shot HDR Recovery via a Video Diffusion Prior,” which treats exposure order as a synthetic temporal axis and reconstructs HDR by generating and fusing a five-frame exposure bracket, DiffHDR instead models actual video sequences and performs radiance inpainting directly in video latent space (Talegaonkar et al., 12 May 2026).

Several limitations are explicit. Reconstructed HDR in clipped regions remains plausible rather than physically verified. The training corpus, although large for HDR video, is synthesized from static HDRIs with camera motion, so a domain gap to true dynamic scenes remains. Dependence on mask quality means incorrect exposure masks can misguide generation. Prompt- and reference-conditioned modes increase controllability but also make the output less strictly objective as a restoration. Precision sensitivity is nontrivial: BF16 VAE decoding introduces banding, requiring FP32 for stable HDR decode (Yu et al., 7 Apr 2026).

This broader comparison suggests a methodological split in the field. Some systems reconstruct HDR by synthesizing or refining interpretable exposure stacks and then fusing them (Bemana et al., 2024, Talegaonkar et al., 12 May 2026, Fekri et al., 11 May 2026). Others adapt LDR-pretrained diffusion pipelines to HDR-friendly operating spaces such as PU21, PQ, or gain maps (Wu et al., 4 Feb 2026, Hu et al., 18 Dec 2025). DiffHDR instead leverages a pretrained video generative prior directly, with Log-Gamma serving as the bridge between HDR radiance and LDR-trained latents (Yu et al., 7 Apr 2026). Within that taxonomy, its main contribution is the combination of video-native diffusion, bounded HDR representation, exposure-aware masks, and inference-time controllability for temporally coherent HDR video reconstruction.