DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models

Abstract: Most digital videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in the over- and underexposed regions. To address this, we present DiffHDR, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task within the latent space of a video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance of the quantized pixels. Our framework further enables controllable LDR-to-HDR video conversion guided by text prompts or reference images. To address the scarcity of paired HDR video data, we develop a pipeline that synthesizes high-quality HDR video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR videos with considerable latitude for re-exposure.

• The paper introduces a generative approach that converts LDR videos to HDR using latent video diffusion and Log-Gamma color mapping.
• The paper employs exposure-aware controllability with region-specific prompts to ensure temporal consistency and precise radiance recovery.
• The paper demonstrates superior performance over prior methods, achieving high fidelity and structural consistency on diverse HDR benchmarks.

DiffHDR: Generative LDR-to-HDR Video Reconstruction via Video Diffusion Models

Introduction

The conversion of low dynamic range (LDR) video into high dynamic range (HDR) content remains a central challenge in computational photography and video post-production. LDR formats, pervasive in digital video acquisition and generative video synthesis, cannot encode the full radiance present in real-world scenes, leading to pronounced loss of detail in highlights and shadows due to quantization and saturation. Existing LDR-to-HDR solutions are either constrained by requirements for multi-exposure input or lack fidelity in the hallucinatory recovery of missing radiance, particularly in temporally coherent video content. The paper "DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models" (2604.06161) presents a generative approach to LDR-to-HDR video conversion, leveraging a latent-space video diffusion model, a domain-aligned Log-Gamma color mapping, and exposure-aware controllability mechanisms, enabling faithful, controllable HDR video reconstruction and re-exposure.

Figure 1: DiffHDR reconstructs lost radiance to convert LDR videos into faithful HDR while maintaining temporal coherence (top), and supports controllable HDR synthesis guided by text or reference images (bottom).

Methodology

DiffHDR reformulates LDR-to-HDR video conversion as a generative inpainting task within the latent space of a pretrained video diffusion model. The overall architecture is modular:

1. Log-Gamma Color Mapping: HDR radiance is compressed to align with the operational manifold of video VAEs pretrained on LDR data, without VAE finetuning or architectural modification.
2. Synthetic HDR Video Curation: To circumvent the scarcity of high-quality HDR supervision, the authors synthesize diverse, temporally-consistent HDR videos by rendering short trajectories from high-resolution panoramic HDR images (e.g., Polyhaven). Data augmentation simulates realistic LDR formation (exposure shift, noise, quantization, clipping).
3. Latent Video Diffusion: A LoRA-adapted (rank-32) VACE backbone is used for efficient adaptation to HDR generation, conditioned on LDR input and exposure masks.
4. Exposure-Aware Controllability: Luminance-based mask detection and context-focused prompting (with local cross-attention routing) enables fine-grained, region-specific reconstruction, driven by text or exemplar images.

   Figure 2: Framework of DiffHDR. LDR input is mapped to Log-Gamma space; a mask detector and context-focused module facilitate controllable reconstruction; the final output is an HDR video.

Log-Gamma Color Encoding

Standard video VAEs fail to encode/decode HDR content directly due to severe value range mismatch. The proposed Log-Gamma mapping compresses HDR values to the LDR-compatible regime while maintaining perceptual and color fidelity. Ablation demonstrates that this mapping yields superior PSNR/SSIM and lower LPIPS compared to linear or log-mapping alternatives, both visually and by error metrics.

Figure 3: Comparison of different color mapping methods by error visualization and metrics; Log-Gamma mapping achieves minimal error.

Training Pipeline

The HDR dataset is constructed by simulating camera intrinsics and motion in Blender over HDR panoramas, followed by LDR formation with realistic variability. Training objectives employ rectified flow-matching in latent space, ensuring temporal coherence and radiance fidelity. Only DiT LoRA adapters are finetuned; the VAE backbone remains fixed for maximum distributional alignment with large-scale video priors.

Exposure Masking and Semantic Control

Mask detection is performed by luminance thresholding with per-pixel EMA smoothing for temporal stability. During inference, context-focused cross-attention utilizes prompts of the form [overexposed: ...]; [underexposed: ...], decoupling guidance in explicitly saturated or shadowed regions. The model's residuals in masked regions are steered towards prompt-specific features, controlled by region-specific scaling coefficients.

Experimental Results

Quantitative Evaluation

On both synthetic (Polyhaven), established benchmarks (SI-HDR, Cinematic Video), in-the-wild videos, and diffusion-generated content (Veo2), DiffHDR delivers consistently stronger performance than prior state-of-the-art:

• SI-HDR: Best PU21-PIQE (19.37), and FID (18.68), with competitive HDR-VDP3.
• Cinematic/Synthetic Video: Best FovVideoVDP, DOVER, MUSIQ, and CLIPIQA.
• Generalization: On in-the-wild and generative videos, DiffHDR retains temporal consistency and radiance fidelity, outperforming LEDiff, SingleHDR, and other competitors across all reported metrics.

Notably, the method maintains temporal stability even under content with large de-exposed regions or dynamic range shifts.

Figure 4: Qualitative comparison on SI-HDR; DiffHDR restores highlight and shadow detail with structural consistency across exposures.

Figure 5: Qualitative comparison on real-world video; DiffHDR reconstructs high-frequency features in saturated regions and produces temporally stable results.

Ablation Studies

• Log-Gamma Mapping: When omitted, color and geometry in HDR reconstruction degrade severely; error maps localize this to areas of clipping and saturation.
• Exposure-Aware Augmentation and Masking: Without these, noise suppression weakens and shadow detail is lost.
• Context-Focused Prompting: Traditional global prompts cannot effectively guide region-specific hallucination; context-focused prompting recovers high-intensity structures (e.g., solar features) and enhances control.

  Figure 6: Ablation on data augmentation and mask detection; lack results in noise or texture loss.

  

  Figure 7: Ablation on context-focused prompting; only context-focused prompting reconstructs true solar and shadow detail.

Controllable Generation

DiffHDR supports text-guided and exemplar-driven HDR synthesis, enabling user-directed hallucination in ambiguous regions. This is critical for production pipelines and creative editing, as underlying radiance in saturated regions is fundamentally one-to-many.

Figure 8: DiffHDR enables text/image-conditioned HDR synthesis in masked regions.

Numerical Fidelity and Precision

FP16/BF16 inference in VAE leads to banding in smooth gradients in HDR; FP32 inference resolves this. Finetuning the VAE on HDR content leads to oversmoothing and high-frequency loss; thus, only LoRA-finetuning of DiT blocks is employed, preserving the backbone's high-fidelity priors.

Implications and Future Directions

DiffHDR represents a shift in LDR-to-HDR video reconstruction from deterministic regression to probabilistic, controllable inpainting in latent space, using foundation-scale temporal priors. The implications span:

• Practical: Enabling creative, semantically-driven re-exposure of legacy and generative LDR footage for HDR delivery, without multi-exposure input or specialized hardware.
• Theoretical: Demonstrating alignment pathways between HDR and LDR statistics via color mapping, and exposing the utility of region-specific cross-attention for controllable inpainting.
• Scalability/Future Work: Application to longer-form content, open-world dynamic range hallucination, and extension to segmentation- or editing-guided workflows are tractable. Integration with larger multimodal diffusion backbones may further improve both quality and controllability.

Conclusion

DiffHDR introduces a generative LDR-to-HDR video pipeline that unifies latent video diffusion, Log-Gamma encoding, and region-aware controllability, achieving state-of-the-art radiance reconstruction and semantic flexibility. By leveraging large-scale video priors and structured guidance, it produces temporally consistent, re-exposable HDR video, substantially improving over prior art in both perceptual and fidelity-oriented metrics. This paradigm is poised to unlock HDR post-production capabilities across legacy and synthetic video archives, and establishes new directions for controllable generative video-to-video translation.