Full Dynamic Range Sky-Modelling For Image Based Lighting

Abstract: Accurate environment maps are a key component to modelling real-world outdoor scenes. They enable captivating visual arts, immersive virtual reality and a wide range of scientific and engineering applications. To alleviate the burden of physical-capture, physically-simulation and volumetric rendering, sky-models have been proposed as fast, flexible, and cost-saving alternatives. In recent years, sky-models have been extended through deep learning to be more comprehensive and inclusive of cloud formations, but recent work has demonstrated these models fall short in faithfully recreating accurate and photorealistic natural skies. Particularly at higher resolutions, DNN sky-models struggle to accurately model the 14EV+ class-imbalanced solar region, resulting in poor visual quality and scenes illuminated with skewed light transmission, shadows and tones. In this work, we propose Icarus, an all-weather sky-model capable of learning the exposure range of Full Dynamic Range (FDR) physically captured outdoor imagery. Our model allows conditional generation of environment maps with intuitive user-positioning of solar and cloud formations, and extends on current state-of-the-art to enable user-controlled texturing of atmospheric formations. Through our evaluation, we demonstrate Icarus is interchangeable with FDR physically captured outdoor imagery or parametric sky-models, and illuminates scenes with unprecedented accuracy, photorealism, lighting directionality (shadows), and tones in Image Based Lightning (IBL).

• The paper introduces Icarus, a deep neural network that models full dynamic range skies using LDR exposure bracketing to overcome high-exposure class imbalance.
• It employs a novel exposure decay training scheduler and multi-pathway discriminators to achieve robust HDR reconstruction and faithful solar illumination.
• Results demonstrate superior visual fidelity and illumination metrics compared to traditional sky-models, offering a drop-in replacement for laboriously captured FDR HDR images.

Full Dynamic Range Sky-Modelling for Image Based Lighting: An Expert Overview

Introduction

The paper "Full Dynamic Range Sky-Modelling For Image Based Lighting" (2603.05758) introduces Icarus, an all-weather deep neural network (DNN) sky-model targeting the generation of environment maps with the exposure range and stochastic properties of physically captured, full dynamic range (FDR) outdoor imagery. This work addresses the fundamental limitations in existing sky-models for image-based lighting (IBL): the inability to accurately model the solar high-exposure class and maintain faithful illumination, structured shadows, and chromatic tones. The proposed Icarus framework enables not only high-fidelity rendering with robust handling of the solar region, but also user-driven control of atmospheric and solar features through learned style codes.

Background and Problem Statement

Environment maps are integral to reproducing realistic outdoor illumination in rendering and virtual environments. The standard for photo-realistic and physically plausible IBL remains FDR imagery acquired through arduous physical capture with HDR cameras and bracketed exposures. Existing numerical and parametric sky-models—including Perez, Preetham, and Hošek-Wilkie—are limited by analytic tractability, offering only clear sky or coarse weather generalization. Recent DNN-based models expanded the representational capacity to weathered skies, but these struggle with class-imbalance, especially for the solar region. High dynamic range (HDR) compression via standard tone mapping operators proves inadequate; the non-linear error amplification from LDR to HDR space significantly degrades illumination and directional lighting when generating or reconstructing environment maps.

Incremental clipping of HDRI reveals that visible differences are often imperceptible, but illumination artifacts—washed out tones, lost shadows—are nontrivial in rendered scenes.

Figure 1: Each column illustrates the impact of increased exposure clipping on the rendering; while sky maps may appear similar, illumination, shadows, and transmitted light degrade with lost dynamic range.

Previous mitigations include parametric solar disk substitution and manual intensity boosting. However, as shown, these either skip atmospheric attenuation (yielding artificial shadows) or are prone to weather-dependent over-exposure and non-robust rendering artifacts.

Methodology

Exposure Bracketing and Fusion

Icarus reframes HDR-to-LDR decomposition as an N-exposure bracketing problem. Rather than relying on a single tone-mapped image or a highly compressed latent representation, HDR images are decomposed into a set of LDR exposures, each capturing a bracketed range of intensities. DNN training then occurs on these LDR brackets—a design that circumvents the high-exposure class-imbalance and enables linear fusion post-generation for HDRI reconstruction.

The exposure value (EV) parameterization, coupled with a specialized measure of integrated illumination ($\oiint_I$), quantifies both the exposure gamut and scene-relevant luminous flux, allowing direct comparison of models irrespective of display characteristics.

Figure 3: Visual representation of an LDR exposure bracket normalized to HDR space, capturing well-separated candle-stick regions without requiring aggressive tone mapping.

The bracketing is invertible; post-generation, multiple LDR exposures are fused using a weighting scheme (either established methods like Robertson fusion or a learned DNN-based fusion module), reconstructing an HDR image whose dynamic range is aligned with the reference FDR imagery. This modularization also decouples the ill-posed end-to-end training from the exposure-dependent instability commonly seen in previous architectures.

Model Architecture and Training

Icarus builds upon style-conditional generative pipelines, extending the SEAN framework to sky-modeling. It incorporates two style modules: a "red" style encoder capable of deterministic texture extraction from labeled (segmented) images, and a conditional "teal" style mapper for stochastic texture synthesis compatible with arbitrary segmentation. Generator and decoder networks are configured to output N-exposure LDR brackets. Critical to its stability is an exposure decay training scheduler: decoder heads are initialized to low (trivial) exposures and incrementally decayed, synchronously, to higher exposures across epochs, allowing the network to progressively acquire feature correlation across brackets without collapse.

The loss is decomposed into three pathways:

• Exposure-wise per-LDR-discriminator (handling each exposure independently),
• Bracket-wise HDR-discriminator (ensuring inter-exposure coherence),
• Auxiliary supervised losses (LPIPS, class-selective $L_1$) for stable spatial and textural structure recovery.

User Control and Editing

Icarus's style code conditioning enables at inference time:

• Placement and luminance adjustment of the sun and solar corona,
• Controlled generation or editing of cloud formations (transfer, removal, addition),
• Weighted mixing of style features for artistic manipulation of sky appearance.

  Figure 2: Icarus can transfer solar illumination between source and target skies, allowing user-guided de-obfuscation and accentuation of the solar region.

  

  Figure 6: Icarus supports high-degree user control in editing cloud formations, transferring or removing weather features between exemplars while preserving naturalness.

Results

Quantitative evaluation on the Laval HDR Sky database demonstrates that the Icarus model outperforms the state-of-the-art AllSky and other GAN-based models in both visual fidelity (FID, MiFID, HDR-VDP3) and illumination metrics (EV, $\oiint_I$, $PL_\Omega$). Notably, the model delivers robust performance as map resolution increases—where high-exposure class imbalance typically causes collapse in alternative architectures. Visual inspection confirms superior shadow structure, tonal accuracy, and photorealistic sky diversity across weather types.

Figure 8: A collection of high-resolution environment maps generated by Icarus, spanning diverse solar and cloud conditions, demonstrating stable rendering from sunrise to sunset.

Evaluation of fusion strategies reveals that both learned and analytic (Robertson) schemes yield visually and numerically equivalent HDR outputs, contingent on training stability and per-exposure noise mitigation.

Implications and Future Directions

The full-dynamic-range exposure-aware design of Icarus sets a new baseline for data-driven sky-modeling. By decoupling the exposure challenge from raw image synthesis and learning to bracket and reconstruct the relevant dynamic range, this method eliminates the central bottlenecks in high-fidelity IBL. Furthermore, the introduction of per-feature style code modulation generalizes controllable content generation beyond mere conditional GANs, allowing intuitive manipulation of both global illumination and fine-grained atmospheric phenomena.

On the practical side, Icarus can act as a drop-in replacement for laboriously captured FDR HDRI, offering interchangeable performance without fixed geo-temporal locality. For graphics and vision applications demanding precise illumination—augmented reality, VFX compositing, daylighting simulation—the method enables dynamic, photorealistic, and editable sky-maps.

The main theoretical implication is that the deep generative modeling of globally high-dynamic-range data benefits from bracketed decomposition and synchronous multiscale representation learning; similar methodology may be relevant to other domains (multi-exposure imaging, astronomy, scientific visualization) where capturing and synthesizing rare, high-intensity phenomena is critical.

Icarus is constrained by the limitations of available FDR datasets—primarily limited in geo-temporal diversity and occasionally artifacted (ghosting). As generative pipelines evolve, acquisition of richer, artifact-corrected FDR sky datasets will be essential to scaling diversity and resolution.

Figure 10: Weighted mixing of style features enables user modulation of solar and cloud properties, supporting both realistic and highly stylized environment maps.

Conclusion

The Icarus architecture as introduced in "Full Dynamic Range Sky-Modelling For Image Based Lighting" (2603.05758) represents a substantial advance in all-weather, exposure-faithful sky-modeling using DNNs. By leveraging LDR exposure bracketing, exposure-decay training, and multi-pathway discriminative learning, Icarus achieves unprecedented fidelity, robust illumination, and fine-grained, user-controlled editability of environment maps for photorealistic IBL. Anticipated future developments include expansion to richer datasets, more granular style code disentanglement, and extension to other high-dynamic-range generative imaging domains.