Inferring HDR environment lighting from casually captured images or video

Develop reliable algorithms to infer high-dynamic-range (HDR) 360-degree environment illumination maps from single casually captured images or videos, achieving accurate recovery of light direction and intensity from indirect visual cues without relying on specialized capture setups.

Background

Illumination estimation is central to physically-based rendering and applications such as virtual object insertion, augmented reality, and scene relighting. A common representation is the HDR environment map, which encodes incident light from all directions but is expensive to capture and scarce in real-world datasets.

Prior supervised methods depend on paired LDR inputs and HDR ground truth, limiting generalization. Generative approaches (e.g., diffusion-based inpainting of chrome balls) reveal implicit illumination priors but are often insufficiently robust, lack direct HDR output, and require costly test-time procedures. The authors highlight that, despite progress, robustly inferring scene lighting from casual imagery remains unresolved.