---
title: 'LUCID: Unified Nighttime Image Restoration'
url: https://www.emergentmind.com/topics/lucid
type: topic
---

# LUCID: Unified Nighttime Image Restoration

LUCID, in the context of "Learning Unified Control for Image Deflaring and Exposure Mastery in Nighttime Photography," refers to a diffusion-based, modular image restoration framework that addresses the entangled degradations—specifically, strong flare and photon-limited noise—in nighttime photographic imagery. Unlike legacy approaches that attempt low-light enhancement, flare removal, or high dynamic range (HDR) synthesis as independent tasks, LUCID formulates nighttime restoration as a single, controllable process, enabling fine-grained manipulation of exposure, flare, noise, and explicit light-source appearance through classifier-free guidance. The system combines a dual-path flare disentanglement U-Net, a conditional latent diffusion pipeline, and a four-mode compositional training regime to achieve state-of-the-art performance and continuous user-driven output control [2606.06901].

## 1. Imaging Model and Problem Statement

LUCID models the observed low-light, flare-corrupted image as:
\[
I_\mathrm{in} = R \cdot L + F + \epsilon,
\]
where:
- $I_\mathrm{in} \in \mathbb{R}^{H \times W \times 3}$ is the observed RGB image,
- $R$ is scene reflectance, $L$ is ambient illumination (Retinex-style, multiplicative),
- $F$ is stray-light (additive lens flare and ghosting),
- $\epsilon$ is photon-limited sensor noise.

The restoration objective is to recover a noise- and flare-free, well-exposed image $I_\mathrm{tgt} \approx R \cdot L$, suppressing $F$ and removing $\epsilon$. Crucially, simply boosting exposure amplifies $F$ and $\epsilon$, while overly aggressive suppression of $F$ destroys true scene highlights.

## 2. Flare Disentanglement Module

LUCID’s front-end is a dual-branch U-Net that encodes $I_\mathrm{in}$ and decodes into spatially aligned flare ($I_\mathrm{flare}$) and background ($I_\mathrm{bg} \approx R \cdot L$) images:
\[
\begin{aligned}
I_\mathrm{flare} &= D_\mathrm{flare}(E_\mathrm{decomp}(I_\mathrm{in})), \\
I_\mathrm{bg}    &= D_\mathrm{bg}(E_\mathrm{decomp}(I_\mathrm{in})).
\end{aligned}
\]
Decoder weights are shared for $m$ layers before channel-wise branching at a split layer $k$, producing divergent feature maps $F^k_\mathrm{flare}$ and $F^k_\mathrm{bg}$. The loss enforces:
- **Orthogonality**: $L_{\mathrm{ortho}} = \| F^k_\mathrm{bg} \odot F^k_\mathrm{flare} \|_2^2$
- **Reconstruction consistency**: $L_{\mathrm{recon}} = \| (I_\mathrm{flare} + I_\mathrm{bg}) - I_\mathrm{in} \|_2^2$
- **Component-wise targets**: $L_{\mathrm{comp}} = \| I_\mathrm{flare} - F_{\mathrm{gt}} \|_2^2 + \| I_\mathrm{bg} - B_{\mathrm{gt}} \|_2^2$

Total disentanglement loss:
\[
L_{\mathrm{decomp}} = L_{\mathrm{comp}} + L_{\mathrm{recon}} + L_{\mathrm{ortho}}.
\]
This disentanglement provides reliable structure guidance for downstream restoration.

## 3. Diffusion-Driven Restoration and Continuous Control

After decomposition, LUCID processes $I_\mathrm{bg}$ and $I_\mathrm{flare}$ using a conditional latent diffusion model:
- Images are encoded via a variational autoencoder: $z_\mathrm{main} = E(I_\mathrm{main})$, $z_\mathrm{ref} = E(I_\mathrm{ref})$.
- Two latent states are concatenated: $z = \mathrm{concat\_state}(z_\mathrm{main}, z_\mathrm{ref})$.
- Cross-state attention is applied in each diffusion step.

Diffusion loss combines denoising, intrinsic feature ($L_\mathrm{intri}$; layer features of the encoder), and LPIPS perceptual similarity:
\[
L_{\mathrm{restore}} = L_{\mathrm{diff}} + L_{\mathrm{intri}} + L_{\mathrm{LPIPS}}.
\]

**Continuous exposure control** is realized via classifier-free guidance (CFG): for scale $\beta$,
\[
\hat{z} = z_\mathrm{neg} + \beta (z_\mathrm{pos} - z_\mathrm{neg}),
\]
where $z_\mathrm{pos}$ and $z_\mathrm{neg}$ are predicted denoised latents under positive (user prompt, exposure up) and negative (omitted prompt, exposure down) conditionings. DECODING $\hat{z}$ produces the restored image, interpolating along an exposure manifold.

## 4. Four-Mode Training for Explicit Controllability

LUCID trains under four distinct modes, each defined by its input/conditioning tuple and control prompt:
- **Mode A**: Positive exposure, no light-source prompt; $(I_\mathrm{bg}, I_\mathrm{in}, I_\mathrm{enh})$, prompt: “nighttime enhancement”
- **Mode B**: Negative exposure, no light-source prompt; $(I_\mathrm{bg}, I_\mathrm{flare}, I_\mathrm{sup})$, prompt: “nighttime suppression”
- **Mode C**: Positive exposure + light-source prompt; $(I_\mathrm{bg}, I_\mathrm{in}, I_\mathrm{enh+l})$, prompt: “light source”
- **Mode D**: Negative exposure + light-source prompt; $(I_\mathrm{bg}, I_\mathrm{flare}, I_\mathrm{sup+l})$, prompt: “light source”

Minibatches randomly select among these, and classifier-free guidance is implemented by dropping prompts probabilistically (0.2 for negative conditioning, 0.5 for light-source). This setup grants explicit, disentangled control over both exposure and appearance of direct light sources.

## 5. Continuous HDR Reconstruction and Exposure Bracketing

LUCID supports single-input HDR by sampling the CFG exposure scale $\beta$ continuously:
\[
z_i = z_\mathrm{neg} + \beta_i (z_\mathrm{pos} - z_\mathrm{neg}), \quad I_i = D(z_i)
\]
for $\beta_i \in [\beta_\mathrm{min}, \beta_\mathrm{max}]$. The resulting set $\{I_i\}$ is fused into a composite HDR image:
\[
H(x) = \sum_i w_i(x) I_i(x), \quad \sum_i w_i(x) = 1,
\]
where $w_i(x)$ measures local contrast and fidelity (Laplacian-pyramid fusion). This yields high-quality highlight recovery and artifact-free shadow detail from a single input.

## 6. Experimental Evaluation and Results

LUCID outperforms previous state-of-the-art (SOTA) methods for low-light enhancement, flare removal, and single-image HDR:

| Metric          | LUCID (β=1.05) | Best competitor (DarkIR) |
|-----------------|----------------|-------------------------|
| CLIPIQA         | 0.4774         | 0.4107                  |
| MANIQA          | 0.3264         | 0.3078                  |
| MUSIQ           | 61.45          | 52.26                   |
| LIQE            | 3.019          | 2.267                   |
| NIMA            | 5.390          | 5.227                   |

On Flare7K, LUCID eliminates streaking and over-subtraction artifacts, yielding backgrounds with realistic light-source fall-off, and sometimes improving over ground-truth. On single-image HDR, LUCID’s exposure-bracketed fusion achieves superior highlight/shadow fidelity without hallucinated structures or color artifacts. The control knob $\beta$ tracks exposure value $\Delta$EV nearly linearly, so β in [0.25, 1.5] covers a roughly -2 to +2 EV range under standard gamma.

## 7. Technical Innovations and Impact

LUCID is the first unified nighttime restoration system with the following properties:
- Explicit flare-background disentanglement via a dual-decoder U-Net with orthogonality and component-wise supervision.
- Latent diffusion restoration, leveraging generative priors within a two-state, globally attentive architecture, allowing joint denoising and de-flaring.
- True continuous user control over exposure and preservation or suppression of direct light sources, realized through four-mode training and classifier-free guidance.
- Generalization to robust single-image HDR synthesis.
- Experimental superiority both in objective (no-reference IQA) metrics and visual quality, verified across diverse synthetic and real nighttime scenes.

These capabilities distinguish LUCID from conventional cascaded or fixed-parameter pipelines, directly addressing the highly entangled nature of nighttime degradations [2606.06901].

Source: https://www.emergentmind.com/topics/lucid