---
title: 'MPLI: Multi-Plane Light Image'
url: https://www.emergentmind.com/topics/multi-plane-light-image-mpli-7555a0e2-0c3f-4f9d-9ec0-81988ed135b4
type: topic
---

# MPLI: Multi-Plane Light Image

A Multi-Plane Light Image (MPLI) is a layered, depth-aligned visual representation for encoding photometrically accurate illumination attributes of a 3D scene. Unlike classical Multi-Plane Images (MPIs), which store view-dependent scene appearance (RGBA) for view synthesis, MPLIs are defined as stacks of per-plane RGB “Light Images” sampled at fixed depths, capturing 3D lighting (source positions, intensities, chromaticities) as projected irradiance distributions. Originally introduced in the context of fine-grained video relighting for generative models, the MPLI provides a physics-inspired, compact visual prompt for deep neural networks, supports arbitrary multi-source illumination, and enables temporally varying lighting control while efficiently generalizing to unseen lighting arrangements [2511.06271].

## 1. Definition and Mathematical Formulation

An MPLI consists of $K$ fronto-parallel “Light Images” $L_k \in \mathbb{R}^{H \times W \times 3}$, each situated at a fixed camera-aligned depth $d_k$ within the scene frustum. Each $L_k$ encodes the total irradiance at depth $d_k$, combining all sources:

\[
L_k(x, y) = \sum_{i=1}^M \frac{I_i \cdot c_i}{\|q - p_i\|^2 / s_1 + s_2}
\]

where:

- $M$: number of point light sources
- $p_i = (x_i, y_i, z_i)^T$: source 3D position
- $I_i$: scalar intensity
- $c_i \in \mathbb{R}^3$: RGB color
- $q = (x, y, d_k)^T$: 3D position of the pixel in plane $k$
- $s_1, s_2$: global normalizers ensuring numerical compatibility with downstream models

The full MPLI is the ordered set $\{\;L_1,\;L_2,\;\ldots,\;L_K\;\}$. The representation naturally supports multi-source superposition and temporal dynamism by providing distinct MPLIs for consecutive temporal blocks.

## 2. Encoding Illumination and Scene Geometry

Each per-plane image $L_k$ encodes physical illumination by integrating source-specific falloff (inverse-square law) and color mixing at sampled depths. The position $p_i$ of each light influences the per-pixel irradiance profile via the squared distance in world coordinates, while intensity $I_i$ and color $c_i$ modulate the magnitude and chromatic bias, respectively. The additive structure allows linear encoding of complex, multi-source lighting fields, and by updating MPLIs per time block, the method accommodates dynamic, time-varying sources.

## 3. Relation to Multi-Plane Image (MPI) and Other Layered Representations

MPIs encode scene appearance as RGBA textures at fixed depths, composed via back-to-front over-operator for view interpolation and novel-view rendering [1905.00413, 2210.01602, 2002.05028, 2205.11733]. Each MPI plane typically contains color and alpha (opacity), and is rendered by warping layers to the target view and compositing:

\[
I_{\text{out}}(x, y) = \sum_{k=1}^K T_k(x, y)\,\alpha_k(x, y)\,C_k(x, y), \quad T_k(x, y) = \prod_{j=1}^{k-1}[1-\alpha_j(x, y)]
\]

MPLI planes, in contrast, have only RGB irradiance channels and do not encode occlusion, instead capturing illumination distributions that are not view-dependent images but “lighting maps” independent of camera pose or occlusion. MPLIs can be injected as control signals for relighting, rather than layered for rendering per se. In [2511.06271], the MPLI, after compression (e.g., by a Video VAE), is injected as a latent visual cue into a frozen Video Diffusion Transformer, enabling direct illumination control.

## 4. Implementation in Generative Video Relighting

The "RelightMaster" framework leverages MPLIs for high-fidelity, controllable video relighting [2511.06271]. Key workflow:

- Four Light Images per four-frame temporal block ($K=4$)
- Each set compressed into a latent vector $z_L$ using a Video VAE
- A Light Image Adapter (LIA), reusing backbone video transformer weights, transforms $z_L$ into tokens injected into each block of a frozen Video Diffusion Transformer (DiT) alongside video latents
- Only the LIA weights, 3D-attention, and a small LoRA adapter are fine-tuned. The DiT’s generative prior is preserved, avoiding catastrophic forgetting

Empirical results confirm control over illumination position, intensity, and color: scenes relit via MPLI-encoded prompts exhibit faithful highlight, shadow, and color reproduction, outperforming text-only visual prompt approaches in physical consistency and control granularity.

## 5. Multi-Source, Generalization, and Temporal Dynamics

The summation in the irradiance formula admits arbitrary numbers of point light sources, as each source’s contribution is independent and linearly combined. Despite a focus on single-source training data, the linear superposition ensures compositional generalization to multi-source scenarios and previously unseen source arrangements in position, intensity, and color. Temporally adaptive lighting is realized by feeding temporal sequences of MPLIs, one per time block, enabling smooth cross-fades, dynamic lighting transitions, and moving shadows.

## 6. Broader Context: Layered Light Representations and Optical MPLI

While the deep learning MPLI is focused on differentiable, learnable relighting, analogous concepts appear in computational and physical optics:

- In broadband diffractive optics, “multi-plane light-image” (MPLI) projection uses a single engineered phase mask to project distinct images at multiple spatial planes (and/or spectral bands), optimizing the mask profile to direct desired intensity distributions to target depths and colors [1901.05943]. The underlying principle is joint spatio-spectral manipulation by the mask’s chromatic-dispersive and depth-propagation properties via the Fresnel integral. Fabricated devices can simultaneously project overt and covert (e.g., IR) images at different depths, with multi-plane efficiency up to 65%.
- In computational imaging, “deep learned optical multiplexing” techniques encode multi-depth (focal) information into single capture images, with neural decoding to recover multi-plane intensity stacks, an approach functionally parallel to software-based MPLI in compressing volumetric information into succinct observables and then reconstructing the stack by learned inversion [1907.01528].

## 7. Applications, Limitations, and Future Directions

### Applications

- **Fine-grained neural video relighting:** Scene-level, physically plausible, temporally coherent lighting edits with fine spatial control, as demonstrated in RelightMaster [2511.06271].
- **Generalization to unseen lighting**: Admits complex, dynamic, multi-source illumination fields, enabling a level of relighting control not feasible with text-only prompts.
- **Physical-optical multiplexing**: Used in anti-counterfeiting, holographic display, spectral/axial image multiplexing, and flat optical device engineering [1901.05943].
- **Live microscopy and computational imaging**: Single-shot multi-plane acquisition with neural recovery for high-speed focal stack imaging [1907.01528].

### Limitations

- MPLI as proposed in [2511.06271] supposes explicit, algebraically defined point lighting fields. Extension to arbitrary indirect illumination, non-point light models, or full global illumination scenarios can be challenging.
- No occlusion or volumetric scattering is modeled in the canonical MPLI; these effects must be handled elsewhere in the pipeline.
- The utility in general 3D view synthesis (not just relighting) is less established than the classical MPI’s role in novel-view rendering.

### Future Directions

- Extension of MPLI to encode indirect lighting and global illumination effects by parameterizing more complex radiance transfer kernels.
- Physical implementation of reconfigurable, dynamic MPLI in nanophotonics or metasurface optics for real-time light field and relighting control.
- Fusion of MPLI with adaptive, learned geometry deformation as in SIMPLI [2210.01602], or with dynamic temporal basis encoding as in temporal-MPI [2111.10533], to yield more compact, expressive, and physically consistent layered scene-light representations.

---

MPLI, as an explicit, depth-aligned illumination encoding, offers a bridge between physically informed light transport and neural rendering pipelines, supporting precise, compositional, and temporally rich control over scene relighting and novel illumination synthesis [2511.06271].

Source: https://www.emergentmind.com/topics/multi-plane-light-image-mpli-7555a0e2-0c3f-4f9d-9ec0-81988ed135b4