Differentiable Light Transport

• Differentiable Light Transport is a framework that quantifies how radiometric measurements change with scene and system parameters, enabling inverse rendering and design.
• It employs techniques like differentiable Monte Carlo rendering, rasterization, and transient simulations to accurately estimate gradients despite discontinuities.
• Innovative methods address challenges such as geometric discontinuities, gradient plateaus, and exploding gradients, leading to more stable and efficient optimization.

Differentiable light transport studies how rendered or transported radiometric measurements vary with respect to scene and system parameters, including camera pose, geometry, appearance, material, lighting, time of flight, and optical design variables. In practice, it encompasses differentiable Monte Carlo rendering, differentiable rasterization and shadowing, transient and volumetric transport, and related inverse-design pipelines. The subject is technically distinct from ordinary forward rendering because visibility terms, boundary events, and stochastic path sampling make the derivative of light transport harder to compute and, in many cases, harder to use for optimization than the primal transport itself (Li, 2019, Yi et al., 2022, Krupa et al., 7 May 2026).

1. Mathematical foundations

A foundational line of work revisits radiance itself. “On the Mathematical Formulation of Radiance” models radiance as a differential 2-form,

$$\Lambda = L(q,\bar{p},\nu,t) \, dA_{\perp} \, d\bar{p} \, d\nu \, dt,$$

with

$$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$

Within this formulation, the cosine term and the area formulation arise from pullbacks and exterior calculus rather than from postulates or heuristic arguments. The same framework is presented as coordinate-independent, intrinsically differentiable, and applicable to both surface and volumetric descriptions of transport (Lessig et al., 2012).

At the rendering level, differentiable light transport is commonly written either as a path-space integral or as a direct light integral. “Inverse Path Tracing for Joint Material and Lighting Estimation” expresses the rendered intensity at pixel $j$ as

$I_{R}^{j} = \int_{\Omega} h_j(X) f(X)\; d \mu(X),$

where $X$ is a sampled light path and $f(X)$ is the measurement contribution function combining emission and BRDF factors along the path (Azinović et al., 2019). In direct-light formulations, “DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer” starts from

$$o(x, o) = \int_\Omega r(x, i, o) \, \ell(x, i) \, |n \cdot i| \; di,$$

and then differentiates either a Monte Carlo estimator or an analytic spherical-Gaussian approximation with respect to geometry, material, and lighting variables (Chen et al., 2021).

These formulations establish the central object of the field: a mapping from scene parameters to measurements whose derivatives are needed for inverse rendering, parameter estimation, or design. A plausible implication is that the modern mathematical view of radiance and the algorithmic view of rendering integrals are complementary rather than competing: one provides rigor for transport quantities, and the other provides implementable estimators and optimization pipelines.

2. Differentiating the rendering equation

A central obstacle is geometric discontinuity. “Differentiable Visual Computing” states that the rendering integral includes visibility terms that are not differentiable, and that naïvely applying automatic differentiation or finite differences to sampled rendering yields biased or highly noisy estimates because samples will almost never land on the discontinuity set responsible for visibility changes. The dissertation addresses this by decomposing image derivatives into a smooth term and a discontinuity term concentrated on geometric edges, with explicit edge sampling for the latter. It further describes the first general-purpose differentiable ray tracer that solves the full rendering equation while correctly taking geometric discontinuities into account, implemented as the C++/CUDA-based wavefront path tracer redner with PyTorch integration (Li, 2019).

Path-space differentiation also underlies inverse rendering. “Inverse Path Tracing for Joint Material and Lighting Estimation” derives gradients of the path contribution with respect to illumination and material parameters by applying the product rule along the path. The method uses a differentiable Monte Carlo renderer, multiple importance sampling, independent samples for value and gradient estimates to avoid bias, and a tailored stochastic gradient descent procedure using ADAM. Its stated goal is accurate and simultaneous retrieval of light sources and physically based material properties from images and coarse geometry, while retaining physically correct global light transport (Azinović et al., 2019).

The resulting picture is that differentiable light transport is not simply “autodiff applied to a renderer.” In the full rendering-equation setting, unbiased or low-bias gradients typically require estimators that explicitly account for discontinuity structure, path measures, or both. This is why the literature repeatedly separates smooth contributions from visibility contributions instead of treating all transport events uniformly (Li, 2019).

3. Plateaus, exploding gradients, and gradient regularization

The existence of gradients does not guarantee that an optimizer can use them. “Plateau-reduced Differentiable Path Tracing” states this explicitly: current differentiable renderers provide light transport gradients with respect to arbitrary scene parameters, but the mere existence of these gradients does not guarantee useful update steps in an optimization. The paper focuses on plateaus, defined as regions of zero gradient caused by discontinuities such as occlusion, shadows, caustics, or complex global illumination. Its remedy is to convolve the rendering function with a kernel over parameter space,

$$Q(\theta) = [\kappa * P](\theta) = \int_{\Theta \times \Omega} \kappa(\tau) f(\mathbf{x}, \theta - \tau)\, d\mathbf{x}\, d\tau,$$

and then estimate gradients of the blurred objective by Monte Carlo. Two estimators are given: one for differentiable renderers and one for black-box renderers. The method is presented as a straightforward extension to both settings, with antithetic sampling, importance sampling, and adaptive bandwidth scheduling used to reduce variance (Fischer et al., 2022).

A different pathology appears in step-based transport. “Exploring the Boundaries of Differentiable Radiation Transport and Detector Simulation” studies automatic differentiation through a Geant4-like electromagnetic transport simulation and reports exploding gradients driven by rare but extreme sensitivities at material boundaries. In this setting the step length is

$L = \min(L_\mathrm{phys},\, L_\mathrm{bnd}),$

and when the step is boundary-limited, a boundary shift $\Delta b$ produces

$$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$0

with catastrophic divergence as the track becomes nearly parallel to the boundary. The proposed mitigation is a targeted stop-gradient rule: for boundary-limited steps, gradients are blocked when $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$1 or $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$2, while the forward simulation remains unchanged. The paper reports suppression of catastrophic gradient outliers by up to 6 orders of magnitude and shows stable, optimization-ready gradients in a detector-design problem. It also states that the issue is general and generalizes to light rays in simulations using discrete, boundary-constrained stepping (Krupa et al., 7 May 2026).

These results correct two common misconceptions. First, differentiability is not synonymous with optimization readiness: plateaued objectives and exploding gradients both arise in transport even when derivatives exist (Fischer et al., 2022, Krupa et al., 7 May 2026). Second, automatic differentiation of a forward simulator is not, by itself, a sufficient stabilization strategy when visibility or boundary geometry dominates the derivative structure (Krupa et al., 7 May 2026).

4. Renderer architectures and efficient approximations

The literature includes both full transport differentiators and architectures that deliberately approximate transport to obtain faster or more stable optimization. Representative examples are summarized below.

Approach	Representative paper	Characteristic
General-purpose differentiable ray tracing	(Li, 2019)	Full rendering equation with explicit discontinuity handling
Hybrid rasterization and ray tracing	(Chen et al., 2021)	Direct light transport with MC or spherical-Gaussian shading
Differentiable shadow mapping	(Worchel et al., 2023)	Secondary visibility gradients via pre-filtered shadow maps
Analytic optical-surface rendering	(Sun et al., 2024)	Flux-based mesh optimization with OT-guided updates

“DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer” exemplifies the hybrid approach. It first differentiably rasterizes a mesh into G-buffers and then performs deferred shading using either Monte Carlo integration or a spherical-Gaussian basis. The method supports environmental lighting, spatially-varying material models, and non-Lambertian effects including diffuse and specular reflections. The SG variant is described as fast, analytic, and differentiable, while the MC variant better handles highly specular cases (Chen et al., 2021).

“Differentiable Shadow Mapping for Efficient Inverse Graphics” combines pre-filtered shadow mapping with differentiable rasterization to obtain differentiable visibility information from the light’s point of view. The paper argues that adding differentiable shadows is crucial because rasterization without shadows often fails to converge in shadow-dependent tasks. Quantitatively, for “Bunny” at $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$3, it reports rotation error $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$4 for the proposed method versus $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$5 for Mitsuba, with runtime $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$6 versus $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$7; more broadly, it characterizes the method as orders of magnitude faster than differentiable path tracing with similar accuracy for direct-shadow problems (Worchel et al., 2023).

“End-to-end Surface Optimization for Light Control” presents a physically-accurate, differentiable rendering model for a freeform optical surface mesh. Light flux is traced from mesh triangles to the target plane, and gradients with respect to mesh vertices are computed by autodiff on the GPU. To address non-convexity, the method alternates between local, rendering-guided optimization and a face-based optimal transport step that makes large global changes to the surface shape. Fabrication-aware terms include mean-curvature regularization, a robust edge smoothness penalty based on the Welsch function, barrier terms for mesh validity and physical constraints, and out-of-bounds penalties (Sun et al., 2024).

A plausible implication is that efficient differentiable transport is increasingly organized around problem structure. When direct shadows dominate, shadow maps may suffice; when direct reflection with complex BRDFs dominates, hybrid deferred shading may suffice; when global transport and discontinuities dominate, full path-space methods remain necessary.

5. Inverse rendering and optical-system design

A major use case for differentiable light transport is inverse rendering. “Inverse Path Tracing for Joint Material and Lighting Estimation” uses a differentiable Monte Carlo renderer to jointly estimate diffuse reflectance, specular reflectance, roughness, and emission in indoor scenes from RGB images, camera poses, and a coarse geometry scan. Because the renderer includes shadows, interreflections, and occlusion, it is designed to disentangle lighting from spatially-varying BRDFs under global illumination rather than under a simplified illumination model (Azinović et al., 2019).

Differentiable transport has also become a system-level design tool in optics. “End-to-end differentiable design of geometric waveguide displays” couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-matrix thin-film solver. The paper states that, with memory-saving strategies, it optimizes more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. On a representative design, it reports an increase in light efficiency from $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$8 to $$dA_{\perp}(\bar{p}) = \bar{p}_1 \, dy \wedge dz + \bar{p}_2 \, dz \wedge dx + \bar{p}_3 \, dx \wedge dy .$$9, with eyebox and field-of-view uniformity improved by $j$0 and $j$1, respectively. The same framework performs automated layer pruning by driving redundant layers to zero thickness under discrete manufacturability constraints (Yang et al., 7 Jan 2026).

Perceptual design objectives can also be made differentiable. “Glare Mitigation using a Differentiable Unified Glare Rating” replaces the discrete UGR threshold with a tunable sigmoid and adds a differentiable optical scattering pass that simulates the Point Spread Function of the human eye. The optimization variables span three radiometric domains: surface-side microgeometry roughening, boundary-side index-of-refraction optimization, and source-side emitter gobo masking. The paper’s stated contribution is to transform a passive perceptual evaluation into an active loss landscape for robust, physics-based glare reduction in global illumination environments (Beresna et al., 6 Jul 2026).

A different design setting appears in “Efficient data transport over multimode light-pipes with Megapixel images using differentiable ray tracing and Machine-learning.” There, a differentiable ray-tracing digital twin is combined with a U-Net decoder and a differentiable mutual-information estimator based on the von-Mises distribution. The method retrieves up to 66 kB using efficient convolutional operations only, and the ray-based twin is reported to be superior to an eigenmode-based twin on experimental data because it can overcome the simulation-to-experiment gap by adjusting to optical imperfections such as the parallelogram angle of the light-pipe (Lim et al., 2023).

Together, these examples show that differentiable light transport no longer serves only parameter estimation inside classical inverse rendering. It also supports end-to-end co-design of coatings, polarization optics, visual-comfort objectives, and optical communication channels.

6. Transient, volumetric, surrogate, and generalized transport regimes

The steady-state assumption is a major boundary of the classical field. “Differentiable Transient Rendering” extends differentiable Monte Carlo rendering to the time-of-flight regime, where scattering events at path vertices are no longer independent and must be treated jointly as a multidimensional, evolving manifold. The paper introduces a correlated importance function,

$j$2

and uses the generalized transport theorem to differentiate transient path integrals with respect to geometry, material, and refractive-index parameters. The applications include optimizing indices of refraction and non-line-of-sight tracking around complex relay-wall configurations (Yi et al., 2022).

The software counterpart of this extension is “mitransient: Transient light transport in Mitsuba 3,” which adds a temporal dimension to Mitsuba 3, supports physically-based simulations of realistic materials and participating media, and includes transient differentiable rendering, time-resolved polarization tracking, and non-line-of-sight tools through Mitsuba 3 and Dr.Jit on CPU or GPU (Royo et al., 29 Oct 2025).

Volumetric and surrogate models raise a different issue: derivative fidelity. “Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes” uses a physically-based differentiable volume renderer for multiple scattering in clouds and backpropagates gradients with respect to light transport in latent space during diffusion posterior sampling, explicitly contrasting this with classic NeRF approaches (Leonard et al., 9 Jan 2025). “DeepLight: A Sobolev-trained Image-to-Image Surrogate Model for Light Transport in Tissue” argues that standard surrogates trained only on function values do not ensure accurate derivatives, then adds a Sobolev loss on directional derivatives. It reports median function-value error reductions of approximately $j$3 on in-distribution samples and $j$4 on out-of-distribution samples, derivative-error reductions of approximately $j$5 and $j$6, and maximum derivative-error reductions of approximately $j$7 in some cases (Haim et al., 14 Jan 2026).

Global illumination can also be reformulated around alternative primitives. “Differentiable Light Transport with Gaussian Surfels via Adapted Radiosity for Efficient Relighting and Geometry Reconstruction” uses Gaussian surfels and an adapted radiosity formulation in spherical-harmonic coefficient space. The paper reports an analytic backward pass that is approximately $j$8 faster and more memory efficient than auto-differentiation, and view-independent rendering at hundreds of FPS because light transport need not be recomputed under viewpoint changes (Jiang et al., 23 Sep 2025).

Related work also connects transport to online learning. “Learning Light Transport the Reinforced Way” shows that Q-learning and the rendering equation share the structure of Fredholm integral equations of the second kind, then uses reinforcement learning to progressively learn where light comes from for importance sampling in path space. The reported effect is a dramatic reduction in zero-contribution paths and much less noisy images within a fixed time budget (Dahm et al., 2017). This suggests that differentiable light transport increasingly overlaps with broader differentiable and adaptive transport methods in which estimation, optimization, and sampling are designed together rather than sequentially.