Differentiable Reflectance Functions

• Differentiable reflectance functions are mathematical models that describe surface reflectance with continuously differentiable parameters for gradient-based rendering optimization.
• They integrate seamlessly into end-to-end differentiable rendering pipelines, allowing gradients to flow through light transport, visibility, and material laws.
• Their applications span inverse rendering, neural appearance fields, and material acquisition, driving advances in photorealistic image synthesis and scene reconstruction.

A differentiable reflectance function is a mathematical or algorithmic model of surface bidirectional reflectance (BRDF or SVBRDF) that is constructed to be continuously differentiable with respect to its parameters and, crucially, is embedded within an end-to-end differentiable rendering pipeline. This property allows gradients of image-space losses to flow backward through the physics of light transport, visibility, and material laws, enabling the recovery or optimization of geometric and photometric parameters via gradient-based algorithms. The differentiable reflectance function is central to modern inverse rendering frameworks, neural appearance fields, and material estimation from images.

1. Mathematical Formulation of Differentiable Reflectance Functions

Differentiable reflectance models universally describe the outgoing radiance $L_o$ at a point $x$ in direction $\omega_o$ as an integration over the incoming hemisphere:

$$L_o(x,\omega_o) = \int_\Omega f_r(x, \omega_i, \omega_o) \, L_i(x, \omega_i) \, (n(x)\cdot\omega_i) \, V(x, \omega_i) \, d\omega_i$$

where:

• $f_r(x, \omega_i, \omega_o)$ is the spatially varying BRDF/SVBRDF, often split into diffuse and specular lobes (Li et al., 2021, Brahimi et al., 2022, Zhou et al., 2024).
• $L_i(x, \omega_i)$ is incident radiance, possibly from environment maps or point lights.
• $n(x)$ is the normal at $x$.
• $V(x, \omega_i)$ is the visibility/shadow term (may be 0/1 or differentiable "soft" form).
• The reflectance function $f_r$ is made up of parametric (Cook–Torrance, Disney/BRDF) or data-driven (MLP/Neural) terms, all differentiable with respect to their parameters and scene state.

A canonical example is the Cook–Torrance microfacet model as used in (Li et al., 2021):

$$f_r(x, \omega_i, \omega_o) = \frac{\rho_d(x)}{\pi} + \rho_s(x)\,\frac{D(h,\alpha)\, G(n,\omega_i,\omega_o)\, F(h, \omega_i)}{4(n\cdot\omega_i)(n\cdot\omega_o)}$$

with $h = (\omega_i+\omega_o)/\|\omega_i+\omega_o\|$, $D$ is the microfacet NDF, $G$ is the Smith geometry term, $F$ the Schlick Fresnel factor, and all terms differentiable in the participating variables and parameters.

Neural approaches (e.g. (Zhou et al., 2024)) employ an MLP that parameterizes either the BRDF directly or an antiderivative (with output $g^\xi(h, d')$), with $f_r^\xi$ recovered by differentiation:

$$f_r^\xi(\omega_i, \omega_o) = \frac{1}{\cos\theta_o\sin\theta_o} \frac{\partial^2 g^\xi(\omega_i, \omega_o)}{\partial\theta_o\, \partial\phi_o}$$

functionally ensuring differentiability with respect to network parameters and directional inputs.

2. Integration with Differentiable Rendering Pipelines

The critical utility of differentiable reflectance functions is their insertion into forward rendering equations, which are then made differentiable end-to-end for optimization via reverse-mode autodifferentiation frameworks. This structure enables reconstruction of geometry and reflectance from images.

In physically-based renderers (e.g. Mitsuba 2 (Li et al., 2021, Zhu et al., 2023)), the rendering integral is solved via Monte Carlo path tracing. All arithmetic operations, including the propagation of gradients through the stochastic sampling (reparameterized sampling, edge sampling), are constructed to allow unbiased gradient transfer back to material, geometry, and lighting parameters:

$$I(u) \approx \Delta t\,\frac{1}{N_s}\sum_{s=1}^{N_s} [\theta_d(x) + \theta_s(x, w_i^s, w_o)]\, L_i(w_i^s; x)\, (n(x)\cdot w_i^s)$$

with sample reparameterization differentiable as in (Li et al., 2021).

In volume-rendered models—e.g. Neural Reflectance Fields (Bi et al., 2020, Brahimi et al., 2022, Bi et al., 2020)—MLP-predicted (or grid-predicted) reflectance parameters are combined with direct or shadowed illumination in a differentiable volumetric rendering integral, often approximated by discrete quadrature or compositing with automatic differentiation applied to all operations.

3. Parameterizations: Analytic, Learned, and Neural BRDFs

Parameterizations fall into three broad classes:

• Analytic (Cook–Torrance, Disney), combining closed-form diffuse (Lambertian) and microfacet (GGX, Beckmann) lobes with explicit roughness, specular albedo, and Fresnel terms (Li et al., 2021, Zhu et al., 2023, Chen et al., 2021).
• Basis or compositional schemes, such as DANI-Net's anisotropic spherical Gaussian bases [$$\rho^s_{ij} = \sum_k c_i^k \exp(-r^x_k(h_j\cdot x_i)^2 - r^y_k(h_j \cdot y_i)^2)$$], decoupling spatial variation and angular dependence but maintaining end-to-end differentiability via MLP basis prediction (Li et al., 2023).
• Neural (MLP) systems, predicting spatially-varying or angularly-varying reflectance either as explicit albedo/roughness fields, direct (Rusinkiewicz) angular mapping, or antiderivatives for physical properties enforcement (Brahimi et al., 2022, Zhou et al., 2024).

Table: Common Differentiable BRDF Parameterizations

Model/paper	Local Parameterization	Angular Param.	Physical Properties
Cook–Torrance (Li et al., 2021)	ρₐ(x), ρₛ(x), α(x)	h, n, ωₒ, ωᵢ	Helmholtz, clamps, opt.
Disney (SupeRVol) (Brahimi et al., 2022)	a(x), s(x), ρ(x) (MLPs)	h, n, v	Isotropic, closed-form
PBNBRDF (Zhou et al., 2024)	Neural antiderivative MLP	Rusinkiewicz angles	Enforces reciprocity, passivity
DANI-Net (Li et al., 2023)	Basis weights (MLP), widths	h, tangent/bitangent	Anisotropy, all diff.

Analytic models facilitate physical interpretability and enforce properties such as reciprocity and energy conservation through explicit constraints or losses (Zhou et al., 2024). In neural models, the architectural design or loss functions are constructed to reflect such physical priors, e.g. energy passivity and chromaticity enforcement (Zhou et al., 2024).

4. Differentiability, Optimization, and Gradient Flow

All elements of the differentiable reflectance function and their usage in rendering pipelines are constructed to ensure differentiability:

• Direct (analytic) BRDF terms: All are composed of analytic, closed-form, smooth functions.
• Neural networks: MLPs use ReLU or softplus activations with positional encoding, guaranteeing almost-everywhere differentiability (Brahimi et al., 2022, Zhou et al., 2024).
• Monte Carlo integration: Differentiable via reparameterized sampling [(Loubet et al. 2019), as cited in (Li et al., 2021)], edge sampling for visibility gradients in presence of discontinuities (Dib et al., 2021, Dib et al., 2019).
• All compositing operations and physical integrals are implemented as computational graphs in autodiff frameworks, enabling standard chain-rule–based gradient backpropagation.

Photometric losses ($\mathcal{L}_{photo}$, $\mathcal{L}_\mathrm{NBRDF}$) are computed between rendered and observed images. The network parameters are updated via gradient descent (Adam) using derivatives computed by backpropagation through the entire rendering pipeline (Li et al., 2021, Brahimi et al., 2022, Chung et al., 2023).

5. Regularization, Physical Constraints, and System Design

Regularization prevents degenerate solutions and encourages physical plausibility:

• Non-negativity and clamping: $ρ_d(x), ρ_s(x)\in[0,1]$ (Li et al., 2021, Chung et al., 2023).
• Spatial smoothness: Penalties on adjacent parameter differences.
• Sparsity and normalization of basis coefficients: e.g. $\ell_1$ norm penalties for compositional bases (Chung et al., 2023).
• Energy passivity: Enforcing that integrated BRDF lobes do not exceed physical energy budgets (Zhou et al., 2024).
• Chromaticity norms: To ensure correct color ratios at low intensities (Zhou et al., 2024).

Losses are designed to combine data fidelity (pixel-wise or logarithmic losses) and such regularization terms, possibly augmented with architectural reciprocities (ensuring Helmholtz reciprocity via input transformation (Zhou et al., 2024)) or normalization losses.

6. Applications and Impact

Differentiable reflectance functions underpin a broad spectrum of applications in computational imaging and graphics:

• Inverse rendering: Joint recovery of geometry and spatially varying reflectance from uncontrolled or natural images (Li et al., 2021, Brahimi et al., 2022).
• Material acquisition: Neural and basis-based differentiable reflectance are used to recover complex reflectances from sparse images (Chung et al., 2023, Li et al., 2023, Zhou et al., 2024).
• Neural scene representations: Combining volumetric rendering, neural fields, and differentiable reflectance enables high-fidelity relightable digitizations (Bi et al., 2020, Bi et al., 2020).
• Realistic image synthesis: Learned or reconstructed reflectance maps support physically plausible relighting, novel view synthesis, and material editing (Brahimi et al., 2022, Chung et al., 2023, Zhu et al., 2023).
• Extensions to non-optical domains: Differentiable BRDFs/reflectance fields have been transferred to radio-frequency ray tracing (Jia et al., 5 Jan 2025), demonstrating generality beyond computer graphics proper.

A common technical advance is the tight integration of differentiable reflectance, geometry, and light transport models with machine learning, enabling optimization of large-scale, photorealistic scene representations and their parameters in practical, scalable workflows.

7. Representative Algorithms and Optimization Schedules

Optimization schemes leverage the block structure of scene parameters—geometry ($\theta_g$), diffuse albedo ($\theta_d$), specular parameters ($\theta_s$), lighting ($\theta_l$)—and often employ an alternating minimization strategy to mitigate memory footprint (Li et al., 2021):

for each parameter group θ ∈ {θ_d, θ_g, θ_s, θ_l} do Render I = Φ(θ_g, θ_d, θ_s, θ_l) Compute gradient ∂ℒ/∂θ Update θ ← θ − η_θ · Adam(∂ℒ/∂θ) Clamp reflectance as needed end

Final renderings for evaluation are carried out at high sample rates (spp), though optimization may proceed at low spp for efficiency (Li et al., 2021, Zhu et al., 2023). In neural reflectance or appearance field systems, all-pipeline joint gradient descent is possible due to modular differentiability (Brahimi et al., 2022, Zhou et al., 2024).

• (Li et al., 2021): “Shape and Reflectance Reconstruction in Uncontrolled Environments by Differentiable Rendering”
• (Brahimi et al., 2022): “SupeRVol: Super-Resolution Shape and Reflectance Estimation in Inverse Volume Rendering”
• (Zhou et al., 2024): “Physically Based Neural Bidirectional Reflectance Distribution Function”
• (Li et al., 2023): “DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering”
• (Chung et al., 2023): “Differentiable Point-based Inverse Rendering”
• (Bi et al., 2020): “Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images”
• (Bi et al., 2020): “Neural Reflectance Fields for Appearance Acquisition”
• (Jia et al., 5 Jan 2025): “Neural Reflectance Fields for Radio-Frequency Ray Tracing”
• (Zhu et al., 2023): “Efficient Multi-View Inverse Rendering Using a Hybrid Differentiable Rendering Method”
• (Chen et al., 2021): “DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer”
• (Dib et al., 2019): “Face Reflectance and Geometry Modeling via Differentiable Ray Tracing”
• (Dib et al., 2021): “Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracing”