Spatially-Varying BRDFs

• Spatially-varying BRDFs are advanced reflectance models that describe surface color and texture variations by assigning a full BRDF function to each spatial location.
• They combine analytical, basis expansion, and neural representations to accurately capture and recover detailed material properties under diverse lighting and view conditions.
• Applications include real-time and inverse rendering, remote sensing, and material editing, while challenges involve anisotropy, resolution, and interoperability across renderers.

A spatially-varying bidirectional reflectance distribution function (SVBRDF) generalizes the classical BRDF by allowing surface reflectance properties to change continuously (or discretely) over a material’s spatial domain. The SVBRDF specifies, for each spatial location $x$ on a surface, a complete 4D (or 6D) function $f_r(x;\omega_i,\omega_o)$ that governs the ratio of outgoing radiance in direction $\omega_o$ to differential incident irradiance arriving from direction $\omega_i$. This fine-grained control over reflectance is essential for accurate depiction and measurement of real-world materials, whose visual appearance can exhibit significant spatial heterogeneity due to pigment, microstructure, wear, or functional design.

1. Mathematical Formulation and Models

Formally, for a surface position $x \in \mathbb{R}^2$ (or $\mathbb{R}^3$ in general geometry), and directions $\omega_i, \omega_o \in \mathbb{S}^2$,

$$f_r(x; \omega_i, \omega_o):\; \mathbb{R}^2 \times \mathbb{S}^2 \times \mathbb{S}^2 \to \mathbb{R}^3$$

The outgoing radiance at $x$ in direction $\omega_o$ is given by the rendering equation:

$$L_o(x, \omega_o) = \int_{\Omega^+} f_r(x, \omega_i, \omega_o)\;\,L_i(x, \omega_i)\,(n(x) \cdot \omega_i)\;d\omega_i$$

Various parameterizations are adopted depending on capture requirements and practical renderer compatibility:

• Analytical Parametric Models:

Disney “principled,” Cook–Torrance, Ward, GGX, Ashikhmin–Shirley. Spatial variation is modeled through 2D texture maps (e.g., basecolor $a(x)$, roughness $\alpha(x)$, metallic $m(x)$, normal $n(x)$, etc.) (Boss et al., 2020, Joy et al., 2022), [2019.12.01].

• Basis and Dictionary Expansions:

The SVBRDF at each $x$ is a nonnegative or affine blend of basis BRDFs $\{B_k\}$:

$$f_r(x; \omega_i, \omega_o) = \sum_{k=1}^K w_k(x) B_k(\omega_i, \omega_o)$$

Common in photometric stereo and inverse rendering frameworks (Hui et al., 2015, Li et al., 2020, Chung et al., 27 Nov 2024).

• Neural Representations and Latent Codes:

Neural fields and MLPs map $(x, \omega_i, \omega_o)$ or spatial latent codes plus angular coordinates to $f_r$. Geometry and reflectance fields are decoded continually per point, admitting unrestricted functional expressiveness and supporting high compression (Zhang et al., 2021, Dou et al., 2023, Fan et al., 2021).

2. Acquisition and Recovery Pipelines

SVBRDF acquisition typically couples shape and reflectance inference with varying illumination and view conditions, requiring joint or alternating estimation.

Methodology	Imaging Setup	BRDF Param.	Spatial Modeling
Photometric Stereo	Multi-light, fixed view	Dictionary or basis	Per-pixel dictionary weights
Inverse Rendering	Multi-view, unknown light	Analytic/Neural	2D textures / neural fields
GAN/Diffusion Priors	Flash image(s), single/multi-view	Microfacet+NN	Generator latent + decoder

• Photometric Stereo/Dictionary Approaches:

Multiple illuminations per view; per-pixel intensities fit via nonnegative combinations of known BRDF atoms, with normals estimated by normal search (possibly refined by gradient descent) (Hui et al., 2015, Hui et al., 2015).

• Differentiable Rendering/Analysis-by-Synthesis:

Minimization of rendering loss (e.g., L$_2$ on rendered-vs-observed images) under differentiable parametric, neural, or basis SVBRDF models (Boss et al., 2020, Zhang et al., 2021, Joy et al., 2022). Regularizers (spatial smoothness, entropy, priors from physical models or measured data) are requisite due to the severely ill-posed nature of disentangling shape, reflectance, and lighting.

• Diffuse & Specular Disentanglement:

Multi-stage optimization is common: first recover a coarse diffuse model and lighting, then jointly refine specular, roughness, and other spatially-varying parameters, often with per-texel or per-pixel losses on angular consistency (Joy et al., 2022).

• Neural Priors and Generative SVBRDFs:

Deep latent generative models (StyleGAN2, diffusion) impose strong statistical regularization on the space of spatially-varying reflectances, supporting data-driven inverse rendering and high-level editing (Guo et al., 2020, Sartor et al., 24 Apr 2024, Xue et al., 25 Apr 2024). The inversion is typically solved in latent space, optionally under differentiable rendering constraints.

3. Neural and Data-Driven SVBRDF Representations

Modern SVBRDF systems leverage neural networks for efficient compression, accelerated evaluation, and flexible spatial encoding:

• Coordinate-Based Neural Fields:

Functions $f_\theta(x)$ parameterized by coordinates $x$ (Fourier-embedded) predict BRDF parameters and geometry, enabling arbitrary resolution and seamless spatial transitions (Boss et al., 2020, Zhang et al., 2021).

• Neural Texture Maps:

Low- to mid-dimensional spatial latent features, learned per texel (e.g., $z(x)\in\mathbb{R}^{32}$), modulates an MLP mapping angles $(\omega_i, \omega_o)$ and spatial location to $f_r$ (Fan et al., 2021, Li et al., 10 Aug 2025).

• Spherical/Feature Grids + Neural Primitives:

Discretization of angular space with codebooks of neural primitives assigned via compact indices per grid point, with a small shared MLP acting as the nonlinearity, enables real-time evaluation and high data compression suitable for SVBRDFs and BTFs (Dou et al., 2023).

• Generative Diffusion and GAN Backbones:

SVBRDFs as 10D signals (diffuse, specular, roughness, normal), synthesized jointly conditioned on input images, textual description, or random noise; backbone and refinement strategies decouple pretraining and per-capture adaptation (Sartor et al., 24 Apr 2024, Xue et al., 25 Apr 2024, Guo et al., 2020).

• Operation Algebra in Latent Space:

Linear operations (interpolation, blend) or nonlinear composition (layering, mixing) are performed by neural networks solely in SVBRDF latent space, affording efficient compositionality and semantic editing (Fan et al., 2021, Guo et al., 2020).

4. Inversion, Regularization, and Disentanglement

SVBRDF recovery and editing systems must mitigate ambiguities and stabilize optimization:

• Gradient Variance Consistency (MVCL):

Ensuring that gradients of reconstruction losses with respect to spatial BRDF maps are consistent across different views where a texel is visible promotes robustness against view-dependent entanglement and improves separation of diffuse and specular signals (Joy et al., 2022).

• Sparsity and Interpretability:

Dynamic control over the number and assignment of basis BRDFs, with entropy regularization on spatial weights, ensures spatial separation and interpretable material lobes, supporting scene editing and physically-based relighting (Chung et al., 27 Nov 2024).

• Smoothness and Data-Driven Priors:

Spatial smoothness (Laplacian/TV) and GLO-type priors on latent BRDF codes (constrained by measured BRDF datasets such as MERL) regularize the fit, maintain physically-bounded variation, and enforce semantically reasonable transitions (Zhang et al., 2021, Boss et al., 2020).

• Parameter Remapping and Standardization:

Image-based SVBRDF remapping algorithms permit BRDF parameter translation between renderer conventions using explicit regression (e.g., roughness mapping, affine specularity scaling), supporting renderer-agnostic asset interoperability even with large SVBRDF atlases (Sztrajman et al., 2018).

5. Applications and Performance Characteristics

Spatially-varying BRDFs have become the standard input for modern physically-based and neural renderers. They drive applications in digital content creation, semantic editing, relightable capture, robotics, and remote sensing:

• Real-Time Rendering:

Neural and parametric SVBRDFs can be evaluated and sampled at 30–220 fps for full HD images with modern hardware (Boss et al., 2020, Dou et al., 2023).

• Inverse Rendering Under Unconstrained Conditions:

Pipelines can recover high-fidelity, relightable 3D meshes from unstructured photo collections, images under unknown environmental or point-light settings, and even from a single RGB image (Boss et al., 2020, Zhang et al., 2021, Li et al., 2019), with reported PSNRs reaching 30 dB (synthetic) and 25 dB (real) for novel-view/novel-light synthesis.

• Acquisition with Commodity Devices:

Practical pipelines recover high-resolution SVBRDFs using only 2–9 smartphone pictures of planar samples under flash/ambient lighting, leveraging clustering and staged optimization for tractability (Li et al., 2019).

• Satellite and Remote Sensing:

SVBRDFs parameterized by semi-empirical models (e.g., RPV) embedded within neural radiance fields improve novel-view synthesis and depthmap accuracy from limited, widely-separated satellite captures (Zhang et al., 18 Sep 2024).

• Editing and Synthesis:

GAN and diffusion-based SVBRDF priors enable plausible, physically-interpretable material interpolation, morphing, and semantic text-to-SVBRDF synthesis, supporting designer-in-the-loop workflows (Guo et al., 2020, Xue et al., 25 Apr 2024).

6. Limitations, Open Challenges, and Future Directions

Despite rapid advances, several challenges remain:

• Complex Materials and Anisotropy:

Most SVBRDF capture pipelines assume isotropy and opaqueness; extension to anisotropic, layered, or subsurface models (BTFs, BTDFs, multi-bounce/participating media) presents ongoing technical difficulty (Chung et al., 27 Nov 2024, Li et al., 10 Aug 2025).

• Dynamic and Non-Static Scenes:

Handling moving point lights, deformable geometry, and multi-object scenes under uncontrolled illumination remains a challenge for direct inversion (Joy et al., 2022).

• Resolution, Detail, and Interactivity:

Many neural and diffusion-based approaches are bottlenecked at $256\times256$ resolution, and efficient upsampling or scalable encodings are required for production applications (Sartor et al., 24 Apr 2024, Xue et al., 25 Apr 2024).

• Standardization and Renderer Interoperability:

The lack of BRDF standardization and variable semantics across renderers impedes direct transfer of SVBRDF assets; parametric regression is needed for reliable model-to-model interchange (Sztrajman et al., 2018).

• Physical Constraints and Expressiveness:

Dictionary-based and neural representations are only as general as their training or source basis; out-of-sample materials require new measurements or dictionary expansion, and certain pathological materials (e.g., precise multilayered composites) may elude existing modeling frameworks (Hui et al., 2015, Fan et al., 2021).

A plausible implication is that future SVBRDF research will emphasize unified frameworks for neural, basis, and analytic representations capable of physical plausibility, interoperability, and scalability, while expanding acquisition methods beyond static, opaque, homogeneous surfaces. Novel regularizers, adaptive spatial frequency models, and multimodal data (e.g., text, hyperspectral) may further bridge generative and physically-based paradigms for material capture and synthesis.