Deferred Shading with Reflection-Baked Maps

• The paper demonstrates the integration of analytic microfacet BRDF models with learned radiance textures to achieve high-fidelity, real-time glossy reflections.
• It details the use of 2D Gaussian splatting and per-pixel G-buffer encoding to efficiently capture material properties and view-dependent lighting effects.
• The technique leverages multi-view consistency losses and reflection-strength priors to enhance material inference and reduce artifacts in dynamic lighting scenarios.

Deferred shading with reflection-baked material maps is a contemporary rendering paradigm that integrates physically-based environment interaction, multi-bounce specular effects, and precise material decomposition directly into the deferred shading pipeline. This technique is driven by advances in 2D Gaussian splatting, microfacet-based BRDF modeling, and reflection texture preintegration. By leveraging analytic normal estimation, multi-level prefiltered environment representations, and learned or precomputed radiance textures, it achieves real-time relightable renderings of glossy and reflective surfaces with high fidelity and minimal per-frame cost.

1. Deferred Shading Architectures with Reflection-Baked Maps

Modern deferred rendering separates scene geometry/material evaluation and radiometric evaluation. In 2D Gaussian splatting approaches such as RGS-DR and MaterialRefGS, geometry and material parameters are represented as a collection of learnable 2D disk-like surfels, each with analytic orientation and size. During the G-buffer pass, these surfels are rasterized with front-to-back alpha blending, populating per-pixel attributes that include diffuse color, roughness, metallic factor, specular tint, indirect color, and precise world-space normals (Kouros et al., 25 Apr 2025, Zhang et al., 13 Oct 2025, Kouros et al., 2 Oct 2025).

The per-pixel G-buffer state typically contains:

Buffer Slot	Range / Type	Purpose
Diffuse color	$\mathbb{R}^3$	Base color, non-specular part
Roughness or gloss	$[0,1]$	Microfacet distribution input
Specular/metallic	$\mathbb{R}^3$ or $[0,1]$	Reflectivity, F₀ adjustment
Indirect/reflection	$\mathbb{R}^3$	Multi-bounce or indirect light
Normal	$\mathbb{R}^3$ unit	Surface orientation
Feature map	$\mathbb{R}^F$	For learned residuals

Analytic normals are computed per surfel as $$\mathbf{n} = \frac{\mathbf{t}_u \times \mathbf{t}_v}{\|\mathbf{t}_u \times \mathbf{t}_v\|}$$, ensuring geometric consistency and sharp specular behavior, even under harsh grazing angles (Kouros et al., 25 Apr 2025).

The subsequent deferred shading phase computes the outgoing radiance using a microfacet BRDF, integrating both explicit material maps and prefiltered environmental contributions. A final residual pass (e.g., a small MLP fed with directional encoding and per-pixel features, or a radiance lookup) corrects for non-modeled, view-dependent effects (Kouros et al., 25 Apr 2025, Zhang et al., 13 Oct 2025).

2. Baking View-Dependent Reflections into Material Maps

Reflection-baked material maps encode view-dependent lighting effects by integrating or approximating environment lighting into spatially and angularly resolved material representations. The pioneering methods use two principal strategies:

Multi-Level Prefiltered Environment Maps

The microfacet BRDF reflection integral,

$$L_o(x, \omega_o) = \int_{\Omega} f_r(x, \omega_i, \omega_o) L_i(\omega_i) (\mathbf{n} \cdot \omega_i)\,d\omega_i$$

is approximated using prefiltered mipmapped cubemaps or parametric PMREM pyramids. The physically-based “split-sum” approximation decouples BRDF weights and environment sampling, enabling efficient real-time evaluation:

$$L_s(\omega_o, \mathbf{n}, \rho) = F(\omega_o, \mathbf{n}, \rho) \, \Phi(\omega_r, \rho, E) + (1-F) \, \Phi(\omega_r, \rho_0, E)$$

where $[0,1]$0 samples the prefiltered cubemap at direction $[0,1]$1 and level determined by roughness $[0,1]$2 (Kouros et al., 25 Apr 2025, Kouros et al., 2 Oct 2025). This provides sharp, view-dependent gloss by selecting environment mip level per roughness.

Radiance Textures and Per-Material Atlases

For highly view-dependent or multi-bounce phenomena, a per-material “radiance texture” stores the result of an offline ray-traced or path-traced bake:

• Each texel encodes outgoing radiance for a discretized range of reflection or incidence angles $[0,1]$3.
• During deferred shading, the current view, surface normal, and G-buffer UV index into a radiance block, returning the pixel’s physically accurate reflection (Fober, 2023).

This approach achieves full multi-bounce detail at run-time cost comparable to standard deferred reflection, with trade-offs in memory footprint and bake-time.

3. Learning and Supervising Reflection-Baked Material Maps

Material inference and baking in neural deferred splatting systems face intrinsic ambiguity due to the intertwined shape, reflectance, and illumination. Leading works resolve this using:

• Multi-view Consistency Losses: Patch-based warp consistency across views $[0,1]$4 enforces coherent material attributes in the face of different inputs, minimizing hallucinations and view-dependent artifacts (Zhang et al., 13 Oct 2025).
• Reflection-Strength Priors: High variance in photometric response across neighboring images signals high reflectance, used to supervise metallic/specular map assignment through $[0,1]$5 (Zhang et al., 13 Oct 2025).
• Normal and Diffuse Priors: Early-stage supervision with StableNormal or StableDelight handles ambiguities in material separation and normal estimation. These priors are phased out as view-consistent reflection signals accumulate (Kouros et al., 2 Oct 2025).

The full loss blends photometric, structural, normal, material, and reflection terms to train the scene representation holistically.

4. Deferred Shading Evaluation and Residual Modeling

After G-buffer population with reflection-baked material maps, the deferred shading pass evaluates the full radiometric output per pixel. In most paradigms:

• Diffuse is computed via prefiltered (Lambertian or Disney) environment interaction.
• Specular is evaluated by cubemap or radiance-texture lookup, combined with BRDF weights (often via LUT sampling for Fresnel-geometry).
• Indirect and occlusion terms (when present) incorporate ray-traced bounces or residual networks (MLPs over directional features) to inject high-frequency, non-local, and multi-bounce effects (Zhang et al., 13 Oct 2025, Kouros et al., 25 Apr 2025).

Residual passes, such as the spherical-mipmap-MLP model or explicit indirect/reflection map, recover subtle shadowing, ambient interreflection, or reflection complexity absent from the split-sum base. This refinement is learned end-to-end and added in screen space to the base shading output.

5. Technical Performance, Data Structures, and Implementation

The main computational and memory bottlenecks in deferred shading with reflection-baked material maps are:

Element	Typical Memory Usage	Scaling Factors
Cube mipmap (512²×6×3)	≈18 MB	Face res × #mips × channels
Spherical mipmap (8×512²×16)	≈134 MB	Level × pixel res × features
Radiance Atlas (e.g. 1k×1k×32×32)	4 GB uncompressed	Base res × angle buckets

Training times for full neural pipelines (e.g. RGS-DR, Spec-Gloss Surfels) are on the order of 1–2 hours per scene on modern GPUs; real-time or interactive rates are achieved for shading and relighting due to per-pixel texture fetches and absence of per-frame path tracing (Kouros et al., 25 Apr 2025, Kouros et al., 2 Oct 2025). Radiance atlas methods rapidly synthesize multi-bounce environment-coupled effects with the only extra cost a single atlas texture fetch (Fober, 2023).

Progressive upsampling of environment maps, early-stage lightweight priors, and BVH-accelerated mesh extraction for indirect bakes are standard for scalable optimization in neural pipelines (Kouros et al., 2 Oct 2025).

6. Comparative Results, Ablations, and Limitations

Quantitative evaluations on benchmarks (e.g., Shiny Synthetic, Glossy Synthetic, Ref-Real) indicate that:

• Cube-mip (split-sum) prefiltering is critical for physically accurate fast specular integrals; ablating this step reduces PSNR by >0.8 dB (Kouros et al., 25 Apr 2025).
• Residual or indirect passes (MLP or explicit bakes) further improve perceptual metrics (SSIM, LPIPS) and correct high-frequency artifacts (Kouros et al., 25 Apr 2025, Zhang et al., 13 Oct 2025).
• Multi-view consistency and reflection priors regularize material decomposition, producing crisper, more coherent highlights (Zhang et al., 13 Oct 2025).

Memory requirements scale steeply with atlas angular resolution and base map size. While radiance atlas approaches generalize to arbitrary material complexity and multi-bounce settings, they are currently best suited to static geometry and pre-baked illumination.

7. Relationships to Related Rendering Technologies

Deferred shading with reflection-baked material maps generalizes and outperforms traditional BRDF-based deferred pipelines by unifying analytic microfacet models, prefiltered environment maps, and machine-learned residuals. It subsumes:

• Classic deferred pipelines with screen-space cubemap or SH reflection.
• Explicit offline reflection bakes per material with runtime lookup (Fober, 2023).
• Neural scene representations (e.g. NeRF), but offers higher geometric and photometric precision on glossy and metallic surfaces due to analytic normal handling and environment coupling (Kouros et al., 25 Apr 2025).
• Standard rasterization-based PBR, with improved efficiency and real-time achievable quality for photorealistic, relightable renderings (Kouros et al., 2 Oct 2025).

A plausible implication is that this paradigm forms the technical foundation for next-generation relightable and editable digital asset rendering, seamlessly bridging the gap between stochastic offline path tracing, GPU deferred shading, and differentiable inverse rendering.

---

References:

• RGS-DR: Reflective Gaussian Surfels with Deferred Rendering for Shiny Objects (Kouros et al., 25 Apr 2025)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference (Zhang et al., 13 Oct 2025)
• Spec-Gloss Surfels and Normal-Diffuse Priors for Relightable Glossy Objects (Kouros et al., 2 Oct 2025)
• Radiance Textures for Rasterizing Ray-Traced Data (Fober, 2023)