---
title: Neural Radiance Caching (NRC) Explained
url: https://www.emergentmind.com/topics/neural-radiance-caching-nrc
type: topic
---

# Neural Radiance Caching (NRC) Explained

Neural Radiance Caching (NRC) is a family of learned radiance-cache methods that approximate expensive global-illumination transport with neural functions evaluated during or around path tracing. Rather than explicitly tracing every indirect-lighting path, an NRC system learns a representation of radiance or scattered illumination from Monte Carlo samples and reuses it to terminate paths, reduce variance, reconstruct local incident-light environments, or supply indirect illumination to differentiable renderers. The term encompasses several distinct parameterizations: outgoing scattered radiance, incident indirect radiance, prefiltered specular illumination, and specialized pre-integrated irradiance. These systems differ in whether they are trained online or offline, whether they replace transport or correct it with an unbiased residual, and whether radiance is represented implicitly by an MLP or explicitly through spatial-angular structures.

## 1. Conceptual foundations and historical antecedents

Radiance caching exploits spatial, directional, and temporal coherence in light transport. Monte Carlo path tracing repeatedly estimates radiance at related points and directions, despite neighboring queries often sharing substantial transport structure. A cache replaces some of these evaluations with an approximation trained or constructed from previously sampled transport.

For a surface point $x$ and outgoing direction $\omega_o$, the rendering equation is

$$
L_o(x,\omega_o)
=
L_e(x,\omega_o)
+
\int_{\Omega}
f_r(x,\omega_i,\omega_o)
L_i(x,\omega_i)
\cos\theta_i
\,d\omega_i.
$$

The quantity learned by an NRC depends on the method. It may be outgoing radiance $L_o(x,\omega_o)$, scattered radiance after local BSDF evaluation, incident indirect radiance $L_{\mathrm{ind}}(x,\omega_i)$, prefiltered incoming radiance for specular shading, or diffuse irradiance $E(x,n)$.

Analytical volumetric radiance caching provides an important predecessor. “Second-Order Occlusion-Aware Volumetric Radiance Caching” approximates radiance from sparse cache points using first-order extrapolation, while a Hessian-based error metric determines validity regions [1805.09258]. Its local segment- and triangle-based representation explicitly accounts for visibility changes, transmittance variation, and radiance curvature. The method reports up to a 30% reduction in computation time relative to previous work in a practical three-dimensional scenario, as well as cache-size reductions in two-dimensional experiments.

The analytical method distinguishes two roles for derivatives: gradients improve extrapolation, whereas Hessians determine whether extrapolation remains valid. For a cache point $x_0$, the second-order Taylor remainder is approximated by

$$
\epsilon(x)
\approx
\frac{
\left|
\Delta x^T H_L(x_0)\Delta x
\right|
}{
2L(x_0)
}.
$$

This yields anisotropic ellipsoidal validity regions whose principal radii decrease with radiance curvature. The approach is not neural, but it establishes physically motivated targets for neural cache placement, uncertainty estimation, adaptive sampling, and hybrid analytical-neural methods.

Neural methods replace explicit cache points, hand-derived derivatives, and manually designed interpolation with learned representations. The principal design choices concern the cached quantity, the conditioning variables, the source of training targets, and the treatment of approximation bias.

## 2. Neural radiance-cache parameterizations

### Scattered-radiance caching

“Real-time Neural Radiance Caching for Path Tracing” represents scattered radiance at a surface point and outgoing direction [2106.12372]. Its network approximates

$$
L_s(x,\omega_o)
=
\int_{S^2}
f_s(x,\omega_o,\omega_i)
L_i(x,\omega_i)
|\cos\theta_i|
\,d\omega_i
$$

with a single shared MLP. The input contains position, scattered direction, surface normal, roughness, diffuse reflectance, and specular reflectance. The encoded feature vector has 62 dimensions, padded to 64 for matrix multiplication. Position uses frequency encoding with 12 sine frequencies; direction, normal, and roughness use one-blob encodings; reflectance attributes are supplied directly. The network contains five hidden layers of width 64 and ReLU activations.

The method terminates paths when an area-spread criterion indicates that cache prediction is sufficiently effective. It trains online without pretraining, using a small fraction of extended training paths. Self-training uses cache predictions at suffix endpoints to construct low-noise targets for earlier vertices, while an unbiased fraction of training suffixes injects correctly sampled source radiance. The method uses four batches of 16,384 records per frame and performs four gradient steps per frame. Exponentially averaged weights stabilize rendering queries.

This formulation is designed for fully dynamic scenes involving changes in camera, geometry, materials, and lighting. The cache is implicit in network weights rather than stored as a set of world-space probes. Reported overhead is approximately 2.6 ms at full-HD resolution. In the reported experiments, the combined ReSTIR and NRC system maintains more than 60 frames per second at full HD and achieves an average 13.6-fold speedup over path tracing with ReSTIR at equal MRSE, with scene-specific speedups ranging from 6.1-fold to 172.7-fold.

### Learned local radiance-map reconstruction

“Deep Radiance Caching: Convolutional Autoencoders Deeper in Ray Tracing” uses a different neural cache representation [1910.02480]. A convolutional autoencoder reconstructs a local hemispherical indirect-radiance map at a surface intersection from a sparse one-sample-per-pixel radiance map plus normal and distance maps. The input has $32\times32$ spatial resolution and seven channels: RGB radiance, three normal components, and distance. The output is an RGB indirect-radiance map.

The reconstructed map is integrated with the actual BRDF or BSDF using multiple importance sampling. Direct illumination is excluded from the learned map and computed independently, preserving hard shadows, light-source boundaries, and other high-frequency direct-light features. The network is a U-Net-like convolutional autoencoder with symmetric encoder and decoder structures, skip connections, three downsampling and three upsampling stages, and a model size of approximately 20 MB.

Unlike image-space neural indirect-lighting systems, this method predicts a local directional radiance representation at ray-traced surface intersections rather than a final image or screen-space lighting layer. It is offline-trained but online at render time: the fixed model is trained on a general collection of scenes, while target scenes are rendered without per-scene training, prebaking, or persistent radiance-cache construction. The reported Bathroom Green benchmark gives 43 seconds and SSIM 0.89 at 16 indirect tasks, compared with approximately 99 minutes 30 seconds for the high-quality path-traced reference.

### Incident-radiance caching

“Neural Two-Level Monte Carlo Real-Time Rendering” introduces Neural Incident Radiance Cache (NIRC), which predicts incident indirect radiance at the current shading point as a function of position and incoming direction [2412.04634]. This differs from the scattered-radiance NRC formulation: NIRC can integrate many neural directional queries at the current point without first tracing an additional ray to locate a later cache-query surface.

NIRC uses multiresolution hash encoding for position and spherical-harmonic encoding for incident direction. The usual configuration has 12 hash levels with two features per level and four spherical-harmonic bands. Normal, albedo, and roughness may be supplied as additional surface features. The standard network has four hidden layers of width 64, ReLU activations, and a scalar or color radiance output.

Its principal contribution is not merely a cache representation but a two-level Monte Carlo estimator. The first level integrates the inexpensive neural approximation; the second estimates the residual between unbiased path-traced incident radiance and the cache prediction:

$$
F_{tl}
\approx
F_c+F_r.
$$

Because

$$
\mathbb{E}[F_c+F_r]
=
\mathbb{E}[f_c]
+
\mathbb{E}[f-f_c]
=
\mathbb{E}[f],
$$

the combined estimator is unbiased under the stated sampling and evaluation conditions. The cache may therefore be inaccurate without introducing systematic bias into the combined estimator, although a poor cache increases residual variance. A neural sample costs between 2% and 25% of a path-tracing sample in the reported implementations, with one implementation reporting a cost as low as 2.69%.

### Diffuse irradiance caching

“Real-time Rendering with a Neural Irradiance Volume” specializes neural caching to indirect diffuse irradiance [2602.12949]. It learns

$$
E_\theta(x,n)\approx E(x,n),
$$

where

$$
E(x,n)
=
\int_{\Omega}
L_i(x,\omega_i)
\langle \omega_i,n\rangle^+
\,d\omega_i.
$$

The cache is therefore a pre-integrated 5D field over three spatial and two angular coordinates. Runtime shading uses

$$
L_{\mathrm{out}}(x)
=
\frac{\rho(x)}{\pi}E_\theta(x,n)
+
D(x)
+
L_e(x).
$$

Unlike general NRC, the method requires no runtime angular integration, ray tracing, cache lookup deferral, or denoising for its intended diffuse use case. It is trained offline for a scene or scene family and receives a G-buffer containing world-space position and normal. The reported operating point uses 1–5 MB for medium-sized scenes and approximately 1 ms per full-HD frame on an RTX 4090. The method is designed for mostly static environments and relatively small dynamic objects whose influence on global transport is limited.

### Prefiltered specular caching

“Neural Caching of Prefiltered Radiance for Specular Lighting” adapts NRC to specular transport [2610.11702]. Instead of predicting final outgoing radiance, the network predicts roughness-dependent prefiltered incoming radiance conditioned on position, reflected viewing direction, and roughness:

$$
\widehat{L}_{\mathrm{pref}}(x,\omega_{\mathrm{refl}},\rho)
=
\operatorname{MLP}
\left(
\phi(x),
\omega_{\mathrm{refl}},
\rho
\right).
$$

The reflected direction is

$$
\omega_{\mathrm{refl}}
=
2(n\cdot\omega_o)n-\omega_o.
$$

The prediction is combined with a precomputed BRDF integration map using the split-sum approximation. The network models filtered scene illumination, while the lookup map supplies BRDF, Fresnel, and view-angle reconstruction. The method retains online training and reports faster convergence and improved specular-lighting quality relative to evaluated NRC baselines on Bunny and Specular Sponza.

## 3. Online training, self-supervision, and bias control

Online adaptation is central to the real-time NRC formulation. A pretrained network must generalize across geometry, materials, lighting, camera viewpoints, and animation states. Online training instead adapts the cache to the current rendering stream.

Training targets may be obtained in several ways. Direct path-traced targets are stochastic and approximately unbiased but noisy. Self-training reduces noise by using cache predictions at later path vertices as targets for earlier vertices. In the original real-time NRC formulation, every sixteenth training suffix is terminated by Russian roulette rather than by a cache query, providing an unbiased source of transport and limiting self-confirming feedback [2106.12372].

Self-training approximates high-order transport through repeated few-bounce updates. If $T$ denotes one-bounce transport and $E$ emitted or directly sampled radiance, the target solution is conceptually

$$
L
=
E+TL
=
E+TE+T^2E+T^3E+\cdots.
$$

Repeated cache updates propagate information through additional effective bounce orders without tracing a long path in a single frame. This mechanism is approximate: it depends on training paths revisiting relevant regions and on the cache being sufficiently accurate at terminal points.

NIRC takes a different approach. It does not use NRC’s self-training strategy because no benefit was observed for the incident-radiance formulation. Instead, a separate training pass uses unbiased path tracing and Russian roulette, typically with approximately 2–3% as many paths as pixels [2412.04634]. The two-level estimator then preserves unbiasedness through an explicit residual.

Other approaches use noisy targets but rely on expectation learning. GSCache trains 3D Gaussian primitives from attenuated volume-path samples using a Noise2Noise rationale: stochastic gradient descent can recover the expected path-subspace radiance when the noisy targets are unbiased [2507.19718]. Its cache is biased at render time because paths are deliberately terminated into cache levels, with a user-controlled coefficient governing the bias–variance–runtime trade-off.

Offline approaches place the optimization burden before rendering. DRC trains once on 43 scenes and applies a fixed model to target scenes without per-scene training [1910.02480]. NIV uses Mitsuba 3 to generate reference irradiance for scene-specific training and converges within at most 50,000 iterations [2602.12949]. Neural radiosity and its glossy extensions also optimize independently for each static scene using rendering-equation residuals.

Training objectives commonly use relative losses to prevent high-radiance samples from dominating. Examples include relative $L^2$ losses with stop-gradient normalization in online NRC, NIRC, JOC, neural radiosity, and NIV. Such objectives improve numerical conditioning but do not eliminate approximation bias, stochastic derivative error, or errors caused by incomplete training support.

## 4. Representations for spatial, angular, and glossy structure

A central NRC design problem is representing high-frequency visibility and specular transport. Position-only or low-dimensional directional inputs often force an MLP to reconstruct narrow angular radiance distributions.

Neural Cone Radiosity addresses this by replacing pointwise glossy queries with reflection-cone queries [2509.07522]. The cone is centered on the mirror reflection direction, and its aperture is determined by roughness and the microfacet normal-distribution function. For a cone traveling distance $t$, the projected footprint radius is

$$
r_{\mathcal C\perp}
=
t\tan\left(\frac{\theta_{\mathcal C}}{2}\right).
$$

Reflected samples are clustered by depth using one-dimensional $K$-means. The reported training configuration uses 128 reflected rays and four clusters, while runtime uses 32 rays. Each cluster yields a scale-aware glossy query whose contribution is weighted by the fraction of rays assigned to that cluster. Scale-aware multiresolution hash-grid features provide finer spatial bandwidth for small footprints and coarser bandwidth for broad footprints.

OctaOctree makes spatial-angular structure explicit through an adaptive 3D octree whose spatial cells contain octahedral directional feature maps [2606.08469]. Fine spatial levels use coarser directional maps, while coarse spatial levels use richer angular maps. Direction is projected onto an unfolded octahedron and interpolated barycentrically. Spatial interpolation uses neighboring octree cells. A learned disparity field reprojects lookup directions toward a virtual reflected point, improving spatial reuse of similar reflected content.

The reported OctaOctree representation uses eight levels, four-dimensional features per level, and a decoder with four hidden layers of width 128. It predicts outgoing radiance directly at visible primary intersections with one decoder evaluation. On the reported scenes, it is faster than the evaluated Neural Cone Radiosity, Neural Radiosity, OIDN, and deferred NR/NRC baselines. Its eight-level configuration, however, requires approximately 1.2 GB of cache features, demonstrating a substantial memory–quality trade-off.

GSCache uses an explicitly spatial but non-neural representation for volume path tracing [2507.19718]. Each path-length level contains a point cloud of 3D Gaussians storing position, covariance, opacity, and RGB radiance. Three levels are used in the principal implementation, with approximately 28 MB of cache storage. Gaussian splatting rasterizes each level once per frame, after which the path tracer reuses the resulting cache images. The reported representation uses spherical-harmonic degree zero and therefore stores view-independent RGB.

These approaches differ in the location of angular complexity:

| Method | Cached quantity | Angular treatment |
|---|---|---|
| Real-time NRC | Scattered radiance | Direction and material inputs to an MLP |
| NIRC | Incident indirect radiance | Directional neural queries integrated by Monte Carlo |
| Neural Cone Radiosity | Prefiltered glossy radiance | Roughness-dependent cones and scale-aware features |
| OctaOctree | Outgoing radiance features | Explicit octahedral maps per spatial cell |
| NIV | Diffuse irradiance | Angular integration performed before learning |
| GSCache | Path-length radiance | Degree-zero RGB Gaussian splats |

## 5. Applications and systems integration

NRC has been integrated into several rendering regimes.

For real-time path tracing, online scattered-radiance NRC shortens paths and reduces indirect-lighting noise. It complements ReSTIR, which primarily addresses direct-lighting noise. The method is implemented as a streaming GPU system with fused neural inference, bounded training buffers, and exponentially averaged weights [2106.12372].

For global-illumination rendering on CPUs, DRC reconstructs local hemispherical maps and integrates them with actual material responses [1910.02480]. It supports diffuse, glossy, perfect-specular, transmission, and ambient-occlusion effects to the extent represented by the local map and ray-traced pipeline. Caustics and high-frequency directional transport remain difficult.

For differentiable inverse rendering, JOC uses an NRC distilled from a pretrained neural field to supply indirect radiance during otherwise single-bounce differentiable Monte Carlo rendering [2305.16800]. The cache predicts scattered radiance conditioned on position, direction, and normal. Blocked rays query the NRC rather than terminating with zero radiance. This reduces the tendency of inverse optimization to bake inter-reflections, shadows, and color bleeding into albedo or other material parameters. JOC jointly optimizes DMTet geometry, Disney-style PBR materials, normal maps, and an HDR environment light. At 128 samples per pixel, the reported prototype incurs approximately 40% longer iteration time and 20% additional GPU memory than NVDIFFRECMC, while avoiding explicit differentiable multi-bounce path storage.

For volume path tracing, GSCache organizes cache levels by path length and uses Gaussian splatting to accelerate repeated queries [2507.19718]. It adapts online to lighting, transfer functions, viewport changes, and slicing operations, although transfer-function changes that alter visible morphology require reinitialization. Its cache-rasterization cost is largely independent of samples per pixel, allowing the per-frame cache state to be reused across many samples.

For diffuse real-time global illumination, NIV provides a compact offline model queried from a G-buffer [2602.12949]. It can illuminate previously unseen dynamic objects within the trained volume, but does not model their complete feedback on global transport. Dynamic ambient occlusion is used to approximate some local self-occlusion and indirect occlusion effects.

For glossy rendering, specular-prefiltered NRC, Neural Cone Radiosity, and OctaOctree modify the cache target or representation to address narrow reflection lobes. These methods demonstrate a general principle: the most effective neural cache need not predict unrestricted radiance with a generic coordinate MLP. It may instead predict a pre-integrated quantity, expose roughness-dependent support, or store angular structure explicitly.

## 6. Limitations, trade-offs, and research directions

NRC methods trade variance, bias, memory, training cost, and generality. Online methods adapt to dynamic scenes but may exhibit training lag, stale predictions, temporal flickering, or local minima after abrupt changes. Offline models can provide stable and inexpensive inference but are constrained by their training scenes, geometry, lighting parameterization, and material distributions.

General-purpose online NRC can struggle with sharp contact shadows, caustics, high-frequency visibility boundaries, thin geometry, glossy surfaces, long specular chains, and effects weakly correlated with network inputs [2106.12372]. Self-training can propagate only transport reached by sampled suffixes. EMA stabilizes inference but does not eliminate temporal artifacts.

NIRC’s unbiased residual corrects cache bias only when the residual is evaluated with unbiased path tracing and correct sampling. If the cache is inaccurate, the residual may be large and sign-changing, increasing variance [2412.04634]. Its request and feature buffers can consume hundreds of megabytes at 1080p.

DRC is biased because its learned maps have finite resolution and are reconstructed from a fixed training distribution [1910.02480]. It has no temporal memory, can flicker in animation, loses glossy detail at limited map resolution, and does not accurately represent caustics. NIV is narrower still: it assumes diffuse indirect transport, mostly static geometry, efficiently computed direct lighting, and limited dynamic feedback [2602.12949].

Analytical volumetric caching is physically interpretable but relies on local smoothness, accurate derivatives, sufficiently resolved angular subdivisions, and assumptions such as homogeneous isotropic media [1805.09258]. Neural methods can relax some of these constraints, but must replace explicit visibility models with learned features, ray-informed inputs, or hybrid analytical supervision.

Explicit spatial-angular representations improve glossy fidelity but increase memory. OctaOctree’s eight-level configuration uses approximately 1.2 GB in the representative experiment [2606.08469]. Neural Cone Radiosity requires scene-specific training lasting approximately 1–5 hours and stores approximately 104 MB per scene [2509.07522]. Specular-prefiltered NRC inherits split-sum approximation error, particularly at grazing angles, and its exact architecture and optimization settings are not specified in the supplied report [2610.11702].

Several research directions follow from these trade-offs. Analytical gradients and Hessians can supervise neural cache validity and sample placement. NIRC-style residual estimators can convert biased neural predictions into unbiased or lower-bias estimators. Cone-based and octahedral representations can supply explicit glossy support to otherwise pointwise NRCs. Diffuse irradiance models can act as compact priors or specialized subfields within broader radiance caches. Hybrid systems can combine neural predictions with path-traced residuals, visibility-aware analytical models, adaptive uncertainty estimates, and material-specific parameterizations.

The resulting design space is not a single algorithm. NRC may denote an online MLP predicting scattered radiance, a neural incident-radiance estimator, a learned local radiance-map reconstructor, a prefiltered specular cache, or a broader class of neural approximations to light transport. Across these variants, the common objective is to replace repeated expensive transport evaluations with reusable learned structure while controlling the resulting bias, variance, memory, and adaptation cost.

Source: https://www.emergentmind.com/topics/neural-radiance-caching-nrc