Adaptive Path Tracing in Monte Carlo Rendering

• Adaptive Path Tracing is a family of Monte Carlo methods that dynamically reallocates computational effort to reduce variance in rendering.
• It integrates techniques such as per-pixel adaptive sampling, learned path-guiding, and adaptive spatial discretizations to optimize resource use.
• Empirical studies indicate that APT methods deliver superior image quality and significant performance gains in real-time, low-sample, and volumetric scenarios.

Searching arXiv for papers on adaptive path tracing and related path-guiding/rendering methods. Adaptive path tracing denotes a family of Monte Carlo rendering methods that allocate computational effort non-uniformly during light transport simulation, rather than tracing a fixed number of rays uniformly over pixels, directions, or spatial cells. In contemporary usage, the term covers several distinct but related forms of adaptivity: per-pixel sample allocation in real-time image synthesis, spatiotemporal state accumulation coupled to neural denoising, path guiding via learned proposal distributions for directional sampling, and adaptive spatial discretizations for volumetric transport. Across these variants, the common objective is variance reduction or equivalent-quality acceleration under constrained budgets, especially in regimes where uniform sampling is inefficient or unstable (Scardigli et al., 2023, Bálint et al., 9 Feb 2026, Dodik et al., 2021, Benyoub et al., 13 Jun 2025).

1. Concept and scope

In path tracing, adaptivity refers to the dynamic redistribution of work toward regions, directions, or volumetric cells that are more difficult, more variable, or more consequential for the final image. The most direct form is image-space adaptive sampling, in which a renderer predicts how many rays to trace per pixel under a fixed samples-per-pixel budget. A recent real-time formulation describes Adaptive Path Tracing as a method that “dynamically allocates ray samples per pixel based on learned scene state and feedback, rather than tracing a fixed number of rays uniformly” (Scardigli et al., 2023).

This broad rendering sense includes at least three technical strata. First, there are adaptive sampling systems for low-sample image synthesis, including sub-1-spp regimes, where the principal question is where to spend an extremely limited ray budget (Bálint et al., 9 Feb 2026). Second, there are path-guiding methods that adapt the directional proposal distribution by learning incident radiance and BSDF structure online, so that future bounces are sampled from a distribution closer to the rendering integrand (Dodik et al., 2021). Third, there are adaptive representations for scene or medium traversal, such as tetrahedral volumetric grids that refine only where density variation or visibility requires it, thereby reducing both memory footprint and traversal cost (Benyoub et al., 13 Jun 2025).

A useful way to organize the area is by the object of adaptation. Some methods adapt pixel budgets, some adapt directional proposals, and some adapt transport geometry. This suggests a unifying interpretation: adaptive path tracing is less a single algorithm than a class of variance-allocation strategies embedded in path tracing.

2. Image-space adaptive sampling for real-time rendering

A central recent formulation combines adaptive sampling, latent-state accumulation, and neural denoising in an end-to-end system for real-time path tracing (Scardigli et al., 2023). At each frame, the system takes as input the warped latent space from the previous frame together with current per-pixel features and motion vectors. A sampling importance network predicts a per-pixel sampling map that determines how many new rays to trace at each pixel under the current spp budget. The raw sampled values are not averaged immediately; instead, all sampled values are fed to a latent space encoder, which fuses them with the warped previous latent state to produce a new spatiotemporal latent space. That latent state is then the sole input to the denoiser, which outputs the final image (Scardigli et al., 2023).

The same paper defines the reinforcement-learning formulation explicitly. The observation space is

$$S \text{ is in the space } [-1;1]^{720\cdot720\cdot39},$$

the action space is

$A \text{ is in the space } [-1;1]^{720\cdot720},$

the transition distribution is

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$

the reward is

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$

and the discount factor is

$\gamma = 0.99.$

Here the reward is tied directly to final reconstructed image quality, since better final images induce higher reward (Scardigli et al., 2023).

The same system departs from earlier learned adaptive sampling by training the sampling network with APPO policy optimization rather than by backpropagating through a numerically approximated sampling gradient (Scardigli et al., 2023). The sampling importance network is a UNet with 39 input channels: 7 additional features—3 normals, 3 albedo, 1 depth—plus the 32-channel warped latent space, and it outputs a single-channel sample map. The sample recommendations $\vec{y}$ are computed from the network output $\vec{x}$ as

$$\vec{y}_{i,j}=\mathrm{round}\left(\mathrm{spp\_budget} \cdot \frac{\vec{x}_{i,j}-\min(x)}{\sum_{a,b}(\vec{x}_{a,b}-\min(x))}\right).$$

A final Tanh activation is used so the outputs remain bounded, because unbounded values caused unstable recommendations (Scardigli et al., 2023).

The paper is especially explicit that it keeps all sampled values rather than averaging them per pixel. The latent encoder receives up to 8 samples per pixel, stored as 24 channels, with missing entries filled with $-1$ if a pixel has fewer than 8 samples. The architecture is intentionally not permutation invariant, allowing the network to extract information from all samples of a pixel and from neighboring pixels (Scardigli et al., 2023). This design is presented as a direct response to average-based methods that discard higher-order statistics.

3. Spatiotemporal latent states and neural reconstruction

The spatiotemporal latent space is a key architectural development in neural adaptive path tracing. Rather than storing only the denoised RGB output from the previous frame, the system stores a 32-channel latent state updated by a CNN encoder. This latent state is also the main input to the sampler (Scardigli et al., 2023). The stated rationale is that denoiser-output feedback compresses temporal information into only 3 channels and loses confidence information such as variance or sample count, whereas the latent representation preserves richer information across frames and removes the need for an explicit temporal loss (Scardigli et al., 2023).

The latent encoder itself takes the 24-channel sample input together with the warped 32-channel latent state and outputs a new latent space. The encoder ends with a Tanh layer to bound latent values, and the state channel count of 32 is described as a performance/latency tradeoff choice (Scardigli et al., 2023). The denoiser is another UNet whose architecture is reused from Intel OIDN-style denoising; kernel prediction is not used because previous work found it was not significantly useful in this context (Scardigli et al., 2023).

Training is split between reinforcement learning for the sampler and conventional backpropagation for the encoder and denoiser. The latent encoder and denoiser use a mixed $\ell_1$-MS-SSIM loss with a ratio of 0.16-to-0.84 (Scardigli et al., 2023). This separation of optimization mechanisms is central: the discrete sample-allocation policy is treated as a control problem, while latent accumulation and reconstruction remain differentiable.

A later sub-1-spp system also couples sampling and reconstruction end to end, but replaces the reinforcement-learning treatment with a stochastic formulation of sample placement designed to enable gradient estimation through discrete decisions (Bálint et al., 9 Feb 2026). In that method, a U-Net predicts a continuous density map $A \text{ is in the space } [-1;1]^{720\cdot720},$0, a softmax normalizes one output channel to form the adaptive density, $A \text{ is in the space } [-1;1]^{720\cdot720},$1 of the total budget is allocated uniformly for stability and $A \text{ is in the space } [-1;1]^{720\cdot720},$2 adaptively, and stochastic rounding discretizes the density into sample requests (Bálint et al., 9 Feb 2026). This indicates that end-to-end adaptive path tracing now includes both policy-optimization and differentiable-relaxation approaches.

4. Extremely low-sample regimes and perceptual optimization

Real-time path tracing increasingly operates under extremely low budgets, often below one sample per pixel. One recent system is explicitly designed for the sub-1-spp regime and argues that the core challenge is not merely denoising but deciding where the few available rays should go (Bálint et al., 9 Feb 2026). The paper further argues that superresolution is a poor fit for full path tracing because it uniformly lowers the effective sampling rate everywhere by rendering at reduced resolution, so difficult regions such as reflections and high-frequency lighting become undersampled while fine spatial details are lost before denoising (Bálint et al., 9 Feb 2026).

The same work formulates sample placement stochastically. For a predicted density $A \text{ is in the space } [-1;1]^{720\cdot720},$3 at pixel $A \text{ is in the space } [-1;1]^{720\cdot720},$4, the method always allocates $A \text{ is in the space } [-1;1]^{720\cdot720},$5 samples, defines the fractional part $A \text{ is in the space } [-1;1]^{720\cdot720},$6, draws $A \text{ is in the space } [-1;1]^{720\cdot720},$7, and allocates one extra sample if $A \text{ is in the space } [-1;1]^{720\cdot720},$8. The extra contribution is

$A \text{ is in the space } [-1;1]^{720\cdot720},$9

and the final estimate is normalized by $$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$0:

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$1

A smooth relaxation using a clipped linear ramp is then introduced:

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$2

with correction factor

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$3

and the paper reports that $$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$4 works well in practice (Bálint et al., 9 Feb 2026).

A notable development in this line is the shift from purely signal-space optimization to perceptual training objectives. The same paper proposes a tonemapping-aware training pipeline with differentiable filmic tone mapping, randomized exposure/contrast/saturation augmentation, and MILO as the perceptual loss in tone-mapped space (Bálint et al., 9 Feb 2026). Its spatial term is

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$5

the temporal term is

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$6

and the final loss is

$$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$7

This makes the sampling policy optimize visually meaningful error rather than only raw HDR discrepancy (Bálint et al., 9 Feb 2026).

The denoiser in that system is also specialized for sparse, nonuniform input. Instead of scatter-style kernel prediction, it uses a gather-based 5-level pyramidal filter, with $$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$8 gather kernels at each level, $$P(s'|s, a) \text{ follows the distribution of } (\omega \circ g) (h(a),s),$$9 learned upsampling kernels between levels, and a full-resolution $$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$0 temporal gather kernel applied to the warped previous HDR frame (Bálint et al., 9 Feb 2026). The method further introduces a learned generalization of albedo demodulation by predicting a 3-channel demodulation image rather than relying on exact renderer-provided decomposition (Bálint et al., 9 Feb 2026).

5. Path guiding as adaptive directional tracing

Adaptive path tracing is not limited to image-space sample counts. In path guiding, the adaptation concerns the directional proposal distribution used to extend light paths. A representative method learns a 5D Gaussian mixture model for incident radiance $$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$1 online during rendering and a pre-trained $$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$2D Gaussian mixture model for BSDFs $$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$3, then conditions and multiplies these models to sample the next path direction more like the true integrand of the rendering equation (Dodik et al., 2021).

The rendering target is written as

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$4

and the desired importance distribution as

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$5

The method approximates these factors separately by

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$6

and

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$7

leading to

$$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$8

This is adaptive path tracing in the sense that paths are traced, a better proposal is learned, and future paths are resampled from that learned distribution (Dodik et al., 2021).

Its key representational innovation is the spatio-directional mixture model, which models radiance jointly over space and direction rather than using a purely directional guide. The paper argues that a 5D model naturally captures correlations present in parallax and caustics and performs especially well on scenes with small, localized luminaires (Dodik et al., 2021). Tangent-space Gaussian parameterization is central. Each component is represented by a unit mean direction $$R(s, a, s') = 10^{1-loss((f \circ g) (h(a),s),gd)},$$9 and covariance $\gamma = 0.99.$0 in the tangent plane, with

$\gamma = 0.99.$1

and density on directions obtained by

$\gamma = 0.99.$2

Approximate product sampling under arbitrarily oriented BSDFs is then performed by covariance transport:

$\gamma = 0.99.$3

A spatial kD-tree over the 3D position dimensions partitions the mixture into local neighborhoods so that conditioning and EM updates operate on local subsets rather than all components globally (Dodik et al., 2021).

This direction of work broadens the meaning of adaptive path tracing. Here the adaptation occurs in the proposal distribution for continuation rays, not in the number of camera samples per pixel. A plausible implication is that “adaptive” in path tracing is best understood as adaptive allocation of variance-reduction effort across the full sampling hierarchy.

6. Adaptive spatial representations for volumetric transport

In volumetric path tracing, adaptivity can also enter through the discretization of the medium itself. One recent example uses adaptive tetrahedral grids constructed via the longest edge bisection algorithm for rendering volumetric data with path tracing (Benyoub et al., 13 Jun 2025). The motivating problem is that regular uniform grids waste memory and computation in empty or low-detail regions, producing very large memory footprints and expensive ray marching through unnecessary cells (Benyoub et al., 13 Jun 2025).

The method begins from a control cube tessellated into 24 tetrahedrons, “one per halfedge of the cube.” Tetrahedra are recursively bisected until subdivision criteria are met, and refinement may propagate to neighbors according to a deterministic rule so that the mesh remains conforming (Benyoub et al., 13 Jun 2025). Two subdivision criteria are used. The first is a density-variation criterion:

$\gamma = 0.99.$4

where $\gamma = 0.99.$5 denotes all density values in the cell volume. The second is a camera-dependency criterion that avoids unnecessary subdivision for tetrahedra outside the view frustum or projecting to less than one pixel in the final image (Benyoub et al., 13 Jun 2025).

The path tracing loop remains structurally standard:

$\gamma = 0.99.$9

The adaptive advantage comes from the representation. Each tetrahedron has exactly 4 neighbors in the interior, or fewer at boundaries, so traversal is bounded and simple. The paper notes that at most 4 candidate intersections per cell are evaluated and that this can be reduced to at most 3 faces using the ray direction and pre-stored face normals; only a few bits are needed because the grids produce only 18 different face orientations (Benyoub et al., 13 Jun 2025).

The memory and performance consequences are substantial. For the cloud dataset, the regular grid reference is $\gamma = 0.99.$6, while the adaptive tetrahedral grid contains 5,393,482 tetrahedrons, described as “roughly $\gamma = 0.99.$7 less cells” (Benyoub et al., 13 Jun 2025). Reported render times are 49.773 s for the regular grid reference and 1.633 s for the adaptive tetrahedral grid at the same $\gamma = 0.99.$8 image and 1024 spp, approximately 30× faster; at 32 samples per pixel the paper states the asset can be rendered in less than 50 ms, “fast enough for real-time pre-visualization” (Benyoub et al., 13 Jun 2025).

7. Empirical results, limitations, and terminological boundaries

The real-time neural adaptive sampling system of 2023 evaluates on Emerald Square, SunTemple, Zero-Day, and Ripple Dreams, with 1400, 1200, 400, and 500 frames respectively, split into clips of 20 frames, and ground truth computed at 1000 spp on an NVIDIA GeForce RTX 2080 Ti (Scardigli et al., 2023). At 4.0 spp, it reports 28.7 dB and 22.5 ms for the full method, compared with 28.0 dB and 18.3 ms for ReSTIR+OIDN, 27.9 dB and 46.4 ms for IMCD, 27.7 dB and 17.4 ms for NTAS, and 27.2 dB and 21.9 ms for DASR (Scardigli et al., 2023). The same paper states that the method is Pareto-optimal across a range of spp budgets and that using 2 spp achieves a 1.6× latency reduction while still slightly improving visual quality relative to the strongest baseline at 4 spp (Scardigli et al., 2023).

Ablation results in that work tie the gains to specific adaptive components: RL vs gradient approximation gives about 0.3 dB, adaptive sampling vs uniform about 0.8 dB, removing the latent encoder hurts by about 0.7 dB, removing temporal feedback by about 1.3 dB, and averaging samples instead of keeping all of them by about 0.5 dB (Scardigli et al., 2023). The paper also states that real-time ray tracing corresponds to latency under 30 ms, usually implying an overall budget of roughly 0 to 4 spp on a modern GPU for million-pixel frames (Scardigli et al., 2023).

The sub-1-spp system of 2026 reports, at 0.25 spp, PSNR 25.41, MS-SSIM 0.9273, HaarPSI 0.6815, MILO 2.677, CVVDP 7.056, and CGVQM 57.63 for the adaptive method, compared with PSNR 24.64 for its uniform counterpart, 23.92 for NPPD, 23.09 for SAUDC, and 22.88 for DLSS 4 Ray Reconstruction (Bálint et al., 9 Feb 2026). At 0.11 spp, it reports 24.26 PSNR for the adaptive method, compared with 23.40 for its uniform variant, 22.39 for SAUDC, and 21.74 for DLSS (Bálint et al., 9 Feb 2026). These numbers establish that adaptive sampling can remain effective even at minimal budgets (Bálint et al., 9 Feb 2026).

Limitations recur across the literature. The 2023 system does not model motion blur or after-effects and may over-focus samples on fast-moving objects; it is intended for entertainment-type rendering and may produce artifacts undesirable in safety-critical or medical VR/AR settings (Scardigli et al., 2023). The 2026 sub-1-spp system reports imperfect temporal dependence, especially for disocclusions and specular motion, and notes that learned demodulation is not universally superior numerically even if it improves specific color and texture artifacts (Bálint et al., 9 Feb 2026). The volumetric tetrahedral-grid work remains specialized to heterogeneous volumetric data rather than generic surface path tracing (Benyoub et al., 13 Jun 2025). Path-guiding with spatio-directional mixtures offers strong performance on localized emitters and caustics but incurs higher computational overhead (Dodik et al., 2021).

The acronym “APT” also has broader usage outside rendering. One 2025 paper titled “APT: Improving Diffusion Models for High Resolution Image Generation with Adaptive Path Tracing” uses the same name for patch denoising trajectories in latent diffusion models rather than for Monte Carlo light transport (Han et al., 29 Jul 2025). This suggests a terminological boundary: in graphics rendering, Adaptive Path Tracing primarily concerns adaptive sampling, guiding, or traversal within a path tracer; elsewhere, the phrase may be used more metaphorically for adaptive trajectory control.