Differentiable Voronoi Ray Tracing Beyond Rasterization Speeds

Abstract: Real-time novel view synthesis is dominated by rasterized explicit primitives. These projection-based pipelines provide high throughput but require specialized extensions for non-pinhole effects such as distortion, rolling shutter, and depth of field. Ray-based rendering expresses these effects naturally but is generally assumed too slow for competitive real-time rendering. We analyze the factors governing throughput in differentiable Voronoi ray tracing and identify traversal length, per-cell work, and memory locality as principal determinants. Guided by this, we introduce VoroTracing, which co-designs the scene representation, optimization, and GPU execution to reduce these costs. Compact octahedral appearance textures reduce memory traffic, while surface-concentrated opacity promotes early termination. The fixed-budget representation is optimized without pruning or densification and rendered with a GPU implementation designed for coherent traversal. On Mip-NeRF 360, VoroTracing renders at 623 FPS on an RTX 5090, providing $3.2\times$ the throughput of the fastest prior ray-based method and $2.8\times$ that of 3D Gaussian Splatting, while maintaining competitive reconstruction quality. Our renderer supports fisheye, rolling-shutter, motion-blur, and depth-of-field effects through ray generation and sampling, requiring no specialized rasterization. These results show that real-time throughput can be achieved with the flexibility of ray-based rendering. We release our source code, see https://research.zenseact.com/publications/vorotracing

• The paper introduces VoroTracing, a differentiable Voronoi radiance field that traverses local Delaunay adjacencies instead of performing repeated global ray-intersection queries, achieving 623 FPS on an RTX 5090 with 28.98 dB PSNR, 0.848 SSIM, and 0.235 LPIPS on Mip-NeRF 360.
• The method combines octahedral appearance textures, exponential scale-invariant density, surface-concentrated opacity, and early cell skipping to reduce memory traffic and traversal work, lowering average traversal depth to 46.1 cells per ray versus 66.9 for Radiant Foam.
• The paper shows that a ray-native renderer can outperform reported rasterized and ray-based baselines in throughput while supporting fisheye, rolling-shutter, depth-of-field, and motion-blur cameras, although fixed site capacity and benchmark-specific hardware remain limitations.

The paper argues that ray-based novel-view synthesis need not incur the throughput penalty conventionally associated with ray tracing. Its central claim is that the relevant comparison is not between “rays” and “rasterization” in the abstract, but between complete systems co-designed around representation, traversal, optimization, memory traffic, and GPU execution. VoroTracing instantiates this position with a differentiable Voronoi radiance field whose cells are traversed by local adjacency walks rather than repeated global intersection queries. On Mip-NeRF 360, the resulting renderer reaches 623 FPS on an RTX 5090, exceeding the reported throughput of both ray-based and rasterized baselines while retaining a ray-native image-formation interface (2608.17682).

Problem setting and thesis

Neural radiance fields established differentiable volume rendering as a flexible formulation for novel-view synthesis, but their per-ray sampling and neural evaluation costs limited real-time deployment. 3D Gaussian Splatting subsequently made explicit radiance representations practical through projected primitives, tile-based processing, sorting, and alpha compositing. This performance advantage came with a structural dependency on projection: non-pinhole cameras, rolling shutter, depth of field, and motion blur require modifications to the rasterization pipeline or additional approximation stages.

VoroTracing revisits the alternative represented by Radiant Foam: an explicit Voronoi partition in which each cell stores density and appearance. A ray begins with one nearest-cell query and subsequently advances through the Delaunay adjacency graph. At each step, the renderer tests the bisector planes associated with neighboring cells and selects the nearest forward crossing. The traversal cost therefore depends primarily on the number of cells visited by a ray, not directly on the total number of sites.

Figure 1: Adjacency-based traversal advances a ray through local Voronoi neighbors, replacing repeated global intersection queries with successive exit-face tests.

This observation defines the paper’s systems-level thesis. Three quantities dominate throughput: traversal length, per-cell evaluation cost, and memory locality. The method reduces traversal length through surface-concentrated opacity and early termination; reduces per-cell appearance traffic through octahedral textures; and improves execution efficiency through Morton ordering, warp-coherent ray scheduling, aligned memory accesses, and low-contribution cell skipping.

The resulting speed-quality trade-off is unusually favorable within the benchmark. Averaged over the seven Mip-NeRF 360 scenes, VoroTracing achieves 28.98 dB PSNR, 0.848 SSIM, and 0.235 LPIPS at 623 FPS. It is approximately 3.2 times faster than the fastest prior ray-based method in the reported comparison and 2.8 times faster than 3DGS. These claims require qualification: the benchmark uses a specific RTX 5090 viewer-style timing protocol, and several baselines have method-specific implementation and training conditions. Nevertheless, the measurements directly support the narrower claim that ray tracing can be competitive with, and in this evaluation faster than, widely used rasterized NVS systems.

Figure 2: VoroTracing occupies a high-throughput, competitive-quality region on the Mip-NeRF 360 speed-quality frontier.

Voronoi traversal as the computational substrate

A Voronoi representation partitions space into convex cells associated with a set of sites. Each cell has constant density, while appearance is evaluated from learned texture fields. The Delaunay graph supplies cell adjacency, so the renderer can walk through the scene without reconstructing a global acceleration query at every segment.

The distinction from BVH ray tracing is computationally important. In a BVH renderer, each traversal repeatedly searches a hierarchical structure for geometric intersections. In Voronoi tracing, the initial nearest-neighbor lookup is followed by local exit-face tests. A cell’s degree determines the number of neighboring bisectors examined at each step, while the number of traversed cells determines the number of iterations. The total site count remains relevant to memory footprint, cache behavior, construction, and initial lookup, but it is not the principal per-frame traversal variable.

The paper reports that VoroTracing visits 46.1 cells per ray on average, compared with 66.9 for Radiant Foam, a 31% reduction. On the Garden scene, the corresponding mean decreases from 73 to 49 cells per ray. The spatial distribution of this cost is also meaningful: traversal depth increases around foliage and object silhouettes, where geometric complexity is high, and remains low on smooth surfaces.

Figure 3: The reduction in traversal depth is distributed across the image, rather than arising only from a small number of unusually inexpensive rays.

This result establishes a direct implication for renderer design: reducing the global cell count is not sufficient. A representation with many sites can remain efficient if rays terminate after short local walks, whereas a smaller representation with diffuse opacity may be slower if rays must composite many semi-transparent cells.

Appearance representation and memory traffic

The paper replaces per-cell spherical harmonics with two $8 \times 8$ octahedral textures. The view-independent texture is indexed by the direction from the cell site to the ray-cell intersection point. Because a Voronoi cell is convex and contains its site, every direction from the site intersects the cell boundary exactly once, giving a bijective parameterization of boundary appearance by directions on the sphere. Octahedral mapping converts this direction to a square texture coordinate and avoids the polar singularities of latitude-longitude parameterizations.

The view-dependent texture is indexed by the camera direction. The two fields are added in logit space and passed through a sigmoid. This decomposition separates spatial variation within a cell from directional variation across viewpoints. The view-independent map can vary across a cell’s image footprint, while the view-dependent map supplies a residual for glossy, reflective, or otherwise view-dependent effects.

Figure 4: Octahedral mapping converts a direction from the cell site to a boundary intersection into a compact texture lookup.

This representation addresses two limitations of spherical harmonics. First, degree-$L$ spherical harmonics require $3(L+1)^2$ RGB values per cell; degree three therefore loads 48 values. VoroTracing performs two bilinear lookups, requiring 24 RGB values, independently of texture resolution. Second, spherical harmonics assign a spatially constant field within each cell for a fixed viewing direction. Fine image-space structure must consequently be represented by increasing the number of cells. The surface texture relaxes this coupling by encoding within-cell spatial detail.

Figure 5: Octahedral textures load a fixed number of neighboring texels, whereas spherical-harmonic traffic grows quadratically with angular degree.

The ablation results demonstrate that the two texture components are complementary rather than interchangeable. A diffuse surface texture alone improves neither quality nor speed sufficiently because it cannot represent view-dependent effects. Adding the view-dependent texture improves LPIPS from 0.278 to 0.250 and increases throughput from 552 to 594 FPS. With the view-dependent regularizer, the final model reaches 28.98 dB PSNR and 0.235 LPIPS at 623 FPS.

This apparently contradictory speed result—adding another texture increases throughput—follows from reduced opacity spreading. A richer appearance model explains directional effects directly, preventing the optimizer from distributing semi-transparent density across additional cells. The representation therefore performs more work for some visited cells but causes fewer cells to be visited and more low-contribution evaluations to be skipped.

Figure 6: Removing the view-dependent texture leaves diffuse regions mostly unchanged but alters reflective objects and foliage.

Surface-concentrated opacity and scale-invariant optimization

VoroTracing explicitly treats opacity concentration as both a reconstruction objective and a rendering optimization. In mostly opaque scenes, the desired configuration is transparent free space followed by a compact, nearly opaque surface. Such a distribution reduces compositing work, enables early termination, and increases the effectiveness of inference-time cell skipping.

The method incorporates the Mip-NeRF 360 distortion loss to penalize spatially dispersed rendering weights. Unlike depth supervision, this loss does not specify the surface position; it only encourages the weights that already explain the image to become compact. The resulting per-cell opacity distribution is strongly bimodal. Twenty-four percent of VoroTracing cells have opacity above 0.9, compared with only 4% for Radiant Foam, where 44% of cells lie in the semi-transparent interval from 0.1 to 0.9. VoroTracing obtains this distribution with approximately 2.0 million cells, versus 4.1 million for Radiant Foam.

Figure 7: Exponential density and distortion regularization shift opacity toward near-transparent and near-opaque cells, enabling shorter traversals.

The distortion weight controls an explicit quality-speed trade-off. Increasing it concentrates opacity more aggressively and increases FPS, but excessive values degrade reconstruction because uncertain or semi-transparent regions lose the thickness needed to model them. The selected operating point is near the quality peak and captures most of the available speedup. This is an important limitation of the computational argument: early termination is not free if the regularizer forces an oversimplified geometric explanation.

The scale-invariant density parameterization addresses a separate optimization bias. With conventional density $\sigma$ and segment length $\delta$, a fixed opacity requires density to scale inversely with $\delta$. The corresponding gradient with respect to density scales with $\delta$, so small cells receive weaker density gradients even when they should acquire the same opacity as larger cells. This creates a bias against finely tiled surface structure and can leave low-density haze in free space.

VoroTracing instead optimizes an unconstrained parameter $\rho$ with $\sigma = \exp(\rho)$. After applying the chain rule, the segment length cancels from the opacity gradient. Cells of different sizes but equal opacity therefore receive identically scaled optimization signals. The ablation supports the proposed mechanism: replacing softplus with exponential density raises PSNR from 27.90 to 28.18 dB, reduces cells per ray from 59.6 to 51.7, and increases throughput from 430 to 496 FPS.

Figure 8: After 1,000 training steps, exponential density produces sharper geometry and color than softplus density under the same initialization.

Fixed-budget training without densification

A notable design choice is the elimination of pruning, densification, progressive resolution changes, and multi-stage schedules. VoroTracing initializes up to 2 million sites from dense RoMa v2 image correspondences. Reference images are selected by camera-pose clustering, neighboring views are matched, correspondences are triangulated, and invalid or high-reprojection-error points are discarded. Density-aware subsampling based on a $128^3$ voxel grid reduces overconcentration in heavily matched regions, while 5,000 background sites provide coverage around the reconstructed cloud.

The model then optimizes site positions, densities, and both texture fields for 20,000 steps at fixed image resolution. Training takes 33–50 minutes per scene, with a mean of approximately 40 minutes on an RTX 5090. This training procedure is substantially simpler than adaptive explicit-field pipelines, and it makes the inference budget predictable. However, the simplification shifts responsibility to initialization. A fixed representation cannot add capacity where the correspondence cloud underrepresents a surface.

The ablation makes this cost explicit. Replacing the adaptive Radiant Foam pipeline with the fixed 2-million-site initialization decreases PSNR from 28.37 to 27.90 dB before the remaining VoroTracing components are introduced. The final texture and opacity formulation recovers and surpasses the original quality, but not uniformly across scenes. The strongest deficits appear in outdoor regions containing foliage, distant structures, and high-frequency texture.

Figure 9: VoroTracing often preserves more structure than competing ray-based methods but remains less detailed than 3DGS in underrepresented high-frequency outdoor regions.

The paper’s claim that densification is unnecessary should therefore be interpreted narrowly. It is unnecessary for the reported benchmark under dense correspondence initialization and a fixed 2-million-site budget. It is not shown to be unnecessary under sparse views, unreliable matching, larger scene scales, or scenes with substantial view-dependent geometry.

Quantitative performance and implementation effects

VoroTracing’s comparison with ray-based methods is strongest on throughput. It reaches 623 FPS averaged over Mip-NeRF 360, compared with 194 FPS for Radiant Foam, 131 FPS for ray-traced PowerFoam, 88 FPS for 3DGRT, and 27 FPS for Instant-NGP in the reported configurations. It also improves over Radiant Foam and PowerFoam in the aggregate quality metrics.

Against rasterized methods, VoroTracing achieves 28.98 dB PSNR and 623 FPS. 3DGS reaches 29.11 dB and 220 FPS; 3DGUT reaches 28.98 dB and 254 FPS; Triangle Splatting reaches 28.78 dB and 151 FPS. Thus, VoroTracing is not the highest-quality method on all metrics—3DGS has a 0.13 dB higher average PSNR, while Triangle Splatting has lower LPIPS—but it is substantially faster under the paper’s timing protocol.

The throughput improvement is not attributable to one representation change. Starting from the VoroTracing representation, the packed half-precision base renderer achieves 230 FPS. Morton ordering increases this to 378 FPS, warp-coherent tiling to 536 FPS, and low-contribution cell skipping to 623 FPS. The complete inference stack therefore produces a 2.71-fold improvement without measurable changes in PSNR or LPIPS.

The cell-skipping threshold is particularly favorable in the tested regime. Increasing it from zero to $L$0 raises throughput from 538 to 623 FPS with no reported change in PSNR or LPIPS. At $L$1, throughput reaches 649 FPS but PSNR falls by 0.79 dB. This establishes that the selected threshold is not merely an aggressive quality sacrifice: there is a substantial interval in which computational savings are effectively invisible under the reported metrics.

Ray-native camera effects and deployment

The renderer’s principal functional advantage is that image formation is specified through rays rather than a projection-specific primitive pipeline. Fisheye and lens distortion require only altered ray directions. Rolling shutter uses row-dependent camera poses. Depth of field samples different ray origins across an aperture, and motion blur averages rays generated at different temporal camera poses.

Figure 10: The same trained representation supports pinhole, fisheye, rolling-shutter, and motion-blur rendering by changing ray generation and sampling.

This flexibility is architectural rather than merely demonstrative: the traversal, appearance lookup, and compositing kernels remain unchanged. The throughput claim must nevertheless be separated from the application claim. Fisheye and rolling-shutter rendering preserve one ray per pixel, whereas depth of field and motion blur require multiple rays per pixel. Their frame rates therefore decrease approximately with the additional sampling workload, and the paper does not provide a controlled quality-speed evaluation for these effects.

The reported mobile deployment further supports the portability of the execution model. A 2-million-site Garden representation runs at an average of 40 FPS on an iPhone 16 at a resolution with 1024 pixels along the longer image dimension. The demonstration uses the same representation as the desktop evaluation, without compression, site-count reduction, or retraining.

Figure 11: The uncompressed 2-million-site Garden model runs interactively on an iPhone 16.

Limitations and open questions

The most important limitation is fixed capacity allocation. Dense correspondence initialization can fail in regions with few observations or unreliable matching, and VoroTracing cannot introduce new sites during optimization to compensate. The qualitative results show that 3DGS can retain sharper detail in foliage and distant textured regions. An adaptive cell-allocation mechanism could address this limitation, but the paper does not establish whether such adaptation can preserve the reported traversal and memory advantages.

The evaluation is also concentrated on Mip-NeRF 360 and one GPU generation. The reported speed depends on CUDA implementation details, RTX 5090 hardware, the chosen resolution, and a viewer-style timing boundary. Zip-NeRF uses a non-matched training setup and approximate RTX 5090 inference timing, while other baselines have implementation-specific evaluation paths. These choices do not invalidate the internal ablations, but they constrain the generality of cross-method FPS comparisons.

The representation remains volumetric despite its surface-concentrated opacity. The distortion loss encourages compact support but does not guarantee a watertight or geometrically faithful surface. Explicit mesh extraction, topology quality, and rendering quality after conversion remain open questions. Likewise, multisampled camera effects are demonstrated qualitatively rather than evaluated under a controlled rate-distortion analysis.

Conclusion

“Differentiable Voronoi Ray Tracing Beyond Rasterization Speeds” (2608.17682) presents VoroTracing as a systems co-design for real-time ray-based novel-view synthesis. Its principal contributions are adjacency-based Voronoi traversal, octahedral surface and view-dependent textures, scale-invariant exponential density, surface-concentrated opacity, fixed-budget dense initialization, and GPU execution optimizations centered on locality and coherence. On Mip-NeRF 360, these components yield 623 FPS with competitive reconstruction quality, outperforming the reported rasterized and ray-based baselines in throughput. The paper’s strongest conclusion is consequently specific and experimentally supported: under the evaluated representation, dataset, hardware, and timing protocol, ray-native rendering can provide both real-time performance and substantially broader camera-model flexibility than projection-dependent rasterization.