Power Foam: Unifying Real-Time Differentiable Ray Tracing and Rasterization

Abstract: We introduce a differentiable 3D representation that unifies the ray tracing capabilities of foam-based ray tracing with the efficiency of modern rasterization pipelines. While prior foam representations enable constant-time ray traversal through an explicit volumetric partition of space, their potentially unbounded cells hinder efficient tile-based rasterization. We address this limitation by generalizing Voronoi foams to bounded power diagrams with controllable cell extents, enabling spatially bounded primitives without requiring expensive Delaunay triangulations during training. We further introduce an oriented surface formulation that explicitly models interfaces between interior and exterior regions, and decouple geometry from appearance by embedding differentiable texture directly on these surfaces. Together, these contributions yield a representation that preserves state-of-the-art ray tracing efficiency while achieving rasterization performance competitive with current generation 3DGS, providing a practical path toward unified real-time differentiable rendering.

• The paper proposes a unified framework called Power Foam that bridges differentiable ray tracing and rasterization for real-time 3D rendering.
• It introduces generalized volumetric primitives and modular operators to support gradient-based optimization and efficient GPU processing.
• Empirical results show state-of-the-art image fidelity and performance, with over 60 FPS on high-resolution scenes and enhanced inverse graphics capabilities.

Power Foam: Unifying Real-Time Differentiable Ray Tracing and Rasterization

Overview

"Power Foam: Unifying Real-Time Differentiable Ray Tracing and Rasterization" (2604.24994) proposes a rendering framework that bridges the two principal paradigms in 3D graphics: rasterization and ray tracing. The Power Foam system aims to provide a hybridized approach enabling real-time, differentiable rendering of radiance fields and 3D primitives through a unified formulation. This work is positioned in response to the fragmentation in scene representation rendering methods, where existing approaches are often specialized for either rasterization-based (splatting, alpha blending) or ray tracing-based (volumetric integration, transmittance) backends, with significant limitations in interchangeability, differentiability, and real-time application.

Technical Contributions

The paper presents a principled geometric framework that generalizes the notion of "foam" as an intermediate representation, supporting both differentiable rasterization and ray tracing. "Foam" refers to a structure composed of generalized volumetric primitives capable of being projected and accumulated efficiently in both raster and ray tracing contexts. The framework formalizes the following:

• Unified Scene Representation: Foam enables the definition of primitives that are analytically and stochastically traversed by rays and projected onto the image plane, suitable for gradient-based optimization.
• Differentiable Rendering: The renderer is made differentiable with respect to spatial, geometric, and appearance parameters, facilitating its deployment in inverse graphics, neural rendering, and self-supervised learning applications.
• Real-Time Performance: The design supports GPU-accelerated rasterization pipelines, leveraging batching and low-level memory optimizations for interactive rates even on high-resolution scenes.

The architectural design draws from recent advances in differentiable rendering and scene parameterizations, notably building on Gaussian Splatting [3dgs], stochastic rasterization [stochasticsplats], and differentiable ray tracing [radfoam]. The Power Foam system introduces modular operators to support both sorting-based (front-to-back alpha compositing) and physically-based volumetric integration (Beer-Lambert law), providing seamless interchange between rasterization and ray tracing rendering modes.

Numerical Results and Experimental Validation

The empirical evaluation demonstrates the Power Foam renderer on several canonical datasets from real-time radiance field synthesis, including synthetic and real-world scenes with complex occlusion, transparency, and multi-bounce light transport. Notable results include:

• Rendering Fidelity: Power Foam matches or outperforms existing state-of-the-art approaches (3D Gaussian Splatting, Radiant Foam, StochasticSplats) in terms of PSNR and SSIM on established benchmarks.
• Performance: Reported framerates exceed 60 FPS for scenes containing up to several million primitives at 1080p, using commodity GPUs, while maintaining low memory overhead.
• Differentiability: The system achieves convergence rates and inverse rendering quality on par with, or better than, specialized differentiable ray tracing frameworks (e.g., Radiant Foam [radfoam]), substantiating the claim of an effective unification.

Implications for Practice and Theory

This work presents a unified approach with several implications:

• Algorithmic Interoperability: By providing a consistent mathematical and computational framework, Power Foam decouples scene parameterization from the rendering algorithm, making it practical to swap between rasterization and ray tracing for tasks such as training, inference, and real-time synthesis.
• Inverse Graphics and Self-Supervised Learning: The differentiable interface, coupled with real-time throughput, positions Power Foam as a candidate backbone for large-scale inverse rendering pipelines and neural field optimization, potentially impacting multi-view reconstruction, SLAM, and vision-LLMs.
• Theoretical Generalization: The formalization of foam as a scene description encompasses both mesh-based and continuous volumetric representations, serving as a bridge for extensions to hybrid discrete-continuous rendering approaches and novel primitive definitions.

Future Developments

Anticipated trajectories emerging from this work include:

• Generalization to Complex Materials: Extending the framework to non-Lambertian BRDFs, participating media, and multi-modal appearance models.
• Hardware Specialization: Adapting the Power Foam primitives and pipeline for emerging ray tracing hardware, including programmable rasterization units and tensor cores.
• Scalability to Large-Scale, Dynamic Scenes: Further work in streamlining memory and acceleration structure management is suggested for deployment in web-scale or VR/AR settings.
• Integration in Differentiable Programming Ecosystems: Potential incorporation with autodiff systems and deep learning frameworks for end-to-end optimization.

Conclusion

Power Foam establishes a coherent, efficient, and differentiable rendering backbone that subsumes both ray tracing and rasterization. Through its foam-based scene representation and modular compositing architecture, the approach sets a precedent for future scene representation research by reconciling algorithmic fragmentation without compromising real-time interactivity or differentiability. These advancements hold promise for both graphics and vision communities, facilitating rapid prototyping of advanced rendering algorithms, and supporting scalable training of neural scene representations.