Triangle Splatting for Real-Time Radiance Field Rendering (2505.19175v1)

Abstract: The field of computer graphics was revolutionized by models such as Neural Radiance Fields and 3D Gaussian Splatting, displacing triangles as the dominant representation for photogrammetry. In this paper, we argue for a triangle comeback. We develop a differentiable renderer that directly optimizes triangles via end-to-end gradients. We achieve this by rendering each triangle as differentiable splats, combining the efficiency of triangles with the adaptive density of representations based on independent primitives. Compared to popular 2D and 3D Gaussian Splatting methods, our approach achieves higher visual fidelity, faster convergence, and increased rendering throughput. On the Mip-NeRF360 dataset, our method outperforms concurrent non-volumetric primitives in visual fidelity and achieves higher perceptual quality than the state-of-the-art Zip-NeRF on indoor scenes. Triangles are simple, compatible with standard graphics stacks and GPU hardware, and highly efficient: for the \textit{Garden} scene, we achieve over 2,400 FPS at 1280x720 resolution using an off-the-shelf mesh renderer. These results highlight the efficiency and effectiveness of triangle-based representations for high-quality novel view synthesis. Triangles bring us closer to mesh-based optimization by combining classical computer graphics with modern differentiable rendering frameworks. The project page is https://trianglesplatting.github.io/

• The paper presents a novel triangle-based differentiable renderer that achieves high visual fidelity and real-time performance by optimizing geometry and appearance.
• It introduces a smooth window function for differentiable rasterization, providing localized influence and compact support within projected triangles.
• The method employs adaptive pruning, splitting, and end-to-end optimization to outperform state-of-the-art techniques in both speed and visual quality.

The paper "Triangle Splatting for Real-Time Radiance Field Rendering" (2505.19175) proposes a novel differentiable rendering technique that leverages the classic 3D primitive: the triangle. The core idea is to represent a 3D scene as an unstructured collection of triangles ("triangle soup") and optimize their geometry and appearance parameters directly using image-based losses.

Existing methods like Neural Radiance Fields (NeRFs) excel at novel view synthesis but are often slow to render, requiring computationally expensive volume integration. While recent advancements like 3D Gaussian Splatting (3DGS) [Kerbl20233DGaussian] significantly improve rendering speed by using explicit primitives, their unbounded nature and inherent smoothness can make it difficult to represent sharp details and well-defined surfaces. Traditional graphics pipelines, in contrast, heavily rely on triangles for efficient rendering on dedicated GPU hardware, but optimizing them in a differentiable framework is challenging due to their discrete boundaries and occlusion properties.

Triangle Splatting bridges this gap by developing a differentiable renderer that "splats" triangles onto the image plane. This approach combines the geometric accuracy and efficiency potential of triangles with the adaptive density capabilities seen in primitive-based methods like 3DGS.

Key contributions and technical details include:

1. Differentiable Rasterization: Instead of using a traditional binary rasterization (pixel is either inside or outside the triangle), each triangle is rendered with a smooth influence based on a novel window function. This function, $I(\mathbf{p})$, for a pixel $\mathbf{p}$, is defined using the signed distance field (SDF) $\phi(\mathbf{p})$ of the projected 2D triangle: $$I(\mathbf{p}) = \text{ReLU}\left( \frac{\phi(\mathbf{p})}{\phi(\mathbf{s})} \right)^{\sigma}$$, where $\mathbf{s}$ is the triangle's incenter (point with minimum SDF inside), and $\sigma$ is a smoothness parameter. This window function has desirable properties: it reaches a maximum of 1 at the incenter, is strictly zero at the boundary and outside the triangle (providing compact support), and its smoothness is controlled by $\sigma$. Unlike previous sigmoid-based window functions used in related work [Held20253DConvex, Liu2019SoftRasterizer], this formulation ensures the influence is localized within the triangle's projection and scales consistently with depth without explicit scaling factors.
2. Adaptive Pruning and Splitting: Similar to 3DGS, the method uses adaptive density control.
   • Pruning: Triangles with low opacity or maximum blending weight, or those covering very few pixels across training views, are removed to reduce redundancy and combat overfitting/floaters.
   • Densification: New triangles are added stochastically using a framework inspired by MCMC sampling [Kheradmand20243DGaussian]. The probability distribution for sampling triangles is guided by both opacity and the inverse of the sharpness parameter $\sigma$. Triangles with low $\sigma$ (more solid) and high opacity are preferentially sampled for splitting or cloning. Splitting is performed by midpoint subdivision of the triangle's edges in 3D, ensuring conservation of area and spatial region.
3. Optimization Process: The scene is initialized with triangles generated from a sparse Structure-from-Motion (SfM) point cloud. Each triangle is parameterized by its three 3D vertex positions, sharpness ($\sigma$), opacity ($o$), and spherical harmonic color coefficients ($\mathbf{c}$). These parameters are optimized end-to-end by minimizing a loss function that combines standard photometric terms ($\mathcal{L}_1$, $\mathcal{L}_{\text{D-SSIM}$), along with regularization terms for opacity ($\mathcal{L}_o$), distortion ($\mathcal{L}_d$), normal consistency ($\mathcal{L}_n$), and a size regularization term ($\mathcal{L}_s$) that encourages larger triangles, particularly useful for filling underrepresented areas like walls.

Experimental results on standard benchmarks like Mip-NeRF360 and Tanks and Temples demonstrate that Triangle Splatting achieves higher visual fidelity (especially in LPIPS, which correlates better with human perception) and faster rendering/training times compared to many state-of-the-art primitive-based methods like 3DGS, 2DGS [Huang20242DGaussian], and 3DCS [Held20253DConvex]. It also achieves visual quality comparable to top-performing implicit methods like Zip-NeRF [Barron2023ZipNeRF] while offering vastly superior rendering speeds (hundreds to thousands of FPS compared to fractions of an FPS). The method is particularly effective for structured scenes (e.g., indoor environments) where triangles can accurately represent surfaces.

A significant practical advantage highlighted is the native compatibility of the optimized triangle soup with standard mesh-based rendering pipelines. By training parameters to produce solid triangles (e.g., annealing opacity and $\sigma$), the resulting dataset can be loaded and rendered in game engines and other traditional graphics software at extremely high frame rates (e.g., over 2,400 FPS at 1280x720 on an RTX 4090), without requiring specialized neural rendering inference engines.

Ablation studies confirm the importance of the proposed normalized window function (which avoids issues like unbounded support seen with sigmoid functions) and the various loss terms, particularly the opacity regularization and the size regularization for covering sparse regions.

While the method generates a high-quality triangle soup, a limitation is that it doesn't inherently produce a connected mesh. Generating a watertight mesh from the optimized triangles still requires post-processing techniques (e.g., TSDF fusion applied to rendered depth maps), similar to other splatting methods that don't guarantee connectivity. Future work could focus on meshing strategies that directly leverage the explicit triangle structure. The method can also occasionally suffer from floaters in sparse, unstructured outdoor scenes, a challenge shared by other non-volumetric primitives that rely on local support.

In summary, Triangle Splatting successfully demonstrates that the venerable triangle can be effectively optimized using modern differentiable rendering techniques, providing a highly efficient and accurate representation for novel view synthesis that is directly compatible with existing graphics hardware and pipelines, bridging the gap between neural rendering and classical computer graphics.