RaySplats: Hybrid Rendering & Reconstruction

• RaySplats frameworks are hybrid systems that model spatial data using 3D Gaussian primitives combined with ray tracing and rasterization.
• They utilize analytic ray–ellipsoid intersections to compute lighting, clipping, and RF field contributions with high precision and efficiency.
• Applications span photorealistic rendering, RF localization, and interactive editing, achieving real-time performance and improved accuracy over traditional methods.

A RaySplats framework refers to a class of hybrid rendering and field reconstruction systems that model and process spatial information using 3D Gaussian primitives coupled with ray tracing, rasterization, or other projection-based operators. While originating in the domain of 3D scene rendering, RaySplats approaches have been generalized—most notably to RF field reconstruction and interactive clipping—involving both photometric and non-photometric modalities. These frameworks leverage the analytic properties and spatial compactness of Gaussian ellipsoids (splat primitives), together with the geometric precision of ray tracing, to address limitations of pure rasterization or mesh-based approaches. This family encompasses representative systems such as "RaySplats" (Byrski et al., 31 Jan 2025), "REdiSplats" (Byrski et al., 15 Mar 2025), "RaRa Clipper" (Li et al., 25 Jun 2025), and "PropSplat" (Bjorndahl et al., 8 May 2026).

1. Mathematical Foundation: Gaussian Primitives and Ray–Ellipsoid Geometry

RaySplats-style frameworks encode spatial data using a set $\{G_i\}_{i=1}^N$ of anisotropic 3D Gaussian primitives. For each $i$, the splat is defined by center $\mu_i\in\mathbb{R}^3$, covariance $$\Sigma_i = R_i \operatorname{diag}(e^{2s_{x_i}}, e^{2s_{y_i}}, e^{2s_{z_i}}) R_i^T$$ (with rotation $R_i$ parameterized by a quaternion), and additional attributes such as color $c_i$, peak offset $o_i$ (for field adjustment), or opacity $\hat\alpha_i$. The density function is:

$$G_i(\mathbf{x}) = \frac{1}{(2\pi)^{3/2}|\Sigma_i|^{1/2}} \exp\left( -\frac{1}{2}(\mathbf{x} - \mu_i)^\top\Sigma_i^{-1} (\mathbf{x} - \mu_i) \right)$$

The volumetric support of each Gaussian extends far beyond its centroid, unlike mesh primitives or voxels. For visibility or field computation, each ray $\mathbf{r}(t) = \mathbf{o} + t\mathbf{d}$ is analytically intersected with the confidence ellipsoid of each Gaussian via quadratic form:

$i$0

where $i$1 is a quantile of $i$2, yielding the entry/exit points $i$3.

This formulation underpins both physical (optics, RF) and synthetic (graphics) field synthesis in all RaySplats-style systems, with splat attributes dictating compositing or correction roles per application (Byrski et al., 31 Jan 2025, Bjorndahl et al., 8 May 2026).

2. System Architectures and Hybridization Strategies

RaySplats frameworks are characterized by their fusion of rasterization and ray tracing in Gaussian scene processing.

• Scene Representation: The fundamental data structure is a set of 3D Gaussians, sometimes approximated by mesh-proxy polygons (octagons, triangle fans) for hardware acceleration (Byrski et al., 15 Mar 2025). Each Gaussian can encode photometric (color, opacity) or non-photometric (RF offset) attributes.
• Rendering Pipeline:
  • Rasterization: Employed to quickly identify primitives intersecting a projection region (image tile, clipping volume) (Li et al., 25 Jun 2025).
  • Ray Tracing: Used for precision evaluation: computing ray–ellipsoid intersections, length of ray segments inside primitives, and field contributions (e.g., line integrals for attenuation, opacity, or field modification) (Byrski et al., 31 Jan 2025, Bjorndahl et al., 8 May 2026).
  • Blending/Compositing: RaySplats use a volume compositing loop, typically of the form $i$4, with transmittance factors $i$5 for accumulating contributions along a ray (Byrski et al., 31 Jan 2025).
• Physical Effects: Lighting (shadows, reflections, transparency) is implemented by spawning additional rays from each intersection and applying BRDF models using analytic Gaussian normals or mesh-derived normals (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025).

A summary of key architectural features in RaySplats variants is given below:

System	Splat Geometry	Ray Tracing Role	Rasterization Role
RaySplats	3D Gaussians	Primary rendering	None
REdiSplats	Mesh-proxy GS	Primary rendering, editing	None
RaRa Clipper	3D Gaussians	Selective (cutoff)	Bulk classification
PropSplat	3D Gaussians	Field projection	None

3. Algorithmic Workflow: From Initialization to Inference

The core RaySplats workflow follows several application-specific stages:

1. Primitive Initialization:
   • For photometric rendering, primitives are fitted via multi-view geometry or point clouds (Byrski et al., 31 Jan 2025).
   • In RF field modeling (PropSplat), splats are initialized along transmitter–receiver segments, with centers sampled in $i$6 relative to each observed path and log-scale set to a fraction of the median path distance (Bjorndahl et al., 8 May 2026).
2. Optimization:
   • Parameters (mean, scale, rotation, peak/opacity) are optimized by minimizing a task-dependent loss (e.g., weighted MSE for field regression, SSIM/LPIPS for image synthesis).
   • End-to-end differentiability is achieved by propagating gradients through both analytic (Mahalanobis) weights and, where mesh proxies are used, mesh vertex positions (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025, Bjorndahl et al., 8 May 2026).
   • For PropSplat, loss is $i$7, with $i$8 emphasizing longer links (Bjorndahl et al., 8 May 2026).
3. Inference:
   • Rendering: At inference, each camera ray is analytically intersected with all (or a subset) of the Gaussians, computing color/field compositing in closed form or with depth ordering (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025).
   • RF Field Synthesis: Given a transmitter and candidate receiver, field prediction is $i$9, where $\mu_i\in\mathbb{R}^3$0 is the Mahalanobis distance from the projected point on the Tx–Rx line to the ellipsoid center (Bjorndahl et al., 8 May 2026).
   • Clipping: In RaRa Clipper, rasterization classifies Gaussians as fully in-front, fully behind, or cutoff by the clipping plane. Only cutoff Gaussians are ray-traced to compute length-weighted opacities, ensuring continuous attenuation (Li et al., 25 Jun 2025).

4. Applications: Rendering, Interactive Editing, RF Reconstruction, and Clipping

RaySplats methods appear in distinct technical domains:

• Photorealistic Rendering and Neural Scene Representations:
  • Provide real-time, high-fidelity synthesis with full lighting effects, including support for shadows, reflections, transparency, and hybrid mesh–splat geometry. Extensively evaluated on benchmarks such as Mip-NeRF360, Tanks & Temples, and Deep Blending, achieving PSNR up to 29.57 dB and SSIM up to 0.900 (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025).
  • Editable scene content is enabled by parameterizing each flat Gaussian as a mesh-polygon, allowing mesh vertex-level manipulation and export to Blender or Nvdiffrast for physics or manual control (Byrski et al., 15 Mar 2025).
• Wireless Field (RF) Reconstruction:
  • PropSplat eliminates the need for floor plans, GIS maps, or clutter data, learning the spatial offset field directly from sparse RF measurements. Inference is performed via a sum of Mahalanobis-weighted Gaussian offsets on-top of a learnable log-distance path-loss backbone, matching or exceeding the accuracy of state-of-the-art neural radiance field baselines for signal prediction and device localization (Bjorndahl et al., 8 May 2026).
• Interactive Clipping and Visualization:
  • RaRa Clipper addresses the artifact-prone nature of naive hard or center-based clipping for volumetric splats by applying ray tracing only to those intersected by the clip plane, ensuring artifact-free, anti-aliased boundaries at frame rates exceeding 77 FPS on multi-million-object scenes (Li et al., 25 Jun 2025).

5. Performance, Quality Metrics, and Comparisons

Quantitative evaluation demonstrates that RaySplats frameworks achieve efficient implementation and high-quality outputs across modalities.

• Rendering and Clipping:
  • Rendering speed exceeds 30 FPS at $\mu_i\in\mathbb{R}^3$1 for 30k Gaussians, with memory footprint below 10 GB on NVIDIA A100 for RaySplats (Byrski et al., 31 Jan 2025).
  • Clipping with RaRa shows negligible overhead (e.g., $\mu_i\in\mathbb{R}^3$281.5 FPS vs. 77.8 baseline at 4M Gaussians); artifact reduction is corroborated by zero $\mu_i\in\mathbb{R}^3$3 error and SSIM=1.0 in unclipped regions (Li et al., 25 Jun 2025).
• RF Field Reconstruction:
  • PropSplat, with only $\mu_i\in\mathbb{R}^3$4 primitives, achieves 5.38 dB RMSE in large-scale outdoor settings with sparse (300 m) training spacing, outperforming WRF-GS+ (5.87 dB), GSRF (7.46 dB), and NeRF$\mu_i\in\mathbb{R}^3$5 (14.76 dB) (Bjorndahl et al., 8 May 2026).
  • Indoor Bluetooth localization error reaches 0.19 m mean (vs. 1.84 m for NeRF$\mu_i\in\mathbb{R}^3$6), at matched RSSI RMSE.
• Perceptual and User Metrics:
  • RaRa Clipper is rated significantly higher than hard-clip baselines on six perceptual metrics (Wilcoxon $\mu_i\in\mathbb{R}^3$7) with up to 91% preference for topological integrity (Li et al., 25 Jun 2025).

Method	Domain	Key Metric	Value
RaySplats	Graphics	SSIM (Mip-NeRF360)	0.846
PropSplat	RF Field	Outdoor RMSE (dB)	5.38
RaRa Clipper	GS Visualization	FPS (4M Gaussians)	81.5
REdiSplats	Graphics	SSIM (Mip-NeRF360)	0.848

6. Relation to Prior and Alternative Approaches

Traditional 3D Gaussian splatting (3DGS) is rasterization-centric: it projects ellipsoids to image space, compositing alpha along depth. However, this approach is limited in incorporating geometric and photometric phenomena that depend on precise ray–scene interactions—such as hard shadows, optical transparency, and physically meaningful occlusions.

• Pure Rasterization: Efficient for direct view rendering but poor at handling light transport, arbitrary clipping, and mesh integration. Suffers from artifact-prone hard boundaries and limited compatibility with simulation and design tools (Byrski et al., 31 Jan 2025).
• Pure Ray Tracing: Accurately computes all per-ray effects but is computationally expensive for massive (10^6+) Gaussian clouds. Historically not real-time for large neural fields (Li et al., 25 Jun 2025).

RaySplats frameworks resolve this trade-off by:

• Employing rasterization for dense, trivial cases (most Gaussians), and ray tracing only for boundary-sensitive cases (clipping, lighting, field compositing) (Li et al., 25 Jun 2025).
• Utilizing mesh-proxy or analytic integration to enable fast, robust intersection tests and compositing (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025).
• Enabling full interoperability with device-native 3D pipelines without bespoke shaders or render engines (Byrski et al., 15 Mar 2025).

A plausible implication is that this hybridization sets a template for scalable, physically precise, and easily editable scene representations across a range of scientific, wireless, and visualization disciplines.

7. Implementation Notes and Extensibility

Implementations of RaySplats systems exploit both algorithmic and hardware accelerations:

• Hardware BVHs and RT cores (NVIDIA OptiX) eliminate the bottleneck for mesh-based or flat-Gaussian intersection (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025).
• Index buffers and forward–backward CUDA kernels avoid redundant traversals during gradient computation.
• Plug-and-play clipping modules can be inserted into arbitrary Gaussian Splatting engines, supporting arbitrary clipping primitives by extension (spheres, convex polyhedra) (Li et al., 25 Jun 2025).
• Minimal reliance on GIS or external geographic priors broadens applicability to domains lacking detailed mapping information, e.g., communication or unknown 3D topography (Bjorndahl et al., 8 May 2026).

The RaySplats framework thus constitutes a generalizable, analytically grounded paradigm for processing, rendering, and editing spatial data using orders of magnitude fewer parameters than pixel- or voxel-based methods, while supporting high fidelity, physical realism, and interactivity (Byrski et al., 31 Jan 2025, Byrski et al., 15 Mar 2025, Li et al., 25 Jun 2025, Bjorndahl et al., 8 May 2026).