Photon Splatting in Graphics & Sensing

• Photon splatting is a method that uses Gaussian kernels to project and accumulate discrete carriers for approximating radiance and signal fields in rendering and sensing.
• It extends traditional point-based rendering by incorporating mathematical accumulation rules and ray-based evaluations to ensure accurate depth and opacity computation.
• The approach supports diverse applications such as SPAD-based scene reconstruction, wireless channel prediction, and holographic wavefield generation, bridging physics and graphics.

Searching arXiv for papers on "photon splatting" and closely related splatting-based methods to ground the article in current literature. Photon splatting denotes a family of splatting-based constructions in which discrete carriers are projected with finite-support or Gaussian-like kernels and accumulated to approximate a target field. Recent arXiv literature suggests that the term spans both a graphics lineage—point splatting, surfels, Gaussian splatting, and related radiance-field renderers—and domain-specific systems in which the accumulated quantity is not RGB radiance alone but also SPAD hit probability, holographic complex amplitude, or wireless channel response (Franke et al., 2024, Teja et al., 26 Jun 2025, Cao et al., 7 Jul 2025, Choi et al., 10 May 2025).

1. Terminology and lineage

In the splatting literature, the basic operation is the projection of a discrete primitive to a footprint in a target domain, followed by weighted accumulation. Traditional point-based rendering and surfels use points or discs with Gaussian alpha masks and normalized blending, whereas screen-space hole filling uses small point splats followed by 2D filtering or interpolation. 3D Gaussian Splatting extends that lineage to fully volumetric Gaussians optimized as a radiance field; each splat carries density and color, and compositing mimics volume rendering (Franke et al., 2024).

The same lineage motivates the photon analogy. One recent formulation states that photon mapping stores discrete photons and that photon splatting projects them into screen space or onto surfaces with Gaussian kernels to estimate radiance, whereas 3DGS and related methods store “radiance carriers” and splat them into the image, integrating contributions along the view direction (Franke et al., 2024). This suggests that “photon splatting” is best understood not as a single fixed algorithm but as a broader kernel-accumulation paradigm.

A parallel systems lineage emphasizes explicit rasterization. A compute-shader point renderer encodes depth and color into a 64-bit integer, uses atomicMin for visibility, and implements a three-pass high-quality splatting scheme that averages fragments whose linear depth is at most $1\%$ larger than the closest depth in a pixel (Schütz et al., 2019). Although that work does not discuss photon mapping explicitly, it provides a screen-space splat renderer whose depth logic and accumulation structure are directly reusable in photon-style pipelines.

Formulation	Primitive	Accumulated quantity
Compute-shader splatting (Schütz et al., 2019)	1-pixel points	Front-most sample or depth-window average
3DGS-style splatting (Franke et al., 2024, Meyer et al., 3 Jun 2025)	3D Gaussians	Alpha-composited color or feature fields
Ray-traced Gaussian splatting (Byrski et al., 31 Jan 2025)	Confidence ellipsoids	Per-ray opacity and color
PhotonSplat (Teja et al., 26 Jun 2025)	3D Gaussians	SPAD photon-arrival parameter and hit probability
Wireless Photon Splatting (Cao et al., 7 Jul 2025)	Surface-attached virtual sources	CIR over angular and delay domains
Gaussian Wave Splatting (Choi et al., 10 May 2025)	3D Gaussians	Complex hologram wavefield

2. Core mathematical structure

A large fraction of the literature shares the same front-to-back accumulation rule. In 3DGS-style rendering, if splats are sorted along the camera direction, accumulated color $C$ and transmittance $T$ evolve as

$$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$

An equivalent formulation used in multi-spectral Gaussian splatting is

$$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$

with $N$ the Gaussians contributing to a pixel (Franke et al., 2024, Meyer et al., 3 Jun 2025).

This accumulation is usually preceded by projection. In standard 3DGS, a 3D Gaussian with mean $\mu$ and covariance $\Sigma$ is projected to a 2D elliptical Gaussian in image space; conceptually, the 3D mean maps to a 2D center, and the covariance is transformed by the projection Jacobian to a 2D covariance $\Sigma_{2D}$ that defines the footprint (Franke et al., 2024). MS-Splatting follows the same construction, explicitly noting the approximation

$$\boldsymbol{\Sigma}^{(2d)} \approx J \,\boldsymbol{\Sigma}\, J^{T},$$

with $C$0 the Jacobian of projection at $C$1 (Meyer et al., 3 Jun 2025).

A ray-based variant replaces rasterized footprints by direct ray–primitive interaction. RaySplats represents Gaussians by confidence ellipsoids, intersects them with rays, and computes per-ray opacity from the maximum Gaussian density along the ray rather than from a screen-space footprint (Byrski et al., 31 Jan 2025). The resulting compositing rule remains the same front-to-back alpha accumulation, but the primitive is now evaluated in ray space rather than image space.

Domain-specific formulations modify only the forward observable. In PhotonSplat, the alpha-composited grayscale output of Gaussians is interpreted as a per-pixel photon-arrival parameter,

$C$2

and the SPAD hit probability becomes

$C$3

with $C$4 a binary SPAD readout (Teja et al., 26 Jun 2025). In wireless Photon Splatting, the final observable is a baseband channel impulse response,

$C$5

so the splatted quantity is no longer radiance but multipath structure in delay and angle (Cao et al., 7 Jul 2025).

3. Photon splatting in radiance-field rendering

In radiance-field rendering, the photon-splatting interpretation is explicit in several recent works. VR-Splatting describes a hybrid foveated renderer in which the periphery is represented by a low-detailed Gaussian field and the fovea by neural points plus a CNN. Its core pipeline renders a full-frame Gaussian image $C$6, queries eye tracking, renders neural points only in a foveal sub-frustum, uses Gaussian depth for occlusion culling, reconstructs a high-detail foveal image $C$7, and blends $C$8 and $C$9 with an edge-aware foveated mask. The paper characterizes this as a splat-based “photon budget” reallocation: few Gaussians with large-ish splats in the periphery, and many fine neural splats with neural post-filtering in the fovea (Franke et al., 2024).

The same work makes the historical connection explicit: Gaussians are treated as volumetric world-space splats, neural points as pixel-scale splats plus a CNN, and the overall system is described as similar in spirit to photon splatting in global illumination, while differing in that the radiance field is optimized from multi-view images rather than estimated on the fly from photon maps (Franke et al., 2024). This framing is especially clear in its use of one splat field as a visibility proxy for another: approximate Gaussian depth is composited as

$T$0

and then used to cull neural points behind Gaussian geometry (Franke et al., 2024).

RaySplats generalizes this family by moving Gaussian splatting from rasterization to ray tracing. It replaces projected screen-space Gaussians with exact ray–ellipsoid intersections, computes per-ray opacity from Gaussian response along the ray, and integrates the result inside NVIDIA OptiX. This enables shadows, reflections, transparency, and mesh integration, while retaining front-to-back compositing over Gaussian primitives (Byrski et al., 31 Jan 2025). The paper does not implement photon mapping, but it explicitly argues that the same ray–Gaussian intersection and per-ray attenuation machinery provides the foundation for photon-splatting-like or path-tracing-like light transport on top of Gaussians (Byrski et al., 31 Jan 2025).

MS-Splatting extends the same radiance-splatting template to multi-spectral data. Each Gaussian is a 3DGS primitive, but color is replaced by a learned feature embedding $T$1 decoded by a shallow MLP conditioned on viewing direction: $T$2 The paper interprets each primitive as a shared latent “spectral radiance packet” rendered by the same alpha-blending rule as 3DGS, enabling joint prediction of RGB, red-edge, NIR, and related bands from one Gaussian field (Meyer et al., 3 Jun 2025).

A further extension, RT-Splatting, factorizes each Gaussian’s geometric occupancy $T$3 from its optical opacity $T$4, so that effective volumetric opacity becomes $T$5. It then uses occupancy for first-hit surface aggregation and $T$6 for transmission, yielding a unified surface-volume representation for semi-transparent specular scenes (Shi et al., 18 May 2026). The paper explicitly relates this factorization to separating geometry from scattering or absorption in classical photon mapping or volume rendering (Shi et al., 18 May 2026).

4. PhotonSplat and SPAD-based scene reconstruction

The named method "PhotonSplat: 3D Scene Reconstruction and Colorization from SPAD Sensors" introduces a direct SPAD-domain interpretation of Gaussian splatting (Teja et al., 26 Jun 2025). A SPAD pixel integrates over a short exposure $T$7 and reports a single bit: $T$8 if at least one photon was detected and $T$9 otherwise. If the number of incident photons is Poisson with parameter $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$0, then

$$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$1

PhotonSplat uses a 3DGS scene representation, but replaces per-Gaussian RGB by a grayscale arrival-rate contribution $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$2, so that the composited output

$$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$3

is interpreted as $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$4, the Poisson parameter for SPAD photon arrival at pixel $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$5 (Teja et al., 26 Jun 2025).

Training is driven by a Bernoulli or BCE loss on binary SPAD frames together with a 3D spatial smoothing regularizer. The smoothing term perturbs camera translation, renders nearby SPAD probability maps, and penalizes their deviation from the average over perturbed viewpoints: $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$6 This suppresses scene-inconsistent speckle without temporally averaging away motion (Teja et al., 26 Jun 2025).

The method proceeds in two stages. Stage 1 optimizes geometry and grayscale SPAD response from multi-view binary SPAD frames. Stage 2 adds colorization, either reference-based from a single blurry RGB image or no-reference using a U-former or UNet restoration model followed by DDColor, after which color is lifted back into the Gaussian scene for multi-view-consistent rendering (Teja et al., 26 Jun 2025). The framework is also extended to dynamic scenes by plugging the same photon model into 4D Gaussian Splatting.

PhotonSplat is evaluated on PhotonScenes, a real-world SPAD dataset captured with the PI Imaging SPAD512$$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$7 sensor at $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$8 resolution and approximately $$\begin{aligned} C &= \sum_{i} T_{i-1}\,\alpha_i\,C_i, \ T_i &= T_{i-1}(1 - \alpha_i),\quad T_0 = 1 . \end{aligned}$$9 fps, with 3,000–5,000 frames per scene and a mean of $$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$0 firing pixels per frame. On simulated SPADized RGB datasets it outperforms Deblur-GS and NANeRF in grayscale reconstruction, and on real PhotonScenes it recovers sharper geometry and better multi-view color consistency (Teja et al., 26 Jun 2025).

5. Beyond image formation: wireless channels and holography

"Photon Splatting: A Physics-Guided Neural Surrogate for Real-Time Wireless Channel Prediction" reuses the splatting vocabulary in a different field (Cao et al., 7 Jul 2025). Here, a “photon” is a surface-attached virtual source at location $$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$1 carrying a directional wave signature

$$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$2

where $$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$3 is total path length from transmitter to photon and $$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$4 is a local transfer matrix (Cao et al., 7 Jul 2025). At runtime, channel impulse responses are predicted by splatting these photons onto the angular domain of the receiver using a geodesic rasterizer. The resulting model predicts CIRs for 900 receivers in approximately 30 ms in a single-building experiment, and evaluates 29 transmitter configurations across 1,000 receivers in 0.98 s on Bistro, compared with more than 147 s per transmitter for ray tracing (Cao et al., 7 Jul 2025).

Gaussian Wave Splatting extends the splatting idea to computer-generated holography. Starting from Gaussian scene representations optimized for novel-view synthesis, it derives a closed-form 2D Gaussian-to-hologram transform that supports occlusions and alpha blending, and an efficient Fourier-domain approximation implemented with custom CUDA kernels (Choi et al., 10 May 2025). The accumulated quantity is a holographic wavefield rather than an intensity image, so splatting occurs in complex amplitude space rather than only in RGB radiance. The paper positions this as a bridge between neural rendering pipelines and holographic display technology (Choi et al., 10 May 2025).

AstroSplat is adjacent rather than nominally photon-splatting, but it illustrates the same movement toward physically grounded splat-based rendering. Built on 2D Gaussian Splatting, it replaces appearance-only spherical harmonic intensity with planetary reflectance models such as Lambert, Lommel–Seeliger, and Lunar-Lambert, driven by surface normal, Sun direction, view direction, and per-Gaussian relative albedo (Nolan et al., 12 Mar 2026). This does not trace photons, but it shifts splatting from purely learned appearance toward explicit light–surface interaction.

6. Misconceptions, limitations, and directions of research

A common misconception is that photon splatting is synonymous with 3D Gaussian Splatting. The literature indicates a looser relationship. In some papers the term is an analogy for radiance carriers projected as splats; in others it names a specific forward model for SPAD binary sensing or for wireless multipath prediction (Franke et al., 2024, Teja et al., 26 Jun 2025, Cao et al., 7 Jul 2025). A second misconception is that splatting is necessarily screen-space rasterization. RaySplats shows that the same primitives can be interpreted through exact ray–ellipsoid intersection, while wireless Photon Splatting splats into an angular receiver domain rather than an image plane (Byrski et al., 31 Jan 2025, Cao et al., 7 Jul 2025).

Current limitations are domain-specific. RaySplats does not implement photon mapping or multi-bounce global illumination; it provides a differentiable geometric and rendering layer for such extensions (Byrski et al., 31 Jan 2025). PhotonSplat still depends on COLMAP or GLOMAP for initialization, degrades in extreme low light, and currently focuses on monochrome SPAD sensing, although the paper identifies color SPADs, better sensor models, and joint pose-plus-scene optimization as natural extensions (Teja et al., 26 Jun 2025). Wireless Photon Splatting assumes static scenes, known Tx/Rx positions, and a single carrier frequency in its experiments, with future work aimed at dynamic environments, online geometry updates from vision or LiDAR, and broader communication-sensing settings (Cao et al., 7 Jul 2025).

Several papers point toward a broader research program. MS-Splatting suggests continuous wavelength conditioning by extending the decoder to $$\mathbf{C} = \sum_{i\in N} \mathbf{c}_i\,\alpha_i\,T_i,\quad T_i = \prod_{j=1}^{i-1} (1 - \alpha_j),$$5, which would make spectral photon-style splatting more directly compatible with physically based spectral rendering (Meyer et al., 3 Jun 2025). RT-Splatting identifies occupancy-opacity factorization as a path toward thicker refractive media and multi-bounce transport (Shi et al., 18 May 2026). Gaussian Wave Splatting shows that Gaussian primitives can be moved from radiance accumulation to wave propagation (Choi et al., 10 May 2025). Taken together, these works suggest that photon splatting is evolving from a narrow graphics metaphor into a broader operator family for accumulating learned or physics-guided kernels across image, sensor, spectral, angular, and wave domains.