3D Gaussian Splatting Representations

Updated 22 June 2026

3D Gaussian Splatting is a continuous parametric framework that models scenes via oriented 3D Gaussians with spatial, color, and opacity parameters.
It organizes millions of anisotropic Gaussians with optimized memory layouts and contribution scoring for progressive, real-time rendering.
The approach integrates advanced compression, view-dependent effects, and dynamic extensions to enable high-fidelity, adaptive 3D visualization.

3D Gaussian Splatting representations define an explicit, continuous parametric framework for high-fidelity 3D scene modeling and real-time photorealistic rendering. In this paradigm, a scene is embodied as a collection of anisotropic 3D Gaussians—each parameterized by spatial location, orientation, color, and opacity—whose screen-space projections are alpha-composited using analytic splatting kernels. The approach has achieved widespread adoption due to its efficiency, scalability, flexibility, and ability to support novel extensions such as semantic labeling, view-dependent effects, dynamic content, as well as advanced compression and streaming protocols (Bao et al., 2024, Kerbl, 30 Oct 2025, Wang et al., 2024).

1. Parametric Structure of 3D Gaussian Splat

Each individual splat is defined as an oriented 3D Gaussian parameterized by four principal attributes: spatial mean μ ∈ ℝ³, covariance matrix Σ ∈ ℝ³×³ (symmetric, positive-definite), RGB color c ∈ ℝ³ (optionally per-view via spherical harmonics), and opacity α ∈ [0,1]. The explicit mathematical form is: $G(x) = \exp\left(-\frac{1}{2}(x-\mu)^\top \Sigma^{-1}(x-\mu)\right)$ Spatial covariance is often factorized Σ = R S Sᵗ Rᵗ (R ∈ SO(3) as a quaternion, S diagonal) to enforce positive-definiteness and efficient storage. In typical models, each splat’s per-pixel image contribution is computed by projecting to a 2D Gaussian in the image plane (mean μ′, covariance Σ′), forming the screen-space weight

$G_i(p) = \exp\left[-\frac{1}{2}(u_p-\mu_i')^\top (\Sigma_i')^{-1} (u_p-\mu_i')\right]$

where u_p is the pixel location (Zoomers et al., 2024, Gao et al., 2 Jan 2025, Kerbl, 30 Oct 2025).

2. Memory Layout and Scene Organization

A 3DGS scene is the union of N splats, typically organized for compute efficiency as either Array-of-Structs (AoS) or Structure-of-Arrays (SoA). For N splats, memory layouts are:

AoS: struct Splat {float μ[3]; float Σ[6]; float c[3]; float α; } splats[N];
SoA: separate arrays for μ_x[N], μ_y[N], … Σ_xx[N], Σ_xy[N], … c_r[N], … α[N]

A vanilla splat without spherical harmonics has a footprint of 52 bytes (13 floats), growing to 100–200 bytes if augmented with 9–27 SH coefficients for view-dependent color (Zoomers et al., 2024, Wang et al., 2024). This explicit storage scale, often millions of splats per scene, motivates downstream research into efficient attribute compression, low-rank factorizations, and hierarchical or octree-based encodings (Wang et al., 30 Nov 2025, Wang et al., 7 Aug 2025).

3. Rendering Model and Compositing Mechanism

Rendering in 3DGS employs analytic splatting and alpha compositing. Each projected Gaussian is rasterized to pixel p as a normalized kernel, and all overlapping splats are sorted by depth. The composite color at pixel p is

$C(p) = \sum_{k=1}^M T_{k}(p)\,\alpha_{i_k}\,c_{i_k}$

with transmittance pre-factor

$T_{k}(p) = \prod_{m<k} (1 - \alpha_{i_m})$

where the index sequence {i_1,...,i_M} is sorted by increasing depth. Each splat’s weight at a pixel thus incorporates both its own opacity and the cumulative attenuation from farther splats (Zoomers et al., 2024, Bao et al., 2024, Kerbl, 30 Oct 2025). Extensions to the compositing model allow for view-dependent opacity (Nowak et al., 29 Jan 2025), photometric losses with structural similarity, and advanced rasterization (EWA filtering, foveated passes) (Kerbl, 30 Oct 2025, Bao et al., 2024).

4. Contribution Scoring, Prioritization, and Progressive Streaming

A key innovation for scalable rendering and networked applications is per-splat contribution scoring. The global importance sᵢ of each splat i is calculated by summing its weighted contribution across all representative pixels and training views: $s_i = \sum_{v} \sum_{p} T_i^v(p) \cdot \alpha_i$ Global scene ordering O is then argsorted on sᵢ (descending). Such ranked ordering supports progressive chunked transmission and rendering: the receiver first renders the most perceptually important subset to produce an immediate coarse scene, then iteratively refines the image by loading subsequent splat chunks, trading off bandwidth, time, and perceptual error (Zoomers et al., 2024). Optional spatial refinements (octree-leaf top-sᵢ selection, frustum culling) enhance warm-start quality for initial paint or view changes.

5. Integration with Compression and Efficient Storage

The progressive protocol is fully compatible with existing 3DGS compression schemes (e.g., vector-quantized codebooks, run-length encoding, octree occupancy, subvector quantization) (Wang et al., 30 Nov 2025, Lee et al., 21 Mar 2025, Tang et al., 29 Mar 2025). The workflow is: compute the global importance ordering O, partition O into independently compressible chunks, feed each chunk to the chosen compressor, and ensure that decompression of each chunk is self-contained. This architecture enables adaptive streaming, zero-overhead partial rendering, and bandwidth-efficient large-scene access (Zoomers et al., 2024, Wang et al., 7 Aug 2025). Empirical results demonstrate significant reductions in streaming latency and memory with negligible loss in metrics such as PSNR, SSIM, or LPIPS.

6. Pseudocode Summary of Key Stages

Below are canonical algorithms for scoring, ordering, and progressive rendering (Zoomers et al., 2024):

Algorithm 1: Estimate Contribution Scores sᵢ

For i = 1 to N: set sᵢ ← 0
For each training view v in V:
    Render view v with full splat set
    For each pixel p in view v:
        Compute top-K overlapping splats {i₁…i_K} and weights w_{i_k}(p)
        For each k = 1…K: s_{i_k} += w_{i_k}(p)
Return s₁…s_N

Algorithm 2: Build Global Ordering O

Let O₀ = argsort_i(sᵢ) descending
(Optional) Subdivide into octree leaves; in each leaf sort and append splats by sᵢ
(Optional) Push out-of-frustum splats to tail
Return O

Algorithm 3: Progressive Render/Stream Loop

Inputs: sorted O = [o₁…o_N], chunk size B
Initialize GPU buffer S ← ∅, k ← 0
While k·B < N:
    k ← k+1
    to_send ← O[ (k–1)B+1 … kB ]
    Fetch/decompress to_send
    Upload to GPU buffer S
    Render current view from S → display
    If user view changes → re-prioritize (goto Algorithm 2)

This staged structure exploits the analytic compositing efficiency of 3DGS to allow monotonic improvement in visual quality as more splats are loaded, directly supporting adaptive streaming scenarios and just-in-time rendering on memory- or bandwidth-constrained clients.

7. Applications, Limitations, and Future Directions

The 3D Gaussian Splatting representation, especially in its prioritized and progressively streamable form, establishes a rigorous foundation for high-fidelity real-time scene rendering across heterogeneous devices and over constrained networks (Zoomers et al., 2024). It also forms the substrate for further research in semantic segmentation, scene graph construction, and live dynamic content delivery (Kerbl, 30 Oct 2025). Ongoing challenges include further reducing the per-splat footprint, extending scoring mechanisms to dynamic or semantic criteria, and hybridizing with other explicit or neural field frameworks for generalized scene understanding and manipulation.

References:

(Zoomers et al., 2024) PRoGS: Progressive Rendering of Gaussian Splats (Kerbl, 30 Oct 2025) The Impact and Outlook of 3D Gaussian Splatting (Bao et al., 2024) 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities (Wang et al., 30 Nov 2025) Smol-GS: Compact Representations for Abstract 3D Gaussian Splatting (Wang et al., 7 Aug 2025) Perceive-Sample-Compress: Towards Real-Time 3D Gaussian Splatting (Lee et al., 21 Mar 2025) Optimized Minimal 3D Gaussian Splatting (Tang et al., 29 Mar 2025) NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representations