---
title: 3D Gaussian Splatting Representations
url: https://www.emergentmind.com/topics/3d-gaussian-splatting-representations
type: topic
---

# 3D Gaussian Splatting Representations

3D Gaussian Splatting representations define an explicit, continuous parametric framework for high-fidelity 3D scene modeling and real-time photorealistic rendering. In this paradigm, a scene is embodied as a collection of anisotropic 3D Gaussians—each parameterized by spatial location, orientation, color, and opacity—whose screen-space projections are alpha-composited using analytic splatting kernels. The approach has achieved widespread adoption due to its efficiency, scalability, flexibility, and ability to support novel extensions such as semantic labeling, view-dependent effects, dynamic content, as well as advanced compression and streaming protocols [2407.17418, 2510.26694, 2501.00342].

## 1. Parametric Structure of 3D Gaussian Splat

Each individual splat is defined as an oriented 3D Gaussian parameterized by four principal attributes: spatial mean μ ∈ ℝ³, covariance matrix Σ ∈ ℝ³×³ (symmetric, positive-definite), RGB color c ∈ ℝ³ (optionally per-view via spherical harmonics), and opacity α ∈ [0,1]. The explicit mathematical form is:
\[
G(x) = \exp\left(-\frac{1}{2}(x-\mu)^\top \Sigma^{-1}(x-\mu)\right)
\]
Spatial covariance is often factorized Σ = R S Sᵗ Rᵗ (R ∈ SO(3) as a quaternion, S diagonal) to enforce positive-definiteness and efficient storage. In typical models, each splat’s per-pixel image contribution is computed by projecting to a 2D Gaussian in the image plane (mean μ′, covariance Σ′), forming the screen-space weight
\[
G_i(p) = \exp\left[-\frac{1}{2}(u_p-\mu_i')^\top (\Sigma_i')^{-1} (u_p-\mu_i')\right]
\]
where u_p is the pixel location [2409.01761, 2501.01003, 2510.26694].

## 2. Memory Layout and Scene Organization

A 3DGS scene is the union of N splats, typically organized for compute efficiency as either Array-of-Structs (AoS) or Structure-of-Arrays (SoA). For N splats, memory layouts are:
- AoS: `struct Splat {float μ[3]; float Σ[6]; float c[3]; float α; } splats[N];`
- SoA: separate arrays for μ_x[N], μ_y[N], … Σ_xx[N], Σ_xy[N], … c_r[N], … α[N]

A vanilla splat without spherical harmonics has a footprint of 52 bytes (13 floats), growing to 100–200 bytes if augmented with 9–27 SH coefficients for view-dependent color [2409.01761, 2501.00342]. This explicit storage scale, often millions of splats per scene, motivates downstream research into efficient attribute compression, low-rank factorizations, and hierarchical or octree-based encodings [2512.00850, 2508.04965].

## 3. Rendering Model and Compositing Mechanism

Rendering in 3DGS employs analytic splatting and alpha compositing. Each projected Gaussian is rasterized to pixel p as a normalized kernel, and all overlapping splats are sorted by depth. The composite color at pixel p is
\[
C(p) = \sum_{k=1}^M T_{k}(p)\,\alpha_{i_k}\,c_{i_k}
\]
with transmittance pre-factor
\[
T_{k}(p) = \prod_{m<k} (1 - \alpha_{i_m})
\]
where the index sequence {i_1,...,i_M} is sorted by increasing depth. Each splat’s weight at a pixel thus incorporates both its own opacity and the cumulative attenuation from farther splats [2409.01761, 2407.17418, 2510.26694]. Extensions to the compositing model allow for view-dependent opacity [2501.17978], photometric losses with structural similarity, and advanced rasterization (EWA filtering, foveated passes) [2510.26694, 2407.17418].

## 4. Contribution Scoring, Prioritization, and Progressive Streaming

A key innovation for scalable rendering and networked applications is per-splat contribution scoring. The global importance sᵢ of each splat i is calculated by summing its weighted contribution across all representative pixels and training views:
\[
s_i = \sum_{v} \sum_{p} T_i^v(p) \cdot \alpha_i
\]
Global scene ordering O is then argsorted on sᵢ (descending). Such ranked ordering supports progressive chunked transmission and rendering: the receiver first renders the most perceptually important subset to produce an immediate coarse scene, then iteratively refines the image by loading subsequent splat chunks, trading off bandwidth, time, and perceptual error [2409.01761]. Optional spatial refinements (octree-leaf top-sᵢ selection, frustum culling) enhance warm-start quality for initial paint or view changes.

## 5. Integration with Compression and Efficient Storage

The progressive protocol is fully compatible with existing 3DGS compression schemes (e.g., vector-quantized codebooks, run-length encoding, octree occupancy, subvector quantization) [2512.00850, 2503.16924, 2503.23162]. The workflow is: compute the global importance ordering O, partition O into independently compressible chunks, feed each chunk to the chosen compressor, and ensure that decompression of each chunk is self-contained. This architecture enables adaptive streaming, zero-overhead partial rendering, and bandwidth-efficient large-scene access [2409.01761, 2508.04965]. Empirical results demonstrate significant reductions in streaming latency and memory with negligible loss in metrics such as PSNR, SSIM, or LPIPS.

## 6. Pseudocode Summary of Key Stages

Below are canonical algorithms for scoring, ordering, and progressive rendering [2409.01761]:

**Algorithm 1: Estimate Contribution Scores sᵢ**
```
For i = 1 to N: set sᵢ ← 0
For each training view v in V:
    Render view v with full splat set
    For each pixel p in view v:
        Compute top-K overlapping splats {i₁…i_K} and weights w_{i_k}(p)
        For each k = 1…K: s_{i_k} += w_{i_k}(p)
Return s₁…s_N
```

**Algorithm 2: Build Global Ordering O**
```
Let O₀ = argsort_i(sᵢ) descending
(Optional) Subdivide into octree leaves; in each leaf sort and append splats by sᵢ
(Optional) Push out-of-frustum splats to tail
Return O
```

**Algorithm 3: Progressive Render/Stream Loop**
```
Inputs: sorted O = [o₁…o_N], chunk size B
Initialize GPU buffer S ← ∅, k ← 0
While k·B < N:
    k ← k+1
    to_send ← O[ (k–1)B+1 … kB ]
    Fetch/decompress to_send
    Upload to GPU buffer S
    Render current view from S → display
    If user view changes → re-prioritize (goto Algorithm 2)
```

This staged structure exploits the analytic compositing efficiency of 3DGS to allow monotonic improvement in visual quality as more splats are loaded, directly supporting adaptive streaming scenarios and just-in-time rendering on memory- or bandwidth-constrained clients.

## 7. Applications, Limitations, and Future Directions

The 3D Gaussian Splatting representation, especially in its prioritized and progressively streamable form, establishes a rigorous foundation for high-fidelity real-time scene rendering across heterogeneous devices and over constrained networks [2409.01761]. It also forms the substrate for further research in semantic segmentation, scene graph construction, and live dynamic content delivery [2510.26694]. Ongoing challenges include further reducing the per-splat footprint, extending scoring mechanisms to dynamic or semantic criteria, and hybridizing with other explicit or neural field frameworks for generalized scene understanding and manipulation.

---
**References**:  
[2409.01761] PRoGS: Progressive Rendering of Gaussian Splats  
[2510.26694] The Impact and Outlook of 3D Gaussian Splatting  
[2407.17418] 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities  
[2512.00850] Smol-GS: Compact Representations for Abstract 3D Gaussian Splatting  
[2508.04965] Perceive-Sample-Compress: Towards Real-Time 3D Gaussian Splatting  
[2503.16924] Optimized Minimal 3D Gaussian Splatting  
[2503.23162] NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representations

Source: https://www.emergentmind.com/topics/3d-gaussian-splatting-representations