Volumetric Gaussian Representation Overview

• Volumetric Gaussian representations are explicit models encoding 3D/4D fields as mixtures of anisotropic Gaussian kernels, enabling continuous density and appearance.
• They bridge dense voxel grids and explicit meshes by leveraging analytic line integrals, differentiable rendering, and efficient ray marching suitable for real-time visualization.
• These representations support dynamic scene compression, neural asset generation, and progressive streaming with proven high compression rates and real-time mobile performance.

Volumetric Gaussian representations provide an explicit, mathematically principled, and highly efficient framework for modeling, rendering, and manipulating 3D and 4D (spatiotemporal) data in graphics, vision, and scientific computing. They encode spatial fields as mixtures of anisotropic Gaussian kernels (“splats” or “blobs”), enabling continuous volumetric density, radiance, and appearance that can be efficiently projected, composited, and optimized. This parametric representation bridges the traditional divide between dense voxel grids and explicit surface or mesh models, and forms the foundation of modern differentiable rendering, real-time visualization, neural 3D asset generation, dynamic scene compression, and analysis-by-synthesis pipelines.

1. Mathematical Definition and Properties

A single volumetric Gaussian is defined by a mean $\mu \in \mathbb{R}^3$, a symmetric positive-definite covariance $\Sigma \in \mathbb{R}^{3\times3}$, and an amplitude (opacity or emission) parameter $A$. The spatial density at $x \in \mathbb{R}^3$ is

$$G(x; \mu, \Sigma) = A \exp\Bigl(-\tfrac12 (x - \mu)^\top \Sigma^{-1} (x - \mu)\Bigr).$$

Attribute channels (e.g., color, opacity, spherical harmonics for view dependence) can be attached per-primitive, leading to a scene model

$$\rho(x) = \sum_{i=1}^K G_i(x), \quad c(x) = \frac{\sum_i G_i(x)c_i}{\sum_j G_j(x)},$$

where $c_i$ are per-Gaussian color or appearance vectors. Covariance $\Sigma$ is often parameterized as $\Sigma = R S S^\top R^\top$ for a rotation $R$ and scale $S=\operatorname{diag}(s_1, s_2, s_3)$ (Huang et al., 29 May 2025, Zheng et al., 22 Sep 2025, Gupta et al., 17 Dec 2025).

The representation’s continuous differentiability supports analytic computation—specifically closed-form line integrals for ray marching, analytic gradients for backpropagation, and convenient manipulation of blur, anisotropy, and spatial locality.

2. Rendering Formulations: Splatting, Ray-Integration, and Path Tracing

Rendering a volumetric Gaussian scene can follow rasterization/splatting or physically based ray integration:

• 3D Gaussian Splatting (3DGS): Each Gaussian is projected to a 2D image footprint via an affine camera. The 3D ellipsoid projects to a 2D ellipse with density

$$\tilde{G}_i(p) = \exp\left(-\frac{1}{2}(p - \tilde{\mu}_i)^\top \tilde{\Sigma}_i^{-1} (p - \tilde{\mu}_i)\right),$$

where $\tilde{\mu}_i$ and $\tilde{\Sigma}_i$ are the projected mean and covariance (Matias et al., 20 Oct 2025, Huang et al., 29 May 2025). Depth sorting and alpha blending approximate the emission-absorption volume rendering integral. Analytic closed-form expressions for the Gaussian line integral enable exact, ray-centric rendering, removing dependence on 2D approximations for challenging camera models (Huang et al., 29 May 2025).

• Ray Marching and Path Tracing: For physical media or radiance fields, each ray $r(t) = o + t \omega$ traverses a sum of Gaussians, with cumulative optical depth and analytic transmittance:

$T(t) = \exp\left(-\int_0^t \rho(r(t')) dt'\right).$

Each Gaussian’s integral along the ray reduces to a 1D Gaussian in $t$, yielding error-function expressions for attenuation and sampling (Zhou et al., 2024, Condor et al., 2024, Sharma et al., 14 Sep 2025). For global illumination, scattering, and path tracing, explicit Gaussian mixtures serve as both phase function and extinction source, supporting unbiased Monte Carlo estimates (Zhou et al., 2024, Condor et al., 2024).

• Alpha Compositing: In real-time or differentiable splatting, the front-to-back accumulation formula

$$C(p) = \sum_{i=1}^K c_i \alpha_i \prod_{j < i}(1 - \alpha_j)$$

with pixel-wise compositing weights $w_i(p)$ provides efficient, parallelizable rendering for large Gaussian sets (Zhang et al., 7 Mar 2025, Rai et al., 26 Feb 2026, Gupta et al., 17 Dec 2025).

3. Representation Construction, Optimization, and Learning

Multiple construction and optimization workflows exist for volumetric Gaussian fields:

• Initialization: Gaussians are seeded from Structure-from-Motion (SfM) point clouds, dense back-projections (from NeRF or FBP-based 3D imaging), or uniform grid centers, depending on target sparsity and initial coverage (Li et al., 2023, Shin et al., 10 Jan 2025, Sharma et al., 14 Sep 2025).
• Parameter Learning: Attributes are refined under multiview image supervision by differentiable photometric losses, optionally augmented by regularization terms for spatial compactness, anisotropy, opacity sparsity, or polynomial smoothness (Matias et al., 20 Oct 2025, Shi et al., 8 Jan 2026, Condor et al., 2024).
• Densification and Pruning: Adaptive schemes periodically split Gaussians with large covariance or high residuals, clone in areas of under-sampling, and prune low-opacity or redundant splats to optimize representational capacity (Matias et al., 20 Oct 2025, Shin et al., 10 Jan 2025, Shi et al., 8 Jan 2026).
• Hybrid Representations: Some pipelines combine Gaussians with triangulated surfaces, either for efficiency (skin via mesh, hair via Gaussians in avatars) or for semi-transparent layer compositing (Gupta et al., 17 Dec 2025).
• Dynamic Scenes and 4DGS: For temporally evolving content, models support per-primitive rigid or nonrigid motion, appearance warping, and direct 4D parameter fields. Key techniques include control graph deformation, motion field MLPs, and compressed UV packing for codec compatibility (Zhang et al., 7 Mar 2025, Zheng et al., 22 Sep 2025, Jiang et al., 9 Sep 2025, Rai et al., 26 Feb 2026).
• Structured and Hierarchical Encodings: Representations include regular grid arrangements (GaussianVolume, (He et al., 2024)), locality-aware neural fields exploiting attribute coherence (Shin et al., 10 Jan 2025), and progressive hierarchical layers for rate-distortion-optimized streaming (Shi et al., 8 Jan 2026, Zheng et al., 22 Sep 2025).

4. Applications in Graphics, Vision, and Scientific Domains

Volumetric Gaussian representations are foundational in a broad range of domains:

Application	Approach/Key Features	Paper Examples
Novel View Synthesis & NVS	Differentiable 3DGS, alpha splatting, adaptive SH for view-dependence	(Matias et al., 20 Oct 2025), [3DGEER]
Dynamic Volumetric Video	4D Gaussians, spatio-temporal keyframing, motion fields, progressive compression	(Zhang et al., 7 Mar 2025, Zheng et al., 22 Sep 2025, Rai et al., 26 Feb 2026)
Text-/Image-to-3D Asset Gen.	Coarse-to-fine diffusion/U-Nets, GaussianVolume generation, attribute prediction	(He et al., 2024)
Scientific Visualization & CT	Sparse sum-of-Gaussians for density fields, analytic ray-integrals, data compression	(Li et al., 2023, Sharma et al., 14 Sep 2025)
Surface Reconstruction & Mesh Extraction	GSDF–Gaussian signed distance fields, hybrid mesh-Gaussian models	(Matias et al., 20 Oct 2025, Gupta et al., 17 Dec 2025)
Avatar Modeling and Animation	Mesh+Gaussian hybrid, SH coefficients for hair/skin, differentiable layering	(Gupta et al., 17 Dec 2025)
Multimodal Rendering & Phys. Media	Path tracing with analytic Gaussian interactions, emission & scattering, global illumination	(Zhou et al., 2024, Condor et al., 2024)

Their differentiability, analytic support, and explicit structure make them suitable for feed-forward learning pipelines, differentiable inverse problems, and hardware-accelerated rasterization.

5. Compression, Progressive Streaming, and Scalability

A central advantage of Gaussian splatting is compact, progressive, and hierarchical coding:

• Progressive Hierarchies: Layered or cluster-parametric decomposition splits Gaussians into “Sketch” (high-frequency, boundary) and “Patch” (low-frequency, smooth) groups for progressive streaming. Transmission order is prioritized by perceptual significance, opacity, or coverage (Shi et al., 8 Jan 2026, Zheng et al., 22 Sep 2025).
• Codec/NV Stream Compatibility: Gaussian attributes can be mapped to 2D UV atlases, packed as multichannel images, and compressed with standard codecs (e.g., FFV1, H.264), supporting real-time streaming and random access without domain-specific decoders (Rai et al., 26 Feb 2026, Jiang et al., 9 Sep 2025).
• Temporal Compression: Differential coding of per-frame displacements, quantization of parameter deltas, and entropy modeling yield $>40\times$ compression rates for dynamic scenes (Zhang et al., 7 Mar 2025, Zheng et al., 22 Sep 2025, Rai et al., 26 Feb 2026).
• Attribute-Aware Compression: Neural field sharing and adaptive SH bandwidth further reduce redundancy by exploiting spatial and frequency coherence (Shin et al., 10 Jan 2025), while multi-resolution keyframing and UV packing address temporal/contextual redundancy (Rai et al., 26 Feb 2026).
• Empirical Rate-Distortion Results: Compression rates up to $175\times$ (with $<0.3$ dB PSNR loss) and real-time decoding and rendering at 60–300 FPS on mobile hardware are routinely reported (Shi et al., 8 Jan 2026, Zhang et al., 7 Mar 2025, Zheng et al., 22 Sep 2025).

6. Extensions: Directional, Structured, and Unified Representations

Recent research generalizes volumetric Gaussians to capture advanced physical or structural effects:

• Direction-Aware (6DGS): 6D Gaussian Splatting represents color and opacity as joint functions of position $x \in \mathbb{R}^3$ and viewing direction $d \in \mathbb{R}^3$ via $6 \times 6$ covariance matrices. This construction supports conditional slicing for angularly sharp features and yields dramatic reductions in splat count for specular, refractive, and view-dependent effects (Gao et al., 2024).
• Unified Primitive for Surface-Volume Coupling: Surface-like and volumetric elements are both modeled as Gaussians with varying anisotropy and weight, enabling seamless modeling of glossy surfaces, fuzzy volumes, and physically accurate scattering from the same primitive type (Zhou et al., 2024, Condor et al., 2024).
• Hybrid Representations: Several pipelines (e.g., GPiCA) unify mesh and Gaussian primitives in a single differentiable rendering system, allocating each primitive type by region (e.g., skin as mesh, hair as Gaussians), compositing semi-transparent layers (Gupta et al., 17 Dec 2025).
• Codec- and UV-aligned Structuring: Gaussian parameters are rearranged into image-space grid (UV) atlases or feature images, supporting direct compatibility with video codecs and optimized for dense storage and streaming (Rai et al., 26 Feb 2026, Jiang et al., 9 Sep 2025).
• Scientific and Multiresolution Data: Conversion pipelines process OpenVDB's spatial tree structure or AMR grids into hierarchical Gaussian sets for SciVis, achieving sparse compression and analytic rendering (Sharma et al., 14 Sep 2025).

7. Limitations, Practical Trade-offs, and Future Directions

Despite significant advances, several challenges persist:

• Memory and Attribute Overhead: Dense scenes require up to $10^5$–$10^6$ Gaussians, leading to gigabyte-scale models. Compression, pruning, and hierarchical representations address but do not eliminate this scaling (Matias et al., 20 Oct 2025, Shi et al., 8 Jan 2026).
• Fidelity Limitations: For extremely thin surfaces, high-frequency texture, or highly specular effects, current parameterizations may underrepresent sharp boundaries or require very small, high-count Gaussians (Gao et al., 2024, Shi et al., 8 Jan 2026).
• Angular/Directional Trade-offs: Direction-aware Gaussians raise per-primitive storage by $50$–$100\%$, and very sharp angular phenomena may still tax the representation (Gao et al., 2024).
• Codec Artifacts and UV Packing: UV-packed atlas approaches must manage layer ordering, quantization, and sparsity fill; post-hoc structuring may lead to lingering temporal artifacts if not accompanied by direct UV-space fitting (Rai et al., 26 Feb 2026).
• Dynamic and Topological Complexity: Highly dynamic or topologically varying scenes require nonrigid appearance warping, spatio-temporal tracking, and dynamic Gaussian activation/deactivation, increasing optimization and decoding complexity (Jiang et al., 9 Sep 2025, Zhang et al., 7 Mar 2025).

Anticipated research directions include hybrid LoD hierarchies, learned initialization and amortized optimization, scalable 4D streaming, and tighter synchronized coupling to neural scene representations.

---

References

• "From Volume Rendering to 3D Gaussian Splatting: Theory and Applications" (Matias et al., 20 Oct 2025)
• "Sketch&Patch++: Efficient Structure-Aware 3D Gaussian Representation" (Shi et al., 8 Jan 2026)
• "3DGEER: Exact and Efficient Volumetric Rendering with 3D Gaussians" (Huang et al., 29 May 2025)
• "4DGCPro: Efficient Hierarchical 4D Gaussian Compression for Progressive Volumetric Video Streaming" (Zheng et al., 22 Sep 2025)
• "EvolvingGS: High-Fidelity Streamable Volumetric Video via Evolving 3D Gaussian Representation" (Zhang et al., 7 Mar 2025)
• "PackUV: Packed Gaussian UV Maps for 4D Volumetric Video" (Rai et al., 26 Feb 2026)
• "Locality-aware Gaussian Compression for Fast and High-quality Rendering" (Shin et al., 10 Jan 2025)
• "Unified Gaussian Primitives for Scene Representation and Rendering" (Zhou et al., 2024)
• "Don’t Splat your Gaussians: Volumetric Ray-Traced Primitives for Modeling and Rendering Scattering and Emissive Media" (Condor et al., 2024)
• "6DGS: Enhanced Direction-Aware Gaussian Splatting for Volumetric Rendering" (Gao et al., 2024)
• "GVGEN: Text-to-3D Generation with Volumetric Representation" (He et al., 2024)
• "PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion" (Zhang et al., 30 Sep 2025)
• "3D Gaussian Modeling and Ray Marching of OpenVDB datasets for Scientific Visualization" (Sharma et al., 14 Sep 2025)
• "Topology-Aware Optimization of Gaussian Primitives for Human-Centric Volumetric Videos" (Jiang et al., 9 Sep 2025)
• "Gaussian Pixel Codec Avatars: A Hybrid Representation for Efficient Rendering" (Gupta et al., 17 Dec 2025)
• "VoGE: A Differentiable Volume Renderer using Gaussian Ellipsoids for Analysis-by-Synthesis" (Wang et al., 2022)
• "3DGR-CT: Sparse-View CT Reconstruction with a 3D Gaussian Representation" (Li et al., 2023)