Explicit 3D Gaussian Splatting

• Explicit 3D Gaussian Splatting is a scene representation method that uses explicit anisotropic Gaussians to capture position, shape, color, and opacity for real-time rendering.
• It leverages a fully differentiable rendering pipeline that projects Gaussian splats via alpha blending, enabling end-to-end optimization with photometric losses.
• The approach supports advanced applications such as semantic mapping, dynamic scene reconstruction, and precise surface extraction while addressing memory efficiency challenges.

Explicit 3D Gaussian Splatting is a class of scene representation and rendering methods in which a set of explicit, learnable 3D anisotropic Gaussians—each with position, shape, color, and opacity—are jointly optimized to model and render complex geometric and photometric content in real time. Unlike implicit neural fields, which define continuous volumetric radiance through coordinate-MLPs, explicit 3D Gaussian Splatting encodes scene information in a dense set of spatially parameterized Gaussian “primitives.” Rendering is performed by projecting (splatting) these Gaussians onto the image plane and compositing their contributions via alpha blending, yielding efficient, differentiable, and highly editable radiance fields (Wu et al., 2024, Chen et al., 2024, Zhou et al., 12 Aug 2025, Matias et al., 20 Oct 2025, Ji et al., 2024).

1. Mathematical Foundations and Representation

Each explicit 3D Gaussian represents an oriented ellipsoidal region in ℝ³, parameterized by:

• Mean (center): $\mu_i \in \mathbb R^3$
• Covariance: $\Sigma_i \in \mathbb R^{3\times3}$ (typically decomposed as $\Sigma_i = R_i S_i^2 R_i^\top$ with $R_i \in SO(3)$, $S_i = \mathrm{diag}(s_{i,x}, s_{i,y}, s_{i,z})$)
• Opacity (weight): $w_i \in [0,1]$
• Color: $c_i \in [0,1]^3$, or spherical harmonic coefficients for view dependence

The unnormalized 3D spatial density is:

$$G_i(\mathbf x) = w_i\,\exp\Big(-\tfrac{1}{2}(\mathbf x-\mu_i)^\top \Sigma_i^{-1} (\mathbf x-\mu_i)\Big)$$

For rendering, this scene-level collection of Gaussians $\{G_i\}_{i=1}^N$ induces a radiance field by alpha compositing their projected ellipsoidal “splats” in screen space (Zhou et al., 12 Aug 2025, Matias et al., 20 Oct 2025).

2. Differentiable Rendering and Splatting Pipeline

The rendering process comprises the following steps:

• Projection: Each Gaussian $G_i$ is projected into the image plane using the camera intrinsics $K$ and extrinsics $(R_\mathrm{cam}, t_\mathrm{cam})$. The projected center and covariance are:

$$\mu_{i,2D} = K(R_\mathrm{cam} \mu_i + t_\mathrm{cam})\,/\,z,\quad \Sigma_{i,2D} = K R_\mathrm{cam} \Sigma_i R_\mathrm{cam}^\top K^\top$$

• Splatting: The 3D ellipsoid is rendered as a 2D elliptical Gaussian on the image plane, typically using a tile-based rasterizer for parallelization (Chen et al., 2024, Wu et al., 2024).
• Alpha Blending: For a pixel $p$, the rendered color is given by the compositional alpha-blending rule:

$$C(p) = \sum_{i\in\mathcal N(p)} \left[\prod_{j<i} (1-G_j(p))\right] G_i(p) c_i$$

where $\mathcal N(p)$ denotes the ordered (front-to-back) set of Gaussians projecting to pixel $p$, and $T_i = \prod_{j<i}(1 - G_j(p))$ is the transmittance.

This fully differentiable rendering architecture allows end-to-end gradient-based optimization of all Gaussian parameters $[\mu_i, \Sigma_i, w_i, c_i]$ with respect to image-space losses (Matias et al., 20 Oct 2025, Zhou et al., 12 Aug 2025, Ji et al., 2024).

3. Initialization, Optimization, and Density Control

Initialization typically uses sparse Structure-from-Motion (SfM) point clouds or monocular depth maps to seed the Gaussians, each initialized with approximate data-driven positions and small isotropic covariances (Zhu et al., 2024, Wang et al., 1 Jul 2025). Joint optimization is then carried out using photometric losses (e.g., $L_1$, SSIM, or LPIPS between rendered and ground-truth images), along with regularization terms such as:

• Scale regularization: penalizing overly large/small Gaussians
• Anisotropy bounding: limiting extreme elongation
• Surface/normal alignment: encouraging Gaussians to align with mesh normals or surface SDFs (Chen et al., 2023, Wang et al., 1 Jul 2025)

Adaptive density control is central to 3DGS:

• Splitting: Large or highly anisotropic Gaussians are split along principal axes using closed-form, moment-conserving formulas to improve uniformity and surface coverage (Feng et al., 2024).
• Cloning/Densification: Additional Gaussians are introduced where photometric gradients are high, enabling coverage of fine structures (Zhou et al., 12 Aug 2025).
• Pruning: Gaussians with persistently low opacity or inconsequential contribution are periodically culled.

Recent techniques employ gradient-direction-aware criteria (e.g., Gradient Coherence Ratio, GCR) to more precisely distinguish between cases requiring splitting or densification, effectively balancing quality against memory and computational cost (Zhou et al., 12 Aug 2025).

4. Advanced Extensions: Semantics, Dynamics, Compression

Explicit 3DGS forms the basis for numerous advanced scene representations:

• Semantic Mapping: High-dimensional semantic or spatial descriptors can be fused into each Gaussian using deep feature fusion (e.g., Mask2Former, DepthAnything), and compressed via encoder-decoder networks for efficient memory usage. Multi-channel supervision then enables dense, robust 3D semantic mapping as seen in NEDS-SLAM, which demonstrated 90.8% Replica mIoU, Depth L1 = 0.47 cm, competitive with (and often exceeding) NeRF/GS-based SLAM (Ji et al., 2024).
• Dynamic Scenes: Dynamic content is captured by associating time-dependent parameters to each Gaussian (position trajectories, covariance deformation, view-dependent appearance). Optical flow decoupling and motion-aware loss functions allow learning explicit 4D Gaussian splats with robust motion priors and joint pose optimization for scenes with complex, non-rigid dynamics (Zhu et al., 2024).
• Compression and Minimality: Memory and bandwidth losses are mitigated by redundancy-aware pruning, compact attribute coding, and sub-vector quantization, as in the OMG pipeline, which achieves nearly 50% storage savings and 600+ FPS while maintaining visual fidelity (Lee et al., 21 Mar 2025).

5. Surface Reconstruction and Geometry Extraction

Explicit 3D Gaussian Splatting is leveraged for high-fidelity surface reconstruction by guiding neural implicit surface fitting (e.g., signed distance fields) with thin, surface-conforming Gaussian splats. Regularizers encourage thin, planar Gaussians and align the smallest-scale axis with surface normals (Chen et al., 2023). Uniform splitting, scale conditioning, and covariance regularization produce “pancake-like” distributions tightly conforming to surfaces, enabling precise point cloud and mesh extraction, stable manifold estimation, and improved downstream segmentation and editing.

6. Applications, Limitations, and Open Challenges

Explicit 3DGS underpins applications in real-time view synthesis, interactive geometry editing, SLAM, semantic mapping, animation, generative 3D content, and robotics (Wu et al., 2024, Zhu et al., 2024, Bao et al., 2024). Its explicit primitives allow for direct operations such as region-specific editing, object insertion or deletion, dynamic environmental updates, and high-level semantic annotation.

Key limitations and areas of ongoing research include:

• High memory requirements for large-scale scenes (mitigated by adaptive pruning, vector quantization, or hybrid approaches)
• Baked lighting (inability to support real-time relighting without scene re-optimization)
• Limited secondary-ray effects, motivating extensions toward integrated ray tracing and BRDF-aware rendering (Matias et al., 20 Oct 2025)
• Managing Gaussian count, overlap, and redundancy in highly unstructured or sparse data scenarios
• Developing feed-forward or few-shot pipelines for instant scene-level 3DGS prediction

Explicit 3D Gaussian Splatting continues to be an active research frontier due to its scalability, real-time performance, rich geometric and semantic extension, and synergy with deep learning pipelines (Wu et al., 2024, Zhu et al., 2024, Matias et al., 20 Oct 2025, Zhou et al., 12 Aug 2025, Ji et al., 2024).