---
title: Gaussian Splatting with Discretized SDF
url: https://www.emergentmind.com/topics/gaussian-splatting-with-discretized-sdf
type: topic
---

# Gaussian Splatting with Discretized SDF

Gaussian splatting with discretized signed distance fields (SDF) combines high-performance explicit 3D Gaussian representations with grid- or point-sampled SDFs to achieve accurate surface geometry, robust relightable asset generation, and efficient real-time rendering. This paradigm overcomes the limitations of pure 3D Gaussian splatting—such as poor geometry regularization and susceptibility to outlier “floaters”—while also mitigating the high computational cost and smoothing artifacts of classical SDF-based volumetric rendering. By tightly coupling a discretized SDF to the Gaussian primitives, modern methods achieve bidirectional supervision, surface-constrained optimization, and scalable mesh or appearance extraction.

## 1. Discretized SDF Formulations

Discretized SDFs underpin this synergy between explicit and implicit geometry. Multiple parameterization strategies are used:

- **Factorized Grid + MLP (Tensor-style):** SDFs are represented on a high-resolution 3D grid (e.g., $400^3$), where a learnable latent field is factorized along axes and composed with small MLPs for local interpolation [2406.18544]. The grid can be factored as:
  $$
  V(p) = \sum_{k=1}^r \left[ v_k^X(p_x) \circ M_k^{YZ}(p_y,p_z) \oplus v_k^Y(p_y) \circ M_k^{XZ}(p_x,p_z) \oplus v_k^Z(p_z) \circ M_k^{XY}(p_x,p_y) \right]
  $$
  followed by $s(p) = \Theta(V(p), p)$ to yield the SDF at $p$.

- **Trilinear Grid:** Pure voxel grids store $s_g$ at each lattice center; SDF queries are trilinearly interpolated, commonly at resolutions such as $N^3=256^3$ or $400^3$ [2507.15602,2507.15629].

- **Hash Grid + MLP:** For better scalability, SDF fields can be implemented with multi-resolution hash grids and compact MLPs, supporting fine geometry and scalable supervision [2404.00409,2403.16964].

- **Octree Discretization:** To economize memory and provide spatial adaptivity, SDFs are encoded at octree leaf corners with trilinear interpolation for arbitrary queries [2406.18199].

Discretization enables both efficient SDF queries during Gaussian optimization and direct extraction of meshes using Marching Cubes after training converges.

## 2. Bidirectional Coupling: Mutual Supervision and Losses

The tight integration between Gaussians and discretized SDFs relies on supervision flows in both directions:

- **SDF $\to$ GS:**
  - **Depth and Normal Consistency:** Rendered depth and normals from both the SDF and the Gaussian splats are forced to match along camera rays. The loss formulation typically includes symmetrical depth and normal agreements:
    $$
    L_\text{depth} = \mathbb{E}_r\left[ |D_\text{sdf}(r) - \text{stopgrad}(D_\text{gs}(r))| + |D_\text{gs}(r) - \text{stopgrad}(D_\text{sdf}(r))| \right]
    $$
    $$
    L_\text{normal} = \mathbb{E}_r\left[ (1-\langle n_\text{sdf}, \text{stopgrad}(n_\text{gs})\rangle ) + (1-\langle n_\text{gs}, \text{stopgrad}(n_\text{sdf})\rangle ) \right]
    $$
    [2406.18544,2404.00409]

  - **Zero-Level Set Attraction:** Each Gaussian center $\mu_j$ is penalized for deviation from the SDF zero-level set: $L_\text{pos} = \sum_j |s(\mu_j)|$.

  - **Covariance-Normal Alignment:** The Gaussian with smallest variance direction $u_j$ is aligned with the SDF surface normal $\nabla s(\mu_j)$:
    $$
    L_\text{cov} = \sum_j \left[1 - \langle u_j, \text{normalize}(\nabla s(\mu_j))\rangle\right]
    $$
    [2406.18544].

- **GS $\to$ SDF:**
  - **Normal Guidance:** The smoother per-pixel normals $n_\text{gs}$ (from splatted Gaussians) are used to regularize the SDF, e.g. via:
    $$
    L_\text{sdf\_normal} = \mathbb{E}_r [1 - \langle \nabla s(p^*), \text{stopgrad}(n_\text{gs}(r)) \rangle]
    $$
    where $p^*$ is the SDF hit point for ray $r$ [2406.18544].

  - **Joint Photometric Losses:** Both representations contribute to RGB rendering or relighting losses with compositional objectives.

  - **Ray Sampling Localization:** Gaussian depth is used to restrict SDF sampling to near-surface intervals, focusing computational resources and supervision [2403.16964,2510.13381].

This bidirectional supervision ensures that explicit and implicit structures co-adapt, yielding high-fidelity geometry alongside photorealistic rendering.

## 3. SDF-Guided Gaussian Optimization, Densification, and Pruning

Discretized SDF fields are used to regularize and dynamically adapt the Gaussian distribution in several ways:

- **Outlier/Floater Pruning:** Gaussians distant from the SDF zero-level set (e.g., $|s(\mu_j)| > s_\epsilon$ where $s_\epsilon$ is an adaptive threshold) are culled to avoid floating, non-surface primitives [2406.18544,2507.15629,2403.16964].

- **Densification:** Grid cells with $|S_{i,j,k}| < \tau_s$ (i.e., near the surface) and under-represented by Gaussians trigger densification by cloning or splitting existing splats or seeding new Gaussians [2405.19671,2510.13381]. SDF-prioritized densification ensures geometric coverage, particularly in textureless or under-sampled regions.

- **Opacity-from-SDF Mapping:** Bell-shaped or logistic SDF-to-opacity transforms (e.g., $o_i = \frac{4 e^{-\gamma s_i}}{(1+e^{-\gamma s_i})^2}$) link a Gaussian’s opacity to its SDF sample, enforcing layer localization directly in splatting [2507.15629,2404.00409,2406.18199].

- **Covariance Regularization:** Additional terms encourage the Gaussians’ shapes to flatten along the principal surface direction, spreading them as thin sheets for surface adherence [2406.18199,2410.14189].

Combined, these mechanisms concentrate Gaussian splats precisely along the learned SDF surface, improving geometric faithfulness and rendering efficiency.

## 4. Projection-Based and Pulling Constraints for Surface Consistency

To circumvent the challenges of gradient-based Eikonal penalties or SDF norm enforcement in discretized settings, several works introduce projection-based constraints:

- **Projected Center Losses:** Each Gaussian center is projected to the zero-level set via $ \mu_j^{\mathrm{proj}} = \mu_j - s_j n_j$, with $n_j$ a surface normal, and differences in rendered depth or geometry are penalized to enforce surface adherence [2507.15629,2410.14189].

- **Pulling Gaussians to SDF Surface:** At every optimization step, Gaussians are “pulled” along the SDF gradient to reside on $f=0$. The operation 
  $$
  \mu_j' = \mu_j - s_j\,\nabla f(\mu_j) / \|\nabla f(\mu_j)\|
  $$
  ensures alignment between the explicit and implicit geometry during rendering [2410.14189].

- **Eikonal-Like Constraints via Projections:** Instead of computing $\|\nabla f\|$, the projection-based loss ensures first-order consistency between the splatted surface and the implicit SDF interface [2507.15629].

Such strategies enable discretized SDFs to robustly constrain piecewise-sampled geometry representations without requiring full continuous SDF access throughout the volume.

## 5. Training Pipelines and Computational Characteristics

Hybrid GS+SDF models exhibit several jointly optimized phases:

- **Phase 1:** Gaussian-only photometric training, possibly with smoothness or normal regularizations, warms up splats for gross appearance and geometry [2406.18544].

- **Phase 2:** SDF-only stage, often with frozen Gaussians, trains the grid or hash-based SDF with both supervised and mutual geometry losses.

- **Phase 3:** Joint fine-tuning, with bidirectional depth, normal, and photometric agreements; SDF-guided pruning and densification occur alongside surface-projected constraints.

These pipelines are computationally efficient. For example, GS-ROR$^2$ completes training in ≈1.5h on RTX 4090 (memory consumption: 100k Gaussians, $400^3$ SDF grid, small MLP; <4 GB), with rendering at 200+ FPS [2406.18544]. The introduction of a discretized SDF typically increases memory usage by a minor amount compared to pure 3DGS—e.g., GS-ROR$^2$ requires ≈22 GB, whereas discretized SDF baking in [2507.15629] matches 3DGS’s 4 GB with no extra networks.

## 6. Applications, Limitations, and Empirical Performance

These hybrid models have been validated across multiple domains:

- **Relightable Asset Creation:** Combining a discretized SDF and Gaussians enables accurate relighting of reflective or complex surfaces, outperforming standard 3DGS [2406.18544,2507.15629].
- **Surface Reconstruction:** High-fidelity meshes are obtained by marching cubes on the trained SDF grid, achieving state-of-the-art Chamfer distances and preserving sharp features absent in pure splatting [2404.00409,2403.16964,2410.14189].
- **Novel View Synthesis:** Geometry-constrained Gaussians deliver photorealistic rendering at high frame rates and with improved robustness to sparse or ambiguous input [2403.16964,2507.15602].
- **Dynamic Scene and SLAM:** Real-time 3D mapping in dynamic/urban and large-scale indoor environments leverages SDF-prioritized Gaussian selection for scalable, artifact-free reconstructions [2509.11574,2510.13381,2405.19671].

Empirical results demonstrate improved PSNR, SSIM, LPIPS, and Chamfer metrics against baselines. For example, [2507.15629] reports PSNR 24.52 vs. 23.39 and CD 0.0107 vs. 0.0140 compared to leading hybrid methods on reflective Blender models, with half the memory footprint.

## 7. Comparative Table of Representative Approaches

| Method           | SDF Discretization      | GS-SDF Coupling Mechanism                   | Key Applications                  |
|------------------|------------------------|---------------------------------------------|-----------------------------------|
| GS-ROR$^2$ [2406.18544]      | Grid+MLP (TensorSDF or trilinear) | Bidirectional depth/normal, zero-set pull, pruning | Reflective object relighting, mesh extraction |
| SurfaceSplat [2507.15602]   | Voxel grid + MLP (coarse/fine)    | Cyclic bootstrapping, image-based SDF refinement | Surface reconstruction from sparse views         |
| 3DGSR [2404.00409]          | Hash grid + MLP                    | SDF-to-opacity, mutual rendering, joint losses   | High-fidelity mesh extraction, view synthesis    |
| GSDF [2403.16964]           | Multi-resolution hash grid + MLP   | Depth localization, SDF-guided densification/pruning, mutual geometry supervision | Fast rendering and mesh extraction              |
| GS-Octree [2406.18199]      | Octree vertex SDF                  | SDF-to-alpha, opacity/scale regularization      | Lighting-robust geometry under specular highlights |
| SplatSDF [2411.15468]       | Hash grid + SDF at anchor points   | Anchor-based embedding fusion at surface hits   | Accelerated SDF-NeRF training, high-fidelity geometry|
| GaussianRoom [2405.19671]   | Grid SDF+MLP (periodically flushed)| SDF-guided densification, pruning, monocular priors | Indoor scene reconstruction                     |
| UGSDF [2510.13381]          | Multi-resolution grid + MLP + hypernet| SDF-guided adaptation, 2D-prior fusion        | Dynamic urban scene rendering                    |

These methods establish that Gaussian splatting with discretized SDF not only advances geometry quality and computational scalability relative to prior pure implicit or pure explicit schemes but also enables relightable asset production and robust scene understanding across complex domains.

Source: https://www.emergentmind.com/topics/gaussian-splatting-with-discretized-sdf