Gaussian Set Surface Reconstruction (GSSR)

• Gaussian Set Surface Reconstruction (GSSR) is a technique that optimizes individual Gaussian primitives to accurately trace true 3D surfaces with well-aligned normals and consistent distribution.
• It employs dual-level consistency losses, matching pixel-level depth gradients and per-Gaussian normals, to enhance geometric precision and surface coherence.
• GSSR integrates opacity regularization and periodic reinitialization to prune redundant elements, resulting in a compact, editable, and computationally efficient 3D representation.

Gaussian Set Surface Reconstruction (GSSR) refers to a methodology for reconstructing 3D scene surfaces by actively optimizing a set of Gaussian primitives so that they densely and accurately adhere to the underlying geometric surface, with well-aligned normals and uniform distribution. The recent work “Gaussian Set Surface Reconstruction through Per-Gaussian Optimization” (Huang et al., 25 Jul 2025) presents a comprehensive approach that addresses shortcomings in earlier Gaussian Splatting techniques, focusing on precise geometry extraction, surface coherence, and editability at the level of individual Gaussian primitives.

1. Fundamental Framework and Motivation

Traditional 3D Gaussian Splatting (3DGS) methods offer photorealistic novel view synthesis via a dense set of ellipsoidal Gaussians, but have significant limitations in geometric surface reconstruction due to non-uniform, redundant, and poorly-aligned Gaussian placement. Even modern variants like PGSR incorporate depth/normal penalties at the rendered map level but fail to enforce per-Gaussian geometric fidelity, often resulting in irregular spatial distributions and deviation from the true surface.

GSSR draws on principles from Point Set Surfaces, arguing that high-quality reconstruction requires each Gaussian primitive to be individually attracted to the latent surface. The approach jointly optimizes location, scale, and orientation (especially the minimum-scale direction, meant to model the local normal) for each Gaussian, enforcing geometrically meaningful, “flattened” splats that trace out the target geometry rather than merely accumulate color information for rendering.

2. Per-Gaussian and Pixel-Level Consistency Losses

Two tiers of geometric consistency are deployed:

• Pixel-Level Normal Consistency: At each rendered pixel, the surface normal computed from rendered depth gradients ($n_\mathrm{depth}(p)$) is matched to the alpha-blended normal derived from all contributing splats ($n_\mathrm{rendered}(p)$). The loss

$$\mathcal{L}_\mathrm{normal} = \frac{1}{|\Omega|} \sum_{p\in\Omega} \| n_\mathrm{depth}(p) - n_\mathrm{rendered}(p) \|_1$$

enforces locally accurate surface orientation.

• Gaussian-Level Normal Consistency: Each Gaussian’s dominant axis (minimum scaling direction, i.e., its “normal”) is encouraged to align with the normal extracted at its projected 2D location. The instance-level loss

$$\mathcal{L}_{\mathrm{normal}-G} = \frac{1}{N} \sum_{i=1}^N \psi(i) \| n_\mathrm{depth}(p_i) - n_i \|_1$$

uses a bilateral weighting $\psi(i) = \alpha_i \cdot r_i$ (opacity and projected radius) to stabilize the penalty.

Multi-view photometric consistency (see below) is incorporated at both pixel and Gaussian levels to ensure that both local detail and global alignment are robust to varying viewpoints.

3. Multi-View Photometric Consistency and Surface Alignment

To refine the distribution and alignment of Gaussians, GSSR introduces strong geometric supervision based on multi-view consistency:

• For a reference pixel $p_r$ and a neighbor view, corresponding points $p_n$ are computed via homography:

$$p_n = H_{rn} p_r \ H_{rn} = K \left( R_{rn} - \frac{T_{rn} n_r^\top}{d_r} \right) K_r^{-1}$$

where $K$ and $K_r$ are camera intrinsics, $R_{rn}$ and $T_{rn}$ relative pose, $n_r$ the surface normal, $d_r$ the depth.

• Local image patches are compared via the Normalized Cross-Correlation (NCC) between views, penalizing photometric disagreement. The patchwise loss is adaptively weighted by forward–backward reprojection error to avoid artifacts at occlusion boundaries or misaligned correspondences.
• This multi-view supervision is imposed both for rendered surface maps (pixel domain) and for splat centroids (primitive domain), robustly pushing the Gaussian set to align with the true multi-view-consistent geometry.

4. Opacity Regularization and Redundancy Pruning

To address redundancy and misallocated splats, GSSR employs an entropy-based opacity regularization:

$$\mathcal{L}_\mathrm{opacity} = \frac{1}{N} \sum_{i=1}^N [\log(\alpha_i) + \log(1 - \alpha_i)]$$

driving each opacity $\alpha_i$ toward 0 (transparent, removable) or 1 (opaque, effective for surface formation).

This mechanism prunes Gaussians whose contributions are negligible, leading to a more compact and surface-faithful representation, with reduced computational waste and improved mesh quality.

5. Periodic Depth- and Normal-Guided Reinitialization

To ensure even surface coverage and to eliminate persistent under-represented regions, GSSR implements a periodic reinitialization based on accumulated opacity and normal cues:

1. A transmittance-based weight map highlights under-sampled regions (where surface coverage is weak).
2. New 3D candidate splat positions are sampled according to this distribution.
3. Depth and normal maps (optionally spatially filtered for stability) are used to initialize the geometry and orientation of new/replaced Gaussians.
4. This process is performed every fixed number of iterations, ensuring ongoing adaptation of the set as optimization proceeds.

This strategy ensures the Gaussians remain uniformly distributed and well-aligned, preventing holes and clustering that typically arise from naive update mechanisms.

6. Experimental Results and Geometric Quality Assessment

Extensive quantitative and qualitative validation on DTU and Tanks and Temples datasets demonstrates the geometric improvements of GSSR:

• Geometric precision (Gaussian centroid accuracy) is improved as shown by lower Chamfer Distance and higher completeness compared to baselines such as 2DGS and PGSR.
• Surface regularity and compactness are enhanced, as shown in density visualizations and centroid error plots.
• Rendering quality for novel view synthesis is preserved, with strong results on PSNR, SSIM, and LPIPS metrics (Mip-NeRF360 dataset).
• Ablation studies confirm the importance of per-Gaussian normal loss, opacity regularization, and the reinitialization scheme.
• GSSR representations facilitate operations such as interactive scene editing and local object manipulation, which benefit from explicit, regularized Gaussian alignment and density.

7. Applications and Broader Implications

The explicit, optimally-aligned Gaussian set produced by GSSR has implications beyond static reconstruction:

• Scene Editing and Animation: Gaussians, being correctly and evenly placed, can be intuitively edited (translated, deleted, reoriented) for object removal, local deformation, or texturing tasks, unlike earlier models which merge/split irregularly or cluster unpredictably.
• Procedural Scene Generation: The uniform placement over the surface enables efficient sampling and meshing for downstream applications such as XR/VR scene authoring, digital twin construction, and mesh-based learning tasks.
• Efficiency: Pruning and uniform re-sampling reduce the overall primitive count while raising the surface coverage, leading to lower compute/memory requirements.

The GSSR approach establishes a new standard for geometry-first optimization of Gaussian splat sets, making them practical for high-fidelity, edit-friendly 3D reconstructions that retain the rendering advantages of the underlying representation.

---

Key Formulas

Aspect	Formula	Description
Pixel-level normal loss	$$\mathcal{L}_\mathrm{normal} = \frac{1}{|\Omega|} \sum_{p\in\Omega} \| n_\mathrm{depth}(p) - n_\mathrm{rendered}(p) \|_1$$	Aligns depth and splat normals
Gaussian-level normal loss	$$\mathcal{L}_{\mathrm{normal}-G} = \frac{1}{N} \sum_{i=1}^N \psi(i) \| n_\mathrm{depth}(p_i) - n_i \|_1$$	Gaussian instance alignment
Opacity regularization	$$\mathcal{L}_\mathrm{opacity} = \frac{1}{N} \sum_{i=1}^N [\log(\alpha_i) + \log(1 - \alpha_i)]$$	Prunes redundant splats
Multi-view photometric loss	$p_n = H_{rn} p_r$ where $$H_{rn} = K [R_{rn} - (T_{rn} n_r^\top)/d_r ] K_r^{-1}$$	Multi-view NCC constraint
Rendering equation	$C(p) = \sum_{i=1}^N T_i \alpha_i c_i$, $$T_i = \prod_{j=1}^{i-1} (1 - \alpha_j \cdot G_j^{2D}(p))$$	Volumetric alpha-blending for rendering

---

GSSR positions itself as a per-Gaussian-optimized, high-fidelity reconstruction method that bridges the accuracy of classical geometric modeling with the efficiency and rendering prowess of neural point-based representations (Huang et al., 25 Jul 2025).