Polarization-Enhanced Gaussian Densification

• The paper introduces a novel integration of polarimetric cues with 3D Gaussian Splatting, significantly enhancing scene geometry by reducing photometric ambiguities.
• It leverages a multi-view PatchMatch depth completion with a composite energy function to fuse polarization and geometric information for robust normal and depth estimation.
• Empirical evaluations demonstrate marked improvements in Chamfer Distance and image quality metrics, validating the effectiveness in both indoor and outdoor challenging environments.

Polarization-Enhanced Gaussian Densification refers to the incorporation of polarimetric imaging cues—specifically, the Degree of Linear Polarization (DoLP) and Angle of Linear Polarization (AoLP)—into the process of 3D Gaussian Splatting (3DGS) for improved scene reconstruction. The fusion of polarization priors with 3DGS addresses the limitations posed by photometric ambiguities in regions with specular reflection or low texture, resulting in enhanced surface completeness, normal accuracy, and separation of diffuse and specular components in complex real-world environments (Guo et al., 30 Nov 2025, Shan et al., 2 Dec 2025).

1. Polarization Fundamentals and Measurement Model

Polarimetric cameras acquire intensity images at four analyzer angles: $I_0$, $I_{45}$, $I_{90}$, and $I_{135}$. The corresponding Stokes parameters are: $$s_0 = I_0 + I_{90},\quad s_1 = I_0 - I_{90},\quad s_2 = I_{45} - I_{135}.$$ From these, the DoLP ($\rho$) and AoLP ($\phi$) are computed as: $$\rho = \frac{\sqrt{s_1^2 + s_2^2}}{s_0}, \qquad \phi = \tfrac12\,\arctan\!\left(\frac{s_2}{s_1}\right).$$ Here, $\rho \in [0,1]$ quantifies the proportion of polarized light; $\phi$ is azimuthal, but ambiguous under $\pi$ and $\pi/2$ rotations. In surface normal estimation, DoLP encodes the tilt (with ambiguity), while AoLP encodes the projected normal’s orientation. The physical optics relating polarization to surface orientation constrain the geometry even under photometric degeneracy.

2. 3D Gaussian Splatting and Baseline Limitations

3DGS represents a scene as a set $\{G_i\}$ of anisotropic 3D Gaussians, each parameterized by a mean $\mu_i \in \mathbb{R}^3$, covariance $\Sigma_i$, opacity $\alpha_i$, and a low-order spherical harmonic (SH) reflectance vector $c_i$. Rendering proceeds by projecting each Gaussian as an ellipse, compositing opacity and color contributions per pixel. Real-time rasterization is achieved via hardware blending and efficient SH evaluation. However, conventional 3DGS struggles where photometric cues are unreliable (e.g., mirror-like surfaces, textureless regions), due to a lack of directional information and misestimation of shape from color-only inputs (Guo et al., 30 Nov 2025, Shan et al., 2 Dec 2025).

3. Polarization-Guided PatchMatch Depth Completion

To resolve the degeneracies inherent in traditional photometric stereo, polarization-enhanced densification integrates polarimetric cues within a multi-view PatchMatch framework:

• Hypothesis set: For each image pixel $p$, hypotheses combine depths and surface normals from both current 3DGS geometry and polarimetric estimation:

$$\mathcal H(p) = \big\{(d,\,n),\;(d^{\mathrm{prt}},\,n),\;(d,\,n_i^{\mathrm{aolp}}),\;(d^{\mathrm{prt}},\,n_i^{\mathrm{aolp}}) \big\}_{i=1}^4,$$

where $n_i^{\mathrm{aolp}}$ are unit normals constructed from AoLP-cued azimuths and the zenith inferred from 3DGS.

• Cost function: For each hypothesis $(d^*, n^*)$, the energy is

$$E(d^*, n^*) = S_{\rm phot}(d^*, n^*) + \lambda_1 S_a(n^*) + \lambda_2 S_{\rm nd}(d^*, n^*),$$

where $S_{\rm phot}$ is bilateral normalized cross-correlation, $S_a$ enforces consistency between candidate normals and AoLP, weighted by DoLP, and $S_{\rm nd}$ penalizes misalignment between normal and depth gradient.

• Optimization loop: PatchMatch propagation, random search, and multi-view aggregation are conducted via this composite energy. Candidates are filtered by both geometric and polarimetric consistency across views, yielding dense depth and normal maps in previously ambiguous regions.

4. Gaussian Back-Projection and Proximity-Based Fusion

Surviving pixels after PatchMatch filtering are back-projected into 3D with: $X_p = K^{-1}[u_p, v_p, 1]^\top d_p^{\mathrm{opt}},$ where $K$ is the camera intrinsic matrix. New 3D Gaussians $G_p$ are constructed at each $X_p$ with:

• Center $\mu_p = X_p$.
• Orientation $R_p$ aligning the shortest axis with $n_p^{\mathrm{opt}}$.
• Covariance $$\Sigma_p = R_p\,\mathrm{diag}(s, s, \epsilon)\,R_p^\top$$, with $\epsilon \ll s$.
• Color from the RGB image and opacity from the learned density prior.

Fusion integrates $G_p$ into the existing 3DGS set by proximity and normal alignment; overlapping Gaussians are merged via weighted averaging, while distinct Gaussians are inserted directly. This enables iterative densification of the 3DGS scene, with newly inferred structure in challenging areas (Guo et al., 30 Nov 2025).

5. Bidirectional Coupling with Advanced Polarization Priors

Systems such as PolarGuide-GSDR establish a bidirectional loop:

• Geometry disambiguates polarization: An initial pure 3DGS-DR pass estimates coarse normals, which are then used to resolve the $\pi$ and $\pi/2$ azimuthal ambiguity of raw polarization normals.
• Polarization supervises 3DGS updates: Disambiguated polarization normals and separated specular/diffuse components (using Fresnel models and SH parameterization) provide supervision for the next stage of Gaussian normal and reflectance optimization.
• Deferred reflection modeling: Gaussian representations encode both diffuse and specular returns, with spherical harmonic coefficients absorbing both lighting and BRDF response. Polarization priors constrain the fitting to maintain plausible separation and orientation (Shan et al., 2 Dec 2025).

This coupling leads to robust normal estimation and explicit separation of physical image components, driving high-quality novel view synthesis and geometric fidelity under complex material and lighting conditions.

6. Empirical Impact and Quantitative Evaluation

Polarization-enhanced densification yields significant improvements in geometric completeness and accuracy, particularly in reflective and textureless regions:

• On NeISF (Teapot), PolarGS densification alone reduces Chamfer Distance (CD) from 3.75 cm to 2.13 cm; the full PolarGS pipeline achieves CD of 1.65 cm.
• As an augmentation, PolarGS improves GOF’s average CD from 3.42 cm to 2.48 cm and PGSR’s from 2.93 cm to 1.97 cm, demonstrating framework-agnostic gains.
• In challenging indoor and outdoor benchmarks, PolarGuide-GSDR consistently improves PSNR, SSIM, and LPIPS over standard 3DGS-DR. Specular-diffuse separation is physically faithful, and normal maps are demonstrably smoother and better aligned with reference polarization normals (Guo et al., 30 Nov 2025, Shan et al., 2 Dec 2025).

Ablation reveals that both polarization-enhanced densification and photometric corrections are necessary for full performance: omission of either degrades CD by up to 50%.

7. Limitations and Future Research

Polarization-enhanced densification’s performance depends on polarization signal strength and model fidelity:

• In near-Lambertian regions where DoLP is extremely low ($\rho \approx 0.01$), azimuthal cues are weak.
• Highly metallic or multi-bounce polarization phenomena may violate the simple geometric polarization model; current approaches do not handle circular polarization (Stokes $s_3$).
• Extensions to volumetric polarization modeling, learnable BRDF priors, and support for transparent or translucent materials are prospective research avenues.

Real-time rendering is preserved (40–100 FPS) despite polarimetric supervision, maintaining practical applicability (Shan et al., 2 Dec 2025). A plausible implication is that broader adoption of polarization-aware acquisition hardware could further generalize the effectiveness of these approaches for uncontrolled real-world scenes.