PolarGuide-GSDR: 3D Polarization Framework

• The paper introduces a framework that integrates polarization-derived priors with 3D Gaussian Splatting, enabling high-fidelity, real-time novel view synthesis and surface reconstruction.
• It employs a four-stage pipeline—polarization preprocessing, 3DGS initialization, ambiguity resolution, and deferred reflection—to accurately recover surface normals and specular reflectance.
• Quantitative evaluations show a 40% reduction in normal estimation error and improved PSNR, demonstrating practical efficiency in reconstructing highly reflective scenes.

PolarGuide-GSDR is a framework that integrates polarization-derived priors with 3D Gaussian Splatting (3DGS) for high-fidelity, real-time novel view synthesis and surface reconstruction on highly reflective, real-world scenes. By establishing a bidirectional coupling between 3DGS geometry and polarization cues, PolarGuide-GSDR enables joint disambiguation and supervision of surface normals and specular reflectance without reliance on environment maps or restrictive material assumptions (Shan et al., 2 Dec 2025).

1. Pipeline Structure and Key Workflow Stages

PolarGuide-GSDR operates in four main stages, beginning with multi-view RGB plus polarization data and culminating in a real-time, high-fidelity 3DGS representation.

Workflow Overview

1. Input: Multi-view images where each pixel contains four polarization channels: $I_{0^\circ}$, $I_{45^\circ}$, $I_{90^\circ}$, $I_{135^\circ}$.
2. Stage 1 — Polarization Preprocessing:
   • Calculate per-pixel Stokes vector $\mathbf S = [S_0, S_1, S_2]^T$, Degree of Linear Polarization (DoLP), and Angle of Polarization (AoP).
   • Decompose the input images into specular ($I_{sp}$) and diffuse ($I_{dp}$) intensity maps using linear-polarizer Mueller-matrix modeling and Fresnel equations.
   • Estimate initial polarization normals $\mathbf n_{pol}(x,y)$.
3. Stage 2 — 3DGS Initialization:
   • Recover coarse camera poses and a sparse point cloud using COLMAP.
   • Initialize $N$ 3D Gaussian primitives, each parameterized by mean $\boldsymbol\mu_i$, covariance $\Sigma_i$, color $\mathbf c_i$, and opacity $\alpha_i$.
   • Perform an initial 10k-iteration optimization using RGB images.
4. Stage 3 — Ambiguity Resolution:
   • For each pixel $p$, cast its ray through the 3DGS scene. Extract the dominant Gaussian and estimate a geometric normal $\mathbf n_{geo}(p)$.
   • Construct candidate normal set $\mathcal{C}$ by rotating/refining $\mathbf n_{pol}$.
   • Disambiguate and select the candidate normal $\hat{\mathbf n}_{pol}(p)$ closest to geometric normal for pixels with DoLP $>\tau$.
5. Stage 4 — Deferred Reflection and Joint Optimization:
   • Extend Gaussians with a scalar specular weight $r_i$ and low-order spherical harmonic (SH) reflectance coefficients.
   • Render separate diffuse ($I_{base}$) and specular ($I_{refl}$) images, combining the results using learned per-pixel blending.
   • Train with combined photometric, specular, diffuse, and normal losses.

During inference, only the trained 3DGS model is required for real-time GPU-based rendering.

2. Polarization Modeling, Priors, and Ambiguity

Polarization cues encode valuable surface normal priors exploitable in photometric scene understanding. The per-pixel Stokes parameters are derived as: $$S_0 = 0.5\,(I_{0^\circ} + I_{45^\circ} + I_{90^\circ} + I_{135^\circ})$$

$$S_1 = I_{0^\circ} - I_{90^\circ},\quad S_2 = I_{45^\circ} - I_{135^\circ}$$

The Degree and Angle of Linear Polarization follow as: $$D = \frac{\sqrt{S_1^2+S_2^2}}{S_0},\qquad \phi = \frac{1}{2} \mathrm{atan2}(S_2, S_1)$$ By inverting the DoLP via the Nayar polarized-diffuse model and Fresnel reflection, the zenith angle $i$ (incidence) and subsequently the refraction angle can be estimated. The resulting (but ambiguous) polarization normal is: $$\mathbf n_{pol} = \begin{bmatrix} \sin i \cos \phi \ \sin i \sin \phi \ \cos i \end{bmatrix}$$ However, ambiguities of $\pi$ and $\pi/2$ in $\phi$ yield multiple possible normal orientations per pixel, motivating geometric disambiguation as described below.

3. 3D Gaussian Splatting Representation

The scene is modeled as a set of $N$ Gaussians $G_i$, each specified by:

• Center $\boldsymbol\mu_i \in \mathbb{R}^3$,
• Covariance $\Sigma_i \in \mathbb{R}^{3 \times 3}$,
• Diffuse color $\mathbf c_i \in \mathbb{R}^3$,
• Opacity $\alpha_i$.

To support view-dependent appearance, each Gaussian is extended with low-order SH coefficients $b_i^{l, m}$, so the outgoing color for view direction $\mathbf v$ is: $$\mathbf c_i(\mathbf v) = \sum_{l=0}^L \sum_{m=-l}^l b_i^{l,m}\, Y_l^m(\mathbf v)$$ Rendering along a ray $\mathbf r(t)$ is performed as: $$I(\mathbf v) = \int_0^\infty T(t)\, \sigma(t)\, \mathbf c(t, \mathbf v)\, dt$$

$T(t)=\exp\left(-\int_0^t \sigma(s) ds \right)$

with the integral discretized by splatting each Gaussian’s projection.

4. Bidirectional Coupling: Geometry and Polarization

PolarGuide-GSDR institutes a feedback loop between polarization cues and the 3DGS geometry:

• Geometry-guided Disambiguation: For each pixel where DoLP $>\tau$, the geometric normal $\mathbf n_{geo}(p)$ extracted from the dominant Gaussian (via covariance principal axis) selects from four candidate normals derived from the polarization model (rotations of $\mathbf n_{pol}$).
• Polarization-supervised Optimization: The selected normal $\hat{\mathbf n}_{pol}(p)$ forms a surface-normal loss:

$$\mathcal L_{pol} = \sum_{p:D(p)>\tau} \|\mathbf n_{pred}(p) - \hat{\mathbf n}_{pol}(p)\|^2$$

biasing the 3DGS model’s normals toward physically consistent solutions.

• Specular Guidance via Decomposition: The decomposed specular intensity ($I_{sp}$) supervises the per-Gaussian specular weights, while SH basis expansion captures view-dependent effects.

This bidirectional mechanism results in explicit, interpretable reflection separation and normal estimation within the fast-optimizing 3DGS structure.

5. Deferred Reflection Module

The deferred reflection module enables the 3DGS backbone to represent complex reflectance phenomena without environment maps:

• Each Gaussian comprises both diffuse and specular (“reflection weight” $r_i$) channels; the latter modulates the SH-based reflectance.
• For each view and pixel, rendering proceeds as:

$$I_{base} = \mathrm{3DGS\_render}(\{\mathbf c_i\}),\qquad I_{refl} = \mathrm{3DGS\_render}(\{r_i \cdot Y(\mathbf v)\})$$

$$I_{final} = (1-r) \cdot I_{base} + r \cdot I_{refl}$$

• The reliance on low-order SH (e.g., $l = 2$) suffices for plausible view-dependent highlights, precluding the need for environment maps.

6. Training, Losses, and Optimization

PolarGuide-GSDR optimization is governed by a set of loss functions, each tailored to the decomposed photometric streams and geometric supervision:

• Photometric loss ($\mathcal L_{rgb}$): weighted sum of $L_1$ distance and D-SSIM between $I_{final}$ and ground truth.
• Specular and Diffuse losses ($\mathcal L_{refl}, \mathcal L_{base}$): same structure, but with isolated specular ($I_{sp}$) or diffuse ($I_{dp}$) targets.
• Normal loss ($\mathcal L_{normal}$): mean cosine distance, using the closest of the four potential normals.
• Total loss: weighted sum of the above.

Training proceeds through:

1. 10k iterations of RGB-only 3DGS pretraining,
2. 30–80k joint iterations leveraging polarization priors,
3. an optional 10k end-to-end refinement.

Convergence is determined by flattening of the photometric and normal losses.

7. Experimental Evaluation and Quantitative Results

PolarGuide-GSDR demonstrates improvements over baseline 3DGS-DR and polarization-NeRF methods in both reconstruction quality and efficiency. Quantitative results on public and self-collected datasets show:

Dataset	Method	PSNR (↑)	SSIM (↑)	LPIPS (↓)	FPS (↑)
Gnome	3DGS-DR	21.13	0.861	0.252	53.7
	PolarGuide-GSDR	22.54	0.890	0.216	43.6
Automotive & Glass	3DGS-DR	18.31	0.763	0.343	118.3
	PolarGuide-GSDR	19.29	0.774	0.339	104.6

• Normals: mean angular error reduced from ~$15^\circ$ (3DGS-DR) to ~$9^\circ$ (PolarGuide-GSDR), representing a ~40% reduction in error.
• Specular PSNR: improvement from ~17 dB (3DGS-DR) to ~19 dB.
• Training Time: PolarGuide-GSDR requires ~1.3 hr/scene compared to ~1 hr for 3DGS-DR and ~6 hr for GNeRP.
• Real-Time Inference: 40–120 FPS on an RTX 4090.
• Qualitative Observation: Crisp, undistorted reflections and smooth normal fields are observed in highly specular scenes (cars, glass, water), versus blurred highlights and noisy normals with vanilla 3DGS-DR.

A plausible implication is that the bidirectional polarization–splatting coupling provides a scalable, interpretable pathway to high-fidelity rendering of reflective scenes with orders-of-magnitude speedup over polarization-NeRF methods.

8. Summary and Context

PolarGuide-GSDR establishes the first paradigm for directly embedding polarization priors into 3D Gaussian Splatting optimization, enabling robust reflection separation and precise surface-normal estimation in real time, without the overhead of environment maps or presupposed material knowledge. It advances the state of the art in specular scene reconstruction, yielding better interpretability and efficiency compared to previous NeRF-based or split-pipeline techniques (Shan et al., 2 Dec 2025).