Polarization-Guided Photometric Correction

• Polarization-guided photometric correction is a computational imaging approach that exploits polarization cues to separate diffuse and specular reflections.
• It integrates physical models, deep network architectures, and optimization frameworks to enhance HDR reconstruction and 3D scene modeling under challenging lighting conditions.
• Empirical evaluations demonstrate improved metrics such as PSNR, SSIM, and reduced Chamfer distance, validating its effectiveness in correcting view-dependent artifacts.

Polarization-guided photometric correction is a class of computational imaging and computer vision techniques that leverages the optical physics of light polarization to enable robust separation of diffuse and specular components, correct view-dependent photometric artifacts, and ultimately facilitate tasks such as high-dynamic-range (HDR) reconstruction, specular reflection separation, and accurate scene modeling in challenging reflective or overexposed environments. These methods exploit the polarization-dependent response of natural and reflected light captured via polarization-sensitive cameras, integrating physical models and polarization-derived priors into deep networks or optimization frameworks to achieve improved radiance recovery and scene interpretation.

1. Physical Foundations and Polarization Measurement

At the core of polarization-guided photometric correction lies the use of polarimetric cues extracted from multi-orientation polarized images. A division-of-focal-plane polarization camera employs on-chip linear micro-polarizers, typically at angles $\{0^\circ, 45^\circ, 90^\circ, 135^\circ\}$, capturing four spatially registered intensity images per frame.

The first three Stokes parameters are derived as

$$S_0 = \frac12 \big(I_{0^\circ} + I_{45^\circ} + I_{90^\circ} + I_{135^\circ}\big), \quad S_1 = I_{0^\circ} - I_{90^\circ}, \quad S_2 = I_{45^\circ} - I_{135^\circ}.$$

From these, the degree of linear polarization (DoLP) and angle of linear polarization (AoLP) are

$$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \quad \phi = \frac12 \mathrm{atan2}(S_2, S_1).$$

Physically, specular reflections yield high DoLP, while diffuse reflections are nearly unpolarized. These polarization cues are fundamental for identifying reflectance ambiguities and irregular exposure.

The irradiance reaching each polarizer is expressed as: $$I_{\theta} = 0.5\,I\, [1 \pm \rho \cos(2\theta)], \quad \text{or} \quad 0.5\,I\, [1 \pm \rho \sin(2\theta)],$$ mapping onto effective exposures and enabling per-pixel photometric discrimination (Ting et al., 2022, Shan et al., 2 Dec 2025, Guo et al., 30 Nov 2025).

2. Mathematical Models for Photometric Correction

Several frameworks utilize these measurements for robust photometric decomposition:

• Specular Reflection Separation: By modeling the observed intensity $I(\varphi_{pol})$ at each pixel $p$ as

$$I_d(p) + I_{sc}(p) + I_{sv}(p)\,\cos2(\varphi_{pol} - \alpha(p)) = I_c(p) + I_{sv}(p)\,\cos2(\varphi_{pol} - \alpha(p)),$$

each channel’s polarization response forms a linear system, allowing least-squares estimation of diffuse $I_d$, unpolarized specular $I_{sc}$, and polarized specular $I_{sv}$ (Wen et al., 2021).

• Polarization Chromaticity Image: An illumination-invariant chromatic descriptor $I_{chro}$ is constructed as

$$I_{chro}(p) = \frac{\tilde I_{rawD}(p)}{s(p)}, \quad s(p) = \sum_{c} I_{rawD}^c(p) + \bar I_{\min},$$

where $I_{rawD}$ is the initial estimate of diffuse color. $I_{chro}$ facilitates clustering for robust cluster-wise separation of diffuse and specular components (Wen et al., 2021).

• Color Refinement Maps (CRMs): In complex 3D modeling, DoLP is used to define per-pixel reflective masks, leading to spatially adaptive diffuse color estimates even in highly specular or saturated regions. Two CRMs—a diffuse intensity map $I_{diff}(x)$ and a chromaticity map $I_{chro}(x)$—are constructed and propagated based on DoLP and local reference intensities (Guo et al., 30 Nov 2025).

3. Deep Network Architectures Leveraging Polarization

Modern HDR reconstruction pipelines, such as Deep Polarimetric HDR Reconstruction (DPHR), integrate polarimetric cues directly into deep convolutional networks:

• Input Representation: Four aligned LDR images, optional DoLP/AoLP maps, and exposureness metrics are concatenated as a 512×512×12 tensor.
• Feature Masking: At each U-Net layer $l$, branch-specific soft masks $M_{l,i}$, derived from DoLP and exposureness $K_i$, gate features to suppress contributions from overexposed or low-polarization regions,

$$X_{l+1,i} = X_{l,i} \odot M_{l,i}, \qquad M_{l+1, i} = (|W_{l, i}|/\|W_{l, i}\|_1 + \epsilon) * M_{l, i},$$

ensuring propagation of valid signals only (Ting et al., 2022).

• Fusion Strategy: Outputs from model-based HDR fusion and learned HDR are blended using a per-pixel confidence weight $\alpha$ computed via DoLP and feature mask statistics.

Losses combine masked $\ell_1$ differences, perceptual losses (VGG feature and style), and HDR-specific metrics.

4. Optimization Approaches and Clustering Based Methods

Certain approaches formulate photometric correction as a cluster-wise optimization task constrained by polarization priors:

• Global Objective: Recovery of diffuse $R_D$ and specular $R_S$ components solves

$$\min_{R_D, R_S} \|I - R_D - R_S\|_F^2 + \|R_D - f(D)\|_F^2 + \lambda \|R_S\|_1,$$

where $f(D)$ enforces closeness to cluster-wise low-rank estimates of the diffuse component (Wen et al., 2021).

• ADMM Solution: Variable splitting and dual update steps iterate between least-squares updates for $R_D$, soft-thresholding for $R_S$, and re-computation of the cluster prior $f(D)$. This framework constraints $R_D$ to remain close to physically meaningful diffuse clusters, robustly separating true albedo from specular outliers.
• Cluster Formation: Chromaticity-based $k$-medoids grouping is used to cluster pixels with similar intrinsic color, each of which undergoes robust PCA (nuclear norm + $\ell_1$ sparse separation) for further refinement.

5. Polarization-Guided Correction in 3D Gaussian Splatting

Integration of polarization cues into 3D Gaussian Splatting (3DGS) extends correction to scene reconstruction and view synthesis:

• Normal Estimation: Surface normals $\mathbf{n}_{\mathrm{pol}}$ inferred from DoLP/AoLP are used to resolve ambiguities ($\pi$ and $\pi/2$ flips) with the help of geometric priors from Gaussian distributions, optimizing

$$E_{\mathrm{normal}} = \frac1N \sum_{i=1}^N 1_{\{{\mathrm{DoLP}}_i>\tau\}}\min_{c\in \mathcal{C}_i}[1-\langle \mathbf{n}_{\mathrm{pred}}(i),\,c\rangle].$$

• Reflectance Separation: Polarization components allow direct estimation of specular and diffuse images for each view,

$$I_{sp}(\phi_{pol}) = \cdots, \quad I_{dp}(\phi_{pol}) = \cdots,$$

which then serve as targets for the SH coefficients of the 3DGS representation (Shan et al., 2 Dec 2025).

• Reflective-Aware Loss: Reflective and overexposed regions are identified via DoLP and intensity masks, inciting a corrective loss that penalizes deviations from the CRM-predicted diffuse values,

$$L_{\mathrm{ref}} = L_s(C,\,I_{diff}) + L_o(C,\,I_{chro}).$$

This technique enforces photometric consistency and suppresses artifacts in the recovered model (Guo et al., 30 Nov 2025, Shan et al., 2 Dec 2025).

• Bidirectional Coupling: 3DGS geometry informs the resolution of ambiguities in polarization-derived normals, while polarization cues are used to supervise updates to 3DGS parameters, leading to superior specular/diffuse separation and improved geometric fidelity (Shan et al., 2 Dec 2025).

6. Implementation Considerations and Evaluation

Robust implementation of polarization-guided pipelines involves careful preprocessing and regularization:

• Data Preparation: RAW images are demosaicked, linearized, and normalized. Polarimetric parameters are estimated per pixel; exposureness or reference intensity maps are computed as needed for masking or CRM construction.
• Network and Optimization: Data augmentation, tailored learning rate schedules, and mask normalization are critical to stable training. Regularization penalties enforce smoothness and prevent overfitting to outlier features.
• Performance Metrics: Methods are evaluated using PSNR, SSIM, LPIPS, Chamfer distance (3D), FSIM, and HDR-specific metrics such as PU-PSNR, HDR-VDP2, and color angular accuracy. Across benchmarks:
  • DPHR achieves $\sim$30.6 dB PU-PSNR and 0.94 PU-SSIM, outperforming single-shot LDR methods by large margins (Ting et al., 2022).
  • PolarGS reduces Chamfer distance by $\sim$20–22% in challenging reflective scenes (Guo et al., 30 Nov 2025).
  • PolarGuide-GSDR surpasses prior NeRF and 3DGS methods by 0.5–1 dB PSNR and attains real-time rendering rates (43–105 FPS) (Shan et al., 2 Dec 2025).
  • Reflection separation accuracy is improved even under near-duplicate illuminations (e.g., PSNR $\approx$32.1 dB, CA $\approx$24.9°, hue SD $\approx$0.034) (Wen et al., 2021).

7. Summary Table: Key Polarization-Guided Photometric Correction Strategies

Method	Core Principle	Key Innovation
DPHR (Ting et al., 2022)	DoLP-guided feature masking in HDR	U-Net with exposure/polarization masks
PolarGuide-GSDR (Shan et al., 2 Dec 2025)	Polarization-coupled 3DGS	Bidirectional coupling for normals/reflectance
PolarGS (Guo et al., 30 Nov 2025)	Reflective-aware correction maps	CRM-based loss for specular/overexposed pixels
PGSRS (Wen et al., 2021)	ADMM with polarization clustering	Chromaticity clusters + robust PCA for separation

Each approach harnesses polarization measurements to resolve photometric ambiguities, enforce consistency, and improve robustness to real-world reflectance and illumination challenges. These systems achieve quantitative and qualitative advances across HDR imaging, reflection separation, and geometric 3D reconstruction even in highly challenging conditions where traditional RGB-only photometric models are ineffective.