Polarization Sensor-Aware Data Augmentation

• The paper presents sensor-aware augmentation methods that enforce physical measurement constraints, improving IoU by 18.1% in urban segmentation and reducing MAE in shape-from-polarization tasks.
• The methodology integrates regularized geometric rotations, diffusion-based synthesis, and analyzer-space noise injection to mimic realistic sensor behavior across various polarimetric imaging systems.
• The approach addresses the limitations of RGB-style augmentation by preserving angular periodicity and adhering to forward measurement models like Malus law to ensure valid polarimetric outputs.

Polarization sensor-aware data augmentation is the class of augmentation procedures that preserve the physical and sensor-specific structure of polarimetric measurements rather than treating them as ordinary scalar image channels. In polarization imaging, the measurement process is constrained by analyzer orientation, periodic angular variables, bounded polarization magnitude, and sensor-dependent formation pipelines; consequently, augmentation that ignores these constraints can generate physically impossible samples and degrade learning. The topic has developed along three closely related lines: regularized geometric augmentation for division-of-focal-plane polarimetric segmentation in urban robotics (Blanchon et al., 2020), diffusion-based synthesis of polarimetric fields followed by sensor-specific forward modeling (Zhang et al., 23 Jul 2025), and analyzer-space blur, noise, and quantization for reducing the synthetic-to-real gap in shape-from-polarization (Li et al., 5 Mar 2026).

1. Physical basis of augmentation in polarimetric imaging

Polarization imaging differs from RGB imaging because the recorded signal is not only intensity. The foundational measurements in the cited works are analyzer-resolved observations at orientations $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$, obtained either directly from a division-of-focal-plane microgrid or reconstructed as analyzer images. A typical DoFP 2×2 super-pixel therefore encodes four orientation-specific samples, denoted in the literature as $P_0, P_{45}, P_{90}, P_{135}$ or $I_0, I_{45}, I_{90}, I_{135}$ (Blanchon et al., 2020).

From these measurements, the standard linear-polarization quantities are the degree of linear polarization and the angle of linear polarization: $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$ with representative Stokes conventions including

$$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$

and, in the SfP training pipeline,

$$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$

This suggests that normalization conventions vary across pipelines, while the dependence of $\mathrm{DoLP}$ and $45^\circ$0 on $45^\circ$1 and $45^\circ$2 remains unchanged (Zhang et al., 23 Jul 2025).

The forward measurement relation used across these works is the linear polarizer model: $45^\circ$3 or equivalently,

$45^\circ$4

where $45^\circ$5 is intensity, $45^\circ$6 is DoP or DoLP, and $45^\circ$7 is AoP or AoLP. This measurement equation is the central constraint that augmentation must respect: if transformed samples no longer satisfy it, the resulting data cease to correspond to realizable sensor outputs (Blanchon et al., 2020).

A widely used representation for augmentation is the HSL mapping

$45^\circ$8

in which hue encodes twice the polarization angle, saturation encodes degree of polarization, and luminance encodes intensity. The use of $45^\circ$9 converts the $90^\circ$0-periodic angle into a $90^\circ$1-periodic hue representation, which is convenient for image-space transformation but does not remove the underlying directional constraint (Blanchon et al., 2020).

2. Why RGB-style augmentation is inadequate

The central negative result of the early literature is that naïve RGB-style augmentation corrupts polarimetric structure. Standard augmentation assumes that pixel values are scalar intensities independent of scene and sensor geometry, whereas polarimetric data contain orientation-dependent measurements and angular variables with nontrivial transformation laws (Blanchon et al., 2020).

One failure mode is analyzer-angle coherence. In a DoFP microgrid, the $90^\circ$2, $90^\circ$3, $90^\circ$4, $90^\circ$5 ordering is tied to the physical 2×2 analyzer layout. Arbitrary rotations or flips permute analyzer orientations and make the channels inconsistent with their labels unless the representation is explicitly corrected. A second failure mode is AoP periodicity and directional behavior: geometric rotation changes the reference frame of the polarization axis, and mirroring flips orientation. Rotating or flipping an HSL polarimetric image without remapping the hue channel therefore corrupts the encoded polarization angle. A third failure mode is violation of DoP bounds. Because $90^\circ$6, nonlinear per-channel photometric transforms in raw analyzer space can disturb the linear sinusoidal dependence on analyzer angle and can distort AoP and DoP (Blanchon et al., 2020).

The empirical consequence is not merely theoretical. In the urban segmentation study, “Standard” augmentation that omits polarization compensation often performs worse than no augmentation, whereas the regularized procedure improves learning. In the SfP study, “post-augmentation,” meaning distortion after Stokes and AoLP/DoLP computation, also underperforms augmentation in analyzer space. These results jointly support the view that the augmentation locus matters: perturbations are effective only when they preserve the measurement model and the sensor’s orientation semantics (Li et al., 5 Mar 2026).

A common misconception is that polarimetric augmentation is an incremental variant of RGB augmentation. The cited work does not support that interpretation. Instead, it treats augmentation as a constrained transformation problem in which analyzer geometry, angle periodicity, and signal formation are part of the data definition itself.

3. Regularized geometric augmentation for division-of-focal-plane data

A canonical formulation of polarization sensor-aware augmentation is the regularized rotation-and-symmetry procedure for semantic segmentation in urban robotics (Blanchon et al., 2020). The target problem is segmentation of specular hazards such as water, windows, and cars, whose reflections are strongly linearly polarized. The model is DeepLabV3+ with an Xception backbone, trained on 178 annotated polarimetric frames expanded to 2136 via augmentation; validation uses 50 balanced images, and testing uses an 8049-frame video at 10 Hz captured with a Trioptics PolarCam 4D Technology V mounted on a Robotnik Summit XL under ROS. Training is performed for 150 epochs with learning rate $90^\circ$7, batch size 8, Adam, and an adapted Sørensen–Dice loss (Blanchon et al., 2020).

The augmentation is implemented on the HSL polarimetric representation. For rotation by $90^\circ$8, the image undergoes anti-clockwise geometric rotation $90^\circ$9, while the hue channel is compensated by subtracting $135^\circ$0: $135^\circ$1 After transformation, the hue is wrapped by

$135^\circ$2

The saturation and luminance channels are rotated geometrically but do not receive angle compensation because DoP and intensity are treated as rotationally invariant scalars with respect to the image plane. For reflection symmetry, the geometric flip is accompanied by

$135^\circ$3

followed again by modulo-$135^\circ$4 wrapping (Blanchon et al., 2020).

The paper interprets these operations through the Stokes rotation law: $135^\circ$5 which is equivalent to $135^\circ$6 modulo $135^\circ$7. In this view, the hue compensation in the $135^\circ$8 domain is not an image-processing heuristic but a representation-space implementation of the appropriate polarization-axis transformation (Blanchon et al., 2020).

The augmentation schedule samples random rotations in $135^\circ$9 increments over $P_0, P_{45}, P_{90}, P_{135}$0 and applies symmetry with probability $P_0, P_{45}, P_{90}, P_{135}$1. Ratliff interpolation is performed before polarimetric-field computation, and augmented outputs may be retained in HSL or converted back to $P_0, P_{45}, P_{90}, P_{135}$2 tensors. Practical checks include enforcing $P_0, P_{45}, P_{90}, P_{135}$3, $P_0, P_{45}, P_{90}, P_{135}$4, valid intensity range for $P_0, P_{45}, P_{90}, P_{135}$5, clamping DoP to $P_0, P_{45}, P_{90}, P_{135}$6, and verifying analyzer-angle labeling by reconstructing $P_0, P_{45}, P_{90}, P_{135}$7 for $P_0, P_{45}, P_{90}, P_{135}$8 (Blanchon et al., 2020).

The reported effect is an average IoU gain of $P_0, P_{45}, P_{90}, P_{135}$9 between non-augmented and regularized augmentation training on real-world data. Regularized augmentation consistently outperforms both “None” and “Standard” augmentation, particularly for hazard classes such as cars and water, while buildings remain poorly detected because of limited training data and physical similarity with the background class. The key finding is therefore not only that augmentation helps, but that physically consistent augmentation helps whereas unregularized augmentation can deteriorate segmentation (Blanchon et al., 2020).

4. Generative augmentation and sensor-specific forward models

A later development replaces direct geometric manipulation of measured polarimetric images with synthesis of polarimetric fields from RGB, followed by explicit sensor modeling. “PolarAnything” finetunes Stable Diffusion v1.5 to predict encoded AoLP and DoLP from a single RGB image and then generates polarization images through a physics-based forward model rather than diffusing raw polarization images directly (Zhang et al., 23 Jul 2025).

The representation is a three-channel field

$I_0, I_{45}, I_{90}, I_{135}$0

where $I_0, I_{45}, I_{90}, I_{135}$1 is AoLP and $I_0, I_{45}, I_{90}, I_{135}$2 is DoLP. This encoding respects the $I_0, I_{45}, I_{90}, I_{135}$3-periodicity of polarization angle and avoids angle wrapping artifacts. At inference, the model decodes

$I_0, I_{45}, I_{90}, I_{135}$4

unnormalizes $I_0, I_{45}, I_{90}, I_{135}$5 to $I_0, I_{45}, I_{90}, I_{135}$6, sets $I_0, I_{45}, I_{90}, I_{135}$7, and computes

$I_0, I_{45}, I_{90}, I_{135}$8

Analyzer images then follow from the forward relation $I_0, I_{45}, I_{90}, I_{135}$9 or the equivalent Malus-law form $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$0 (Zhang et al., 23 Jul 2025).

Within this framework, augmentation becomes sensor-aware by construction. For DoFP cameras, the synthetic $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$1 can be placed into a 2×2 micro-polarizer mosaic, corrupted with per-orientation gains, spectral weighting, PSF blur, Poisson shot noise, additive Gaussian read noise, angle misalignment offsets, extinction-ratio variations, and then demosaiced before Stokes reconstruction. For rotating-polarizer systems, one can synthesize arbitrary angle sets $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$2 and model exposure changes or pose drift; for division-of-amplitude systems, the same forward relation is used with per-arm throughput differences (Zhang et al., 23 Jul 2025).

The training data comprise 1,148 real polarization images at $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$3 from more than 100 objects and 19 lighting environments, combined with a small public set for training and 33 images held out for testing. Finetuning uses AdamW with $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$4, $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$5, weight decay $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$6, learning rate $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$7, batch size 16, 600 steps on $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$8A100, and random $$\mathrm{DoLP} = \frac{\sqrt{S_1^2 + S_2^2}}{S_0}, \qquad \mathrm{AoLP} = \tfrac{1}{2}\operatorname{atan2}(S_2, S_1),$$9 crops (Zhang et al., 23 Jul 2025).

The evaluation emphasizes representation choice. Diffusing raw polarization images degrades polarization properties; diffusing AoLP/DoLP improves performance; and encoded AoLP/DoLP is best, with PSNR $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$0, SSIM $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$1, AoLP MAngE $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$2, and DoLP MAbsE $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$3 on the real-world dataset. For multiview SfP on NERSP “Shisa,” generated polarization images yield normal MAngE $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$4 versus $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$5 with real data and Chamfer distance $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$6 versus $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$7; for single-view SfP retraining, augmenting Stanford-ORB with PolarAnything reduces mean angular error on the “PN” dataset from $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$8 to $$S_0 = I_0 + I_{90}, \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135},$$9 (Zhang et al., 23 Jul 2025).

This line of work broadens the meaning of sensor-aware augmentation. Rather than only transforming existing polarimetric measurements, it enables large-scale augmentation of RGB corpora into synthetic but physically constrained polarization data, provided that the final stage reimposes sensor-specific formation and calibration effects.

5. Analyzer-space augmentation for shape-from-polarization

In object-level shape-from-polarization, the dominant issue addressed by polarization sensor-aware augmentation is the synthetic-to-real domain gap. The cited study attributes this gap to unrealistic prior synthetic objects and to the fact that real polarization cameras exhibit shot or read noise, blur, and limited ADC bit depth, all of which strongly affect polarization signals, especially AoLP. The proposed remedy is to augment in analyzer space before polarization signal processing, so that AoLP inherits realistic, spatially structured noise patterns rather than artificial perturbations added after the fact (Li et al., 5 Mar 2026).

The training data are rendered with Mitsuba3 in polarized mode using a physically based polarimetric BRDF from Baek et al. (2018). The new DTC-p dataset contains 1,954 3D-scanned real-world objects with geometry-consistent textures and is used to generate 40K training scenes, 1K validation scenes, and 1K test scenes. Each scene includes 1–10 objects on a ground plane, 827 HDR environment maps for training, 10 for evaluation and test, random environment rotation, and hemisphere-sampled cameras at resolution $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$0 (Li et al., 5 Mar 2026).

The augmentation pipeline is simple and explicitly heuristic. Starting from ideal rendered Stokes maps $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$1, the method first inverts them to analyzer images $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$2. It then applies a shared Gaussian defocus blur with random kernel size, adds zero-mean additive Gaussian noise to each analyzer image, and quantizes to the target sensor ADC bit depth, which is 12-bit for Sony PolarSens/IMX250MYR in the real-data setup: $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$3 Only after these operations are Stokes, DoLP, and AoLP recomputed and provided to the network. The paper explicitly contrasts this “pre-augmentation” with “post-augmentation,” where blur or noise is applied after Stokes and AoLP/DoLP computation (Li et al., 5 Mar 2026).

The results show that augmentation in analyzer space is crucial. Average MAE over three real datasets is $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$4 for the full model with pre-augmentation, $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$5 for post-augmentation, and $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$6 with no augmentation. The full model also reports $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$7 below $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$8 and $$S_0 = \tfrac{1}{2}(I_0 + I_{45} + I_{90} + I_{135}), \qquad S_1 = I_0 - I_{90}, \qquad S_2 = I_{45} - I_{135}.$$9 below $\mathrm{DoLP}$0, compared with $\mathrm{DoLP}$1 and $\mathrm{DoLP}$2 without augmentation. In broader ablations, the RGB-only variant has average MAE $\mathrm{DoLP}$3, and the full model outperforms the RGB-only VFM MoGe-2, which reports $\mathrm{DoLP}$4 average MAE, despite using far less data and a smaller model (Li et al., 5 Mar 2026).

The paper is also explicit about what the augmentation does not model. It does not separately parameterize shot noise, read noise, dark current, analyzer angle misalignment, finite extinction ratio, diattenuation or retardance, per-channel crosstalk, DoFP mosaic or demosaicing artifacts, lens vignetting, rolling shutter, chromatic dispersion, exposure, gamma, white balance, or temperature drift. This limitation is important because it establishes the present method as an efficient approximation rather than a complete camera simulator (Li et al., 5 Mar 2026).

6. Comparative structure, recurring design principles, and open directions

Across the three representative formulations, the augmentation target shifts from measured polarimetric images, to synthesized polarimetric fields, to analyzer-space perturbation before Stokes reconstruction. The common principle is nonetheless stable: polarimetric augmentation is valid only if it respects the transformation law of polarization and the sensor’s acquisition model.

Approach	Augmentation locus	Reported effect
Regularized geometric augmentation (Blanchon et al., 2020)	HSL polarimetric representation with $\mathrm{DoLP}$5	Average IoU gain of $\mathrm{DoLP}$6 over non-augmented training
PolarAnything-based augmentation (Zhang et al., 23 Jul 2025)	Encoded $\mathrm{DoLP}$7 plus forward sensor model	Best representation reaches PSNR $\mathrm{DoLP}$8, SSIM $\mathrm{DoLP}$9, AoLP MAngE $45^\circ$00, DoLP MAbsE $45^\circ$01
SfP sensor-aware augmentation (Li et al., 5 Mar 2026)	Analyzer space before Stokes and AoLP/DoLP computation	Pre-augmentation MAE $45^\circ$02 versus $45^\circ$03 post-augmentation and $45^\circ$04 no augmentation

Several recurring design rules emerge. First, periodic angular variables should be represented in a form that avoids wrapping artifacts, such as $45^\circ$05 or $45^\circ$06. Second, augmentation should be applied where the measurement model remains meaningful: geometric transforms require angle compensation, and sensor noise is most effective when injected before Stokes reconstruction rather than after derived-field computation. Third, the forward model must remain enforceable, whether through Malus-law reconstruction, Stokes-domain consistency, or explicit analyzer re-packing into the DoFP mosaic.

Several misconceptions are corrected by the literature. It is not sufficient to rotate or flip polarimetric images as if they were RGB. It is not generally beneficial to perturb AoLP or DoLP directly, because derived quantities inherit sensor artifacts in structured, not arbitrary, ways. It is also not enough to generate visually plausible polarization maps if the final samples are not passed through a sensor-specific forward model. These are not philosophical distinctions; they are directly tied to the observed gaps between “Standard” and “Regularized” augmentation in segmentation, between raw-image diffusion and encoded polarimetric diffusion in synthesis, and between pre- and post-augmentation in SfP.

The open directions named in the cited works are likewise consistent. One line is richer sensor modeling, including distortion and noise addition, learned calibration modules, and direct learning from raw $45^\circ$07 or Stokes channels instead of HSL mappings (Blanchon et al., 2020). A second is broader physical scope, especially circular polarization $45^\circ$08 or DoCP, spectral-band-aware color polarization, and stronger physics-aware losses such as Stokes positivity and Malus consistency during generation (Zhang et al., 23 Jul 2025). A third is broader scene and material coverage, including transparent and conductive materials, scene-level geometry, and more detailed acquisition artifacts beyond blur, Gaussian noise, and 12-bit quantization (Li et al., 5 Mar 2026).

Taken together, these works define polarization sensor-aware data augmentation as a physically constrained interface between augmentation, rendering, and measurement. The unifying objective is not merely data diversity but preservation of analyzer-angle semantics, angular periodicity, bounded polarization magnitude, and realistic sensor corruption, so that augmented samples remain valid instances of the modality on which downstream learning depends.