PolarAnything: Diffusion for Polarimetric Imaging

• PolarAnything is a diffusion-based framework that synthesizes polarimetric images from RGB input by predicting an encoded representation (cos(2Φ), sin(2Φ), P) to overcome angular discontinuities.
• The framework employs a finetuned Stable Diffusion v1.5 encoder/decoder and U-Net architecture to accurately render polarization images at arbitrary analyzer angles using a physical imaging model.
• Empirical evaluations show improved PSNR, SSIM, and downstream task performance in shape from polarization compared to conventional synthesis methods.

PolarAnything is a diffusion-based framework for synthesizing polarimetric images from a single RGB input. It finetunes Stable Diffusion v1.5 to predict an encoded representation of polarization properties, $[\cos(2\Phi),\sin(2\Phi),P]$, where $\Phi$ is the Angle of Linear Polarization and $P$ is the Degree of Linear Polarization, and then renders polarization images at arbitrary analyzer angles through a physical imaging equation (Zhang et al., 23 Jul 2025). A broader interpretation is also suggested by adjacent polarimetry literature: beyond the specific 2025 model, “PolarAnything” can plausibly denote an end-to-end polarimetric environment spanning acquisition, calibration, storage, radiative-transfer modeling, monitoring, and downstream inference across computer vision, laboratory scattering studies, exoplanet atmospheres, and autonomous space instrumentation (Xiao et al., 2017, Rodriguez et al., 2023, Lietzow et al., 2020, Poch et al., 2018).

1. Polarimetric state and image formation

PolarAnything operates in the regime of linear polarization. In the underlying formulation, the polarization state of light is described by the Stokes vector $S = [I,Q,U,V]^\top$, with $I$ the total intensity, $Q$ and $U$ the linear-polarization terms, and $V$ the circular-polarization term. The method focuses on the Degree of Linear Polarization and the Angle of Polarization,

$$\text{DoLP} = P = \frac{\sqrt{Q^2+U^2}}{I}, \qquad \text{AoLP} = \Phi = \frac{1}{2}\arctan2(U,Q),$$

with $\Phi$ defined modulo $\Phi$0. For a linear polarizer at angle $\Phi$1, the transmitted intensity is written as

$\Phi$2

so that once $\Phi$3 and $\Phi$4 are known, polarization images at arbitrary analyzer angles follow from a single forward model rather than independent image synthesis (Zhang et al., 23 Jul 2025).

This representation is aligned with the measurement model used by micro-grid polarization cameras. In division-of-focal-plane sensors, each $\Phi$5 super-pixel contains analyzers at $\Phi$6, from which one reconstructs

$\Phi$7

and then

$\Phi$8

This establishes the practical bridge between raw sensor measurements and the AoLP/DoLP fields used by PolarAnything and related toolchains (Rodriguez et al., 2023).

2. Encoded representation and diffusion architecture

The central design choice in PolarAnything is not to diffuse the four analyzer-angle images directly, and not to regress $\Phi$9 in raw angular form, but to generate the three-channel field

$P$0

The motivation is geometric and statistical. AoLP is $P$1-periodic, so direct regression of $P$2 creates discontinuities at the wrap-around; the pair $P$3 instead embeds AoLP on a continuous unit-circle representation. DoLP is physically bounded in $P$4 and is linearly rescaled to $P$5 for compatibility with the pretrained VAE and Stable Diffusion ranges. The resulting tensor is treated as an image, encoded by a frozen Stable Diffusion v1.5 VAE, and denoised in latent space by a fully finetuned U-Net conditioned on RGB through a dedicated image encoder (Zhang et al., 23 Jul 2025).

The architecture retains the Stable Diffusion text encoder and latent-diffusion machinery, but the dominant conditioning is image-based. The VAE encoder/decoder $P$6 remains frozen; the RGB condition encoder $P$7 consists of a shallow conv+SiLU feature extractor followed by an encoder with the same structure as the U-Net encoder; and the denoising model $P$8 predicts latent noise conditioned on $P$9, the timestep $S = [I,Q,U,V]^\top$0, the text embedding $S = [I,Q,U,V]^\top$1, and hierarchical RGB features. Training uses the standard latent DDPM noise-prediction objective,

$S = [I,Q,U,V]^\top$2

No auxiliary Stokes, Fresnel, or normal-consistency loss is introduced; physical structure is instead carried by the chosen representation, real polarimetric supervision, and the rendering step via the linear-polarizer equation (Zhang et al., 23 Jul 2025).

The training set is built solely from real captured polarization images. The authors report a new dataset of 1,148 images at resolution $S = [I,Q,U,V]^\top$3, covering more than 100 objects across diffuse versus specular, dielectric versus conductive, and transparent versus opaque categories under 19 illumination conditions, with Morimatsu’s data and 1,115 of their own images used for training and 33 reserved for evaluation. Training uses random $S = [I,Q,U,V]^\top$4 crops, AdamW with $S = [I,Q,U,V]^\top$5, $S = [I,Q,U,V]^\top$6, weight decay 0.001, learning rate $S = [I,Q,U,V]^\top$7, batch size 16, 600 finetuning steps, and 8 NVIDIA A100 GPUs for about 10 hours (Zhang et al., 23 Jul 2025).

3. Empirical performance and downstream use

The empirical case for PolarAnything rests on two claims: that the generated outputs are visually plausible as polarization images, and that they remain useful as polarization for downstream inference. The representation ablation is the clearest quantitative result. Diffusing four polarization images directly yields PSNR 23.23, SSIM 0.9165, AoLP MAngE $S = [I,Q,U,V]^\top$8, and DoLP MAbsE 0.1233; diffusing AoLP and DoLP directly yields PSNR 40.57, SSIM 0.9904, AoLP MAngE $S = [I,Q,U,V]^\top$9, and DoLP MAbsE 0.1100; and diffusing encoded AoLP and DoLP yields PSNR 41.74, SSIM 0.9927, AoLP MAngE $I$0, and DoLP MAbsE 0.1075. This establishes the encoded representation not merely as a convenience for angular periodicity, but as the most accurate of the tested targets (Zhang et al., 23 Jul 2025).

The method is also compared qualitatively to Mitsuba 2 on two reconstructed objects, “Elephant” and “Money Jar.” The reported conclusion is that AoLP and DoLP from PolarAnything are visually closer to ground truth than Mitsuba’s for both materials, with Mitsuba limited by imperfect geometry, PBR parameters, and pBRDF assumptions. Generalization is evaluated on held-out scenes, public Pandora and NeRSP data, and scene-level images with multiple objects; the reported finding is that predicted AoLP/DoLP closely match ground truth and generalize well to unseen environments and public datasets despite modest training size. A Restormer baseline trained to predict AoLP/DoLP directly is reported to be less accurate than PolarAnything, which the authors attribute to the stronger pretrained priors of diffusion models under limited-data conditions (Zhang et al., 23 Jul 2025).

Downstream evaluations are more consequential than image-fidelity scores alone. In multi-view shape from polarization with PISR on the NeRSP object “Shisa,” PISR using real polarization images yields normal error $I$1 and Chamfer Distance 0.6765, whereas PISR using polarization images generated by PolarAnything yields normal error $I$2 and Chamfer Distance 0.6564; PolarAnything’s own AoLP error and DoLP MAE on that case are 33.68° and 0.1563. In single-view SfP, the authors construct PolarStanford-ORB by sampling 300 RGB images from Stanford-ORB and synthesizing polarization for them, then retrain DeepSfP. On the out-of-distribution PN test set, original DeepSfP has mean error about $I$3, DeepSfP+MSO has 24.22°, and DeepSfP+PSO has 22.93°, with DeepSfP+PSO also best or near-best on DP+PN combined. The reported implication is that PolarAnything-generated data can be as useful as, and sometimes better than, Mitsuba-rendered data for SfP training, while requiring only single RGB images and no 3D assets (Zhang et al., 23 Jul 2025).

4. From model to ecosystem: acquisition, calibration, and autonomous processing

A broader interpretation of PolarAnything is suggested by complementary systems literature in which polarization is not a single model output but a complete operational stack. On the instrumentation side, the PSI POLAR Data Center for the space-borne hard X-ray Compton polarimeter POLAR manages an automated pipeline from file ingestion to calibrated science products. POLAR itself targets linear polarization measurements in the 50–500 keV range for Gamma-Ray Bursts and Solar Flares, and telemetry may reach up to 50 GB daily, with science data of 10–100 GB/day, average about 50 GB/day, housekeeping around 20 MB/day, and platform data around 60 MB/day updated every 500 ms. The PPDC periodically synchronizes raw files via rsync, verifies MD5 checksums, stores metadata in MySQL, processes platform and science data into ROOT files, and maintains level-0 through level-3 products plus level-2B calibration subsets. Level-0 generation is typically completed within 30 minutes after raw-data arrival, and quick-look plus initial GRB analysis within about 2 hours; the system is reported to have run in a continuous manner for more than two years without human intervention (Xiao et al., 2017).

That operational model is mirrored, at a much smaller scale, in computer-vision tooling. Pola4All surveys polarimetric applications and provides an open-source toolkit for communication with and processing of most existing micro-grid polarization cameras. Implemented in C++ on top of OpenCV, ROS, Qt5, and matplotlib-cpp, it standardizes raw split images, polarized color images, Stokes reconstruction, and $I$4–DoLP–AoLP products. It also implements per-pixel calibration parameters $I$5 to compensate pixel gain, polarizing efficiency, and effective orientation, addressing manufacturing deviations from the ideal analyzer states $I$6. In this sense, the broader “PolarAnything” idea is not limited to synthesis from RGB: it includes the acquisition and calibration layer required to make polarimetric features numerically stable and interoperable across downstream tasks (Rodriguez et al., 2023).

5. Physical modeling across scales

PolarAnything, understood as a polarimetric program rather than only a generative model, sits within a wider family of physically grounded forward and inverse models. In laboratory planetary science, POLICES measures the degree of polarization in visible light scattered by water-ice surfaces at phase angles from $I$7 to $I$8. Its phase curves show that the amplitude and shape of the negative polarization branch vary with particle size and the degree of metamorphism of the ice, and that fresh frost exhibits resonances interpreted as Mie oscillations from transparent micrometer-sized particles with narrow size distributions and spherical shape. The reported comparative interpretation is that Europa is possibly covered by relatively coarser ($I$9–$Q$0m) and more sintered grains than Enceladus and Rhea, which are more likely covered by frost-like particles of few micrometers in average. The broader significance is methodological: polarization is treated as a sensitive probe of microstructure, not merely of intensity (Poch et al., 2018).

In exoplanetary atmospheres, POLARIS extends 3D Monte Carlo radiative transfer to polarized continuum scattering in atmospheres, surfaces, and local planetary environments such as rings. The framework propagates the full Stokes vector, applies Mueller matrices for Rayleigh and Mie scattering, avoids locally plane-parallel approximations, includes extended stellar illumination, and supports inhomogeneous clouds and circumplanetary rings. One illustrative result is that a circumplanetary ring consisting of small water-ice particles can strongly increase reflected flux at larger phase angles when the orbit is seen edge-on, because the particles tend to scatter forwards, while decreasing the degree of polarization at those phase angles. These studies define the physically explicit end of the spectrum to which PolarAnything is adjacent: learned synthesis offers accessibility and scale, whereas radiative-transfer solvers offer explicit control over scattering physics, geometry, and multiple scattering (Lietzow et al., 2020).

6. Limitations, misconceptions, and future directions

The principal misconception to avoid is that PolarAnything provides an exact physical simulator. The paper does not claim this. It explicitly states that no explicit Fresnel equations or material models are embedded, no explicit Stokes constraint such as $Q$1 is enforced, and no explicit normal–AoLP mapping is modeled; rather, the method learns these relationships implicitly from real data while using the forward imaging equation only at the rendering stage. Its physical guarantees are therefore implicit rather than analytical. Additional stated limitations are the modest size of the training corpus, domain and camera dependence, diffusion inference cost relative to single-forward-pass CNNs, and the absence of explicit 3D or multi-view consistency, since each view is synthesized independently from its RGB input (Zhang et al., 23 Jul 2025).

A second misconception is that the broader polarimetric field is already standardized. The opposite is emphasized by Pola4All, which identifies a lack of standards and publicly available tools, limited large-scale datasets, calibration complexity, unresolved physical-modeling gaps for mixed reflection modes and material diversity, and open questions about how best to fuse polarization with RGB, IR, depth, and geometry in learned systems. This suggests that PolarAnything’s importance lies not only in being the first diffusion-based polarimetric image generator, but also in clarifying a research agenda: expand datasets to broader scenes and cameras, incorporate more explicit physics constraints into loss functions or architectures, improve interoperability and calibration across hardware, and apply synthesized or acquired polarization to a wider range of tasks such as reflection control, transparent segmentation, and shape reconstruction (Rodriguez et al., 2023, Zhang et al., 23 Jul 2025).