Geometric Parallax Rectification

• Geometric parallax rectification is a family of methods that compensate for spatial mismatches introduced by displaced viewpoints, asynchronous sensors, and non-linear projection geometries.
• Techniques range from homography and affine transformations to layer-wise deformable sampling and cross-modal attention, ensuring aligned coordinate systems for improved inference.
• Applications include RGB-event fusion, remote sensing, underwater imaging, and diffraction tomography, with empirical results showing enhanced metrics such as mIoU and accuracy improvements.

Geometric parallax rectification denotes a family of methods that compensate spatial mismatch introduced when measurements, images, or features are acquired from displaced viewpoints, asynchronous sensors, refractive interfaces, or depth-dependent projection geometries. Across contemporary literature, the object of rectification ranges from pixel grids and stereo scanlines to multimodal feature tensors and tomographic sinograms, but the common objective is to map observations into a coordinate system in which corresponding structures are better aligned and downstream inference is less biased by viewpoint-induced displacement (Peng et al., 10 Jul 2026, Monsalve et al., 23 Sep 2025, Guo et al., 2024, Modregger et al., 2024).

1. Conceptual scope and sources of parallax

In multimodal vision, parallax appears as a discrepancy between dense RGB frames and sparse event streams, or more abstractly as a mismatch between 3D geometry and 2D visual priors. In remote sensing, off-nadir passive microwave measurements assign cloud-originating signals to incorrect surface coordinates. In underwater imaging, refraction at the water–glass or water–air interface bends rays and induces object displacement, scaling, and non-linear warping. In diffraction tomography, signals from different sample depths reach the detector with a lateral offset. These cases differ in sensing physics, yet all instantiate a geometric mismatch between where information is observed and where it should be registered for analysis (Peng et al., 10 Jul 2026, Tuo et al., 18 Sep 2025, Monsalve et al., 23 Sep 2025, Guo et al., 2024, Modregger et al., 2024).

The literature also makes clear that rectification is not restricted to a single representational level. Some methods operate directly on images through homographies, affine rectification, or isometric flattening. Others rectify intermediate features by attention, deformable sampling, or layer-wise cross-modal fusion. This suggests that “geometric parallax rectification” is best understood as a broader alignment principle rather than a single algorithmic template (Tuo et al., 18 Sep 2025, Peng et al., 10 Jul 2026).

2. Learned cross-modal rectification in modern vision systems

The paper "Weaving Light and Time: Unified Harmonic-Geometric Representation Learning for Dense RGB-Event Parsing" introduces Evita, a unified backbone for dense RGB-event parsing in which Geometric Parallax Rectification is embedded directly into every encoder layer (Peng et al., 10 Jul 2026). Its motivation is explicit: RGB-event fusion is impaired by sensor spatial parallax and asynchronous delays, and the gap is compounded by the divide between dense intensity grids and sparse kinematic spikes. Evita addresses this with a cross-modal, offset-predicting operator inspired by deformable convolutions. RGB features are first projected to event-channel dimensionality,

$\mathbf{X}'_r = \psi(\mathbf{X}_r),$

then concatenated with event features and their difference,

$$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$

from which a learnable generator predicts offsets and a modulation mask. Aligned event features are produced by modulated deformable sampling,

$$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$

The effect is to warp event features so that kinematic edges and patterns are aligned to RGB structures despite spatial or temporal desynchronization (Peng et al., 10 Jul 2026).

Evita’s integration strategy is equally important. Geometric Parallax Rectification is re-estimated at every hierarchy level rather than applied once as a preprocessing step. Pretraining exposes the network to both strictly aligned and synthetically-misaligned RGB-event pairs, specifically 60% aligned and 40% misaligned, and the framework can additionally use a geometric coherence penalty when pixel correspondences are available. In ablation, the DDD17 baseline without GPR or Harmonic Spectral Resonance reports mIoU $=76.94\%$; adding only GPR raises it to $78.09\%$, only Harmonic Spectral Resonance to $78.26\%$, and both together to $79.11\%$. On DSEC, GPR alone yields a $+1.08\%$ improvement and the combined setting $+2.01\%$. Under random large spatial shifts, CMX-B4 drops by $-2.43\%$, whereas Evita-L drops by only $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$0 (Peng et al., 10 Jul 2026).

A related but distinct learned formulation appears in "Seeing 3D Through 2D Lenses: 3D Few-Shot Class-Incremental Learning via Cross-Modal Geometric Rectification" (Tuo et al., 18 Sep 2025). CMGR targets 3D Few-Shot Class-Incremental Learning and identifies semantic blurring, modality mismatch, and catastrophic forgetting as consequences of texture-biased projections and indiscriminate fusion. Its Structure-Aware Geometric Rectification module hierarchically aligns 3D part structures with CLIP intermediate spatial priors through layer-adaptive cross-attention,

$$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$1

with cross-attention applied only at selected layers $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$2. A self-masking mechanism retains only a top $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$3 fraction of attention weights, and a regularization term preserves similarity structure between masked and unmasked features. Final rectified features are then aggregated with the original 2D and 3D streams. The paper explicitly relates this hierarchy to parallax rectification: the procedure aligns 3D part hierarchies with 2D spatial priors and performs a multi-scale geometric calibration between modalities. Quantitatively, on ShapeNet $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$4 CO3D, CMGR reports final-task accuracy $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$5, compared with $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$6 for FILP-3D and $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$7 for C3PR; average accuracy $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$8; and performance drop $$\mathbf{Z} = [\mathbf{X}'_r \,\|\, \mathbf{X}_e \,\|\, (\mathbf{X}'_r - \mathbf{X}_e)],$$9. In ablation, enabling SAGR alone raises accuracy from $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$0 to $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$1 (Tuo et al., 18 Sep 2025).

3. Projective, affine, and developable-surface rectification

A classical line of work treats rectification as the recovery of a projective or metric warp. "A Geometric Approach to Obtain a Bird's Eye View from an Image" shows that a ground-plane rectifying homography can be parameterized by only four parameters specifying the horizon line and the vertical vanishing point, or only two if the field of view or focal length is known (Abbas et al., 2019). The method introduces a bounded stereographic encoding for vanishing points and vanishing lines so that a CNN can regress finite quantities even when the underlying geometry lies at infinity. The resulting homography is composed from roll removal, tilt removal, translation, and optional alignment, and the paper reports $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$2 AUC on Horizon Lines in the Wild and a runtime of about $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$3 ms per image on a GTX 1050 Ti GPU (Abbas et al., 2019).

When radial distortion is significant, rectification must jointly model lens distortion and plane-induced projective deformation. "Rectification from Radially-Distorted Scales" proposes the first minimal solvers that jointly estimate lens distortion and affine rectification from repetitions of rigidly transformed coplanar local features (Pritts et al., 2018). The method uses the division model,

$$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$4

and enforces equal-scale constraints after undistortion and affine rectification. Minimal configurations are denoted $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$5, $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$6, and $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$7, producing degree-4 polynomial equations in $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$8. Solver construction uses Gröbner basis techniques with sampled monomial bases for numerical stability; for the $$\hat{\mathbf{X}}_e(\mathbf{p}) = \sum_{k \in \Omega} w_k \cdot \mathbf{X}_e\!\left(\mathbf{p} + \mathbf{p}_k + \mathbf{\Delta}_k(\mathbf{p})\right) \cdot \mathbf{M}_k(\mathbf{p}).$$9 configuration the elimination template is $=76.94\%$0 with $=76.94\%$1 solutions (Pritts et al., 2018).

For planar objects with Manhattan structure, "Fast Projective Image Rectification for Planar Objects with Manhattan Structure" estimates two dominant vanishing points and derives a metric rectification homography from the corresponding camera rotation (Shemiakina et al., 2019). The calibration matrix is written as

$=76.94\%$2

and the final homography has the form

$=76.94\%$3

On MIDV-500, the reported runtime is around $=76.94\%$4 ms on a Core i7 3610qm CPU, and accuracy is described as better or equal to the-state-of-the-art if the background occupies no more than half of the image (Shemiakina et al., 2019).

Rectification becomes more complex when the surface itself is non-planar but developable. "Geometric Rectification of Creased Document Images based on Isometric Mapping" formulates rectification as simultaneous optimization of a 3D document mesh $=76.94\%$5 and a 2D unfolded mesh $=76.94\%$6, linked by an isometric mapping (Luo et al., 2022). Developability is enforced through face-wise constraints on diagonals and inner products, summarized by an energy $=76.94\%$7, while textural cues are introduced through straightness and projection constraints for feature lines such as text baselines or document boundaries. The full objective combines developability, data fidelity to a point cloud, fairness terms on both meshes, and line/ray constraints, and optimization uses repeated refinement with an LBFGS solver. The reported result is the best or near-best performance on metrics including MS-SSIM, local/line distortion, LPIPS, and OCR-based CER/WER, especially under severe creases and foldings (Luo et al., 2022).

4. Stereo, scanline, and rolling-shutter formulations

Stereo rectification adds an epipolar constraint: corresponding points should lie on the same scanline after warping. "Robust Uncalibrated Stereo Rectification with Constrained Geometric Distortions (USR-CGD)" proposes a nine-parameter homography family with five rotations, two vertical translations, and two focal lengths, explicitly designed to reduce geometric distortion while maintaining small rectification error (Ko et al., 2016). The rectifying homographies are parameterized as

$=76.94\%$8

with analogous form for the right image. USR-CGD defines several distortion measures, including modified aspect ratio, skewness, rotation angle, size ratio, and orthogonality, and combines them with Sampson error in an adaptive cost function

$=76.94\%$9

The weights are activated only when their corresponding errors exceed thresholds. The reported outcome is best or near-best rectification error together with markedly improved geometric fidelity, and the vertical disparity is described as below $78.09\%$0 px, which is critical for 1D scanline matching (Ko et al., 2016).

Rolling-shutter cameras violate the assumption of a single pose per frame. "Image Stitching and Rectification for Hand-Held Cameras" derives a differential homography for scanline-varying camera poses and extends As-Projective-As-Possible warping to an RS-aware spatially-varying homography field (Zhuang et al., 2020). Under a differential motion model, the homogeneous flow obeys

$78.09\%$1

where $78.09\%$2 depends on scanline locations and an acceleration parameter $78.09\%$3. A minimal 5-point solver estimates the model inside RANSAC, after which a local RS-aware APAP field is fitted for non-planar scenes and camera parallax. The stated effect is RS-aware stitching and rectification at one stroke, with superior performance over state-of-the-art methods, especially for images captured by hand-held shaking cameras (Zhuang et al., 2020).

5. Physics-informed parallax correction in remote sensing, underwater imaging, and tomography

In passive microwave precipitation retrieval, parallax is a literal ground-location error caused by off-nadir observation of elevated hydrometeors. "Quantifying the Effect of a Parallax Correcting Algorithm for Passive Microwave Satellite Precipitation Retrievals across the Continental United States" formalizes the horizontal displacement as

$78.09\%$4

where $78.09\%$5 is the cloud-intercept height and $78.09\%$6 is the field-of-view zenith angle (Monsalve et al., 23 Sep 2025). Corrected coordinates are obtained by Great Circle Distance formulas from the assigned surface location. The paper argues that, for GMI retrievals, the physically appropriate intercept height is the Freezing Level derived from ERA5 temperature profiles. Over CONUS, the FL-based correction gives an average RMSE reduction of $78.09\%$7 across the year; in July the reduction reaches $78.09\%$8–$78.09\%$9, and correlation improves by up to $78.26\%$0. A cloud-top-based correction instead increases RMSE by $78.26\%$1. Event-level RMSE reductions reach up to $78.26\%$2 in small, isolated convective nuclei (Monsalve et al., 23 Sep 2025).

Underwater rectification is dominated by refraction rather than off-nadir observation. "NeuroPump: Simultaneous Geometric and Color Rectification for Underwater Images" explicitly embeds Snell’s law into a NeRF pipeline so that geometry and color are rectified jointly (Guo et al., 2024). Refraction is written as

$78.26\%$3

and the refracted ray is parameterized from a shifted origin $78.26\%$4 and underwater direction $78.26\%$5. A depth-dependent pixel remapping $78.26\%$6 is used for pose correction, and the rendering model includes attenuation and back-scatter:

$78.26\%$7

The method is described as the first NeRF-based method to simultaneously and explicitly rectify both geometric and color distortions, and it introduces an underwater 360 benchmark with real paired images with and without water (Guo et al., 2024).

In angular sensitive powder diffraction tomography, the relevant effect is a depth-induced detector offset. "Parallax in angular sensitive powder diffraction tomography" derives

$78.26\%$8

for the lateral shift at the detector, and expresses the rotation-dependent angular offset as

$78.26\%$9

A central result is that parallax is additive to other offset contributions, so correction is straightforward. A second central result is that for full $79.11\%$0 scans parallax has no impact on reconstructions of angular information because the average of the parallax term over the full rotation vanishes. The paper therefore distinguishes sharply between limited-angle experiments, where correction is necessary, and full-rotation experiments, where the effect cancels in reconstruction (Modregger et al., 2024).

6. Limits, misconceptions, and adjacent meanings of rectification

A recurrent misconception is that parallax rectification is equivalent to applying a single global homography. The literature surveyed here contradicts that assumption in multiple ways. Rolling-shutter stitching requires scanline-varying homographies (Zhuang et al., 2020); underwater imaging requires ray bending under Snell’s law rather than straight-line projection (Guo et al., 2024); RGB-event fusion uses per-layer deformable alignment (Peng et al., 10 Jul 2026); and creased documents require isometric flattening of a developable surface rather than planar rectification (Luo et al., 2022).

A second misconception is that any physically plausible height proxy improves spatial correction. The passive microwave study shows the opposite: freezing-level-based correction improves GPROF retrievals, whereas cloud-top-based correction increases RMSE by $79.11\%$1 (Monsalve et al., 23 Sep 2025). A third misconception is that parallax necessarily contaminates all tomographic reconstructions. In angular sensitive powder diffraction tomography, full $79.11\%$2 rotation cancels the parallax term in reconstructions of angular information, though not in raw projection geometry (Modregger et al., 2024).

The term “rectification” also has adjacent meanings outside spatial alignment. In "Third-order rectification in centrosymmetric metals," rectification denotes the conversion of AC fields into DC currents via third-order nonlinear optical responses, with mechanisms including Berry curvature quadrupole, Fermi surface injection, and shift effects (Sarkar et al., 29 Jan 2025). In "Origin of Robust Rectification in Geometric Diodes," rectification refers to current-flow preference produced by asymmetric bias-induced barrier lowering, with an intrinsic rectification ability at up to $79.11\%$3 THz (Bai et al., 2021). These works are geometrically grounded and concern rectification, but they do not address parallax in the spatial-registration sense. Their inclusion is therefore terminological rather than methodological.