PriOr-Flow: Dual-Branch Panoramic Flow Estimation

• PriOr-Flow is a dual-branch optical flow estimation framework that compensates for severe, latitude-dependent distortions in panoramic (ERP) images.
• It employs a primitive and an orthogonal branch with collaborative modules like DCCL and ODDC to fuse multi-scale features and mitigate polar distortions.
• Evaluations on datasets such as MPFDataset and FlowScape demonstrate that PriOr-Flow significantly reduces error rates, especially in challenging polar regions.

PriOr-Flow is a dual-branch, panoramic optical flow estimation framework specifically designed to address the critical problem of severe, latitude-dependent distortions induced by sphere-to-plane projections such as equirectangular projection (ERP). These distortions, most pronounced at the poles, are a major obstacle to accurate motion estimation with conventional, perspective-based optical flow methods. PriOr-Flow leverages orthogonal views as spatial priors and introduces specialized collaboration modules that compensate for polar region distortions, achieving state-of-the-art results across challenging panoramic datasets.

1. Motivation: Panoramic Optical Flow and Polar Distortions

Panoramic (360°) imaging systems are increasingly important in domains such as autonomous driving, mobile robotics, VR, and environmental monitoring, necessitating robust wide-field motion estimation. ERP, the most widespread sphere-to-plane mapping, suffers from highly non-uniform distortions—latitudes near the poles are severely stretched, leading to degraded optical flow accuracy, especially in those regions. Prior methods, including ERP warping, tangent plane rendering, or weighted cost aggregation, offer only partial mitigation and generally do not explicitly address the most challenging polar irreconcilabilities.

PriOr-Flow was developed to directly counteract these limits by using information from an orthogonal spherical rotation: the polar regions in ERP become equatorial in the orthogonal view and vice versa, meaning distortion is now minimal in the same polar areas previously at greatest risk. The framework is founded on the principle that fusing cost information from both view arrangements yields a richer, more robust motion representation.

2. Dual-Branch Collaborative Architecture

The core of PriOr-Flow is a dual-path network structure:

• Primitive branch: operates on the standard ERP image pairs, following the distortion signature typical of current panoramic pipelines.
• Orthogonal branch: takes as input a 90° (typically x-axis) rotated version of the ERP image. The orthogonalization operator $T_p^o$ spatially rotates each pixel’s spherical coordinates, performing bilinear interpolation to generate a view with low distortion in former ERP poles.

Formally, the orthogonal image is generated as

$$I^o = T_p^o(I^p) = \{ \text{Bi}(I^p, \mathcal{R}(-90^\circ, \mathbf{x})) ~|~ \mathbf{x} \in I^p \}$$

where Bi denotes bilinear interpolation and $\mathcal{R}$ is spherical rotation.

Each branch independently extracts multi-scale feature pyramids and builds 4D cost volumes encoding patchwise similarities across flow candidates. During each refinement stage, the branches interact through proposed collaboration mechanisms, significantly improving robustness and polar performance with minimal architecture changes.

3. Dual-Cost Collaborative Lookup (DCCL): Distortion-Resilient Correlation

The DCCL operator is at the heart of PriOr-Flow’s polar distortion compensation strategy. At each refinement step, for every primitive-branch pixel, the operator:

1. Computes the correspondence position in the next frame, handling horizontal wrap-around:

$$\hat{\mathbf{x}}^p = \left(u^p + f^p_1(\mathbf{x}^p)\bmod W, v^p + f^p_2(\mathbf{x}^p)\right)$$

1. Constructs spatial neighborhoods around this coordinate in both ERP and (rotated) orthogonal views.
2. Performs cost volume lookups:
   • In the primitive (ERP) pyramid: $\mathcal{C}^p$
   • In the orthogonal pyramid, after conversion to ERP coordinates: $\mathcal{C}^{o2p}$
3. Aggregates both cue sets and supplies them to the primitive flow update module, ensuring polar flow estimation directly benefits from the low-distortion, high-confidence signals of the orthogonal branch.

This collaborative lookup leads to highly robust and accurate similarity estimation, especially in regions where single-view approaches would be severely compromised by projection artifacts.

4. Ortho-Driven Distortion Compensation (ODDC)

To further refine flow in areas vulnerable to distortion (chiefly poles), PriOr-Flow deploys the Ortho-Driven Distortion Compensation (ODDC) module. Key steps include:

1. Mapping the orthogonal branch's current estimated flow $\mathcal{F}^o$ into primitive coordinates: $\mathcal{F}^{o2p} = T_o^p(\mathcal{F}^o)$.
2. Computing confidence maps for both primitive and transferred orthogonal flows using group-wise correlation between feature embeddings.
3. Extracting motion features as concatenations of encoded cost cues, confidence maps, and both flow fields.
4. Employing a ConvGRU-based update module to yield a residual motion estimate, adaptively fusing branch predictions. High-confidence, low-distortion cues from the orthogonal path dominate updates in polar regions, further suppressing error propagation from the distortion-afflicted primitive branch.

This mechanism is designed to be iteratively applied; at every update step, ODDC reinforces the parts of the primitive flow that are most susceptible to distortion with orthogonally-derived, distortion-suppressed alternatives.

5. Performance Evaluation: Datasets, Metrics, and Empirical Results

PriOr-Flow was benchmarked on leading panoramic scene flow datasets:

• MPFDataset: Synthetic, city and event scenes with pixelwise ground truth.
• FlowScape: Four synthetic scenarios with severe occlusions and complex small-object motion.
• OmniPhotos and ODVista: Real-world panoramic video datasets for qualitative assessment.

Evaluation Metrics:

• EPE (End-Point Error) and SEPE (Spherical End-Point Error, i.e., geodesic discrepancy).

Results:

• MPFDataset: PriOr-RAFT (RAFT-based PriOr-Flow) achieves an EPE of 2.22, a 30% reduction compared to SLOF (RAFT-based, 3.17), and a SEPE improvement of 20.9%.
• FlowScape: PriOr-RAFT achieves an EPE of 2.33, outperforming SLOF (7.59), PanoFlow, and in polar regions, outperforms PanoFlow by 39.7%.
• Ablation studies: Both DCCL and ODDC modules contribute measurable improvements individually; their combination yields the largest accuracy increases in polar segments.

Compatibility and Overhead:

• The PriOr-Flow strategy is compatible with mainstream iterative optical flow architectures (RAFT, GMA, SKFlow). Runtime overhead is modest relative to the improvement in accuracy, even when iterative steps are reduced for efficiency.

Method	Baseline	MPF EPE ↓	FlowScape EPE ↓	Polar FlowScape EPE ↓
SLOF	RAFT	3.17	7.59	6.25
PanoFlow	RAFT	N/A	3.38	6.25
PriOr-Flow	RAFT	2.22	2.33	4.13

6. Significance, Impact, and Future Directions

PriOr-Flow establishes a new paradigm for panoramic flow by systematically correcting for projection-induced artifact zones. Its main innovations—the use of dual branches with mutual, region-adaptive information exchange—are general and can inspire new designs for 360° tasks beyond optical flow, including segmentation and depth estimation.

Key outcomes and implications:

• Generalization: Demonstrated robustness across both synthetic and real-world scenes, with strong potential for wide adoption in deployable panoramic vision systems.
• Universal Compatibility: Dual-branch, cross-view compensation can be plugged into most perspective-based pipelines, raising baseline performance across the board.
• Benchmark Advancement: Sets new state-of-the-art on panoramic optical flow datasets, with largest gains in formerly most-challenging regions.
• Scalable innovation: Overhead is moderate for the performance gain, making the approach viable for practical scene understanding systems.

Anticipated directions include optimizations for real-time scenarios, extensions to spherical segmentation and dense geometry, and deeper paper of dynamic view-based cost fusion for domain-adaptable, high-fidelity motion reasoning.

---

Official implementation: https://github.com/longliangLiu/PriOr-Flow Key datasets: MPFDataset, FlowScape, OmniPhotos, ODVista