U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation

Abstract: Unsupervised optical flow methods typically lack reliable uncertainty estimation, limiting their robustness and interpretability. We propose U$^{2}$Flow, the first recurrent unsupervised framework that jointly estimates optical flow and per-pixel uncertainty. The core innovation is a decoupled learning strategy that derives uncertainty supervision from augmentation consistency via a Laplace-based maximum likelihood objective, enabling stable training without ground truth. The predicted uncertainty is further integrated into the network to guide adaptive flow refinement and dynamically modulate the regional smoothness loss. Furthermore, we introduce an uncertainty-guided bidirectional flow fusion mechanism that enhances robustness in challenging regions. Extensive experiments on KITTI and Sintel demonstrate that U$^{2}$Flow achieves state-of-the-art performance among unsupervised methods while producing highly reliable uncertainty maps, validating the effectiveness of our joint estimation paradigm. The code is available at https://github.com/sunzunyi/U2FLOW.

• The paper introduces a recurrent framework combining optical flow estimation with per-pixel uncertainty measurement, improving robustness in unsupervised settings.
• It employs augmentation consistency with an MLE-based Laplacian objective to decouple uncertainty learning from flow regression, enhancing training stability.
• Empirical evaluations on KITTI and Sintel benchmarks demonstrate significant performance gains and reliable uncertainty quantification compared to previous methods.

U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation

Introduction

The paper "U$^{2}$Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation" (2604.10056) introduces a novel recurrent framework for joint optical flow and per-pixel uncertainty estimation in unsupervised settings. Unlike previous methods, which typically limit themselves to point estimates of flow, this work systematically addresses both the estimation of dense optical flow and the quantification of its epistemic uncertainty, thereby enhancing the robustness and interpretability of motion prediction under the absence of ground-truth supervision.

Methodological Contributions

Decoupled Uncertainty-Learning via Augmentation Consistency

Central to U$^{2}$Flow is the decoupling of flow and uncertainty learning objectives. The framework exploits predictive inconsistencies under spatial and appearance augmentations as a supervisory signal for uncertainty estimation. Specifically, an MLE-based Laplacian objective is employed on the distance between flows predicted before and after augmentation, imposing a per-pixel probabilistic constraint without reliance on annotations.

This uncertainty objective is detached from main flow regression, thereby mitigating gradient interference and optimizing training stability. The lack of such decoupling in prior works is shown to be detrimental based on ablation results.

Uncertainty-Aware Recurrent Refinement

In each iteration of the recurrent backbone (a RAFT-style all-pairs correlation network), a dedicated uncertainty estimation head predicts log-variance maps. These are fed back into the update module to dynamically scale feature maps via a sigmoid-transformed weighting, suppressing unreliable regions during refinement. The approach contrasts with standard recurrent refinement strategies which indiscriminately process spatial features, ignoring model confidence, and results in consistent accuracy gains across benchmarks.

Uncertainty-Guided Regional Smoothness

Smoothness constraints for unsupervised optical flow are conventionally regularized using either edge-aware terms or homography-based regional constraints, often excluding unreliable pixels based on occlusion masks. U$^{2}$Flow proposes a more principled uncertainty-based selection: only low-uncertainty regions participate in homography estimation and loss computation, leading to improved flow coherence especially in domains (e.g., KITTI) where local rigidity and planarity predominate.

Uncertainty-Driven Bidirectional Flow Fusion

The framework further integrates a lightweight fusion network to adaptively combine predicted forward and mapped-backward flows. Crucially, the fusion is guided not by heuristic occlusion masks but by per-pixel uncertainty estimates—fusion occurs only where the forward estimate is deemed uncertain and the backward estimate is confident. This mechanism yields superior error localization compared to traditional occlusion-based fusion strategies.

Empirical Evaluation

Extensive evaluations on the KITTI and Sintel benchmarks demonstrate that U$^{2}$Flow achieves superior performance among unsupervised models, with notable improvements in both average endpoint error (EPE) and Fl-all error rate.

• KITTI-2015: U$^{2}$Flow achieves Fl-all=6.13%, outperforming other state-of-the-art unsupervised approaches such as UPFlow (9.38%) and even surpassing several multi-frame and semantic-aware methods.
• Sintel: Both clean and final passes show improved EPE compared to prior unsupervised baselines, confirming robustness and cross-domain generalization.

(Figure 1)

Figure 1: Qualitative results on KITTI (frames #5, #9) and Sintel (ambush_3/23, cave_3/16), showing flow predictions and uncertainty maps in comparison to SMURF.

Ablation studies delineate the impact of each architectural component. The adoption of uncertainty-driven fusion, precise uncertainty-weighted homography losses, and explicit uncertainty-guided refinement all yield additive benefits.

(Figure 2)

Figure 2: Detailed test results of U$^{2}$Flow on Sintel, highlighting strong regional and boundary accuracy.

Uncertainty Estimation Reliability

Quantitative evaluation of uncertainty estimation uses established metrics:

• AUSE (Area Under Sparsification Error Curve): U$^{2}$Flow yields AUSE of 0.11/0.12 on Sintel/KITTI, significantly lower than PDC-Net+ (0.18/0.16) and heuristic baselines.
• Spearman Correlation Coefficient (CC): U$^{2}$Flow attains 0.66/0.64 on Sintel/KITTI, outpacing all compared approaches.

These metrics strongly evidence that the predicted uncertainty maps closely track true endpoint error spatially, validating the effectiveness of augmentation-consistency-based supervision in capturing error modes absent ground truth.

Practical and Theoretical Implications

The transition to uncertainty-aware, decoupled unsupervised optical flow estimation significantly impacts both the deployment of optical flow in downstream vision pipelines and the theoretical modeling of self-supervised learning:

• Practical: Reliable spatially-dense uncertainty quantification is invaluable in safety-critical applications such as autonomous driving, SLAM, adaptive tracking, and 3D scene reconstruction. Models can now better modulate outputs, fuse estimates, and trigger fallback mechanisms based on predicted confidence.
• Theoretical: The self-supervised use of augmentation-induced inconsistency as a surrogate for uncertainty is extensible to other dense prediction regimes. This approach could generalize to unsupervised depth, stereo, or even dense semantic prediction, provided adequate augmentation regimes can be formulated.

Furthermore, the demonstrated domain generalization suggests that uncertainty estimation based on model perturbations captures not only dataset-specific artifacts but also fundamental epistemic uncertainty.

Future Directions

The limitations acknowledged in the manuscript center on augmentation domain coverage: not all real-world perturbations (e.g., extreme motion blur, atmospheric effects) are adequately simulated in the current regime, possibly constraining uncertainty generalization further. Leveraging generative models for augmentation, or self-adaptive online augmentation scheduling, could further expand the expressivity of uncertainty supervision.

More broadly, the joint modeling of epistemic and aleatoric uncertainty, and the integration of temporal context or limited pseudo-supervised signals, are notable directions for advancing the reliability and interpretability of self-supervised vision.

Conclusion

U$^{2}$Flow establishes a new paradigm for self-supervised optical flow estimation by coupling recurrent, uncertainty-aware refinement with a decoupled augmentation-consistency learning strategy. The approach provides state-of-the-art unsupervised flow predictions and robust, quantitatively validated uncertainty maps. These technical innovations set a strong foundation for future advances in trustworthy and robust motion estimation, with direct applicability to downstream vision and robotics systems.