CoProU-VO: Combined Projected Uncertainty in VO

• The paper introduces a novel end-to-end unsupervised VO framework that fuses target and projected reference uncertainties to effectively mask dynamic regions.
• It employs a rigorous probabilistic formulation with Vision Transformer backbones to achieve robust depth and pose estimation under challenging, real-world conditions.
• Empirical results on benchmarks like KITTI and nuScenes demonstrate significant improvements in VO accuracy by reducing error metrics through combined uncertainty propagation.

Combined Projected Uncertainty VO (CoProU-VO) is an end-to-end framework for unsupervised monocular visual odometry that introduces principled cross-frame uncertainty propagation. Unlike previous uncertainty-aware methods that rely solely on single-frame information, CoProU-VO jointly leverages both target and reference frame uncertainties projected into a common domain, leading to improved masking of uncertain regions—particularly in dynamic scenes where the static scene assumption is violated. This paradigm enables robust and accurate camera pose and depth estimation under challenging, real-world conditions.

1. Motivation and Challenges in Unsupervised Visual Odometry

Unsupervised monocular visual odometry (VO) methods are attractive due to the elimination of ground-truth pose and depth data requirements. Most such approaches employ photometric consistency losses, synthesizing the target view from a reference view using estimated depth and pose, then supervising via the photometric reconstruction error. A key assumption is scene staticity: if all scene elements are static and there are no occlusions or specularities, the pixel-wise residuals become meaningful indicators of model performance.

However, real-world environments frequently contain moving objects, occlusions, and non-Lambertian surfaces, fundamentally invalidating this assumption. Regions influenced by dynamics typically cause irregular residuals, which, if not handled, degrade both depth and pose estimation. Traditional uncertainty-based masking attempts to identify unreliably reconstructed pixels, but prior methods typically predict per-pixel uncertainty using only the target frame—neglecting uncertainty introduced when warping the reference image to synthesize the target view. This omission leads to insufficient filtering of dynamic or occluded regions and consequent performance deterioration, especially in the presence of severe motion or rapid scene changes (Xie et al., 1 Aug 2025).

2. Probabilistic Formulation and Cross-Frame Uncertainty Propagation

CoProU-VO’s central innovation is the combination of target and projected reference uncertainties in a rigorous probabilistic framework. Let Σₜ(pₜ) denote the uncertainty (usually standard deviation) at pixel pₜ in the target frame, and Σₜ′(pₜ′) the uncertainty at the corresponding projected pixel pₜ′ in the reference frame. The latter is mapped into the target frame coordinates by warping along the estimated depth and pose.

Assuming independent Laplacian noise models for both target and reference images, the effective uncertainty for the photometric loss at pixel pₜ is:

$$σ_\text{eff}(pₜ) = \sqrt{Σₜ(pₜ)^2 + Σ_{t′→t}(pₜ)^2}$$

where Σ_{t′→t}(pₜ) is the reference frame uncertainty after projection into the target frame. The negative log-likelihood of the residual $r(I_t, I_{t′→t})$ then becomes:

$$\mathcal{L}_p = \frac{r(I_t, I_{t′→t})}{σ_\text{eff}} + \log σ_\text{eff} + \text{const}$$

This loss formulation robustly down-weights uncertain regions in both frames, effectively masking dynamic, occluded, or otherwise irrecoverable pixels during training.

3. Model Architecture and Vision Transformer Backbones

The architecture employs pre-trained Vision Transformer (ViT) backbones such as DINOv2 or DepthAnythingV2, known for strong capacity in extracting global and semantic features. Both the target and reference images are encoded via a frozen ViT, producing features that are re-used for depth and per-pixel uncertainty estimation using a decoder (adapted DPT). Estimated uncertainty maps are produced alongside depth for both frames.

A subsequent pose network (typically a ResNet-18 backbone) estimates interframe camera motion. These components are trained end-to-end, using only photometric loss—weighted by the combined uncertainty as above—without ground-truth depth or pose labels.

Notable design decisions include using only two consecutive frames at each training iteration, preserving low computational complexity and memory usage, and keeping transformer backbones frozen to leverage strong pre-trained features while maintaining real-time feasibility (Xie et al., 1 Aug 2025).

4. Empirical Results and Ablation Studies

Evaluation on KITTI and nuScenes benchmarks demonstrates substantial improvements in VO accuracy and robustness over prior unsupervised monocular, two-frame methods. Specific metrics used include Absolute Trajectory Error (ATE), relative translation error (t_err, as %), and relative rotation error (r_err, as °/100m).

Key findings:

• CoProU-VO outperforms earlier state-of-the-art methods (such as SC-Depth and DF-VO), especially in dynamic and highway scenes where competing methods often fail (e.g., due to large moving vehicles or complex occlusion).
• The method achieves competitive results even when trained and evaluated with only two consecutive frames, without performance loss from the absence of longer sequences.
• Ablation studies confirm that propagating uncertainty from the reference into the target frame—rather than using a single-frame uncertainty—significantly lowers both training/validation losses and VO errors. The combined uncertainty leads to more effective dynamic-object masking.
• Further ablations on encoder choice indicate that the uncertainty fusion mechanism, rather than improvements in depth alone, is primarily responsible for increased robustness.

These results are robust across variations in backbone, sequence content, and evaluation frequency.

5. Significance and Theoretical Implications

The projected uncertainty paradigm introduced by CoProU-VO establishes a theoretically grounded method for uncertainty-aware, unsupervised VO under real-world conditions. By coupling uncertainties across frames using a rigorous probabilistic model, it surpasses single-frame masking or ad-hoc filter strategies. The formula

$$σ_\text{eff}(pₜ) = \sqrt{Σₜ(pₜ)^2 + Σ_{t′→t}(pₜ)^2}$$

emerges directly from the variance addition law for independent Laplacian variables and justifies combined masking in a principled manner.

This approach also facilitates principled training: masking relies not on heuristics such as a fixed threshold, but on dynamically learned, spatially varying per-pixel uncertainties rooted in the actual distribution of photometric errors.

6. Limitations and Future Directions

While CoProU-VO demonstrates pronounced gains, some limitations and research directions remain:

• Current pose estimation is tied to a ResNet-18-based PoseNet, which introduces a representational bottleneck. Upgrading to more advanced pose architectures could yield further accuracy improvements.
• The uncertainty model is two-frame based; extending to multi-frame trajectories (e.g., temporal uncertainty propagation through recurrent modules) may offer improved dynamic modeling, particularly in rapidly changing scenes.
• Integration with geometry-aware occlusion models and fusing uncertainty propagation with advanced depth reconstruction frameworks (e.g., DUSt3R, VGGT) is a promising direction.
• Although frozen transformer backbones deliver strong results, future work could explore end-to-end fine-tuning with larger and more varied datasets to boost scene adaptability.

7. Impact on Uncertainty-Aware VO and Related Research

CoProU-VO marks a significant advance for unsupervised monocular visual odometry, particularly in dynamic, unstructured, and adverse conditions. The cross-frame uncertainty fusion paradigm has influenced subsequent works across VO, SLAM, and related sensor fusion domains. By formalizing projected and combined uncertainty, the framework offers a robust foundation for developing reliable, self-supervised navigation systems in safety-critical autonomous and robotics applications (Xie et al., 1 Aug 2025).

This approach highlights that careful, mathematically-grounded uncertainty integration is crucial—not only for accuracy, but also for interpretability, mask efficiency, and error calibration—thereby reshaping best practices in end-to-end structure-from-motion and VO research.