Leveraging implicit differentiation to couple feature learning with long‑term geometric consistency

Develop a principled method that leverages implicit differentiation to couple feature representation learning with long‑term geometric consistency in visual odometry and visual–inertial odometry pipelines, while maintaining robustness and efficiency when training and operating on real‑world, unconstrained video streams.

Background

The paper argues that learned feature extractors and matchers are typically trained independently of downstream geometric optimization, which leads to a mismatch with the reprojection objectives used in VO/VIO. Integrating global bundle adjustment or similar geometric modules into learning pipelines is challenging due to implicit and iterative solvers.

Implicit differentiation offers a way to backpropagate through optimization defined by equilibrium conditions without unrolling, and has been applied in some robotics problems. However, applying it to couple feature learning with long‑term geometric consistency for VO/VIO remains underexplored, raising the open challenge of how to realize this coupling while preserving robustness and efficiency on unconstrained video data.