Relative Pose Estimation of Calibrated Cameras with Known $\mathrm{SE}(3)$ Invariants (2007.07686v1)

Abstract: The $\mathrm{SE}(3)$ invariants of a pose include its rotation angle and screw translation. In this paper, we present a complete comprehensive study of the relative pose estimation problem for a calibrated camera constrained by known $\mathrm{SE}(3)$ invariant, which involves 5 minimal problems in total. These problems reduces the minimal number of point pairs for relative pose estimation and improves the estimation efficiency and robustness. The $\mathrm{SE}(3)$ invariant constraints can come from extra sensor measurements or motion assumption. Different from conventional relative pose estimation with extra constraints, no extrinsic calibration is required to transform the constraints to the camera frame. This advantage comes from the invariance of $\mathrm{SE}(3)$ invariants cross different coordinate systems on a rigid body and makes the solvers more convenient and flexible in practical applications. Besides proposing the concept of relative pose estimation constrained by $\mathrm{SE}(3)$ invariants, we present a comprehensive study of existing polynomial formulations for relative pose estimation and discover their relationship. Different formulations are carefully chosen for each proposed problems to achieve best efficiency. Experiments on synthetic and real data shows performance improvement compared to conventional relative pose estimation methods.

• The paper introduces a novel approach for relative pose estimation by integrating SE(3) invariants, specifically rotation angle and screw translation, to improve solution efficiency.
• It rigorously evaluates various polynomial formulations and minimal problem cases, demonstrating enhanced robustness against noise compared to traditional methods.
• Experimental validations on synthetic and real datasets confirm that using SE(3) invariants significantly reduces required point correspondences and computational overhead.

Understanding Relative Pose Estimation with $SE(3)$ Invariants

The paper addresses the challenging problem of relative pose estimation for calibrated cameras, specifically focusing on concise solutions constrained by known $SE(3)$ invariants. This work introduces a methodological framework that takes advantage of these invariants, such as rotation angle and screw translation, to enhance the efficiency and robustness of pose estimation processes. This approach is beneficial because it reduces the number of point correspondences necessary for reliable camera pose estimation. Thus, it directly addresses computational overhead and potential inaccuracies stemming from extraneous data points.

Key Contributions

The authors delineate their contributions as follows:

1. Definition and Use of $SE(3)$ Invariants: The paper elaborates on the integration of rotation angle and screw translation as $SE(3)$ invariants in minimal relative pose estimation algorithms. Typically, conventional methods might require extrinsic calibration to incorporate additional constraints into the camera frame. However, this work circumvents such dependencies, thus enhancing solver flexibility in practical applications.
2. Comprehensive Analysis of Polynomial Formulations: The paper conducts an extensive examination of existing polynomial formulations utilized in relative pose estimation. This includes a detailed exploration of the relationships and features of various formulations, aiming to identify optimal formulations for distinct relative pose problems leveraging $SE(3)$ invariants.
3. Experimental Validation: Through synthetic and real data experiments, the efficacy of the proposed methods is illustrated. The experiments demonstrate performance improvements over traditional relative pose estimation approaches, marking a significant advancement in the field.

Methodological Insights

Relative pose estimation is a critical task in computer vision applications, from augmented reality to autonomous navigation. The common objective is to determine the relative orientation and position of a camera with respect to an observed scene, using minimal point correspondences.

The paper extensively tests five minimal problem cases categorized based on the incorporation of rotation angle and screw translation. Here are some specific cases:

• 4P-RA: Utilizes known rotation angle to solve the pose problem with four point correspondences.
• 4P-ST0: Employs zero screw translation as a constraint, facilitating four point solutions.
• 3P-RA-ST0: Combines both the rotation angle and zero screw translation, allowing for a three-point solution.

Each of these setups is approached with various polynomial formulations, such as SIR (Solving Isolated Rotation) of different orders and NullSpace-based methods, to ascertain their applicability and efficiency.

Numerical and Practical Implications

Numerically, the robustness of these methods is tested against noise and varying motion assumptions in synthetic datasets, with performance benchmarks extensively detailed. Practically, the approaches are validated on real-world datasets involving indoor mobile robots and outdoor autonomous vehicles. The results consistently indicate that knowledge of $SE(3)$ invariants can significantly enhance estimation accuracy, especially when traditional assumptions of camera motion might not hold.

Speculation on Future Developments

The methodology proposed suggests several promising directions for further research. One potential avenue is the exploration of dynamic environments where additional $SE(3)$ invariant constraints might arise from sensor fusion, such as integrating LIDAR with visual data in automated systems.

Another exciting development could be the extension of these concepts to more complex vision systems involving generalized or omnidirectional cameras, where perspectives on invariance might differ due to broader fields of view and non-standard lens architectures.

In summary, the paper presents a rigorous investigation into reducing computational burdens and improving accuracy in relative pose estimation through strategic use of $SE(3)$ invariants. It opens pathways for developing more robust vision algorithms, crucial for advancing not only academic research but also real-world technological applications.