Dive Deeper into Rectifying Homography for Stereo Camera Online Self-Calibration (2309.10314v4)

Abstract: Accurate estimation of stereo camera extrinsic parameters is the key to guarantee the performance of stereo matching algorithms. In prior arts, the online self-calibration of stereo cameras has commonly been formulated as a specialized visual odometry problem, without taking into account the principles of stereo rectification. In this paper, we first delve deeply into the concept of rectifying homography, which serves as the cornerstone for the development of our novel stereo camera online self-calibration algorithm, for cases where only a single pair of images is available. Furthermore, we introduce a simple yet effective solution for global optimum extrinsic parameter estimation in the presence of stereo video sequences. Additionally, we emphasize the impracticality of using three Euler angles and three components in the translation vectors for performance quantification. Instead, we introduce four new evaluation metrics to quantify the robustness and accuracy of extrinsic parameter estimation, applicable to both single-pair and multi-pair cases. Extensive experiments conducted across indoor and outdoor environments using various experimental setups validate the effectiveness of our proposed algorithm. The comprehensive evaluation results demonstrate its superior performance in comparison to the baseline algorithm. Our source code, demo video, and supplement are publicly available at mias.group/StereoCalibrator.

• The paper introduces a novel algorithm that improves stereo camera self-calibration by leveraging rectifying homography for precise extrinsic parameter estimation.
• It employs a two-pronged approach with single-pair calibration using dual rotation matrices and global optimization for multiple image pairs.
• Evaluation on custom and public datasets demonstrates reduced angular errors and variances, enhancing real-time performance in dynamic conditions.

Insights into Stereo Camera Online Self-Calibration and Rectifying Homography

The paper "Dive Deeper into Rectifying Homography for Stereo Camera Online Self-Calibration" presents advancements in the online self-calibration of stereo cameras by exploring the concept of rectifying homography. The authors propose an innovative algorithm designed specifically to enhance the estimation of extrinsic parameters, which are crucial for the successful execution of stereo matching algorithms. Significantly, this research moves away from traditional approaches that cast self-calibration as a mere visual odometry problem, instead emphasizing the principles of stereo rectification.

Methodological Contributions

The authors introduce a two-pronged methodological approach:

1. Single-Pair Calibration: This involves using rectifying homography to recalibrate the stereo camera in scenarios where only a single image pair is available. The strategy hinges on optimizing two independent rotation matrices, $\boldsymbol{R}_l$ and $\boldsymbol{R}_r$, representing the rotations of the left and right camera coordinate systems, respectively. By accumulating residuals in the vertical direction, the approach avoids a positive semidefinite coefficient matrix that can emerge when jointly optimizing rotation and translation, thereby increasing robustness against initial estimation errors.
2. Global Optimization for Multi-Pair Cases: For scenarios involving multiple stereo image pairs, the authors propose a straightforward yet effective solution for achieving global optimality in extrinsic parameter estimation. This is derived from a novel energy function that measures cosine similarities between estimated and actual normalized vectors. The global optimum rotation axis and translation vector are estimated without resorting to more complex visual odometry backend optimization techniques, like bundle adjustment or Kalman filters.

Evaluation Metrics and Results

Recognizing issues with existing evaluation metrics, the authors propose four new metrics to quantify the performance of their algorithm:

• Angular Error in Translation ($e_{\boldsymbol{t}}$) and Rotation Vector Distance ($e_{\boldsymbol{\theta}}$): These quantify the discrepancy between estimated and ground-truth extrinsics, providing a measure of accuracy.
• Standard Deviation of Angular Errors ($\sigma_{\boldsymbol{t}}$) and Standard Deviation of Rotation Vector Distances ($\sigma_{\boldsymbol{\theta}}$): These serve as indicators of the robustness of single-pair extrinsic estimation.

Extensive experiments, consisting of both custom datasets capturing indoor and outdoor scenarios, and established public datasets like KITTI 2015 and Middlebury 2021, demonstrate the superior performance of the proposed algorithm against the baseline. The new approach consistently results in lower angular errors and reduced variance in the evaluation metrics, underscoring its robustness and accuracy even under non-ideal conditions such as motion blur or dynamic scenes.

Implications and Future Directions

This research has significant implications for real-world applications such as autonomous navigation and robotics, where maintaining accurate depth perception through stereo cameras is critical. The ability to perform adaptive self-calibration in real-time can improve system reliability and safety.

Theoretically, this work provides a new framework for understanding stereo camera calibration by leveraging rectifying homography more fully. Future developments could expand on this framework, integrating it with machine learning approaches to further enhance robustness and adaptability. Additionally, further investigation into real-time performance optimization and hardware integration would help in broadening the practical applicability of the proposed methods in real-world robotic systems.