Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement (1908.11542v1)

Abstract: We propose an approach to estimate the 6DOF pose of a satellite, relative to a canonical pose, from a single image. Such a problem is crucial in many space proximity operations, such as docking, debris removal, and inter-spacecraft communications. Our approach combines machine learning and geometric optimisation, by predicting the coordinates of a set of landmarks in the input image, associating the landmarks to their corresponding 3D points on an a priori reconstructed 3D model, then solving for the object pose using non-linear optimisation. Our approach is not only novel for this specific pose estimation task, which helps to further open up a relatively new domain for machine learning and computer vision, but it also demonstrates superior accuracy and won the first place in the recent Kelvins Pose Estimation Challenge organised by the European Space Agency (ESA).

• The paper introduces a novel fusion of deep learning and nonlinear optimization for precise 6DOF satellite pose estimation.
• It leverages HRNet for accurate landmark detection and SA-LMPE for robust pose refinement, achieving rotation error of 0.7277° and translation error of 0.0359m.
• The approach outperforms previous methods in handling variations in orientation, scale, and lighting, making it effective for critical satellite operations.

Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement

The paper presents a sophisticated approach to estimating the six degrees of freedom (6DOF) pose of a satellite from a single image using deep learning and geometric optimization. This methodology is crucial for various space operations, such as satellite docking, debris removal, and establishing communication links between spacecraft. The approach combines the power of machine learning for feature detection with geometric principles for accurate pose estimation, illustrating a successful integration of these fields.

Methodology

The authors propose a novel method centered on three main components that contribute to the pose estimation pipeline:

1. 3D Model Reconstruction: The initial phase involves reconstructing a 3D model of the satellite by applying multi-view triangulation to a subset of training images. Each image is annotated with ground truth poses and keypoints, allowing for the optimization of a 3D point cloud that represents the satellite's structure.
2. Landmark Prediction Using Deep Learning: A deep network, specifically the High-Resolution Network (HRNet), is then trained to predict the 2D locations of pre-defined landmark points on new images. The HRNet maintains high-resolution feature representations, enhancing spatial precision during the landmark prediction process. This aspect of the methodology highlights leveraging robust and high-capacity neural architectures for precise feature localization.
3. Nonlinear Pose Refinement: Finally, these 2D image points are associated with the known 3D structure points to compute the satellite pose. The authors employ a robust optimization framework utilizing Levenberg-Marquardt methods enhanced by a simulated annealing scheme (SA-LMPE) to iteratively refine the pose estimate. This method adapts the optimization parameters dynamically, providing robustness against outliers in the predicted correspondences.

Results and Evaluation

The proposed method demonstrates superior accuracy in the Kelvins Pose Estimation Challenge organized by the European Space Agency (ESA), securing the first place. Key results include:

• Pose Estimation Accuracy: The mean error for rotation was found to be 0.7277 degrees, and the translational error was 0.0359 meters, indicating a high precision in estimating satellite poses.
• Comparison with Previous Works: The method significantly outperforms previous approaches, such as the Spacecraft Pose Network (SPN). Metrics such as the Intersection Over Union (IOU) of the bounding boxes also showed substantial improvements.
• Robust Handling of Variations: The method efficiently manages variations in object size, orientation, and lighting by leveraging the high-resolution representation and robust optimization techniques.

Implications and Future Work

The implications of this research are substantial in the domain of space operations where reliable and efficient pose estimation is a prerequisite. The integration of deep learning with classical geometric optimization techniques can be further explored across different applications beyond satellite imagery, potentially extending to more generalized object pose estimation tasks.

Future developments could include enhancing the scalability of the approach to handle larger varieties of objects and employing real-time processing capabilities to facilitate its deployment in active satellite missions. Additionally, exploring the integration of other data modalities, such as thermal imaging or multispectral data, may provide insights into improving pose estimation under more challenging conditions.

In conclusion, this paper presents a well-rounded and rigorously evaluated approach to satellite pose estimation, contributing significantly to the fields of machine learning and computer vision as applied to space technologies.