GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence (2311.14155v2)

Abstract: We present GigaPose, a fast, robust, and accurate method for CAD-based novel object pose estimation in RGB images. GigaPose first leverages discriminative "templates", rendered images of the CAD models, to recover the out-of-plane rotation and then uses patch correspondences to estimate the four remaining parameters. Our approach samples templates in only a two-degrees-of-freedom space instead of the usual three and matches the input image to the templates using fast nearest-neighbor search in feature space, results in a speedup factor of 35x compared to the state of the art. Moreover, GigaPose is significantly more robust to segmentation errors. Our extensive evaluation on the seven core datasets of the BOP challenge demonstrates that it achieves state-of-the-art accuracy and can be seamlessly integrated with existing refinement methods. Additionally, we show the potential of GigaPose with 3D models predicted by recent work on 3D reconstruction from a single image, relaxing the need for CAD models and making 6D pose object estimation much more convenient. Our source code and trained models are publicly available at https://github.com/nv-nguyen/gigaPose

• The paper presents GigaPose, which decouples template generation and patch correspondences to achieve state-of-the-art speed and robustness.
• It employs a reduced two-DoF template space for out-of-plane rotation and contrastive learning for reliable feature extraction.
• Evaluations on BOP challenge datasets show a 35-fold speed improvement and enhanced resilience to segmentation errors.

Overview of GigaPose for 6D Object Pose Estimation

The paper introduces GigaPose, a novel approach to 6D object pose estimation, particularly suited for novel objects using CAD models from RGB images. The method stands out for its exceptional speed, robustness, and accuracy, addressing some of the major limitations found in previous approaches like MegaPose. In essence, GigaPose gracefully combines discriminative templates with patch correspondences to streamline the estimation process.

Traditional methods in this domain often suffer from scalability issues due to the massive number of templates required or the computational cost of evaluating each image-template pair. GigaPose circumvents these issues by simplifying the template generation process. The method uses templates to determine the out-of-plane rotation by rendering them in a reduced two-DoF space, notably speeding up the inference. The method's design choice to decouple the in-plane rotation and scale, and instead resolve them through direct patch correspondences, not only accelerates the process but also enhances robustness to segmentation errors.

Technical Contributions and Methodology

GigaPose's framework splits the 6D pose estimation problem into manageable sections: estimating out-of-plane rotation using template matching and determining the remaining parameters with patch correspondences. This decoupling allows GigaPose to process and onboard novel objects in a significantly less template-intensive manner.

The key technique used in facilitating this process is the sublinear nearest-neighbor search among the templates in feature space, drastically reducing computation compared to MegaPose’s linear complexity with template numbers. By employing contrastive learning for feature extraction, they ensure both robustness to scale and in-plane rotation, while maintaining sensitivity to out-of-plane changes.

For practical implementation, the templates leverage dense features extracted from the Vision Transformers (ViTs), helping to streamline the nearest template retrieval process. Meanwhile, the 2D-to-2D correspondences between patches yield 2D translation, scale, and in-plane rotation without reverting to the 3D model, showcasing the flexibility of their approach.

Evaluation and Implications

The paper details rigorous evaluations on the BOP challenge datasets, demonstrating that GigaPose achieves state-of-the-art results with a significant increase in speed—35 times faster than MegaPose for coarse estimation. The results emphasize GigaPose's robustness, especially in settings with challenging occlusions or inaccurate segmentations, where traditional templates may falter.

Moreover, by integrating with existing refiners, such as those from MegaPose, GigaPose further refines poses, illustrating its versatility across different methodological frameworks. The methodology also extends to using 3D models inferred from a single RGB image, reducing dependency on precise CAD models and opening pathways for efficient deployment in scenarios where CAD data is less accessible.

Future Implications and Developments

The introduction of GigaPose presents a significant advancement in CAD-based pose estimation for novel objects, particularly in industrial and robotics settings. This work prompts further exploration in several areas:

1. Integration with Multi-Modal Systems: The method could potentially benefit from combining visual data with other sensory inputs like depth or thermal imaging, further enhancing robustness.
2. Generalization with Limited Data: Future methods could extend the use of patch correspondences to handle variations in object categories under different illumination or context variations.
3. Real-Time Applications: The rapid inference speed positions GigaPose well for real-time applications, encouraging developments in robotics that necessitate fast and reliable pose estimation.

In conclusion, GigaPose's comprehensive approach to 6D object pose estimation through a combination of innovative template use and robust patch matching sets a new benchmark in efficiency and accuracy, providing a pivotal tool for future AI research and applications in pose estimation.