SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (2509.21927v1)

Abstract: Recent 6D pose estimation methods demonstrate notable performance but still face some practical limitations. For instance, many of them rely heavily on sensor depth, which may fail with challenging surface conditions, such as transparent or highly reflective materials. In the meantime, RGB-based solutions provide less robust matching performance in low-light and texture-less scenes due to the lack of geometry information. Motivated by these, we propose SingRef6D, a lightweight pipeline requiring only a single RGB image as a reference, eliminating the need for costly depth sensors, multi-view image acquisition, or training view synthesis models and neural fields. This enables SingRef6D to remain robust and capable even under resource-limited settings where depth or dense templates are unavailable. Our framework incorporates two key innovations. First, we propose a token-scaler-based fine-tuning mechanism with a novel optimization loss on top of Depth-Anything v2 to enhance its ability to predict accurate depth, even for challenging surfaces. Our results show a 14.41% improvement (in $\delta_{1.05}$) on REAL275 depth prediction compared to Depth-Anything v2 (with fine-tuned head). Second, benefiting from depth availability, we introduce a depth-aware matching process that effectively integrates spatial relationships within LoFTR, enabling our system to handle matching for challenging materials and lighting conditions. Evaluations of pose estimation on the REAL275, ClearPose, and Toyota-Light datasets show that our approach surpasses state-of-the-art methods, achieving a 6.1% improvement in average recall.

• The paper presents a novel method that estimates 6D object poses using only a single RGB reference, bypassing the need for CAD models and expensive depth sensors.
• It employs a token-scaler approach in DPAv2 combined with a depth-aware matching process via a modified LoFTR to enhance spatial context and accuracy.
• Experimental results show a 6.1% increase in average recall on datasets like REAL275, demonstrating improved performance especially with transparent and low-texture surfaces.

SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference

This paper introduces a novel approach to 6D object pose estimation, emphasizing a minimalistic requirement of only a single RGB image as a reference for new objects. The methodology leverages a fine-tuned version of Depth-Anything v2 (DPAv2) and a depth-aware matching process integrated with LoFTR for enhanced pose estimation.

Introduction

Traditional 6D pose estimation methods often depend on accurate depth sensors and CAD models, which can be limited by transparency and reflectivity issues, or substantial computational and training overhead. SingRef6D circumvents these limitations by bypassing expensive depth sensors and multi-view setups. It employs a lightweight pipeline for depth prediction and matching, designed to be resource-efficient and applicable under constrained scenarios.

Methodology

Depth Prediction with Token-Scaler

At the core of SingRef6D is an innovative depth prediction mechanism based on DPAv2. The paper introduces a token-scaler, a novel fine-tuning approach that adapts the depth model to enhance feature accuracy across varied surface conditions, including transparent materials.

Figure 1: Visualized pipeline for inference; the depth model estimates metric depth, and depth-aware matching establishes robust correspondences.

The token-scaler mechanism adjusts features across multiple hierarchy levels in DPAv2, emulating human depth perception and allowing for consistent depth predictions.

Loss Function Design

The optimization of depth prediction involves a comprehensive loss scheme:

• Global Loss: A combination of scale-shift invariant (SSI) and BerHu loss is used to penalize large residuals effectively.
• Local Losses: These include scale alignment, edge-emphasize, and normal consistency measures that guide the model to maintain edge definition and align surfaces accurately.

Depth-Aware Matching

The second stage of SingRef6D integrates depth data into the matching process. LoFTR is modified to include depth values in its matching strategy, leading to improved spatial context understanding and increased accuracy, especially in low-texture areas.

Experimentation and Results

SingRef6D was evaluated on three datasets: REAL275, ClearPose, and Toyota-Light. It demonstrated significant improvements over traditional methods, achieving a 6.1% increase in average recall for pose estimation.

Figure 2: Depth prediction and projected point clouds' comparisons; the proposed method handles transparency and scale better.

In depth estimation tasks, the model consistently showed enhanced $\delta_{1.05}$ accuracy metrics compared to baselines like Unidepth and vanilla DPAv2.

Figure 3: Comparison with other depth estimation models highlights clear depth map outputs with retained details.

Comparison with Other Methods

The paper provides a detailed comparison of input data requirements and computational overhead against other contemporary methodologies. SingRef6D outperforms its counterparts in scenarios with limited resources by eliminating the need for extensive novel view synthesis or deep neural fields.

Trade-Offs and Limitations

While SingRef6D presents an efficient approach, its reliance on RGB segmentation masks holds back its generalizability. Moreover, the model's performance depends on the pre-trained capabilities of DPAv2 and LoFTR, limiting its applicability under extreme lighting conditions.

Conclusion

SingRef6D offers a robust framework for pose estimation using minimal references. By aligning depth perception with human-like spatial reasoning, it lays the groundwork for efficient real-world applications without extensive computational demands or data requirements.

As future directions, expanding the model's domain could involve integrating vision-LLMs for improved object localization and applying the token scaler for broader vision-transformer applications in low-resource settings.

Figure 4: Predicted 6D poses highlight reduced rotation errors and translation shifts across datasets.