SingRef6D: Minimal-Reference Monocular 6D Pose

• SingRef6D is a monocular 6D pose estimation pipeline that uses a single RGB image to estimate both rotation and translation without needing CAD models or multi-view data.
• It incorporates a token-scaler-based fine-tuning strategy that enhances depth prediction, achieving a 14.41% improvement in accuracy on the REAL275 dataset.
• The integration of depth-aware LoFTR matching yields robust keypoint correspondences, improving average recall by 6.1% in challenging sensing environments.

SingRef6D is a monocular 6D pose estimation pipeline which requires only a single RGB reference image for each target object. In contrast to traditional pose estimation methods that depend on precise CAD models, sensor depth input, multi-view acquisition, or volumetric neural field synthesis, SingRef6D operates under a strict minimal reference constraint. Its architecture incorporates two major innovations: a token-scaler-based fine-tuning strategy for improved monocular depth prediction and a depth-aware matching process that integrates spatial relationships within the LoFTR correspondence framework. These advances yield robust pose predictions for objects with challenging surface properties and make SingRef6D particularly suitable for resource-limited or adverse sensing scenarios.

1. Minimal Reference Monocular 6D Pose Estimation

SingRef6D is designed to estimate the full 6D pose (three rotational and three translational degrees of freedom) using only a single monocular RGB reference per object category. This eliminates the need for dense 3D geometry input (such as CAD meshes), multi-view data collection, or specialized sensors (e.g., time-of-flight, structured light). The framework is tailored for cases where only a single image of an object is available, facilitating ease of deployment in factory automation, AR, or robotics, especially for objects with reflective or transparent surfaces where depth sensors often fail.

2. Token-Scaler-Based Fine-Tuning for Depth Prediction

Depth estimation in SingRef6D begins with Depth-Anything v2 (DPAv2), a state-of-the-art monocular depth predictor. Recognizing its deficiencies on challenging surfaces (incorrect scales, boundary blur, normal misalignment), SingRef6D introduces a token scaler network which dynamically reweights transformer-layer features at multiple scales. Specifically, for four abstraction levels, local and global features $F_l$ are adaptively fused:

$F'_l = \mathcal{F}_l(F_l, \text{Scaler}(F'_{l+1}))$

where low-level features benefit from localized attention and high-level features incorporate scene context via InceptConv, yielding improved scene- and object-scale recovery.

Training leverages a composite loss:

• Global scale alignment (scale-shift-invariant, BerHu penalty, regularization)
• Local edge consistency (edge loss)
• Surface normal consistency

This yields a relative improvement of 14.41% in $\delta_{1.05}$ threshold accuracy on the REAL275 dataset compared to a fine-tuned DPAv2 baseline. Accurate monocular metric depth is fundamental for robust pose matching and scale recovery.

3. Depth-Aware Matching with LoFTR Integration

Traditional RGB-only matching is susceptible to false correspondence—especially with objects lacking texture or in adverse lighting. SingRef6D extends LoFTR, a transformer-based dense matching model, by fusing DPAv2-predicted depth with RGB features. Depth cues are embedded at multiple levels, augmenting latent correspondences with spatial metrics. This enables reliable keypoint matching even in low-light or textureless regions and distinguishes foreground-background ambiguities that are challenging in monocular vision.

During feature matching, LoFTR weights remain frozen, preserving its established representation power, while the depth-aware fusion improves robustness to material and illumination variation. The overall effect is a denser and more accurate set of correspondences feeding the pose computation module.

4. Quantitative Benchmark Evaluation

SingRef6D achieves demonstrable improvements over prior approaches:

• Depth prediction: 14.41% improvement in $\delta_{1.05}$ threshold compared to DPAv2 (fine-tuned head)
• Pose estimation: 6.1% gain in average recall (AR) over state-of-the-art matching methods on REAL275, with further improvements of +15.3% over SIFT-based techniques
• On ClearPose, which contains transparent objects, depth accuracy rises from 31.23% to 54.30% ($\delta_{1.05}$)
• On Toyota-Light, competitive results are achieved, marginally lower than text-prompted methods but with much lower computation and resource demand

The core metrics employed are threshold accuracy ($\delta_x$), RMSE, Abs.Rel., Sq.Rel. for depth prediction, and VSD, MSSD, MSPD, ADD for pose estimation, following common evaluation standards.

5. Applications

SingRef6D is suited for scenarios where dense depth, multiple views, or full 3D models are impractical, such as:

• Industrial and warehouse robotics, enabling pose estimation for transparent or reflective goods
• Augmented reality deployment in consumer-grade devices (camera-only hardware)
• Mobile manipulation or inspection tasks in field robotics, especially under adverse lighting or limited vantage points

The computational efficiency (low FLOPs, small model size) enables deployment on edge devices and in real-time robotics pipelines.

6. Limitations and Considerations

Current limitations include:

• Requirement for an object mask or segmentation at inference: Without reliable segmentation, initial localizations may fail.
• Dependency on the quality of frozen DPAv2 and LoFTR weights: Scenes or objects substantially different from training domains may degrade performance.
• Possible failure in extremely dark scenes with little RGB signal, and reliance on accurate token-scaler calibration for the specific sensor/camera setup.

Performance is bounded by the accuracy of pre-trained backbones and the diversity of training data. Potential future research directions may include self-supervised mask estimation, joint backbone fine-tuning, or adaptation for articulated objects.

7. Comparative Advantages

SingRef6D offers a unique blend of robustness (via improved depth and spatial-aware matching) and efficiency (minimal reference, low computational cost). When evaluated against volumetric or neural field alternatives, its pipeline requires much less input and infrastructure (no CAD/model synthesis, no multi-view aggregation). This positions SingRef6D as a practical method for real-world pose estimation in constrained environments.

Method	Required Reference	Depth Use	Keypoint Matching	AR/Accuracy Improvement
SingRef6D	1 RGB image	Enhanced, token-scaled DPAv2	Depth-aware LoFTR	+6.1% AR over SOTA
Oryon, RoMA	Multi-view/CAD	Dense/Neural fields	Volumetric/semantic	Lower AR, higher FLOPs
SIFT-based	1 RGB (classical)	N/A	Appearance only	Baseline

This table summarizes reference requirements, matching strategy, and performance as described in the benchmark analysis.

SingRef6D marks a significant contribution for single-image-referenced, monocular object pose estimation, particularly for objects and environments where acquiring depth is difficult or precision CAD models are unavailable (Wang et al., 26 Sep 2025).