MR6D: Benchmark for Mobile 6D Pose Estimation

• MR6D is an instance-level 6D pose dataset designed for benchmarking object pose estimation in industrial mobile robotics.
• The dataset features 92 real-world scenes and 16 standardized objects captured under conditions like long-range sensing and heavy occlusion.
• Experimental results highlight segmentation errors as a major bottleneck, underscoring the need for improved 2D segmentation techniques.

Searching arXiv for MR6D and closely related 6D pose-estimation benchmark papers. Searching arXiv for "MR6D Benchmarking 6D Pose Estimation for Mobile Robots". MR6D is an instance-level 6D pose dataset for industrial mobile robotics, introduced to benchmark object pose estimation under the operating conditions of mobile platforms rather than the more common robot-arm manipulation setting. In the 6D pose literature, object pose estimation denotes estimation of an object’s 3D translation in camera coordinates together with its 3D rotation (Pereira et al., 2021). MR6D targets that problem in industrial environments characterized by larger objects, long-range sensing, low and diverse viewpoints, heavy occlusion and self-occlusion, and both static and dynamic interactions, and it is explicitly framed as a response to an evaluation gap in existing benchmarks (Gouda et al., 19 Aug 2025).

1. Motivation and problem setting

MR6D is motivated by the observation that established 6D pose datasets such as LM, T-LESS, ITODD, HB, YCB-V, IC-BIN, TUD-L, HOPE, HANDAL, and HOT3D mainly emphasize small household objects, close-range views, robot arm or tabletop interactions, static well-controlled scenes, and often category-level or household-centric evaluation (Gouda et al., 19 Aug 2025). The paper argues that these conditions are a poor match for mobile industrial robots, which typically operate without manipulators or with specialized grippers, handle larger objects such as pallets and bins, and observe scenes from longer distances and unusual viewpoints.

The benchmark is therefore centered on the perception regime specific to mobile robotics. The targeted challenges include long-range perception, large object sizes, diverse camera viewpoints, occlusion and self-occlusion, similar-looking objects, stacked or closely packed object configurations, static and dynamic interactions, weak or missing depth at long range, and low-angle and mobile-robot viewpoints (Gouda et al., 19 Aug 2025). The resulting problem setting is not category-level recognition in household scenes, but instance-level pose estimation for known objects in industrial and logistics environments.

A central claim of MR6D is that current datasets and pipelines may appear strong on household-style benchmarks while still failing in real mobile robotics settings (Gouda et al., 19 Aug 2025). This claim is operationalized by evaluating unseen-object 6D pose estimation pipelines under sensing conditions that are intentionally harder than standard manipulation datasets because the objects are often much farther away and are viewed from nonstandard angles.

2. Dataset composition and object inventory

MR6D contains 92 real-world scenes and 16 unique objects organized into four subsets: Validation Static, Dynamic Test, O³dyn Static Test, and Mobile Robot-Like (MR) Static (Gouda et al., 19 Aug 2025). The dataset is available at https://huggingface.co/datasets/anas-gouda/mr6d.

The object selection is tailored to mobile industrial robotics. The objects are chosen so that they are not graspable by standard two-finger or suction grippers, and they are also standardized or globally available to support reproducibility (Gouda et al., 19 Aug 2025). The inventory includes a Euro pallet, three Euro-standard KLT storage bins of different sizes and colors, an Amazon Basics suitcase, and 11 IKEA storage items. The paper notes that the IKEA objects are expected to remain available through at least end of 2026.

All objects are associated with high-quality 3D mesh models. The Euro pallet model is manually designed, whereas the other meshes are reconstructed with BundleSDF using Zivid 2 RGB-D capture for accurate geometry (Gouda et al., 19 Aug 2025). This combination of standardized industrial objects and explicit mesh fidelity is important because the benchmark is intended for evaluation of precise 6D pose pipelines rather than coarse category-level localization.

A plausible implication is that MR6D is designed not only as a dataset of images and poses, but as an evaluation substrate for reproducible instance-level industrial perception. The emphasis on globally available objects, accurate geometry, and explicit industrial semantics distinguishes it from household-centered benchmarks.

3. Subsets and acquisition regimes

MR6D is divided into four subsets with distinct sensing and motion characteristics (Gouda et al., 19 Aug 2025).

Subset	Setting	Salient properties
Validation Static	Research hall / hangar	Static objects, tracked moving camera
Dynamic Test	Research hall	Both camera and objects move
O³dyn Static Test	Robot-mounted low-angle camera	Indoors/outdoors, sunlight, weak depth
MR-like Static	Robot-like moving viewpoint	Far-to-near approach sequence

The Validation Static subset is captured in a research hall or hangar with arena size 22 × 10 m², equipped with 55-camera VICON motion capture, and recorded at 30 FPS using an Intel RealSense D435i (Gouda et al., 19 Aug 2025). The camera is tracked by VICON while the objects remain static, and the camera moves on a half-circular path around the objects. Its annotation process uses a marker object to record an approximate object pose, manual refinement with the BOP annotation tool, and projection of refined poses to individual frames using the known camera trajectory. The subset is intended as a high-quality static benchmark.

The Dynamic Test subset is captured in a similar research hall, but with a larger arena of 30 × 15 m² and also at 30 FPS with an Intel RealSense D435i (Gouda et al., 19 Aug 2025). It is the only subset in which both the camera and the objects move. It includes human interaction—moving, transporting, picking, and stowing objects—and is designed to simulate collaborative settings, for example a robot picking an object from a person. Object tracking uses VICON markers, and additional calibration aligns the tracked object frame with the geometric mesh frame.

The O³dyn Static Test subset uses the robot’s built-in Intel RealSense L515 LiDAR camera mounted about 40 cm above the ground, yielding a distinctly low-angle perspective (Gouda et al., 19 Aug 2025). It is recorded indoors and outdoors under varying lighting, including sunlight and shadows, and is described as very challenging because of large detection distances, depth degradation in sunlight, and sometimes nearly empty depth images. Collection sites include industrial-like environments such as a warehouse prototype and workshop floor. Camera trajectory is initially estimated from robot odometry, then recovered or refined with VGGT for multi-view reconstruction; scale is corrected using odometry, and manual refinement is used when odometry is noisy. Final 6D poses are annotated manually.

The MR-like Static subset is also recorded at 30 FPS with an Intel RealSense D435i (Gouda et al., 19 Aug 2025). It uses a moving robot-like viewpoint similar to O³dyn and simulates a wheeled robot such as evoBOT. The motion sequence starts far away, approaches the objects, and ends focusing on one target object, thereby mimicking a robot preparing to grasp an object.

4. Annotation, calibration, and geometric conventions

MR6D uses multiple annotation strategies chosen according to the acquisition regime: VICON motion capture for validation and dynamic subsets; SfM, VGGT, and robot odometry for the robot-mounted subsets; fiducial markers; and manual refinement with the BOP toolkit when necessary (Gouda et al., 19 Aug 2025). This mixed strategy reflects the fact that no single annotation method is equally suitable for tracked indoor halls, dynamic human-object interaction, and robot-mounted low-angle data collected under sunlight and shadow.

Two calibration transformations are formalized in the paper. The first is the eye-in-hand calibration, which aligns the VICON-tracked camera frame to the camera optical frame, written as $^{\text{cam\_optical}T_{\text{cam\_MoCap}$. The second is the object calibration, which aligns the VICON-tracked object frame to the object mesh or geometric frame, written as $^{\text{Obj\_Geo}T_{\text{Obj\_MoCap}$ (Gouda et al., 19 Aug 2025). These calibrations are required especially for the dynamic subset so that tracked poses correspond precisely to the CAD model frame.

The benchmark uses the standard BOP evaluation metrics AR, AR_MSPD, AR_MSSD, and AR_VSD (Gouda et al., 19 Aug 2025). The paper further identifies MSPD as maximum symmetry-aware projection distance, MSSD as maximum symmetry-aware surface distance, and VSD as visible surface discrepancy. These metrics capture different aspects of 6D pose correctness, particularly in the presence of symmetry and occlusion.

This calibration and evaluation design places MR6D squarely within the BOP-style 6D pose ecosystem while changing the sensing regime. The result is methodological continuity at the metric level together with a significant change in the distribution of viewpoints, object scales, and depth quality.

5. Benchmark protocol and experimental findings

MR6D is evaluated as a benchmark for unseen-object 6D pose estimation in mobile-robot contexts (Gouda et al., 19 Aug 2025). The paper reports two experimental pipelines. Setup A uses ground-truth 2D masks and runs FoundationPose for 6D pose estimation, thereby isolating the quality of the pose estimator itself. Setup B uses CTL (Centroid Triplet Loss) for 2D segmentation and identification, followed by FoundationPose, so that both segmentation and pose estimation operate on unseen objects. Segment Anything 2 was considered for segmentation and classification, but CTL was chosen over DINOv2 because of better segmentation performance.

For the three test subsets, the average results with GT masks + FoundationPose are AR = 0.3462, AR_MSPD = 0.3498, AR_MSSD = 0.3210, and AR_VSD = 0.3679 (Gouda et al., 19 Aug 2025). With CTL masks + FoundationPose, the average results are AR = 0.1841, AR_MSPD = 0.1836, AR_MSSD = 0.1722, and AR_VSD = 0.1964. Using predicted masks instead of ground-truth masks therefore causes a large drop, roughly cutting AR almost in half.

The hardest subset is O³dyn. With ground-truth masks it reaches AR = 0.2927, and with CTL masks AR = 0.1673 (Gouda et al., 19 Aug 2025). This is consistent with the subset’s sensing conditions: long-range perception, poor depth, low-angle viewpoints, and outdoor lighting effects that can produce nearly empty depth images.

The principal empirical conclusion is that current 6D pose pipelines generalize only partially to mobile robotics and that 2D segmentation is a major bottleneck (Gouda et al., 19 Aug 2025). The paper attributes this bottleneck to propagation of 2D segmentation errors into pose estimation, over-segmentation of objects into parts by Segment Anything-like models, visual similarity between instances, and clutter and occlusion that make identification difficult. The results therefore suggest that end-to-end performance in mobile robotics can be dominated by segmentation quality even when the underlying pose estimator has some capacity to generalize.

6. Failure modes, significance, and research directions

The paper identifies several qualitative failure cases that persist even with ground-truth masks (Gouda et al., 19 Aug 2025). One is occlusion-induced misidentification, in which the model predicts the pose of a nearby object instead of the target. A second is confusion caused by closely stacked similar objects. A third is orientation errors under partial visibility, where a visible dominant face together with similar nearby textures leads to a completely wrong orientation. These cases show that the challenge is not reducible to segmentation alone; geometric ambiguity under realistic occlusion remains a substantive obstacle.

MR6D is significant because it shifts 6D pose benchmarking toward the actual operating regime of mobile robots: larger objects, industrial settings, long-range sensing, low and diverse viewpoints, motion and collaboration with humans, and heavy occlusion together with poor depth (Gouda et al., 19 Aug 2025). The benchmark is therefore directly relevant to warehouse automation, pallet handling, mobile manipulation, logistics robots, and industrial object localization. Its core contribution is not only a new dataset, but a redefinition of the benchmark regime for 6D pose estimation.

The paper also outlines several future directions. One is better 2D segmentation for unseen objects, potentially supported by background removal and monocular depth estimation (Gouda et al., 19 Aug 2025). Another is entity-level segmentation, based on the argument that objects should be segmented as whole physical entities rather than broken into parts; this is especially important for pallets, bins, suitcases, and storage containers. The authors additionally suggest evaluation metrics tailored to mobile robotics, for example BOP-style metrics that penalize pose errors according to tracking distance, since close objects matter more in grasping and far objects may be less critical for immediate action. Finally, MR6D opens work on long-range 6D pose, low-angle and mobile-view pose estimation, occlusion and self-occlusion robustness, dynamic human-object interactions, and industrial instance-level pose estimation.

Taken together, MR6D defines a benchmark in which industrial mobile robots observe large known objects at distance, from challenging viewpoints, under clutter, motion, and weak depth (Gouda et al., 19 Aug 2025). The reported experiments indicate that existing unseen-object pose pipelines are promising but far from sufficient, and that segmentation is a major limiting factor. This suggests that progress on MR6D will depend not only on stronger pose regressors, but also on perception systems that are explicitly designed for mobile sensing geometry and industrial scene structure.