A Dataset for Improved RGBD-based Object Detection and Pose Estimation for Warehouse Pick-and-Place (1509.01277v2)

Abstract: An important logistics application of robotics involves manipulators that pick-and-place objects placed in warehouse shelves. A critical aspect of this task corre- sponds to detecting the pose of a known object in the shelf using visual data. Solving this problem can be assisted by the use of an RGB-D sensor, which also provides depth information beyond visual data. Nevertheless, it remains a challenging problem since multiple issues need to be addressed, such as low illumination inside shelves, clutter, texture-less and reflective objects as well as the limitations of depth sensors. This paper provides a new rich data set for advancing the state-of-the-art in RGBD- based 3D object pose estimation, which is focused on the challenges that arise when solving warehouse pick- and-place tasks. The publicly available data set includes thousands of images and corresponding ground truth data for the objects used during the first Amazon Picking Challenge at different poses and clutter conditions. Each image is accompanied with ground truth information to assist in the evaluation of algorithms for object detection. To show the utility of the data set, a recent algorithm for RGBD-based pose estimation is evaluated in this paper. Based on the measured performance of the algorithm on the data set, various modifications and improvements are applied to increase the accuracy of detection. These steps can be easily applied to a variety of different methodologies for object pose detection and improve performance in the domain of warehouse pick-and-place.

• The paper introduces the Rutgers APC RGBD Dataset as its main contribution to advance robotic perception in complex warehouse environments.
• It meticulously annotates over 10,000 images with both RGB and depth data, including precise 6DOF object poses, to evaluate detection algorithms.
• The evaluation demonstrates enhancements such as masking, post-processing, and temporal smoothing to boost pose estimation reliability in cluttered settings.

Improved RGBD-Based Object Detection in Warehouse Automation

This paper presents a comprehensive dataset aimed at advancing RGBD-based object detection and pose estimation in warehouse automation tasks, particularly focusing on the challenges inherent in pick-and-place operations inside shelving units. The dataset, named the Rutgers APC RGBD Dataset, serves as a valuable resource for researchers in the field to test and improve robotic perception algorithms effectively.

Overview

In warehouse environments, the accurate identification and manipulation of objects within shelves is paramount. The utilization of RGBD sensors, which offer both color and depth information, presents a promising approach to address these challenges. However, real-world applications often encounter various issues such as low illumination, clutter, texture-less and reflective objects, and limitations in depth sensor range. This dataset provides a controlled environment to paper these challenges, containing over 10,000 images with corresponding ground truth data for objects used during the Amazon Picking Challenge (APC).

Dataset Features

The Rutgers APC RGBD Dataset stands out for its extensive image collection, capturing objects in diverse poses and clutter arrangements across warehouse shelf bins. Such diversity includes images in clutter-free scenarios, as well as ones with varying degrees and types of additional objects. Alongside RGB images, depth data is meticulously annotated with 6DOF object poses. This rich labeling is intended for evaluating and improving algorithms that aim to perform reliably in challenging environments characterized by darkness and clutter.

Evaluation and Potential Enhancements

The paper provides an example evaluation using the LINEMOD algorithm, illustrating the dataset's utility in assessing pose estimation performance. As outlined, using the dataset allows researchers to pinpoint deficiencies in current algorithms, especially in cluttered shelf environments and with texture-less or transparent objects.

Several proposed enhancements for pose detection with algorithms like LINEMOD are discussed, including:

• Masking: Utilizes precise shelf location calibration to improve detection reliability by isolating the bin of interest.
• Post-processing: Implements quadrant-based hue-saturation histogram comparisons to refine object orientation estimates.
• Temporal Smoothing: Aggregates multiple frame estimates to counteract sensor noise, employing a quality measure that favors the most consistent pose hypothesis.

These improvements provide a basis for enhancing the robustness of pose estimation systems in warehouse tasks by increasing the detection rate and refining pose accuracy.

Implications and Future Work

The dataset promotes the development of pose detection algorithms that harness both RGB and depth data effectively for varied objects and positions within warehouse settings. The availability of 3D models alongside depth images invites exploration into machine learning approaches, 3D reconstruction techniques, and potential algorithmic fusion strategies that could enhance perception capabilities significantly.

Notably, challenges such as handling occlusions, leveraging cloud computation for uncertainty management, and integrating classical camera-based methods highlight potential areas for further research. The application of novel machine learning frameworks could open new pathways for intelligent and adaptive robotic systems dealing with complex object recognition and manipulation in logistics environments.

Conclusion

The Rutgers APC RGBD Dataset is a pivotal contribution to the field of robotic perception, presenting a rich source of data to foster innovation in warehouse automation. By addressing specific challenges through targeted dataset controls and illustrative algorithm evaluations, it lays the groundwork for significant advancements in object detection and pose estimation strategies, ultimately aiming to enhance efficiency and reliability in automated warehouse operations.