Detection, localisation and tracking of pallets using machine learning techniques and 2D range data (1803.11254v3)

Abstract: The problem of autonomous transportation in industrial scenarios is receiving a renewed interest due to the way it can revolutionise internal logistics, especially in unstructured environments. This paper presents a novel architecture allowing a robot to detect, localise, and track (possibly multiple) pallets using machine learning techniques based on an on-board 2D laser rangefinder only. The architecture is composed of two main components: the first stage is a pallet detector employing a Faster Region-based Convolutional Neural Network (Faster R-CNN) detector cascaded with a CNN-based classifier; the second stage is a Kalman filter for localising and tracking detected pallets, which we also use to defer commitment to a pallet detected in the first stage until sufficient confidence has been acquired via a sequential data acquisition process. For fine-tuning the CNNs, the architecture has been systematically evaluated using a real-world dataset containing 340 labeled 2D scans, which have been made freely available in an online repository. Detection performance has been assessed on the basis of the average accuracy over k-fold cross-validation, and it scored 99.58% in our tests. Concerning pallet localisation and tracking, experiments have been performed in a scenario where the robot is approaching the pallet to fork. Although data have been originally acquired by considering only one pallet as per specification of the use case we consider, artificial data have been generated as well to mimic the presence of multiple pallets in the robot workspace. Our experimental results confirm that the system is capable of identifying, localising and tracking pallets with a high success rate while being robust to false positives.

• The paper introduces a novel approach using CNNs and Kalman filtering to detect, localise, and track pallets with high accuracy.
• The methodology converts 2D laser scans into bitmaps and applies Faster R-CNNs for precise region proposals and classification.
• The research achieves 99.58% detection accuracy, providing a robust and efficient solution for autonomous mobile robotics in industrial environments.

Detection, Localisation, and Tracking of Pallets Using Machine Learning Techniques and 2D Range Data

The paper presents a novel approach for the deployment of autonomous vehicles in industrial environments focusing on the critical task of detecting, localising, and tracking pallets using a 2D laser rangefinder and machine learning techniques. The research leverages advancements in convolutional neural networks (CNNs), particularly Faster Region-based CNNs (Faster R-CNNs), alongside established techniques in object tracking to provide a comprehensive, autonomous solution suited to dynamic and unstructured environments.

System Architecture and Methodology

The proposed architecture is delineated into three primary phases: data acquisition, neural network training, and online object tracking. The approach begins with the acquisition of laser scans, converting these data points into 2D bitmaps that serve as input for further processing. The first phase employs a Faster R-CNN to propose regions of interest (ROIs) in the generated images, identifying potential pallet candidates. Following this, a secondary CNN operates on these ROIs to classify them into pallet and non-pallet categories.

For tracking, a Kalman filter is utilized to predict the motion of detected pallets. The algorithm offers a significant innovation by introducing a sequential classification process, evaluating continued evidence rather than making immediate decisions. This deferment mechanism mitigates the risk of false positives and ensures robustness, especially in cluttered environments with dynamic obstacles.

Numerical Results

The system’s efficacy is evident in the achieved detection accuracy of 99.58% as determined via k-fold cross-validation during CNN training on a dataset comprising 340 labeled 2D scans. The research demonstrates the potential for detecting multiple pallets concurrently and confirms the system's capability to function as a real-time application under the operating conditions defined in an industrial warehouse located in Tortona, Italy.

Implications and Future Directions

The practical implications of this research are substantial for the domain of autonomous mobile robotics in industrial settings. The methodology reduces dependency on a priori knowledge of the environment, which aligns with the increasing demand for flexible automated systems under the Industry 4.0 paradigm. Moreover, the data-driven approach potentially reduces the typical challenges faced with model-based strategies that are often limited by predefined structural assumptions of the environment.

From a theoretical perspective, the application of CNNs to 2D rangefinder data reveals new avenues for machine learning research in sparse and noisy data environments. The success of this method may prompt further exploration into the integration of alternative machine learning techniques suitable for 2D sensors.

Looking ahead, the authors suggest improvements in orientation estimation, system optimization for enhanced operational frequencies, and adaptation for broader object recognition tasks. Additionally, enhancements such as unique object identification within a simultaneous tracking scenario could further expand its applicability and impact.

In conclusion, this research provides a comprehensive framework for pallet detection and tracking using minimal sensor data, achieving a balance between detection accuracy and computational efficiency, and highlights promising advancements in the practical application of machine learning in industrial automation.