Robotic Autonomous Trolley Collection with Progressive Perception and Nonlinear Model Predictive Control (2110.06648v2)

Abstract: Autonomous mobile manipulation robots that can collect trolleys are widely used to liberate human resources and fight epidemics. Most prior robotic trolley collection solutions only detect trolleys with 2D poses or are merely based on specific marks and lack the formal design of planning algorithms. In this paper, we present a novel mobile manipulation system with applications in luggage trolley collection. The proposed system integrates a compact hardware design and a progressive perception and planning framework, enabling the system to efficiently and robustly collect trolleys in dynamic and complex environments. For the perception, we first develop a 3D trolley detection method that combines object detection and keypoint estimation. Then, a docking process in a short distance is achieved with an accurate point cloud plane detection method and a novel manipulator design. On the planning side, we formulate the robot's motion planning under a nonlinear model predictive control framework with control barrier functions to improve obstacle avoidance capabilities while maintaining the target in the sensors' field of view at close distances. We demonstrate our design and framework by deploying the system on actual trolley collection tasks, and their effectiveness and robustness are experimentally validated.

• The paper presents an advanced robotic system that autonomously collects airport trolleys using progressive perception and NMPC for precise control.
• It employs a bifurcated perception approach with YOLOv5 for initial detection and LiDAR for fine localization, achieving centimeter-level accuracy.
• The NMPC framework, enhanced by control barrier functions, ensures reliable obstacle avoidance and robust performance in dynamic, cluttered environments.

Robotic Autonomous Trolley Collection with Progressive Perception and Nonlinear Model Predictive Control

The research paper at hand presents an advanced mobile manipulation system designed for the autonomous collection of trolleys in airport environments. The system integrates a sophisticated perception pipeline and a model predictive control framework to enable precise navigation and manipulation tasks in dynamic and cluttered spaces. Through innovative mechanical design and a robust autonomy framework, the presented robot addresses significant challenges associated with trolley retrieval operations.

System Overview

The robotic system is comprised of three key components: a mechanical chassis, an array of sensors, and a specialized manipulator for trolley collection. The chassis, consisting of independent driving wheels and a suspension system for universal wheels, ensures stable locomotion across uneven surfaces. The sensor suite, including multi-modal LiDARs and RGB-D cameras, provides comprehensive environmental perception necessary for both long and short-range object detection. The manipulator, featuring a feedback-enabled fork design, facilitates secure capture and transport of the trolleys.

Perception Methodology

The perception framework consists of a bifurcated approach to handle varying distances from the target. At longer distances, a monocular vision-based method combines 2D object detection (utilizing a trained YOLOv5 model) with keypoint localization to determine the 3D pose of trolleys. This step is crucial for initial detection and rough localization. For fine localization at close proximity, a LiDAR-based plane detection method is employed, ensuring centimeter-level accuracy in trolley pose estimation. The efficacy of the perception system is demonstrated through its ability to maintain accuracy despite environmental challenges such as occlusions and lighting variations.

Nonlinear Model Predictive Control (NMPC)

The motion planning strategy leverages a nonlinear model predictive control framework, augmented with control barrier functions (CBFs) to enforce safety constraints. This ensures robust obstacle avoidance and maintains the target object within the sensor's field of view throughout the docking process. The NMPC framework is formulated to optimize trajectory costs while adhering to dynamic constraints, thereby producing efficient and collision-free paths. The application of CBFs offers enhanced safety guarantees by maintaining state constraints that prevent collision with dynamic obstacles and ensure continuous monitoring of the target trolley.

Experimental Validation

The implementation of the system in real-world scenarios demonstrates the robustness and applicability of the developed platform. The robot exhibited autonomous operation capabilities, successfully navigating complex environments, accurately docking with target trolleys, and transporting them to specified locations. The experiments validate the effectiveness of the perception and control mechanisms in adapting to dynamic changes and human interaction in airport settings.

Implications and Future Directions

This work contributes to the growing field of autonomous robotic manipulation, presenting a comprehensive solution for efficient trolley retrieval. The integration of progressive perception with NMPC frameworks expands the utility of autonomous robots in dense, real-world environments, highlighting potential applications beyond airport trolley collection, such as warehouse logistics and urban service robotics.

Future research directions could focus on enhancing the system's scalability and expanding its decision-making abilities for multi-robot coordination and cooperative transport tasks. Additionally, exploring adaptive learning-based approaches could further enhance the robot's capacity to understand and interact within diverse operational contexts, thus broadening its deployment scope.

In conclusion, this paper presents a significant advancement in robotic autonomous systems by effectively addressing the challenges posed by dynamic and complex environments, ensuring accuracy, safety, and efficiency in autonomous trolley collection operations.