Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments (2005.13857v1)

Abstract: Deep Reinforcement Learning has been successfully applied in various computer games [8]. However, it is still rarely used in real-world applications, especially for the navigation and continuous control of real mobile robots [13]. Previous approaches lack safety and robustness and/or need a structured environment. In this paper we present our proof of concept for autonomous self-learning robot navigation in an unknown environment for a real robot without a map or planner. The input for the robot is only the fused data from a 2D laser scanner and a RGB-D camera as well as the orientation to the goal. The map of the environment is unknown. The output actions of an Asynchronous Advantage Actor-Critic network (GA3C) are the linear and angular velocities for the robot. The navigator/controller network is pretrained in a high-speed, parallel, and self-implemented simulation environment to speed up the learning process and then deployed to the real robot. To avoid overfitting, we train relatively small networks, and we add random Gaussian noise to the input laser data. The sensor data fusion with the RGB-D camera allows the robot to navigate in real environments with real 3D obstacle avoidance and without the need to fit the environment to the sensory capabilities of the robot. To further increase the robustness, we train on environments of varying difficulties and run 32 training instances simultaneously. Video: supplementary File / YouTube, Code: GitHub

Deep Reinforcement Learning for Real Autonomous Mobile Robot Navigation in Indoor Environments

The paper under review explores the advancement of deep reinforcement learning (DRL) methodologies applied to autonomous mobile robots operating in indoor environments. The research addresses the challenging domain of real-world robot navigation where traditional reinforcement and deep learning approaches often falter due to safety, robustness, and reliance on structured settings.

Key Contributions and Methods

The authors propose a system utilizing an Asynchronous Advantage Actor-Critic (GA3C) network for autonomous navigation in unknown environments. The innovation in this approach lies in autonomous navigation devoid of a prior map or planner, leveraging only sensory data from a 2D laser scanner and an RGB-D camera. The fusion of these sensors captures comprehensive environmental data which, when processed through the GA3C network, generates linear and angular velocity commands to guide robot movement.

Significantly, the GA3C network is initially trained in a simulation environment designed to be highly parallel and efficient. This simulation phase allows for rapid accumulation of training experiences, addressing a common bottleneck in reinforcement learning. By introducing randomness through Gaussian noise and varying environmental complexities during training, the authors strive to enhance network robustness and avoid overfitting. The final integration of the network onto the physical robot ensures that it operates effectively in real-world conditions.

Results and Evaluation

The research presents training results indicating efficient navigation capabilities with a collision-free navigation rate of 95% in complex scenarios. The parallel simulation technique employed enables fast learning, a distinct improvement over conventional simulators like Gazebo in terms of speed—claimed to be up to 1000 times faster. Despite achieving high performance during the simulated training, the real-world deployment of the model further verifies the feasibility and effectiveness of the proposed system, showcasing commendable obstacle avoidance and generalization ability.

Theoretical Implications and Future Directions

The research contributes to the literature on autonomous mobile robots by demonstrating a viable pathway for transitioning from simulation-based training to real-world deployment. By addressing the need for extensive training data and providing a framework for rapid environmental simulation, the paper enhances the practical applicability of DRL in robotics.

Looking ahead, the authors suggest the exploration of alternative network architectures like DDPG and LSTMs, as well as the potential for multi-robot coordination (swarm robotics) to extend the application of their framework. These areas hold promise for expanding both the capability and complexity of autonomous systems navigating in real environments.

In summary, this paper sheds light on a sophisticated DRL framework integrated with high-fidelity simulation capabilities for enhancing robot autonomy. The combination of robust training methodologies with sensor fusion positions this paper as a substantive contribution to the ongoing evolution of robotics, offering a foundation for future explorations in autonomous navigation.