Deep Drone Racing: Learning Agile Flight in Dynamic Environments (1806.08548v3)

Abstract: Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired speed. This information is then used by the planner to generate a short, minimum-jerk trajectory segment and corresponding motor commands to reach the desired goal. We demonstrate our method in autonomous agile flight scenarios, in which a vision-based quadrotor traverses drone-racing tracks with possibly moving gates. Our method does not require any explicit map of the environment and runs fully onboard. We extensively test the precision and robustness of the approach in simulation and in the physical world. We also evaluate our method against state-of-the-art navigation approaches and professional human drone pilots.

• The paper proposes a hybrid CNN and trajectory planning framework that predicts waypoints from raw images for agile flight in dynamic racing environments.
• The paper validates the approach through extensive simulations and real-world tests, showing competitive performance against state-of-the-art baselines and human pilots.
• The paper highlights future directions in few-shot learning and deeper control integration to optimize high-speed agility and model generalization.

Agile Drone Flight Through Deep Learning in Dynamic Environments

The paper under consideration presents a methodology for autonomous drone flight, specifically focusing on vision-based navigation through dynamic environments for drone racing. The research addresses a significant challenge in robotics: enabling a quadrotor to navigate at high speeds using onboard computations without reliance on static environmental maps.

Methodological Framework

The methodology involves a hybrid approach that integrates a Convolutional Neural Network (CNN) with advanced path-planning and control systems. The CNN processes raw camera images to predict waypoints and desired speeds. The outputs are leveraged by a trajectory planning module to generate minimum-jerk trajectories, which translate into precise motor commands for agile drone maneuvering.

The architecture's robustness stems from its ability to avoid comprehensive global mapping, relying on localized, body-frame data instead. This approach minimizes the computational burden and enhances the system's adaptability to dynamic obstacles and environments, as often encountered in professional drone racing contexts.

Experimental Evaluation

The researchers validated their approach through extensive simulations and real-world experiments. Notably, the CNN was trained through imitation learning in simulation, tracking optimal trajectories in varied static conditions, which surprisingly generalized well to dynamic settings. This adaptability was demonstrated by the quadrotor's ability to contend with dynamically moving gates, a scenario where traditional navigation systems falter.

Simulated experiments revealed that the method could sustain high performance across a range of speeds, though performance naturally tapered with increasing velocity due to platform dynamics. In real-world scenarios, the proposed system outperformed state-of-the-art visual-inertial odometry baselines and exhibited competitive performance against human pilots, although the latter demonstrated quicker lap times at the expense of failure rates.

Implications and Future Work

This research has notable implications in the field of robotics and unmanned aerial vehicle navigation, particularly in environments that require high-speed adaptability and precision. While the system performs commendably, there is room for further optimization in the areas of high-speed agility and the generalization of learned models to new, unseen environments.

The authors suggest several future directions, including reducing the necessity for extensive pre-collected training datasets through techniques like few-shot learning. Moreover, integrating learning more deeply into the control system could further enhance the navigation capabilities for complex maneuvers.

In conclusion, this paper presents a compelling vision-based flight control system capable of navigating highly dynamic environments efficiently. The integration of CNNs for perception with advanced trajectory generation marks a notable stride toward fully autonomous drone racing. This work lays a foundation for future advancements in building more adaptive, perception-driven robotic systems.