Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations (1909.06993v2)

Abstract: Machines are a long way from robustly solving open-world perception-control tasks, such as first-person view (FPV) aerial navigation. While recent advances in end-to-end Machine Learning, especially Imitation and Reinforcement Learning appear promising, they are constrained by the need of large amounts of difficult-to-collect labeled real-world data. Simulated data, on the other hand, is easy to generate, but generally does not render safe behaviors in diverse real-life scenarios. In this work we propose a novel method for learning robust visuomotor policies for real-world deployment which can be trained purely with simulated data. We develop rich state representations that combine supervised and unsupervised environment data. Our approach takes a cross-modal perspective, where separate modalities correspond to the raw camera data and the system states relevant to the task, such as the relative pose of gates to the drone in the case of drone racing. We feed both data modalities into a novel factored architecture, which learns a joint low-dimensional embedding via Variational Auto Encoders. This compact representation is then fed into a control policy, which we trained using imitation learning with expert trajectories in a simulator. We analyze the rich latent spaces learned with our proposed representations, and show that the use of our cross-modal architecture significantly improves control policy performance as compared to end-to-end learning or purely unsupervised feature extractors. We also present real-world results for drone navigation through gates in different track configurations and environmental conditions. Our proposed method, which runs fully onboard, can successfully generalize the learned representations and policies across simulation and reality, significantly outperforming baseline approaches. Supplementary video: https://youtu.be/VKc3A5HlUU8

• The paper introduces a novel cross-modal VAE framework that bridges the sim-to-real gap in drone navigation by learning invariant latent representations.
• The methodology combines supervised and unsupervised data to generate robust visual and state features for effective policy training via imitation learning.
• Experimental results in both simulation and real-world flights demonstrate superior performance, especially in handling dynamic environmental challenges.

Overview of Learning Visuomotor Policies for Aerial Navigation Using Cross-Modal Representations

The paper by Bonatti et al. presents a methodology for training visuomotor policies for aerial navigation tasks, particularly focusing on sim-to-real transfers in contexts like drone racing. The authors address the limitations of using either purely simulated or real-world data to train such policies, advocating for a novel use of cross-modal representations that leverage simulated data exclusively, thereby mitigating the challenges of data collection and labeling typical in real-world environments.

Methodology and Approach

The proposed approach is built around a cross-modal learning architecture using Variational Auto Encoders (VAEs) to generate compact representations that are invariant to visual and environmental variabilities. This is crucial for bridging the sim-to-real gap. The architecture combines supervised data - such as explicit state information like the relative pose of obstacles - with unsupervised data from raw sensory inputs. Jointly encoding these modalities provides a robust state representation for training the control policies.

In practical terms, the approach involves two main stages:

1. State Representation Learning: The system employs a cross-modal VAE framework to construct a low-dimensional latent space, synthesizing different data modalities. This ensures smooth and continuous encoding of essential task-specific information, such as gate positions in drone racing scenarios.
2. Policy Training: Using this latent space, the authors train control policies via imitation learning from expert trajectories generated in simulation. This avoids the sample complexity challenges faced by end-to-end learning models that operate directly on high-dimensional sensory inputs.

Experimental Validation

The authors validate their approach through extensive simulation and real-world experiments. Key findings indicate that policies utilizing cross-modal latent representations significantly outperform those relying on traditional unsupervised representations or direct sensor-to-velocity mappings.

• Simulated Environment Testing: Different latent architectures, including constrained and unconstrained versions, were tested for navigating a simulated track with stochastic gate positions. Policies built on cross-modal representations demonstrated superior performance and robustness over increasing levels of track difficulty.
• Real-World Deployment: To showcase generalization capabilities, the learned policies were deployed on physical drone platforms navigating real-world tracks. The cross-modal representation strategy enabled successful flight across diverse conditions, including adverse weather scenarios unseen during training.

Implications and Future Directions

The implications of this research are twofold. Practically, the ability to train robust navigation policies without real-world data collection presents a significant advancement in deploying autonomous systems in dynamic, complex environments. Theoretically, the cross-modal approach offers a pathway towards more generalized robotics policies capable of operating across diverse domains without extensive redesigns.

Future developments could extend cross-modal representation frameworks to integrate additional sensor modalities beyond visual data, potentially enhancing robustness and performance in varied operational contexts. Moreover, investigating adversarial training methods to minimize domain discrepancies in simulated and real-world latent spaces could further enhance policy transferability, fostering more reliable and versatile autonomous systems.

The open-source contributions accompanying this work also foster further exploration and practical applications in robotics, urban planning, and other fields that could benefit from sim-to-real navigation advancements.