Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation

Abstract: Multisensory polices are known to enhance both state estimation and target tracking. However, in the space of end-to-end sensorimotor control, this multi-sensor outlook has received limited attention. Moreover, systematic ways to make policies robust to partial sensor failure are not well explored. In this work, we propose a specific customization of Dropout, called \textit{Sensor Dropout}, to improve multisensory policy robustness and handle partial failure in the sensor-set. We also introduce an additional auxiliary loss on the policy network in order to reduce variance in the band of potential multi- and uni-sensory policies to reduce jerks during policy switching triggered by an abrupt sensor failure or deactivation/activation. Finally, through the visualization of gradients, we show that the learned policies are conditioned on the same latent states representation despite having diverse observations spaces - a hallmark of true sensor-fusion. Simulation results of the multisensory policy, as visualized in TORCS racing game, can be seen here: https://youtu.be/QAK2lcXjNZc.

• The paper introduces Sensor Dropout, a novel regularization technique that mitigates over-dependence on any sensor in DRL training.
• It refines DRL algorithms like NAF and DDPG to effectively integrate low- and high-dimensional sensor data for improved navigation policies.
• Experiments in TORCS show a reduced performance drop (9.8% vs 51.7%) under sensor noise, indicating high resilience in sensor fusion.

Overview of "Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation"

The paper "Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation" addresses the development of robust autonomous navigation systems through the implementation of end-to-end multisensory sensorimotor control. The primary contribution is the introduction of a novel regularization technique called Sensor Dropout (SD), which enhances the robustness of multisensory policies against partial sensor failures. The research leverages Deep Reinforcement Learning (DRL) algorithms and specifically tailors them to utilize multisensory inputs effectively in order to achieve robust autonomous navigation.

Key Contributions and Methodologies

1. Sensor Dropout (SD): SD is a tailored version of the conventional dropout technique that aids in regularizing training by randomly disabling some sensor inputs. This helps mitigate the over-dependence on any specific sensor and ensures policy robustness even when some sensors fail or provide noisy data. The technique is adeptly integrated into the feedback loop of DRL systems.
2. DRL Algorithms Optimization: The study incorporates and refines two continuous action DRL algorithms, namely Normalized Advantage Function (NAF) and Deep Deterministic Policy Gradient (DDPG), to work in conjunction with SD for improved multisensory input handling. These algorithms cover different aspects of policy learning, from value-based approaches to actor-critic methods, showcasing the versatility of SD across varying algorithmic structures.
3. Environment and Experimental Setup: The research utilizes the TORCS simulator, a robust platform for testing vehicular autonomous navigation systems, to evaluate their policy's efficacy. It employs a combination of low-dimensional (odometry and GPS) and high-dimensional (camera images and lidar scans) sensor data, aligning with realistic vehicular sensor configurations.
4. Auxiliary Loss Integration: Besides SD, an additional auxiliary loss is introduced to further stabilize the learning by minimizing variance in action outputs across different sensor configurations, ensuring smooth transitions and policy switching.

Numerical Results and Practical Implications

The experimentation within the TORCS environment yielded significant numerical results. Notably, policies trained with Sensor Dropout exhibited only a 9.8% performance drop in noisy conditions, a marked improvement compared to a 51.7% drop for baseline policies without SD. The methodology demonstrated its capacity to generalize across unseen sensor conditions and simulate failures, vital for real-world implementation in uncertain environments.

SD's implementation exhibits substantial potential for enhancing the resilience of autonomous navigation systems, especially in safety-critical scenarios. Furthermore, the approach of addressing sensor fusion and robustness in dynamically changing sensor conditions showcases promising paths for extending similar methodologies to other robotics domains and complex, real-world environments.

Future Directions

Future research could explore refining SD's adaptability during runtime to cope with dynamically changing sensor reliability. Moreover, extending the Sensor Dropout framework across diverse navigation environments and problem settings such as urban traffic scenarios could illustrate further applications. Additionally, research could explore augmenting the reward function for comprehensive driving tasks including obstacle avoidance and dynamic path planning.

The integration of SD with advanced DRL algorithms like Trust Region Policy Optimization (TRPO) and its adaptation to real-world robotic systems are prospective developments that could transform the approach's theoretical and applied facets within autonomous systems.