Mitigate catastrophic forgetting in continual robot learning via data mixing

Develop methods to mix different proportions of prior-data distributions with newly collected data during fine-tuning to alleviate catastrophic forgetting in continual learning of embodied robotic policies, and establish criteria for choosing mixing ratios under diverse tasks and environments.

Background

The paper emphasizes that continual learning is crucial for deploying robot learning policies in diverse real-world environments, but remains largely unexplored and existing solutions are often task- or platform-specific. Within this context, the authors enumerate concrete open research problems that must be addressed to make continual learning practical for embodied AI.

One explicitly stated avenue is to mitigate catastrophic forgetting that occurs when models are fine-tuned on new data by mixing different proportions of prior distributions with the latest data. This problem is central to maintaining previously acquired skills while adapting to new tasks and conditions in embodied settings.

References

Open research problems and viable approaches include: 1) mixing different proportions of prior data distribution when fine-tuning on the latest data to alleviate catastrophic forgetting , 2) developing efficient prototypes from prior distributions or curricula for task inference in learning new tasks, 3) improving training stability and sample efficiency of online learning algorithms, 4) identifying principled ways to seamlessly incorporate large-capacity models into control frameworks, potentially through hierarchical learning or slow-fast control, for real-time inference.

— Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI  (2407.06886 - Liu et al., 2024) in Section 8, Challenges and Future Directions – Continual Learning