2000 character limit reached
Online Continual Learning For Interactive Instruction Following Agents (2403.07548v2)
Published 12 Mar 2024 in cs.AI, cs.LG, and cs.RO
Abstract: In learning an embodied agent executing daily tasks via language directives, the literature largely assumes that the agent learns all training data at the beginning. We argue that such a learning scenario is less realistic since a robotic agent is supposed to learn the world continuously as it explores and perceives it. To take a step towards a more realistic embodied agent learning scenario, we propose two continual learning setups for embodied agents; learning new behaviors (Behavior Incremental Learning, Behavior-IL) and new environments (Environment Incremental Learning, Environment-IL) For the tasks, previous 'data prior' based continual learning methods maintain logits for the past tasks. However, the stored information is often insufficiently learned information and requires task boundary information, which might not always be available. Here, we propose to update them based on confidence scores without task boundary information during training (i.e., task-free) in a moving average fashion, named Confidence-Aware Moving Average (CAMA). In the proposed Behavior-IL and Environment-IL setups, our simple CAMA outperforms prior state of the art in our empirical validations by noticeable margins. The project page including codes is https://github.com/snumprlab/cl-alfred.
- Memory aware synapses: Learning what (not) to forget. In ECCV, 2018a.
- Task-free continual learning. arXiv:1812.03596, 2018b.
- Online continual learning with maximally interfered retrieval. In NeurIPS, 2019a.
- Gradient based sample selection for online continual learning. In NeurIPS, 2019b.
- Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In CVPR, 2018.
- Rainbow memory: Continual learning with a memory of diverse samples. In CVPR, 2021.
- Multi-level compositional reasoning for interactive instruction following. In AAAI, 2023.
- Continual lifelong learning in natural language processing: A survey. arXiv:2012.09823, 2020.
- Class-incremental continual learning into the extended der-verse. In IEEE TPAMI, 2022a.
- Transfer without forgetting. In ECCV, 2022b.
- Dark experience for general continual learning: a strong, simple baseline. In NeurIPS, 2020.
- Modeling missing annotations for incremental learning in object detection. In CVPR, 2022.
- Superensemble classifier for improving predictions in imbalanced datasets. In Communications in Statistics: Case Studies, Data Analysis and Applications, 2020.
- Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 2018.
- A simple framework for contrastive learning of visual representations. In ICML, 2020.
- Autoaugment: Learning augmentation policies from data. In CVPR, 2019.
- Embodied question answering. In CVPR, 2018.
- Robothor: An open simulation-to-real embodied ai platform. In CVPR, 2020.
- Manipulathor: A framework for visual object manipulation. In CVPR, 2021.
- Minedojo: Building open-ended embodied agents with internet-scale knowledge. In NeurIPS, 2022.
- Cril: Continual robot imitation learning via generative and prediction model. In IROS, 2021.
- Dialfred: Dialogue-enabled agents for embodied instruction following. In RA-L, 2022.
- Iqa: Visual question answering in interactive environments. In CVPR, 2018.
- Online continual learning for embedded devices. arXiv:2203.10681, 2022.
- Long-tailed continual learning for visual food recognition. arXiv preprint arXiv:2307.00183, 2023.
- Rethinking task-incremental learning baselines. In ICPR, 2022.
- Stay on the path: Instruction fidelity in vision-and-language navigation. In ACL, 2019.
- Incremental object detection via meta-learning. In IEEE TPAMI, 2021.
- Environments for lifelong reinforcement learning. arXiv:1811.10732, 2018.
- Agent with the big picture: Perceiving surroundings for interactive instruction following. In Embodied AI Workshop @ CVPR, 2021.
- Context-aware planning and environment-aware memory for instruction following embodied agents. In ICCV, 2023.
- Overcoming catastrophic forgetting in neural networks. In PNAS, 2017.
- Online continual learning on class incremental blurry task configuration with anytime inference. In ICLR, 2022.
- Online boundary-free continual learning by scheduled data prior. In ICLR, 2023.
- Ai2-thor: An interactive 3d environment for visual ai. arXiv:1712.05474, 2017.
- Beyond the nav-graph: Vision-and-language navigation in continuous environments. In ECCV, 2020.
- Carm: hierarchical episodic memory for continual learning. In DAC, 2022.
- Regularization shortcomings for continual learning. arXiv:1912.03049, 2019.
- Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges. In Information fusion, 2020.
- An exponential learning rate schedule for deep learning. arXiv:1910.07454, 2019.
- Learning without forgetting. In IEEE TPAMI, 2017.
- Libero: Benchmarking knowledge transfer for lifelong robot learning. arXiv:2306.03310, 2023.
- Long-tailed class incremental learning. In ECCV, 2022.
- Gradient episodic memory for continual learning. In NeurIPS, 2017.
- Sgdr: Stochastic gradient descent with warm restarts. arXiv:1608.03983, 2016.
- Self-monitoring navigation agent via auxiliary progress estimation. In ICLR, 2019.
- Online continual learning in image classification: An empirical survey. In Neurocomputing, 2022.
- Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, 1989.
- Lifelong inverse reinforcement learning. In NeurIPS, 2018.
- Knowledge distillation for incremental learning in semantic segmentation. In CVIU, 2021.
- Film: Following instructions in language with modular methods. In ICLR, 2022.
- Mapping instructions and visual observations to actions with reinforcement learning. In EMNLP, 2017.
- Look wide and interpret twice: Improving performance on interactive instruction-following tasks. In IJCAI, 2021.
- Teach: Task-driven embodied agents that chat. In AAAI, 2022.
- Continual lifelong learning with neural networks: A review. In Neural networks, 2019.
- Class-incremental learning for action recognition in videos. In ICCV, 2021.
- Episodic transformer for vision-and-language navigation. In ICCV, 2021.
- Cora: Benchmarks, baselines, and metrics as a platform for continual reinforcement learning agents. In CoLLAs, 2022.
- Gdumb: A simple approach that questions our progress in continual learning. In ECCV, 2020.
- Computationally budgeted continual learning: What does matter? In CVPR, 2023.
- Roger Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. In Psychological review, 1990.
- icarl: Incremental classifier and representation learning. In CVPR, 2017.
- Experience replay for continual learning. In NeurIPS, 2019.
- Gradient projection memory for continual learning. arXiv:2103.09762, 2021.
- Habitat: A Platform for Embodied AI Research. In ICCV, 2019.
- Encoders and ensembles for task-free continual learning. arXiv:2105.13327, 2021.
- Online class-incremental continual learning with adversarial shapley value. In AAAI, 2021.
- Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In CVPR, 2020.
- Factorizing perception and policy for interactive instruction following. In ICCV, 2021.
- vclimb: A novel video class incremental learning benchmark. In CVPR, 2022.
- Voyager: An open-ended embodied agent with large language models. arXiv:2305.16291, 2023.
- Learning to prompt for continual learning. In CVPR, 2022.
- Visual room rearrangement. In CVPR, 2021.
- Continual world: A robotic benchmark for continual reinforcement learning. In NeurIPS, 2021.
- Large scale incremental learning. In CVPR, 2019.
- Lifelong robotic reinforcement learning by retaining experiences. In CoLLAs, 2022.
- Evaluations of the gap between supervised and reinforcement lifelong learning on robotic manipulation tasks. In CoRL, 2022.
- Task-free continual learning via online discrepancy distance learning. In NeurIPS, 2022.
- Continual learning through synaptic intelligence. In ICML, 2017.
- Energy aligning for biased models. arXiv:2106.03343, 2021a.
- When video classification meets incremental classes. In ACM MM, 2021b.
- A model or 603 exemplars: Towards memory-efficient class-incremental learning. arXiv:2205.13218, 2022a.
- Forgetting and imbalance in robot lifelong learning with off-policy data. In CoLLAs, 2022b.
- Visual semantic planning using deep successor representations. In ICCV, 2017.
Sponsor
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.