Self-Initiated Open World Learning for Autonomous AI Agents (2110.11385v3)
Abstract: As more and more AI agents are used in practice, it is time to think about how to make these agents fully autonomous so that they can learn by themselves in a self-motivated and self-supervised manner rather than being retrained periodically on the initiation of human engineers using expanded training data. As the real-world is an open environment with unknowns or novelties, detecting novelties or unknowns, characterizing them, accommodating or adapting to them, gathering ground-truth training data, and incrementally learning the unknowns/novelties are critical to making the agent more and more knowledgeable and powerful over time. The key challenge is how to automate the process so that it is carried out on the agent's own initiative and through its own interactions with humans and the environment. Since an AI agent usually has a performance task, characterizing each novelty becomes critical and necessary so that the agent can formulate an appropriate response to adapt its behavior to accommodate the novelty and to learn from it to improve the agent's adaptation capability and task performance. The process goes continually without termination. This paper proposes a theoretic framework for this learning paradigm to promote the research of building Self-initiated Open world Learning (SOL) agents. An example SOL agent is also described.
- Towards open world recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1893–1902.
- Towards a Unifying Framework for Formal Theories of Novelty. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 15047–15052.
- Lifelong machine learning. Morgan & Claypool Publishers.
- Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. Machine Learning 1–50.
- Zero-Shot Out-of-Distribution Detection Based on the Pretrained Model CLIP. In Proceedings of the AAAI conference on artificial intelligence.
- Breaking the closed world assumption in text classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 506–514.
- Learning cumulatively to become more knowledgeable. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1565–1574.
- Langley, P. 2020. Open-world learning for radically autonomous agents. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 13539–13543.
- Liu, B. 2020. Learning on the job: Online lifelong and continual learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-2020).
- Lifelong and continual learning dialogue systems: learning during conversation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-2021).
- An Application-Independent Approach to Building Task-Oriented Chatbots with Interactive Continual Learning. NeurIPS-2020 Workshop on Human in the Loop Dialogue Systems .
- A unifying view on dataset shift in classification. Pattern recognition 45(1): 521–530.
- Hippocampal networks habituate as novelty accumulates. Learning & Memory 20(4): 229–235.
- Deep learning for anomaly detection: A review. ACM Computing Surveys (CSUR) 54(2): 1–38.
- Continual lifelong learning with neural networks: A review. Neural Networks .
- Open-world Machine Learning: Applications, Challenges, and Opportunities. arXiv preprint arXiv:2105.13448 .
- Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 .
- Unseen class discovery in open-world classification. arXiv preprint arXiv:1801.05609 .
- Novelty assessment in the brain and long-term memory encoding. Psychonomic Bulletin & Review 2(3): 387–390.
- Open-world learning and application to product classification. In The World Wide Web Conference, 3413–3419.
- Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 .
- Bing Liu (212 papers)
- Eric Robertson (7 papers)
- Scott Grigsby (6 papers)
- Sahisnu Mazumder (21 papers)