Emergent Braitenberg-style Behaviours for Navigating the ViZDoom `My Way Home' Labyrinth
Abstract: The navigation of complex labyrinths with tens of rooms under visual partially observable state is typically addressed using recurrent deep reinforcement learning architectures. In this work, we show that navigation can be achieved through the emergent evolution of a simple Braitentberg-style heuristic that structures the interaction between agent and labyrinth, i.e. complex behaviour from simple heuristics. To do so, the approach of tangled program graphs is assumed in which programs cooperatively coevolve to develop a modular indexing scheme that only employs 0.8\% of the state space. We attribute this simplicity to several biases implicit in the representation, such as the use of pixel indexing as opposed to deploying a convolutional kernel or image processing operators.
- Benchmarking genetic programming in a multi-action reinforcement learning locomotion task. In Proceedings of the Genetic and Evolutionary Computation Conference (Companion), pages 522–525. ACM, 2022.
- Caleidgh Bayer. Evaluating simple reactive agents in visual reinforcement learning tasks. Master’s thesis, Faculty of Computer Science, Dalhousie University, 2023.
- Finding simple solutions to multi-task visual reinforcement learning problems with tangled program graphs. In Genetic Programming Theory and Practice XVIII, pages 1–19. Springer, 2021.
- Deep reinforcement learning on a budget: 3d control and reasoning without a supercomputer. In IEEE International Conference on Pattern Recognition, pages 158–165, 2020.
- Valentino Braitenberg. Vehicles: Experiments in Synthetic Psychology. MIT Press, 1984.
- Linear Genetic Programming. Springer, 2007.
- Odor source localization algorithms on mobile robots: a review and future work. Robotics and Autonomous Systems, 112(2):123–136, 2019.
- Carlos Gershenson. Emergence in artificial life. Artifical Life, 29(2):153–167, 2023.
- Exploration in deep reinforcement learning: From single-agent to multiagent domain. IEEE Transactions on Neural Networks and Learning Systems, pages 1–21, 2023.
- A model-based sound localization system and its application to robot navigation. Robotics and Autonomous Systems, 27(4):199–209, 1999.
- S. Kelly and M. I. Heywood. Emergent tangled graph representations for Atari game playing agents. In European Conference on Genetic Programming, volume 10196 of LNCS, pages 64–79, 2017.
- Emergent solutions to high-dimensional multitask reinforcement learning. Evolutionary Computation, 26(3):347–380, 2018.
- Emergent policy discovery for visual reinforcement learning through tangled program graphs: A tutorial. In Genetic Programming Theory and Practice XVI, pages 37–57. Springer, 2018.
- Emergent tangled program graphs in partially observable recursive forecasting and ViZDoom navigation tasks. ACM Transactions on Evolutionary Learning and Optimization, 1, 2021.
- ViZDoom: A Doom-based AI research platform for visual reinforcement learning. In IEEE Conference on Computational Intelligence and Games, pages 1–8, 2016.
- Human-level control through deep reinforcement learning. Nature, 518(7540):529–533, 2015.
- Episodic curiosity through reachability. CoRR, abs/1810.02274, 2018.
- Braitenberg vehicles as computational tools for research in neuroscience. Frontiers in Bioengineering and Biotechnology, 8:1–7, 2020.
- Scaling tangled program graphs to visual reinforcement learning in ViZDoom. In Proceedings of the European Conference on Genetic Programming, volume 10781 of LNCS, pages 135–150. Springer, 2018.
- Christopher J. C. H. Watkins and Peter Dayan. Q-learning. Machine Learning, 8:279–292, 1992.
Paper Prompts
Sign up for free to create and run prompts on this paper.
