Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Evolvability and Behavior Landscapes in Neuroevolutionary Divergent Search (2306.09849v1)

Published 16 Jun 2023 in cs.NE

Abstract: Evolvability refers to the ability of an individual genotype (solution) to produce offspring with mutually diverse phenotypes. Recent research has demonstrated that divergent search methods, particularly novelty search, promote evolvability by implicitly creating selective pressure for it. The main objective of this paper is to provide a novel perspective on the relationship between neuroevolutionary divergent search and evolvability. In order to achieve this, several types of walks from the literature on fitness landscape analysis are first adapted to this context. Subsequently, the interplay between neuroevolutionary divergent search and evolvability under varying amounts of evolutionary pressure and under different diversity metrics is investigated. To this end, experiments are performed on Fetch Pick and Place, a robotic arm task. Moreover, the performed study in particular sheds light on the structure of the genotype-phenotype mapping (the behavior landscape). Finally, a novel definition of evolvability that takes into account the evolvability of offspring and is appropriate for use with discretized behavior spaces is proposed, together with a Markov-chain-based estimation method for it.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. Kenneth O Stanley. Why open-endedness matters. Artificial life, 25(3):232–235, 2019.
  2. Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv preprint arXiv:1712.06567, 2017.
  3. Evolution strategies as a scalable alternative to reinforcement learning. arXiv preprint arXiv:1703.03864, 2017.
  4. Novelty search and the problem with objectives. Genetic programming theory and practice IX, pages 37–56, 2011.
  5. Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI, page 40, 2016.
  6. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909, 2015.
  7. Richard Dawkins. The evolution of evolvability. In Artificial life, pages 201–220. Routledge, 2019.
  8. Reconciling explanations for the evolution of evolvability. Adaptive Behavior, 23(3):171–179, 2015.
  9. Evolvability es: scalable and direct optimization of evolvability. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 107–115, 2019.
  10. Massimo Pigliucci. Is evolvability evolvable? Nature Reviews Genetics, 9(1):75–82, 2008.
  11. Evolvability search: directly selecting for evolvability in order to study and produce it. In Proceedings of the Genetic and Evolutionary Computation Conference 2016, pages 141–148, 2016.
  12. On the critical role of divergent selection in evolvability. Frontiers in Robotics and AI, 3:45, 2016.
  13. Novelty search makes evolvability inevitable. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pages 85–93, 2020.
  14. Improving evolvability through novelty search and self-adaptation. In 2011 IEEE congress of evolutionary computation (CEC), pages 2693–2700. IEEE, 2011.
  15. When novelty is not enough. In EvoApplications (1), pages 234–243, 2011.
  16. How evolvable is novelty search? In 2014 IEEE International Conference on Evolvable Systems, pages 125–132. IEEE, 2014.
  17. Quality evolvability es: Evolving individuals with a distribution of well performing and diverse offspring. arXiv preprint arXiv:2103.10790, 2021.
  18. Extinction events can accelerate evolution. PloS one, 10(8):e0132886, 2015.
  19. The evolutionary origins of modularity. Proceedings of the Royal Society b: Biological sciences, 280(1755):20122863, 2013.
  20. On the entanglement between evolvability and fitness: an experimental study on voxel-based soft robots. In ALIFE: PROCEEDINGS OF THE ARTIFICIAL LIFE CONFERENCE. MIT press, 2022.
  21. Novelty search: a theoretical perspective. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 99–106, 2019.
  22. Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Transactions on Evolutionary Computation, 22(4):623–630, 2017.
  23. Yuval Davidor. Epistasis variance: A viewpoint on ga-hardness. In Foundations of genetic algorithms, volume 1, pages 23–35. Elsevier, 1991.
  24. A comprehensive survey on fitness landscape analysis. Recent advances in intelligent engineering systems, pages 161–191, 2012.
  25. Discovering the elite hypervolume by leveraging interspecies correlation. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 149–156, 2018.
  26. Policy manifold search: Exploring the manifold hypothesis for diversity-based neuroevolution. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 901–909, 2021.
  27. Discovering representations for black-box optimization. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pages 103–111, 2020.
  28. Geodesics, non-linearities and the archive of novelty search. arXiv preprint arXiv:2205.03162, 2022.
  29. A study of fitness landscapes for neuroevolution. In 2020 IEEE Congress on Evolutionary Computation (CEC), pages 1–8. IEEE, 2020.
  30. Neuroevolution in deep neural networks: Current trends and future challenges. IEEE Transactions on Artificial Intelligence, 2(6):476–493, 2021.
  31. Peter F Stadler. Fitness landscapes. In Biological evolution and statistical physics, pages 183–204. Springer, 2002.
  32. John H Gillespie. Molecular evolution over the mutational landscape. Evolution, pages 1116–1129, 1984.
  33. Towards a general theory of adaptive walks on rugged landscapes. Journal of theoretical Biology, 128(1):11–45, 1987.
  34. Openai gym, 2016.
  35. Multi-goal reinforcement learning: Challenging robotics environments and request for research, 2018.
  36. Mujoco: A physics engine for model-based control. In 2012 IEEE/RSJ international conference on intelligent robots and systems, pages 5026–5033. IEEE, 2012.
  37. Heterogeneous team deep q-learning in low-dimensional multi-agent environments. In 2016 IEEE Conference on Computational Intelligence and Games (CIG), pages 1–8. IEEE, 2016.
  38. A population initialization method for evolutionary algorithms based on clustering and cauchy deviates. Expert Systems with Applications, 60:294–310, 2016.
  39. Quality and diversity optimization: A unifying modular framework. IEEE Transactions on Evolutionary Computation, 22(2):245–259, 2017.
  40. Quality diversity through surprise. IEEE Transactions on Evolutionary Computation, 23(4):603–616, 2018.
  41. Lyapunov design for safe reinforcement learning. Journal of Machine Learning Research, 3(Dec):803–832, 2002.
  42. Exploring conflicting objectives with madns: Multiple assessment directed novelty search. In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pages 23–24, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now