Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Structured mutation inspired by evolutionary theory enriches population performance and diversity (2302.00559v2)

Published 1 Feb 2023 in cs.NE

Abstract: Grammar-Guided Genetic Programming (GGGP) employs a variety of insights from evolutionary theory to autonomously design solutions for a given task. Recent insights from evolutionary biology can lead to further improvements in GGGP algorithms. In this paper, we apply principles from the theory of Facilitated Variation and knowledge about heterogeneous mutation rates and mutation effects to improve the variation operators. We term this new method of variation Facilitated Mutation (FM). We test FM performance on the evolution of neural network optimizers for image classification, a relevant task in evolutionary computation, with important implications for the field of machine learning. We compare FM and FM combined with crossover (FMX) against a typical mutation regime to assess the benefits of the approach. We find that FMX in particular provides statistical improvements in key metrics, creating a superior optimizer overall (+0.48\% average test accuracy), improving the average quality of solutions (+50\% average population fitness), and discovering more diverse high-quality behaviors (+400 high-quality solutions discovered per run on average). Additionally, FM and FMX can reduce the number of fitness evaluations in an evolutionary run, reducing computational costs in some scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (50)
  1. P. Alberch and E. A. Gale. 1985. A developmental analysis of an evolutionary trend: digital reduction in amphibians. Evolution 39, 1 (1 1985), 8–23. https://doi.org/10.1111/J.1558-5646.1985.TB04076.X
  2. James Bergstra and Yoshua Bengio. 2012. Random Search for Hyper-Parameter Optimization. J. Mach. Learn. Res. 13, null (feb 2012), 281–305. https://dl.acm.org/doi/10.5555/2188385.2188395
  3. P. M. Brakefield and J. C. Roskam. 2006. Exploring Evolutionary Constraints Is a Task for an Integrative Evolutionary Biology. The American Naturalist 168, S6 (2006), S4–S13. https://doi.org/10.1086/509049 arXiv:https://doi.org/10.1086/509049 PMID: 17109328.
  4. Identification and analysis of evolutionarily cohesive functional modules in protein networks. Genome Research 16, 3 (3 2006), 374–382. https://doi.org/10.1101/GR.4336406
  5. Pedro Carvalho. 2023a. AutoLR Facilitated Mutation Grammar. https://github.com/soren5/autolr/blob/master/grammars/adaptive_autolr_grammar_mutate_level.txt
  6. Pedro Carvalho. 2023b. AutoLR MNIST Model. https://github.com/soren5/autolr/blob/master/models/json/mnist_model.json
  7. Pedro Carvalho. 2023c. AutoLR Original Grammar. https://github.com/soren5/autolr/blob/master/grammars/adaptive_autolr_grammar_original.txt
  8. Evolving Adaptive Neural Network Optimizers for Image Classification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 13223 LNCS (2022), 3–18. https://doi.org/10.1007/978-3-031-02056-8{_}1
  9. Context Matters: Adaptive Mutation for Grammars. In Genetic Programming, Gisele Pappa, Mario Giacobini, and Zdenek Vasicek (Eds.). Springer Nature Switzerland, Cham, 117–132.
  10. Evolved Optimizer for Vision. In First Conference on Automated Machine Learning (Late-Breaking Workshop). https://openreview.net/forum?id=jK_eS5BxOuu
  11. Jacob Cohen. 2013. Statistical power analysis for the behavioral sciences. Routledge.
  12. Automatic Composition of Music by Means of Grammatical Evolution. In Proceedings of the 2002 Conference on APL: Array Processing Languages: Lore, Problems, and Applications (Madrid, Spain) (APL ’02). Association for Computing Machinery, New York, NY, USA, 148–155. https://doi.org/10.1145/602231.602249
  13. Structural variation in the human genome. Nature Reviews Genetics 2006 7:2 7, 2 (2 2006), 85–97. https://doi.org/10.1038/nrg1767
  14. Evolvability ES: Scalable and direct optimization of evolvability. In GECCO 2019 - Proceedings of the 2019 Genetic and Evolutionary Computation Conference. Association for Computing Machinery, Inc, 107–115. https://doi.org/10.1145/3321707.3321876
  15. John Gerhart and Marc Kirschner. 2007. The theory of facilitated variation. Proceedings of the National Academy of Sciences of the United States of America 104, SUPPL. 1 (5 2007), 8582–8589. https://doi.org/10.1073/pnas.0701035104
  16. Hemichordates and the origin of chordates. Current Opinion in Genetics I& Development 15, 4 (2005), 461–467. https://doi.org/10.1016/j.gde.2005.06.004
  17. John C Gerhart and Marc W Kirschner. 1997. Cells, Embryos and Evolution. Blackwell Science 1997.
  18. Jens Gottlieb and Christoph Eckert. 2000. A Comparison of Two Representations for the fixed charge transportation problem. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 1917 (2000), 345–354. https://doi.org/10.1007/3-540-45356-3_34
  19. Jens Gottlieb and Günther R. Raidl. 2000. Locality in decoder-based EAs for the multidimensional knapsack problem. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 1829 (2000), 38–52. https://doi.org/10.1007/10721187_3
  20. Frederic Gruau. 1994. Automatic definition of modular neural networks. Adaptive behavior 3, 2 (1994), 151–183. https://doi.org/10.1177/105971239400300202
  21. Contrasting properties of gene-specific regulatory, coding, and copy number mutations in saccharomyces cerevisiae: Frequency, effects, and dominance. PLoS Genetics 8, 2 (feb 2012). https://doi.org/10.1371/journal.pgen.1002497
  22. Evolution strategies. Springer handbook of computational intelligence (2015), 871–898.
  23. Andrea Hartsock and W. James Nelson. 2008. Adherens and tight junctions: Structure, function and connections to the actin cytoskeleton. Biochimica et Biophysica Acta (BBA) - Biomembranes 1778, 3 (2008), 660–669. https://doi.org/10.1016/j.bbamem.2007.07.012
  24. Genr8: Architects’ experience with an emergent design tool. In The Art of Artificial Evolution. Springer, 167–188. https://doi.org/10.1007/978-3-540-72877-1_8
  25. Conservation and Novelty in the Evolution of Cell Adhesion and Extracellular Matrix Genes. Science 287, 5455 (2000), 989–994. https://doi.org/10.1126/science.287.5455.989
  26. John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA. https://doi.org/10.1007/BF00175355
  27. Michael A Lones. 2019. Instruction-level design of local optimisers using push GP. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 1487–1494. https://doi.org/10.1145/3319619.3326806
  28. Michael A. Lones. 2020. Optimising Optimisers with Push GP. In Lecture Notes in Computer Science. Springer International Publishing, 101–117. https://doi.org/10.1007/978-3-030-44094-7_7
  29. Michael A Lones. 2021. Evolving continuous optimisers from scratch. Genetic Programming and Evolvable Machines 22, 4 (2021), 395–428. https://doi.org/10.1007/s10710-021-09414-8
  30. Unveiling the properties of structured grammatical evolution. Genetic Programming and Evolvable Machines 17, 3 (9 2016), 251–289. https://doi.org/10.1007/S10710-015-9262-4/FIGURES/21
  31. Iñigo Martincorena and Nicholas M. Luscombe. 2013. Non-random mutation: the evolution of targeted hypermutation and hypomutation. Bioessays 35, 2 (2 2013), 123–130. https://doi.org/10.1002/bies.201200150
  32. Grammar-based genetic programming: a survey. Genetic Programming and Evolvable Machines 11, 3 (2010), 365–396. https://doi.org/10.1007/s10710-010-9109-y
  33. Eric Medvet. 2017. A comparative analysis of dynamic locality and redundancy in grammatical evolution. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 10196 LNCS (2017), 326–342. https://doi.org/10.1007/978-3-319-55696-3{_}21/COVER
  34. Mutation bias reflects natural selection in Arabidopsis thaliana. Nature 2022 (1 2022), 1–5. https://doi.org/10.1038/s41586-021-04269-6
  35. Facilitated Variation: How Evolution Learns from Past Environments To Generalize to New Environments. PLoS Computational Biology 4, 11 (2008), 1000206. https://doi.org/10.1371/journal.pcbi.1000206
  36. Franz Rothlauf and Marie Oetzel. 2006. On the locality of grammatical evolution. In Genetic Programming: 9th European Conference, EuroGP 2006, Budapest, Hungary, April 10-12, 2006. Proceedings 9. Springer, 320–330.
  37. Learning internal representations by error propagation. Technical Report. California Univ San Diego La Jolla Inst for Cognitive Science. 318–362 pages. https://ieeexplore.ieee.org/document/6302929
  38. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. Open Ai (2017), 1–13. arXiv:1703.03864 http://arxiv.org/abs/1703.03864
  39. David L. Stern and Virginie Orgogozo. 2009. Is Genetic Evolution Predictable? Science 323, 5915 (2009), 746–751. https://doi.org/10.1126/science.1158997 arXiv:https://www.science.org/doi/pdf/10.1126/science.1158997
  40. Conservation and evolvability in regulatory networks: The evolution of ribosomal regulation in yeast. Proceedings of the National Academy of Sciences of the United States of America 102, 20 (5 2005), 7203–7208. https://doi.org/10.1073/pnas.0502521102
  41. Epithelial Cell Polarity and Cell Junctions in Drosophila. Annual Review of Genetics 35, 1 (2001), 747–784. https://doi.org/10.1146/annurev.genet.35.102401.091415 arXiv:https://doi.org/10.1146/annurev.genet.35.102401.091415 PMID: 11700298.
  42. Developmental bias and evolution: A regulatory network perspective. Genetics 209, 4 (2018), 949–966. https://doi.org/10.1534/genetics.118.300995
  43. The road to modularity. Nature Reviews Genetics 8, 12 (2007), 921–931. https://doi.org/10.1038/nrg2267
  44. Richard A. Watson and Eörs Szathmáry. 2016. How Can Evolution Learn? Trends in Ecology and Evolution 31, 2 (2016), 147–157. https://doi.org/10.1016/j.tree.2015.11.009
  45. Carolyn A. Wessinger and Lena C. Hileman. 2016. Accessibility, constraint, and repetition in adaptive floral evolution. Developmental Biology 419, 1 (11 2016), 175–183. https://doi.org/10.1016/J.YDBIO.2016.05.003
  46. Peter Alexander Whigham. 1996. Grammatical Bias for Evolutionary Learning. (1996). https://doi.org/10.26190/unsworks/6784
  47. Examining the ”best of both worlds” of grammatical evolution. In GECCO 2015 - Proceedings of the 2015 Genetic and Evolutionary Computation Conference. Association for Computing Machinery, Inc, 1111–1118. https://doi.org/10.1145/2739480.2754784
  48. Rasmus G. Winther. 2001. Varieties of modules: Kinds, levels, origins, and behaviors. Journal of Experimental Zoology 291, 2 (8 2001), 116–129. https://doi.org/10.1002/JEZ.1064
  49. Gregory A. Wray. 2007. The evolutionary significance of cis-regulatory mutations. Nature Reviews Genetics 2007 8:3 8, 3 (3 2007), 206–216. https://doi.org/10.1038/nrg2063
  50. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. https://doi.org/10.48550/ARXIV.1708.07747

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now