Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Performance Analysis of Lexicase-Based and Traditional Selection Methods in GP for Symbolic Regression

Published 31 Jul 2024 in cs.NE | (2407.21632v2)

Abstract: In recent years, several new lexicase-based selection variants have emerged due to the success of standard lexicase selection in various application domains. For symbolic regression problems, variants that use an epsilon-threshold or batches of training cases, among others, have led to performance improvements. Lately, especially variants that combine lexicase selection and down-sampling strategies have received a lot of attention. This paper evaluates the most relevant lexicase-based selection methods as well as traditional selection methods in combination with different down-sampling strategies on a wide range of symbolic regression problems. In contrast to most work, we not only compare the methods over a given evaluation budget, but also over a given time budget as time is usually limited in practice. We find that for a given evaluation budget, epsilon-lexicase selection in combination with a down-sampling strategy outperforms all other methods. If the given running time is very short, lexicase variants using batches of training cases perform best. Further, we find that the combination of tournament selection with informed down-sampling performs well in all studied settings.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Sneha Aenugu and Lee Spector. 2019. Lexicase selection in learning classifier systems. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 356–364.
  2. The Problem Solving Benefits of Down-sampling Vary by Selection Scheme. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (Lisbon, Portugal) (GECCO ’23 Companion). Association for Computing Machinery, New York, NY, USA, 527–530.
  3. Untangling the Effects of Down-Sampling and Selection in Genetic Programming. In ALIFE 2024: Proceedings of the 2024 Artificial Life Conference. MIT Press.
  4. Informed Down-Sampled Lexicase Selection: Identifying productive training cases for efficient problem solving. Evolutionary computation (2024), 1–31.
  5. A Static Analysis of Informed Down-Samples. In Proceedings of the Companion Conference on Genetic and Evolutionary Computation (Lisbon, Portugal) (GECCO ’23 Companion). Association for Computing Machinery, New York, NY, USA, 531–534.
  6. Shu-Heng Chen. 2012. Genetic algorithms and genetic programming in computational finance. Springer Science & Business Media.
  7. Batch tournament selection for genetic programming: the quality of lexicase, the speed of tournament. In Proceedings of the genetic and evolutionary computation conference (GECCO ’19). ACM, 994–1002.
  8. Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine learning research 7 (2006), 1–30.
  9. Going faster and hence further with lexicase selection. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 538–541.
  10. Lexicase selection at scale. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 2054–2062.
  11. Probabilistic Lexicase Selection. In Proceedings of the Genetic and Evolutionary Computation Conference (Lisbon, Portugal) (GECCO ’23). ACM, 1073–1081.
  12. Li Ding and Lee Spector. 2021. Optimizing neural networks with gradient lexicase selection. In International Conference on Learning Representations.
  13. Emily Dolson. 2023. Calculating lexicase selection probabilities is NP-Hard. In Proceedings of the Genetic and Evolutionary Computation Conference. 1575–1583.
  14. Yongsheng Fang and Jun Li. 2010. A Review of Tournament Selection in Genetic Programming. In Advances in Computation and Intelligence. Lecture Notes in Computer Science, Vol. 6382. Springer Berlin Heidelberg, 181–192.
  15. Characterizing the Effects of Random Subsampling on Lexicase Selection. In Genetic Programming Theory and Practice XVII. Springer International Publishing, 1–23.
  16. DEAP: Evolutionary algorithms made easy. The Journal of Machine Learning Research 13, 1 (2012), 2171–2175.
  17. Down-Sampled Epsilon-Lexicase Selection for Real-World Symbolic Regression Problems. In Proceedings of the Genetic and Evolutionary Computation Conference (Lisbon, Portugal) (GECCO ’23). ACM, 1109–1117.
  18. A Comprehensive Comparison of Lexicase-Based Selection Methods for Symbolic Regression Problems. In European Conference on Genetic Programming (Part of EvoStar). Springer, 192–208.
  19. Thomas Helmuth and Amr Abdelhady. 2020. Benchmarking parent selection for program synthesis by genetic programming. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion (GECCO ’20). ACM, 237–238.
  20. Effects of Lexicase and Tournament Selection on Diversity Recovery and Maintenance. In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion (GECCO ’16 Companion). ACM, 983–990.
  21. Lexicase selection for program synthesis: a diversity analysis. In Genetic Programming Theory and Practice XIII. Springer International Publishing, 151–167.
  22. Lexicase selection of specialists. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’19). ACM, 1030–1038.
  23. On the importance of specialists for lexicase selection. Genetic Programming and Evolvable Machines 21, 3 (2020), 349–373.
  24. Thomas Helmuth and Lee Spector. 2015. General Program Synthesis Benchmark Suite. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (GECCO ’15). ACM, 1039–1046.
  25. Thomas Helmuth and Lee Spector. 2020. Explaining and exploiting the advantages of down-sampled lexicase selection. In ALIFE 2020: The 2020 Conference on Artificial Life. MIT Press, 341–349.
  26. Thomas Helmuth and Lee Spector. 2021. Problem-Solving Benefits of Down-Sampled Lexicase Selection. Artificial life 27, 3-4 (2021), 183–203.
  27. Solving Uncompromising Problems with Lexicase Selection. IEEE Transactions on Evolutionary Computation 19, 5 (2014), 630–643.
  28. Fast, accurate, and transferable many-body interatomic potentials by symbolic regression. npj Computational Materials 5, 1 (2019), 112.
  29. Random subsampling improves performance in lexicase selection. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’19). ACM, 2028–2031.
  30. An Exploration of Exploration: Measuring the ability of lexicase selection to find obscure pathways to optimality. In Genetic Programming Theory and Practice XVIII. 83–107.
  31. Dorit S Hochbaum and David B Shmoys. 1985. A best possible heuristic for the k-center problem. Mathematics of operations research 10, 2 (1985), 180–184.
  32. Lia Jundt and Thomas Helmuth. 2019. Comparing and combining lexicase selection and novelty search. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 1047–1055.
  33. Improving Genetic Programming with Novel Exploration - Exploitation Control. In European Conference on Genetic Programming. Springer, 64–80.
  34. John R. Koza. 1992. On the programming of computers by means of natural selection. A Bradford book, Vol. 1. MIT Press.
  35. Krzysztof Krawiec and Una-May O’Reilly. 2014. Behavioral programming: a broader and more detailed take on semantic GP. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO ’14). ACM, 935–942.
  36. A probabilistic and multi-objective analysis of lexicase selection and epsilon-lexicase selection. Evolutionary Computation 27, 3 (2019), 377–402.
  37. Contemporary Symbolic Regression Methods and their Relative Performance. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
  38. Epsilon-Lexicase Selection for Regression. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 (GECCO ’16). ACM, 741–748.
  39. A flexible symbolic regression method for constructing interpretable clinical prediction models. NPJ Digital Medicine 6, 1 (2023), 107.
  40. Jared M. Moore and Adam Stanton. 2017. Lexicase selection outperforms previous strategies for incremental evolution of virtual creature controllers. In ECAL 2017, the Fourteenth European Conference on Artificial Life. 290–297.
  41. Jared M. Moore and Adam Stanton. 2018. Tiebreaks and Diversity: Isolating Effects in Lexicase Selection. In The 2018 Conference on Artificial Life. MIT Press, Cambridge, MA, 590–597.
  42. DALex: Lexicase-Like Selection via Diverse Aggregation. In European Conference on Genetic Programming (Part of EvoStar). Springer, 90–107.
  43. The Use of an Analytic Quotient Operator in Genetic Programming. IEEE Transactions on Evolutionary Computation 17, 1 (2013), 146–152.
  44. Where are we now? A large benchmark study of recent symbolic regression methods.. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 1183–1190.
  45. Specialization and Elitism in Lexicase and Tournament Selection. In Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO ’18). ACM, 1914–1917.
  46. Thu Pham-Gia and Tran Loc Hung. 2001. The Mean and Median Absolute Deviations. Mathematical and Computer 34, 7-8 (2001), 921–936.
  47. A field guide to genetic programming. Lulu Press.
  48. Effects of the Training Set Size: A Comparison of Standard and Down-Sampled Lexicase Selection in Program Synthesis. In 2022 IEEE Congress on Evolutionary Computation (CEC). IEEE, 1–8.
  49. Dominik Sobania and Franz Rothlauf. 2021. A generalizability measure for program synthesis with genetic programming. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’21). ACM, 822–829.
  50. Dominik Sobania and Franz Rothlauf. 2022. Program Synthesis with Genetic Programming: The Influence of Batch Sizes. In European Conference on Genetic Programming. Springer, 118–129.
  51. A Comprehensive Survey on Program Synthesis With Evolutionary Algorithms. IEEE Transactions on Evolutionary Computation 27, 1 (2023), 82–97.
  52. Lee Spector. 2012. Assessment of Problem Modality by Differential Performance of Lexicase Selection in Genetic Programming: A Preliminary Report. In Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO ’12). ACM, 401–408.
  53. Relaxations of Lexicase Parent Selection. In Genetic Programming Theory and Practice XV. Springer, 105–120.
  54. Alexander R. M. Wagner and Anthony Stein. 2021. Adopting lexicase selection for michigan-style learning classifier systems with continuous-valued inputs. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, 171–172.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up