Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
97 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning (2404.08239v1)

Published 12 Apr 2024 in cs.NE and cs.AI

Abstract: Evolutionary computation (EC) algorithms, renowned as powerful black-box optimizers, leverage a group of individuals to cooperatively search for the optimum. The exploration-exploitation tradeoff (EET) plays a crucial role in EC, which, however, has traditionally been governed by manually designed rules. In this paper, we propose a deep reinforcement learning-based framework that autonomously configures and adapts the EET throughout the EC search process. The framework allows different individuals of the population to selectively attend to the global and local exemplars based on the current search state, maximizing the cooperative search outcome. Our proposed framework is characterized by its simplicity, effectiveness, and generalizability, with the potential to enhance numerous existing EC algorithms. To validate its capabilities, we apply our framework to several representative EC algorithms and conduct extensive experiments on the augmented CEC2021 benchmark. The results demonstrate significant improvements in the performance of the backbone algorithms, as well as favorable generalization across diverse problem classes, dimensions, and population sizes. Additionally, we provide an in-depth analysis of the EET issue by interpreting the learned behaviors of EC.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. A novel flexible inertia weight particle swarm optimization algorithm. PloS one 11, 8 (2016), e0161558.
  2. Layer normalization. In Proceedings of the 30th Conference on Neural Information Processing Systems.
  3. Improving differential evolution through Bayesian hyperparameter optimization. In Proceedings of the 2021 IEEE Congress on Evolutionary Computation. 832–840.
  4. Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning. In The Twelfth International Conference on Learning Representations.
  5. Optimal contraction theorem for exploration–exploitation tradeoff in search and optimization. IEEE Transactions on Systems, Man, and Cybernetics 39, 3 (2009), 680–691.
  6. A hybrid particle swarm optimizer with sine cosine acceleration coefficients. Information Sciences 422, - (2018), 218–241.
  7. Maurice Clerc and James Kennedy. 2002. The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation 6, 1 (2002), 58–73.
  8. Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1487–1495.
  9. Genetic learning particle swarm optimization. IEEE Transactions on Cybernetics 46, 10 (2016), 2277–2290.
  10. Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems. to be published (2024), to be published.
  11. Protein–protein docking benchmark version 4.0. Proteins: Structure, Function, and Bioinformatics 78, 15 (2010), 3111–3114.
  12. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International conference on machine learning. 448–456.
  13. James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of the International Conference on Neural Networks. 1942–1948.
  14. Pomo: Policy optimization with multiple optima for reinforcement learning. Advances in Neural Information Processing Systems 33, - (2020), 21188–21198.
  15. Discovering Attention-Based Genetic Algorithms via Meta-Black-Box Optimization. In Proceedings of the Genetic and Evolutionary Computation Conference.
  16. Discovering Evolution Strategies via Meta-Black-Box Optimization. In Proceedings of the 11th International Conference on Learning Representations.
  17. A self-adaptive dynamic particle swarm optimizer. In Proceedings of the 2015 IEEE Congress on Evolutionary Computation. 3206–3213.
  18. Jane-Jing Liang and Ponnuthurai Nagaratnam Suganthan. 2005. Dynamic multi-swarm particle swarm optimizer. In Proceedings of 2005 IEEE Swarm Intelligence Symposium. 124–129.
  19. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).
  20. An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning. In Proceedings of the 2019 IEEE Congress on Evolutionary Computation. 815–822.
  21. Nandar Lynn and Ponnuthurai Nagaratnam Suganthan. 2017. Ensemble particle swarm optimizer. Applied Soft Computing 55, 3 (2017), 533–548.
  22. Efficient Neural Neighborhood Search for Pickup and Delivery Problems. In Proceedings of the 31st International Joint Conference on Artificial Intelligence. 4776–4784.
  23. MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning. Advances in Neural Information Processing Systems 36 (2023).
  24. Florian Mischek and Nysret Musliu. 2022. Reinforcement Learning for Cross-Domain Hyper-Heuristics. In Proceedings of the 31st International Joint Conference on Artificial Intelligence. 4793–4799.
  25. Problem definitions and evaluation criteria for the cec 2021 on Single Objective Bound Constrained Numerical Optimization. In Proceedings of the 2021 IEEE Congress on Evolutionary Computation.
  26. Fuzzy Self-Tuning PSO: A settings-free algorithm for global optimization. Swarm and Evolutionary Computation 39, - (2018), 70–85.
  27. Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Transactions on Evolutionary Computation 8, 3 (2004), 240–255.
  28. A new reinforcement learning-based memetic particle swarm optimizer. Applied Soft Computing 43, 3 (2016), 276–297.
  29. Erhard Schmidt. 1907. Zur Theorie der linearen und nichtlinearen Integralgleichungen. Math. Ann. 63, 4 (1907), 433–476.
  30. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).
  31. Deep reinforcement learning based parameter control in differential evolution. In Proceedings of the Genetic and Evolutionary Computation Conference. 709–717.
  32. Yuhui Shi and Russell Eberhart. 1998. A modified particle swarm optimizer. In Proceedings of the IEEE World Congress on Computational Intelligence. 69–73.
  33. Yuhui Shi and Russell C Eberhart. 1999. Empirical study of particle swarm optimization. In Proceedings of the 1999 Congress on Evolutionary Computation. 1945–1950.
  34. Adam Slowik and Halina Kwasnicka. 2020. Evolutionary algorithms and their applications to engineering problems. Neural Computing and Applications 32, 16 (2020), 12363–12379.
  35. NL-SHADE-LBC algorithm with linear parameter adaptation bias change for CEC 2022 Numerical Optimization. In Proceedings of the 2022 IEEE Congress on Evolutionary Computation. 01–08.
  36. Rainer Storn and Kenneth Price. 1997. Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11, 4 (1997), 341–359.
  37. Learning adaptive differential evolution algorithm from optimization experiences by policy gradient. IEEE Transactions on Evolutionary Computation 25, 4 (2021), 666–680.
  38. Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.
  39. Zhiping Tan and Kangshun Li. 2021. Differential evolution with mixed mutation strategy based on deep reinforcement learning. Applied Soft Computing 111, - (2021), 107678.
  40. Differential evolution with hybrid parameters and mutation strategies based on reinforcement learning. Swarm and Evolutionary Computation 75, - (2022), 101194.
  41. Ryoji Tanabe and Alex S Fukunaga. 2014. Improving the search performance of SHADE using linear population size reduction. In Proceedings of the 2014 IEEE Congress on Evolutionary Computation. 1658–1665.
  42. Self regulating particle swarm optimization algorithm. Information Sciences 294, 10 (2015), 182–202.
  43. Evolution Transformer: In-Context Evolutionary Optimization. arXiv e-prints (2024), arXiv–2403.
  44. Attention is all you need. In Proceedings of the 31th Conference on Neural Information Processing Systems. 5998–6008.
  45. Particle swarm optimization algorithm: an overview. Soft Computing 22, 2 (2018), 387–408.
  46. Di Wu and G Gary Wang. 2022. Employing reinforcement learning to enhance particle swarm optimization methods. Engineering Optimization 54, 2 (2022), 329–348.
  47. A fitness-based multi-role particle swarm optimization. Swarm and Evolutionary Computation 44, - (2019), 349–364.
  48. Yue Xu and Dechang Pi. 2020. A reinforcement learning-based communication topology in particle swarm optimization. Neural Computing and Applications 32, 14 (2020), 10007–10032.
  49. Multi-agent Dynamic Algorithm Configuration. In Proceedings of the 36th Conference on Neural Information Processing Systems.
  50. Dynamic multi-strategy integrated differential evolution algorithm based on reinforcement learning for optimization problems. Complex & Intelligent Systems (2023), 1–33.
  51. RLEPSO: Reinforcement learning based Ensemble particle swarm optimizer*. In Proceedings of the 4th International Conference on Algorithms, Computing and Artificial Intelligence. 1–6.
  52. Big bird: Transformers for longer sequences. In Proceedings of the 33th Conference on Neural Information Processing Systems. 17283–17297.
  53. Jingqiao Zhang and Arthur C Sanderson. 2009. JADE: adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation 13, 5 (2009), 945–958.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zeyuan Ma (17 papers)
  2. Jiacheng Chen (37 papers)
  3. Hongshu Guo (15 papers)
  4. Yining Ma (31 papers)
  5. Yue-Jiao Gong (23 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now