Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Constructing Algorithm Portfolios in Algorithm Selection for Computationally Expensive Black-box Optimization in the Fixed-budget Setting (2405.10976v1)

Published 13 May 2024 in cs.LG and cs.AI

Abstract: Feature-based offline algorithm selection has shown its effectiveness in a wide range of optimization problems, including the black-box optimization problem. An algorithm selection system selects the most promising optimizer from an algorithm portfolio, which is a set of pre-defined optimizers. Thus, algorithm selection requires a well-constructed algorithm portfolio consisting of efficient optimizers complementary to each other. Although construction methods for the fixed-target setting have been well studied, those for the fixed-budget setting have received less attention. Here, the fixed-budget setting is generally used for computationally expensive optimization, where a budget of function evaluations is small. In this context, first, this paper points out some undesirable properties of experimental setups in previous studies. Then, this paper argues the importance of considering the number of function evaluations used in the sampling phase when constructing algorithm portfolios, whereas the previous studies ignored that. The results show that algorithm portfolios constructed by our approach perform significantly better than those by the previous approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Gaussian Process Surrogate Models for the CMA Evolution Strategy. Evol. Comput. 27, 4 (2019), 665–697. https://doi.org/10.1162/EVCO_A_00244
  2. Experiments on Greedy and Local Search Heuristics for ddimensional Hypervolume Subset Selection. In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference, Denver, CO, USA, July 20 - 24, 2016, Tobias Friedrich, Frank Neumann, and Andrew M. Sutton (Eds.). ACM, 541–548. https://doi.org/10.1145/2908812.2908949
  3. Surrogate Assisted Feature Computation for Continuous Problems. In Learning and Intelligent Optimization - 10th International Conference, LION 10, Ischia, Italy, May 29 - June 1, 2016, Revised Selected Papers (Lecture Notes in Computer Science, Vol. 10079), Paola Festa, Meinolf Sellmann, and Joaquin Vanschoren (Eds.). Springer, 17–31. https://doi.org/10.1007/978-3-319-50349-3_2
  4. Algorithm selection based on exploratory landscape analysis and cost-sensitive learning. In GECCO. ACM, 313–320. https://doi.org/10.1145/2330163.2330209
  5. New features for continuous exploratory landscape analysis based on the SOO tree. In FOGA. ACM, 72–86. https://doi.org/10.1145/3299904.3340308
  6. Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Inf. Sci. 180, 10 (2010), 2044–2064. https://doi.org/10.1016/J.INS.2009.12.010
  7. Nikolaus Hansen. 2019. A global surrogate assisted CMA-ES. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13-17, 2019, Anne Auger and Thomas Stützle (Eds.). ACM, 664–672. https://doi.org/10.1145/3321707.3321842
  8. COCO: Performance Assessment. CoRR abs/1605.03560 (2016). arXiv:1605.03560 http://arxiv.org/abs/1605.03560
  9. Anytime Performance Assessment in Blackbox Optimization Benchmarking. IEEE Trans. Evol. Comput. 26, 6 (2022), 1293–1305. https://doi.org/10.1109/TEVC.2022.3210897
  10. Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In GECCO. ACM, 1689–1696. https://doi.org/10.1145/1830761.1830790
  11. COCO: a platform for comparing continuous optimizers in a black-box setting. Optim. Methods Softw. 36, 1 (2021), 114–144. https://doi.org/10.1080/10556788.2020.1808977
  12. Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Technical Report. INRIA.
  13. Anja Jankovic and Carola Doerr. 2020. Landscape-aware fixed-budget performance regression and algorithm selection for modular CMA-ES variants. In GECCO. ACM, 841–849. https://doi.org/10.1145/3377930.3390183
  14. The impact of hyper-parameter tuning for landscape-aware performance regression and algorithm selection. In GECCO. ACM, 687–696. https://doi.org/10.1145/3449639.3459406
  15. Automated Algorithm Selection: Survey and Perspectives. Evol. Comput. 27, 1 (2019), 3–45.
  16. Leveraging TSP Solver Complementarity through Machine Learning. Evol. Comput. 26, 4 (2018). https://doi.org/10.1162/EVCO_A_00215
  17. Detecting Funnel Structures by Means of Exploratory Landscape Analysis. In GECCO. ACM, 265–272. https://doi.org/10.1145/2739480.2754642
  18. Pascal Kerschke and Heike Trautmann. 2019a. Automated Algorithm Selection on Continuous Black-Box Problems by Combining Exploratory Landscape Analysis and Machine Learning. Evol. Comput. 27, 1 (2019), 99–127. https://doi.org/10.1162/EVCO_A_00236
  19. P. Kerschke and H. Trautmann. 2019b. Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-package flacco. In Applications in Statistical Computing – From Music Data Analysis to Industrial Quality Improvement. Springer, 93–123.
  20. S.T.E.P.: The Easiest Way to Optimize a Function. In Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, Orlando, Florida, USA, June 27-29, 1994. IEEE, 519–524. https://doi.org/10.1109/ICEC.1994.349896
  21. Evolution by adapting surrogates. Evolutionary computation 21, 2 (2013), 313–340.
  22. The algorithm selection competitions 2015 and 2017. Artif. Intell. 272 (2019), 86–100. https://doi.org/10.1016/J.ARTINT.2018.10.004
  23. Exploratory landscape analysis. In GECCO. ACM, 829–836. https://doi.org/10.1145/2001576.2001690
  24. Mario A. Muñoz and Michael Kirley. 2016. ICARUS: Identification of complementary algorithms by uncovered sets. In IEEE Congress on Evolutionary Computation, CEC 2016, Vancouver, BC, Canada, July 24-29, 2016. IEEE, 2427–2432. https://doi.org/10.1109/CEC.2016.7744089
  25. Exploratory Landscape Analysis of Continuous Space Optimization Problems Using Information Content. IEEE Trans. Evol. Comput. 19, 1 (2015), 74–87.
  26. Raphael Patrick Prager and Heike Trautmann. 2024 (in press). Pflacco: Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems in Python. Evol. Comput. (2024 (in press)).
  27. Parallelized bayesian optimization for expensive robot controller evolution. In PPSN. Springer, 243–256.
  28. J. R. Rice. 1976. The Algorithm Selection Problem. Adv. Comput. 15 (1976), 65–118.
  29. Ryoji Tanabe. 2022. Benchmarking feature-based algorithm selection systems for black-box numerical optimization. IEEE Trans. Evol. Comput. 26, 6 (2022), 1321–1335.
  30. Neural Network Design: Learning from Neural Architecture Search. In 2020 IEEE Symposium Series on Computational Intelligence, SSCI 2020, Canberra, Australia, December 1-4, 2020. IEEE, 1341–1349. https://doi.org/10.1109/SSCI47803.2020.9308394
  31. Recent Advances in Bayesian Optimization. ACM Comput. Surv. 55, 135 (2023), 1–36.
  32. SATzilla: Portfolio-based Algorithm Selection for SAT. J. Artif. Intell. Res. 32 (2008), 565–606. https://doi.org/10.1613/JAIR.2490

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now