A Survey of Meta-features Used for Automated Selection of Algorithms for Black-box Single-objective Continuous Optimization (2406.06629v1)

Abstract: The selection of the most appropriate algorithm to solve a given problem instance, known as algorithm selection, is driven by the potential to capitalize on the complementary performance of different algorithms across sets of problem instances. However, determining the optimal algorithm for an unseen problem instance has been shown to be a challenging task, which has garnered significant attention from researchers in recent years. In this survey, we conduct an overview of the key contributions to algorithm selection in the field of single-objective continuous black-box optimization. We present ongoing work in representation learning of meta-features for optimization problem instances, algorithm instances, and their interactions. We also study machine learning models for automated algorithm selection, configuration, and performance prediction. Through this analysis, we identify gaps in the state of the art, based on which we present ideas for further development of meta-feature representations.

• The paper provides a comprehensive survey of meta-features used to guide algorithm selection in black-box optimization.
• It evaluates diverse feature types, including problem landscape, algorithm, and trajectory-based features for performance prediction.
• It identifies challenges in generalizability and computational cost while suggesting future research paths to enhance automated selection methods.

Survey of Meta-features Used for Automated Selection of Algorithms for Black-box Single-objective Continuous Optimization

A comprehensive survey has been conducted to elucidate the current state of research on meta-features and their application in the automated selection of algorithms, specifically targeting single-objective continuous black-box optimization problems. This survey analyses the existing body of work on algorithm selection, identifying gaps, and proposing potential avenues for future research. This essay provides an insightful overview of the paper, curated for an audience of experienced researchers.

Introduction

Algorithm selection (AS) for black-box optimization involves selecting the most appropriate algorithm for solving a given optimization problem instance from a set of candidates. This selection is driven by leveraging the complementary performance characteristics of various algorithms. However, the challenge lies in determining the optimal algorithm for unseen problem instances, which has been a pivotal focus in recent research.

Researchers have utilized meta-learning to train supervised Machine Learning (ML) models for this purpose. These ML models predict the performance of algorithm instances on problem instances or directly classify problems by indicating the best-fitting algorithm. This process relies on input features—both from the problem instances and the algorithms themselves. The quality of these features critically influences the success of AS methods.

Components of an Algorithm Selection Pipeline

The standard pipeline for algorithm selection comprises several components, each contributing crucial information:

1. Problem Portfolio: A diverse set of optimization problem instances. For academic evaluations, commonly used benchmarks include BBOB and CEC.
2. Algorithm Portfolio: Various algorithm instances like Differential Evolution, CMA-ES, and PSO.
3. Algorithm Evaluation: Methods like fixed-budget and fixed-target evaluation to measure algorithm performance.
4. Problem Landscape Features: Numerical features describing the optimization problem instance.
5. Algorithm Features: Characteristics of algorithm instances.
6. Trajectory-based Features: Interaction data between the problem instance and the algorithm instance during execution.
7. Machine Learning: Models trained to perform AS using features from Problem Landscape, Algorithm, and their trajectory interactions.

Problem Landscape Features

Problem landscape features are well-researched and can be broadly categorized into high-level features (like FLA) and low-level features (like ELA), each capturing different aspects of problem instances:

1. High-level Features: Include properties like multi-modality, separability, and search space homogeneity. These features are often interpretable.
2. Low-level Features: Include ELA, TLA, and deep learning approaches. ELA features, though prevalent, face challenges related to computational cost and sensitivity to sample strategies and problem transformations.

Exploratory Landscape Analysis (ELA)

ELA features are derived from statistical measures applied to samples from the decision space. These include characteristics such as convexity, distribution, local search properties, levelset descriptions, meta-model fits, and more. There are 343 different ELA features provided by the flacco R package. However, ELA features face challenges including robustness to sampling methods and invariance under problem transformations.

Topological and Deep Learning Features

• TLA: Provides robustness against transformations like scaling and shifting. Shows promise for problem classification.
• Deep Learning Features: Include CNNs and transformers trained on fitness maps or direct problem samples, respectively. These approaches offer strong potential in extracting high-quality features, though they may lack interpretability.

Algorithm Features

Algorithm features, often underrated, play a significant role:

1. Source Code Features: Dependent on the actual implementation and programming language.
2. Performance-based Features: Performance2Vec creates representations based on algorithm performance across benchmarks.
3. Explainable Models: SHAP values from performance prediction models provide insight into algorithm behavior.
4. Graph Embeddings: Utilizing knowledge graphs for generating algorithm features based on interactions with problem instances.

Trajectory-Based Features

Trajectory-based features capture real-time interactions between algorithms and problem instances, which can be crucial for dynamic AS. Several methods have been proposed:

1. Internal Algorithm Parameters: Time-series features derived from internal parameters of algorithms.
2. Trajectory-ELA: Features based on the optimization trajectory rather than pre-sampled candidate solutions.
3. DynamoRep and Opt2Vec: Utilize simple statistics and autoencoders for trajectory representation.
4. Iterative-based ELA: ELA features calculated iteratively to capture dynamics.
5. Probing Trajectories: Short runs of initial algorithms to determine subsequent algorithm selection.

Studies Across Benchmark Suites

A significant portion of research utilizes the BBOB benchmark suite. Comparative studies highlight the necessity to enhance the generalizability of AS models beyond BBOB to other benchmarks like CEC and Nevergrad. The table in the paper provides a detailed overview of studies using various features for different learning tasks.

Discussion and Open Challenges

While significant progress has been made, several challenges remain:

1. Issue of Generalization: Models limit their applicability to specific benchmarks. Future work needs to focus on enhancing generalizability.
2. Comprehensive Comparison of Features: Little work has compared deep learning features with traditional ELA, limiting understanding of their relative effectiveness.
3. Online Feature Calculation: Trajectory-based features should be explored for online AS to dynamically adapt to problem instances.

Conclusion

The survey provides a comprehensive review of meta-features used in algorithm selection for single-objective continuous black-box optimization. Key observations indicate that, while substantial groundwork has been laid, significant opportunities lie in improving the generalizability and effectiveness of AS models through better features and ML methods. Future research should emphasize developing invariant features, extensive comparative studies, and continual learning strategies to create robust, generalizable AS models.