Exploratory Landscape Analysis (ELA)

• Exploratory Landscape Analysis (ELA) is a sample-based method that computes numerical features from initial evaluations to characterize optimization landscapes.
• ELA extracts descriptors such as modality, ruggedness, separability, and conditioning to inform problem understanding and guide algorithm selection.
• It integrates techniques like PCA, level-set analysis, and clustering, making it applicable to continuous, mixed-variable, and hyperparameter optimization scenarios.

Exploratory Landscape Analysis (ELA) is a sample-based methodology for characterizing black-box optimization problems through numerical features computed from an initial design of evaluated points. Given a set of decision vectors and their objective values, ELA produces low-level descriptors that act as proxies for high-level properties such as modality, ruggedness, separability, conditioning, convexity, funnel structure, and global structure. In continuous single-objective optimization, these descriptors are used for problem understanding, benchmark-set analysis, performance prediction, and automated algorithm selection or configuration; later work extends the same basic idea to hyperparameter optimization, neural architecture search, mixed-variable domains, and quality-diversity settings (Kerschke, 2017).

1. Core definition and workflow

In its standard form, ELA starts from a modest sample of input–output pairs

$$S=\{(x_i,y_i)\mid x_i\in\mathcal X,\; y_i=f(x_i)\},$$

typically drawn uniformly, by Latin Hypercube Sampling, or by quasi-random designs. The goal is not to solve the optimization problem directly, but to characterize the landscape induced by $f$ before or alongside optimization. A common workflow is: generate an initial design, evaluate the objective, normalize inputs and sometimes outputs, compute feature sets on the resulting sample, and feed the feature vector into a downstream model such as a classifier or regressor for algorithm selection, configuration, or high-level property prediction (Kerschke, 2017).

This workflow is explicitly tied to supervised meta-learning in much of the literature. By mapping each problem instance to a point in a feature space, one can compare instances structurally, cluster benchmark suites, or predict whether a new problem is multimodal, badly conditioned, separable, or likely to favor a particular solver. In this sense, ELA functions as a “white-box” diagnostic tool for black-box optimization: it does not recover the analytic form of $f$, but it turns finite observations into structured geometric and statistical summaries (Tanabe, 2021).

The practical convention $N=50\cdot D$, where $D$ is the dimension of the decision space, recurs across several studies. It is used for classical continuous ELA pipelines, for comparisons between BBOB and hyperparameter-optimization landscapes, and for mixed-variable preprocessing pipelines. Higher budgets are also used in benchmark analysis; for example, $n=250\cdot D$ was used in the ELA study of GKLS, BBOB, and CEC 2014, while large-sample reference distributions such as $100D$ or $1000D$ appear in robustness studies (Schneider et al., 2022).

2. Principal feature families

The classical ELA portfolio is implemented in the flacco ecosystem and contains over 300 features organized into 17 sets. Among the most widely used families are y-distribution, level-set, meta-model, dispersion, information-content, nearest-better clustering, curvature, convexity, local-search, PCA, cell-mapping, generalized cell-mapping, and barrier-tree features (Kerschke, 2017).

Feature family	Representative quantities	Intended structure
y-distribution	skewness, kurtosis, number of peaks	bias, heavy tails, multimodality proxies
meta-model	adjusted $R^2$, intercept, coefficient magnitudes, condition numbers	smoothness, curvature, polynomial approximability
level-set	classifier errors on thresholded elites	separability and shape of good regions
dispersion	pairwise-distance ratios among elite subsets	clustering of good solutions
information content	entropy-based symbolic measures	ruggedness and neutrality
nearest-better clustering	nearest-neighbor vs nearest-better statistics	funnels and local basin structure
PCA	explained-variance ratios, number of PCs	anisotropy and effective dimension
cell-mapping / barrier	transition, angle, convexity, basin-tree measures	local and global topology

For y-distribution features, sample moments of the objective values are central. Typical definitions include skewness and excess kurtosis,

$$\mathrm{skew}=\frac{1}{n}\sum_{i=1}^n\Bigl(\frac{y_i-\bar y}{s}\Bigr)^3,\qquad \mathrm{kurtosis}=\frac{1}{n}\sum_{i=1}^n\Bigl(\frac{y_i-\bar y}{s}\Bigr)^4-3,$$

with $f$0 the sample variance. These descriptors are used in continuous optimization, hyperparameter optimization, NAS landscape studies, and mixed-variable ELA alike (Schneider et al., 2022).

Meta-model features fit simple surrogates to $f$1. The most basic linear approximation is

$f$2

while quadratic variants add second-order terms or pairwise interactions. ELA records descriptors such as $f$3, adjusted $f$4, coefficient magnitudes, intercepts, and condition numbers of Hessian-like matrices. These features measure how well the landscape can be approximated by low-order polynomials and therefore provide coarse information on nonlinearity, global smoothness, and conditioning (Tanabe, 2021).

Level-set features partition the sample by objective thresholds, often empirical quantiles such as $f$5, $f$6, $f$7, or $f$8. One then trains discriminants such as LDA, QDA, or MDA to separate “good” and “bad” regions and records cross-validation errors or related misclassification statistics. In the flacco-based formulation emphasized for large-scale optimization, level-set features quantify the nonlinearity of these thresholded regions through 10-fold cross-validation errors of linear, quadratic, or mixture discriminant analyses (Tanabe, 2021).

Dispersion features compare the spread of elite subsets to that of the full sample. A recurring pattern is to compute distances among all points and among the top $f$9-fraction by fitness, then record either differences or ratios, such as

$f$0

Small elite-set distances indicate that good solutions are clustered; broader elite subsets suggest weak concentration or broad basins (Stein et al., 2020).

Information-content features are based on symbolic encodings of fitness differences along a sequence or random walk. With threshold $f$1, one defines symbols

$f$2

and then computes entropy-like quantities such as

$f$3

Derived measures include $f$4, settling sensitivities, partial information content, and $f$5-ratios. These descriptors target ruggedness, neutrality, and scale-dependent structure (Schneider et al., 2022).

Nearest-better clustering (NBC) features construct a graph by linking each point to the closest strictly better point. Statistics of distances to the nearest neighbor and nearest better neighbor, ratios between them, or correlations with fitness differences are then used to diagnose funnel structure and local basin organization. A typical quantity is

$f$6

Funnel-like landscapes tend to produce characteristic nearest-better patterns, which is why NBC has become a standard ingredient of ELA-based algorithm selection (Schneider et al., 2022).

3. Software ecosystem and standard practice

The main software substrate for ELA is the R package flacco, which consolidates feature computation and visualization for continuous single-objective black-box optimization and also provides a GUI. It implements classical ELA feature sets, information-content, dispersion, nearest-better clustering, cell-mapping, generalized cell-mapping, barrier-tree features, and PCA-based descriptors, and it exposes a unified interface for building feature objects, computing sets selectively or jointly, and visualizing cell maps, barrier trees, and information-content plots (Kerschke, 2017).

A Python ecosystem has emerged around the same methodology. The large-scale ELA study refers to pflacco as the Python wrapper of flacco and uses it to analyze runtime scaling and reduced-space feature computation. Later work on mixed-variable ELA also centers its implementation on pflacco, focusing on the feature subsets ela_distr, ela_meta, disp, ic, and nbc (Tanabe, 2021).

Standard preprocessing conventions depend on the application. Search-space normalization to $f$7 or $f$8 is common; for example, NAS studies rescale hyperparameters to $f$9 to align with BBOB, while HPO comparisons standardize objective values to zero mean and unit variance after scaling the inputs to $N=50\cdot D$0. A later line of work argues that objective-value min–max normalization to $N=50\cdot D$1 should be applied before computing any ELA feature so that features become invariant under affine transformations $N=50\cdot D$2; this is particularly important when training and testing across transformed benchmark instances (Prager, 2023).

The literature also emphasizes that ELA feature computation is not a purely abstract mathematical mapping; it is tied to concrete sampling and normalization choices. That dependence is methodological rather than incidental. A plausible implication is that feature vectors are best interpreted as properties of a problem under a specified observation protocol rather than as absolute invariants of the underlying objective function.

4. Sampling dependence and scalability

A central empirical result is that ELA is strongly sensitive to the sampling strategy. Comparative experiments with uniform random sampling, Latin hypercube variants, and Sobol’ low-discrepancy sequences show that increasing the number of sampled points reduces estimator variance, but feature approximations produced by different sampling strategies do not converge to the same value. This implies that approximated feature values cannot be interpreted independently of the sampling strategy, and that classifiers trained for algorithm selection should use the same design method at training and deployment time. In the reported classification experiments on BBOB, Sobol’-based feature computation achieved the highest accuracy under fixed budgets (Renau et al., 2020).

High-dimensional scalability is a second major issue. For large-scale optimization, flacco feature classes such as ela_level, ela_meta, and cell-mapping become computationally prohibitive. The large-scale analysis reports empirical runtime laws in pflacco of approximately

$N=50\cdot D$3

where $N=50\cdot D$4 denotes the dimension and $N=50\cdot D$5 the sample size, with $N=50\cdot D$6 typical. In the reported workstation experiments, ela_level took over 1.5 hours at $N=50\cdot D$7, ela_meta over 1.1 hours at $N=50\cdot D$8, both became intractable for $N=50\cdot D$9, and cell-mapping exploded even at moderate dimensions because of the $D$0 dependence (Tanabe, 2021).

To address this bottleneck, a PCA-based dimensionality reduction framework was proposed for large-scale ELA. One samples $D$1 points in the original space, optionally reweights them by rank-based weights

$D$2

extracts the top $D$3 principal components, projects the design to $D$4, and computes the expensive feature sets on the reduced data $D$5. This produces reduced-space variants such as d_ela_level, d_ela_meta, and d_cm_*. At $D$6, the reported wall-clock times changed from 1.5 hours to 2 minutes for ela_level and from 1.1 hours to 1 minute for ela_meta, while formerly infeasible cell-mapping features became computable in a few minutes. On the 24 large-scale noiseless BBOB functions up to $D$7, reduced-space features retained much of the predictive power of the original features for seven high-level properties, and Kendall’s $D$8 between original and reduced meta-model coefficients exceeded $D$9 for many key features (Tanabe, 2021).

This PCA-based result does not generalize to all forms of dimensionality reduction. A later study on Random Gaussian Embeddings in $n=250\cdot D$0 reports that only a small subset of features remains stable under random projection; many level-set, quadratic meta-model, NBC, and information-content features exhibit large relative shifts, and apparently robust features may still reflect projection-induced artifacts rather than intrinsic landscape characteristics. That study therefore recommends restricting projected ELA to features that depend largely on objective values or coarse global geometry if random projections are unavoidable (Rodríguez et al., 14 Apr 2026).

A distinct recent line tackles budget-constrained sampling rather than post hoc projection. In $n=250\cdot D$1 with only $n=250\cdot D$2 evaluations, random linear embedding sampling can improve the robustness of some feature classes. The reported recommendations are feature-class dependent: Full/ELA$n=250\cdot D$3 is best for ela_meta.lin_simple, Sliced/ELA$n=250\cdot D$4 with $n=250\cdot D$5 often performs best for disp, ic, and ela_distr, and All_in/ELA$n=250\cdot D$6 with $n=250\cdot D$7 ranks best for ela_level. This suggests that the geometry of the sampling design and the geometry of the feature definition interact in nontrivial ways (Rodríguez et al., 8 Jul 2026).

5. Applications to optimization domains and benchmark analysis

ELA has been used to compare and organize benchmark suites. A detailed study of the GKLS generator computes flacco features on GKLS-simple, GKLS-hard, and GKLS-mod in $n=250\cdot D$8, as well as on BBOB and CEC 2014. After removing constant, invalid, and highly correlated features, 14 descriptors from six groups remained. The GKLS feature ranges were very narrow; for example, number_of_peaks was always in $n=250\cdot D$9 and quadratic adjusted $100D$0 was in $100D$1. In contrast, BBOB and CEC 2014 exhibited much broader ranges, such as number_of_peaks up to 18 and 26, respectively. The study concludes that GKLS instances cluster tightly and are structurally dominated by a paraboloid-like component with one small global basin, which questions their representativeness for broader multimodal benchmarking (Kudela et al., 2023).

Hyperparameter optimization provides a real-world application domain in which ELA is used both for structural comparison and for optimizer transfer. In one study, 30 continuous XGBoost HPO problems in 2D, 3D, and 5D were contrasted with 360 BBOB instances. PCA of approximately 100 ELA features indicated that principal components loaded on multimodality and effective dimensionality; k-means with $100D$2 then separated strongly multimodal problems from unimodal or weakly multimodal ones. The HPO problems fell exclusively into the low- to moderate-multimodality clusters, and their nearest BBOB neighbors were typically unimodal, well-conditioned functions such as Sphere, Ellipsoid, and one-funnel Rotated Rosenbrock variants. Optimizer rankings on HPO and on these matched BBOB subsets were reported to be comparably similar, with MBO and CMA-ES leading on both (Schneider et al., 2022).

ELA has also been applied to neural architecture search. For a 23-dimensional CNN design space over MNIST, Fashion-MNIST, and CIFAR-10, 20 ELA features from five groups were bootstrapped over $100D$3-point Latin Hypercube designs. The reported distributions distinguish the three datasets: CIFAR-10 exhibits the highest $100D$4, Fashion shows tighter clustering of elite points via $100D$5, and $100D$6 for all three, indicating strong nonlinearity. When NAS landscapes were embedded jointly with random instances of the 24 BBOB functions, all NAS replicates formed a cluster separate from every BBOB cluster, and the nearest BBOB functions were Bent Cigar, Attractive Sector, and Sharp Ridge. The authors therefore identify NAS landscapes as a distinct problem class and motivate dedicated benchmark augmentation and optimizer studies for NAS (Stein et al., 2020).

Preliminary work has also extended ELA to quality-diversity optimization. When ELA features are computed on archives produced by CVT-MAP-Elites rather than on random LHS samples, median trajectories of many features diverge significantly from the random baseline. The reported shifts depend on the variation operator, behavior function, archive size, and genotype dimensionality, suggesting that ELA can characterize not only static objective landscapes but also archive-induced sampling distributions in QD (Mosphilis et al., 2024).

6. Mixed-variable extensions and deep-learning alternatives

Classical ELA was originally centered on continuous domains. Mixed-variable problems complicate this because categorical variables have no inherent Euclidean geometry and hierarchical dependencies can create inactive regions and plateaus. A 2024 mixed-variable extension addresses this by numerical encodings prior to applying standard ELA feature sets. Two central schemes are one-hot encoding and target encoding. In the target-encoding formulation,

$100D$7

with $100D$8, so that each category is mapped to a single real value reflecting its response profile. Using this preprocessing on 702 YAHPO Gym instances, agglomerative clustering in ELA space produced three stable clusters with different dominant algorithms, and a random-forest algorithm selector closed approximately $100D$9 of the gap between the single best and virtual best solver over the benchmark suite. Target encoding was reported as both faster and better than one-hot encoding in that study (Prager et al., 2024).

A subsequent study investigated SHAP-based encodings for mixed-variable ELA. Here, a random-forest surrogate is first fitted on the raw mixed-variable sample, SHAP values are computed for categorical features, and the resulting contribution values replace the category labels before standard ELA computation. On eight mixed-variable HPO scenarios from YAHPO Gym, target-encoded and SHAP-encoded ELA features yielded nearly identical mean relERT of about $1000D$0, but they showed strong complementarity across instances. A virtual best choice between the two encodings reduced relERT to about $1000D$1, while implementable meta-selection strategies based on either a binary meta-model or prediction confidence reduced relERT to about $1000D$2 and $1000D$3, respectively (Dietrich et al., 2024).

Alongside these preprocessing-based extensions, several strands of research replace or augment hand-crafted ELA by learned representations. One feature-free approach converts initial samples into point clouds or 2D “fitness maps” and trains deep models such as ShuffleNet V2 or a Point Cloud Transformer to predict high-level properties on the BBOB testbed. In the reported 2D results, classical ELA achieved macro-$1000D$4 overall, ShuffleNet reached $1000D$5, and the PCT also reached $1000D$6; the point-cloud method was reported to remain strong at higher dimensions, while PCA-based image reductions degraded for $1000D$7 (Seiler et al., 2022).

DoE2Vec pursues a different route: a variational autoencoder learns latent codes directly from normalized objective values on a fixed Sobol design of size $1000D$8. Trained on $1000D$9 random surrogate functions in $R^2$0, the learned codes reconstruct the 24 BBOB functions in 2D and support downstream classification of multimodality, global structure, and funnel presence. Classical ELA alone remained very strong, especially in lower dimensions, but combined ELA + VAE-32 features consistently gave the best macro-$R^2$1, including $R^2$2–$R^2$3 for funnel detection across all tested dimensions and $R^2$4–$R^2$5 for multimodality (Stein et al., 2023).

Deep-ELA generalizes the learned-representation perspective to single- and multi-objective continuous optimization by pretraining transformer encoders in a self-supervised way on millions of random problems. It was proposed explicitly to address two drawbacks of classical ELA: strong feature correlations and limited applicability to multi-objective settings. The reported correlation matrices show much lower inter-feature correlation for Deep-ELA embeddings than for classical ELA, and the learned features support high-level property prediction, single-objective algorithm selection, and multi-objective algorithm selection without requiring manually designed summary statistics (Seiler et al., 2024).

7. Limitations, interpretability, and open directions

ELA is not free of methodological caveats. Sampling sensitivity means that feature values are contingent on the design method; lack of invariance can produce misleadingly high accuracy if benchmark transformations leak through raw objective scales; high-dimensional sparsity inflates estimator variance; and some feature families become computationally infeasible long before function evaluation itself dominates the cost (Renau et al., 2020).

Interpretability within ELA is itself a research topic. An exhaustive feature-selection study on BBOB showed that a surprisingly small number of flacco features—often fewer than four—can suffice to achieve $R^2$6 classification accuracy under repeated subsampling, and that the minimal subset size can decrease as dimension increases. The intercept of a simple linear model was often highly discriminative, while information-content ratios appeared in nearly all minimal subsets of size at least two. However, leave-one-instance-out performance dropped sharply for non-invariant singleton features, reinforcing the importance of transformation-invariant descriptors when generalization across instances is required (Renau et al., 2021).

Current open directions are diverse but coherent. For large-scale ELA, the literature proposes nonlinear dimensionality reduction, hybrid reduced/full-space schemes, and genuinely large-scale-aware features operating directly on compressed representations. For mixed-variable ELA, open issues include hierarchical activity patterns, mixed-variable space-filling designs, and feature definitions that incorporate categorical semantics directly rather than via preprocessing. For learned landscape representations, remaining themes include interpretability of latent dimensions, invariance to transformations, adaptation to arbitrary custom designs, and integration with online or dynamic algorithm selection. More broadly, recent work on automatic generation of benchmark functions from target ELA vectors treats ELA not merely as a diagnostic language, but as a constructive design space for synthesizing new test landscapes (Tanabe, 2021).