Area Over the Convergence Curve (AOCC)

• AOCC is a scalar metric that summarizes the entire optimization trajectory by averaging normalized, best-so-far objective values.
• The methodology transforms, clips, normalizes, and inverts performance values before averaging over a fixed evaluation budget.
• Using AOCC aids in comparing PSO configurations and topologies, supporting explainable ML models for algorithm recommendation.

Searching arXiv for the cited AOCC paper and closely related benchmarking/explainer work. (Gupta et al., 8 Sep 2025) Searching (Gupta et al., 8 Sep 2025) "An Explainable Framework for Particle Swarm Optimization using Landscape Analysis and Machine Learning" Area over the Convergence Curve (AOCC) is a scalar performance measure that summarizes the full optimization trajectory of an algorithm from the sequence of best-so-far objective values, rather than only its terminal performance. In the Particle Swarm Optimization (PSO) study “An Explainable Framework for Particle Swarm Optimization using Landscape Analysis and Machine Learning,” AOCC is the central benchmarking quantity inside an adapted IOHxplainer workflow and the target signal for downstream machine-learning models that recommend PSO configurations and topologies from Exploratory Landscape Analysis (ELA) features (Gupta et al., 8 Sep 2025). In that setting, AOCC is defined as a discrete, normalized average over the optimization horizon, computed from transformed and clipped best-found function values.

1. Definition and mathematical form

The study gives AOCC explicitly as

$$AOCC(\bar y) = \frac{1}{B} \sum_{i=1}^{B} \left[1- \frac{(\min(\max((y_i),lb),ub)-lb)}{(ub-lb)}\right].$$

Here, $\bar y$ is the series of best-found function values during a run, $y_i$ is the recorded best-so-far function value at evaluation or iteration index $i$, $B$ is the evaluation budget or horizon, and $lb, ub$ are lower and upper bounds on the transformed function-value range (Gupta et al., 8 Sep 2025).

The formula embodies four design choices stated in the study. First, each logged value is clipped by $\min(\max(y_i,lb),ub)$, so contributions are truncated to a prescribed range. Second, the clipped value is linearly normalized by $\frac{\min(\max(y_i,lb),ub)-lb}{ub-lb}$ into $[0,1]$. Third, the transformation is inverted by $1-\cdot$, so lower objective values yield larger contributions. Fourth, the contributions are averaged over the full budget $\bar y$0, making AOCC a normalized area-over-curve quantity for the entire run (Gupta et al., 8 Sep 2025).

The operational implication is direct: higher AOCC corresponds to better best-so-far values maintained for a larger fraction of the run, whereas lower AOCC corresponds to worse values over much of the budget, slower convergence, stagnation, or improvement that occurs only late in the trajectory. This makes AOCC a trajectory-sensitive metric rather than an endpoint-only score (Gupta et al., 8 Sep 2025).

2. Convergence-curve interpretation and ECDF relation

In the study, AOCC is computed from the convergence curve of best-so-far objective values. The framework records the “best-so-far fitness value” at high frequency, often after every evaluation, thereby producing a complete record of performance convergence over time. AOCC then uses the resulting series of best-found function values $\bar y$1 within a given budget $\bar y$2 (Gupta et al., 8 Sep 2025).

The paper also places AOCC in an ECDF-based benchmarking interpretation. It states that the measure “builds upon the empirical cumulative distribution function (ECDF),” and that AOCC “represents the integrated area beneath the ECDF curve across an infinite set of target values within the specified bounds” (Gupta et al., 8 Sep 2025). In that interpretation, AOCC inherits an anytime-performance perspective: the scalar summarizes how favorable the convergence profile is over the full run, not merely whether a single target is attained.

At the same time, the implemented quantity is not a continuous integral. The formula is a summation over $\bar y$3, so the “area” is a discretized estimate over the optimization horizon. This discrete character matters because the recorded trajectory is sampled through the logging scheme and budget parameter, and the study’s wording alternates between “iterations” and “function evaluations” when discussing that horizon (Gupta et al., 8 Sep 2025).

3. Normalization, preprocessing, and experimental computation

The reported experiments compute AOCC only after logarithmic transformation and clipping of function values. The paper states: “Following standard BBOB benchmarking practices, all function values were logarithmically transformed prior to the AOCC calculation, with values cut to the $\bar y$4 range for $\bar y$5 and $\bar y$6 problems.” For those experiments, this implies $\bar y$7 and $\bar y$8 (Gupta et al., 8 Sep 2025).

A concise summary of the symbols and preprocessing used in the study is given below.

Quantity	Meaning
$\bar y$9	Series of best-found function values
$y_i$0	Best-so-far function value at index $y_i$1
$y_i$2	Evaluation budget or horizon
$y_i$3	Bounds on transformed function-value range
Preprocessing	Log transform, then clipping to $y_i$4

The experimental workflow is also stated explicitly. A run begins with a specific PSO configuration drawn from a space that includes topology—Star, Ring, or Von Neumann—and hyperparameters from Table 2: $y_i$5, $y_i$6, $y_i$7, $y_i$8, and topology-specific settings $y_i$9, $i$0 for Ring and $i$1 for Von Neumann. The paper states that 1,728 PSO configurations were examined on the BBOB suite of 24 noiseless functions in dimensions $i$2 and $i$3 (Gupta et al., 8 Sep 2025).

The computational budget is reported as 100 and 500 iterations, although the narrative also refers to a fixed budget of function evaluations. The implementation pseudocode uses optimizer.optimize(func, iters=budget). The safest reading is therefore that AOCC is defined over a generic run horizon $i$4, while the experiments discretize that horizon through the budget used by the optimization loop (Gupta et al., 8 Sep 2025).

Within each run, the framework logs the best-so-far fitness values at high frequency, often after every evaluation, yielding $i$5. Each transformed and clipped $i$6 contributes

$i$7

and AOCC is the average of these contributions across the full horizon. The resulting run-level AOCC values are then aggregated for later comparison over instances, repetitions, functions, and configurations. The prose reports “five independent runs across the first five instances of each benchmark function,” while Algorithm 1 uses iids = [1,2,3,4,5] and reps = 3; the study is therefore explicit about run-level AOCC construction but not fully consistent about repetition counts (Gupta et al., 8 Sep 2025).

4. Function within explainability and configuration recommendation

AOCC is the main quantitative response variable for the study’s explainability analysis. The paper states that configurable PSO is integrated into the IOHxplainer framework “to generate Area over the Convergence Curve (AOCC) performance data,” and that SHAP-based plots are used to analyze parameter contributions, with SHAP values indicating how each parameter affects AOCC (Gupta et al., 8 Sep 2025). In this usage, each run or configuration receives an AOCC score, and explainable ML methods attribute variation in that score to topology and hyperparameters.

AOCC is also the supervisory signal in the machine-learning stage. The study states that Decision Tree and Random Forest classifiers are trained on the run data to predict performance classes, that the workflow is for “classifying PSO AOCC performance using an expanded set of ELA features,” and that the toolbox supports predicting “the most suitable algorithm configuration ... based on observed characteristics of the optimization landscape” (Gupta et al., 8 Sep 2025). The text mixes regression and classification language, but the directly stated target is AOCC-based performance for selecting or recommending PSO configurations.

The input side of that predictive mapping consists of ELA features computed from sampled landscapes. The paper states that 1000 points are sampled from each BBOB instance and cites expanded feature sets including nbc.nb_fitness.cor, ela_meta.quad_simple.cond, disp.diff_mean_10, ela.distr.skewness, disp.diff.mean_02, ela_meta.lin_simple.coef.max_by_m, and nbc.nn_nb.mean_ratio (Gupta et al., 8 Sep 2025). AOCC thereby serves as the scalar notion of performance quality that links benchmarking, explainability, and algorithm recommendation.

This role of AOCC also clarifies why the metric is treated as preferable to final objective value alone in the paper’s framework. Because the average is taken over the full trajectory, AOCC rewards early progress, stable improvement, and sustained solution quality throughout the budget. Two algorithms with the same terminal value may therefore receive different AOCC scores if one attains good values earlier (Gupta et al., 8 Sep 2025).

5. Benchmarking results and topology-specific interpretation

The empirical analyses compare AOCC across Random Forest (RF), Decision Tree (DT), Single Best (SB), and Average Best (AB) configurations for PSO over all topologies and all BBOB functions, under both LoFo and LoIo validation. Figure 1 reports “AOCCs of RF, DT, SB, and AB configurations for PSO using all topologies on all functions using LoFo,” Figure 2 reports the corresponding LoIo comparison, and Figures 11–13 report “AOCC performance loss” for RF, shallow DT, and AB against the best single run (Gupta et al., 8 Sep 2025).

From those AOCC analyses, the study concludes that Star topology benefits more from model-guided configuration and that RF often performs best there; Ring topology shows similar performance among AB, DT, and RF, with AB sometimes slightly more stable; and Von Neumann topology often has AB as the strongest and most stable performer. The paper also states that LoIo validation generally gives tighter AOCC distributions than LoFo, suggesting better generalization across unseen instances than across unseen functions (Gupta et al., 8 Sep 2025).

The topology-level interpretations are likewise tied to AOCC-based benchmarking and convergence analysis. The study characterizes Star as exhibiting fast convergence and being especially good for simple unimodal problems; Ring as better at diversity maintenance and preferable on rugged multimodal landscapes; and Von Neumann as delivering balanced exploration–exploitation, stronger robustness, effectiveness on complex or deceptive functions, and the best time efficiency (Gupta et al., 8 Sep 2025). Because AOCC is the explicit metric used to summarize convergence behavior over the run, these topology conclusions are not restricted to endpoint quality.

A plausible implication is that AOCC acts as a unifying measure across several decision layers in the framework: it supports topology comparison, parameter-importance explanation, and data-driven recommendation under a shared trajectory-sensitive definition of optimization efficiency. That unification is stated most clearly in the paper’s compact summary, which identifies AOCC as the core performance target for comparison across Star, Ring, and Von Neumann topologies, for SHAP-based hyperparameter importance, and for interpretable ML models trained from ELA features (Gupta et al., 8 Sep 2025).

6. Scope, caveats, and distinction from other area-based criteria

The paper states several limitations and ambiguities around AOCC. First, AOCC depends on preprocessing because it is not computed on raw objective values but on log-transformed and clipped values. This improves comparability across benchmark functions and dimensions, but it also means that the metric depends on the chosen transformed-value scale (Gupta et al., 8 Sep 2025). Second, AOCC is budget-dependent: since it averages over a fixed horizon $i$8, changing that horizon can alter rankings because the metric rewards early progress (Gupta et al., 8 Sep 2025).

Third, the study uses mixed terminology about the discretization axis, alternating between “iterations” and “function evaluations,” even though the formal definition is generic in $i$9. Fourth, although the paper clearly defines AOCC per run from best-so-far data, it does not provide a separate formal equation for averaging across runs or instances, nor for constructing the AOCC-based class thresholds used in the classification stage (Gupta et al., 8 Sep 2025). Fifth, one discussion passage reportedly refers to a model “achiev[ing] the lowest AOCC” as if lower were better, which is inconsistent with the metric’s formula and with the dominant interpretation in the paper that higher AOCC is better; this is most plausibly a wording inconsistency rather than a redefinition of the measure (Gupta et al., 8 Sep 2025).

The distinction between AOCC and more general area-based heuristics is sharpened by comparison with “Extended Linear Regression: A Kalman Filter Approach for Minimizing Loss via Area Under the Curve” (R, 2023). That paper uses an area criterion tied to a weight-versus-loss curve, computes the area under that curve by a trapezoidal procedure after sorting points by weight, and selects the linear regression equation with minimum area (R, 2023). It is therefore related to AOCC only in a loose sense: both use a scalar area associated with a performance-related curve.

The conceptual difference is explicit. AOCC in the PSO study is computed from the time series of best-so-far function values over a fixed optimization budget, whereas the other paper’s AUC is a geometric quantity over weight-loss pairs rather than a convergence trajectory over iteration, evaluation, or time. This suggests that AOCC is more appropriately classified as an anytime-style benchmarking measure rooted in convergence analysis, while the weight-loss AUC criterion is a post hoc model-selection heuristic in parameter-loss space (Gupta et al., 8 Sep 2025, R, 2023).

In the benchmarking and explainability setting where it is defined most clearly, AOCC is therefore best understood as a normalized, discrete summary of the entire best-so-far objective-value trajectory under a fixed budget. Its primary significance lies in converting a run’s convergence history into a single scalar that can be compared across configurations, explained with SHAP, and predicted from landscape descriptors, while preserving sensitivity to early and sustained progress rather than terminal performance alone (Gupta et al., 8 Sep 2025).