Adaptive Sampling CMA-ES

• The paper introduces Adaptive Sampling CMA-ES, an extension of CMA-ES that allocates candidate evaluation times based on predicted sorting difficulty to mitigate noise.
• It adaptively calibrates measurement precision by mapping candidate distances to evaluation durations, ensuring a target signal-to-noise ratio for consistent ranking.
• Empirical results demonstrate up to 65% faster convergence and significant cost reduction in applications like robotic exoskeleton tuning.

Adaptive Sampling CMA-ES (AS-CMA) is an extension of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) tailored for optimization tasks where evaluation of candidate solutions is expensive and subject to significant measurement noise. In robotic optimization contexts, such as exoskeleton control policy tuning, evaluation durations are subject to a speed-accuracy tradeoff: shorter measurement times expedite optimization but increase noise-induced rank errors; longer measurements improve signal fidelity but impede progress. AS-CMA addresses this paradigm by adaptively assigning evaluation times to candidate solutions based on predicted sorting difficulty, calibrating evaluation precision dynamically within each generation. This approach pursues consistent candidate ranking accuracy while minimizing total experimental cost and setup complexity (Martin et al., 14 Jan 2026).

1. Motivation and Problem Formulation

AS-CMA was developed to address the challenge of optimizing in environments where each function evaluation (e.g., a robotic control trial) requires multiple minutes and is corrupted by stochastic measurement noise. Standard CMA-ES updates a Gaussian sampling distribution using only the rank order of batch-evaluated candidates. Noisy evaluations risk rank-order errors, especially when candidates are near-tied, degrading adaptation and convergence. The core insight motivating AS-CMA is that only those pairwise comparisons most vulnerable to noise—specifically, candidates close in (expected) fitness—require high-precision measurements. Conversely, large separations in parameter space (under smoothness assumptions) likely reflect large fitness differences, allowing efficient, lower-precision evaluation without sacrificing selection fidelity. AS-CMA formalizes this intuition by adaptively mapping each candidate’s estimated sorting difficulty into a minimal, custom evaluation duration that satisfies a predefined signal-to-noise threshold.

2. Algorithmic Foundations and Mathematical Formulation

Predicted Sorting Difficulty

Let $\mathbf{x}_1, \ldots, \mathbf{x}_\lambda$ denote the new candidate points generated in generation $g$, with $\lambda$ the population size. For each candidate $i$, the normalized nearest-neighbor distance in parameter space is computed:

$$d_i = \min_{j \neq i} \frac{\|\mathbf{x}_i - \mathbf{x}_j\|_2}{d_{max}},$$

where $d_{max}$ is the Euclidean diameter of the domain (Eq. 6).

This distance is mapped to an estimated fitness difference using a local-slope proxy $k_{avg}$:

$\hat{\delta}_i = k_{avg} \cdot d_i.$

Noise Model and Sample Time Assignment

To achieve a fixed target signal-to-noise ratio $\beta$ (empirically robust at $\beta \approx 1.3$), AS-CMA assumes multiplicative noise such that for two comparable candidates with average fitness $y_{avg}$ and standard deviation (percent error) $\epsilon_i$, the difference distribution’s standard deviation is:

$$\sigma_{diff, i} = \sqrt{2} \, y_{avg} \, \epsilon_i.$$

Solving for $\epsilon_i$ under the requirement that the estimated signal ($\hat{\delta}_i$) exceeds noise by at least $\beta$,

$$\epsilon_i = \frac{k_{avg} \cdot d_i}{\sqrt{2}\, \beta \, y_{avg}}.$$

Each candidate’s sampling/evaluation time $t_i$ is then set by inverting a pre-measured function $\mathcal{A}(t)$, which gives expected percent error as a function of duration:

$t_i = \arg\min_t |\mathcal{A}(t) - \epsilon_i|.$

(Algorithm 1, Eq. 16.)

State Variable Updates

At each generation, $y_{avg}$ is updated via averaging, and the local slope $k_{avg}$ is re-fit by least squares over all candidate pairs:

$$k_{avg} = \arg\min_k \left\|[y_i - y_j]_{i<j} - k \cdot [\|\mathbf{x}_i - \mathbf{x}_j\|_{2}]_{i<j}\right\|_2^2.$$

Integration with CMA-ES

AS-CMA fully preserves standard CMA-ES updates (mean, step-size, covariance) apart from the assignment of nonuniform evaluation times $t_i$ to each candidate.

3. Algorithmic Workflow

The following schema summarizes the AS-CMA process for each generation:

Step	Description	Mathematical Expression
1	Candidate creation	$$\mathbf{x}_i \sim m_g + \sigma_g \mathcal{N}(0, C_g)$$
2	Sorting-difficulty-based $t_i$ assignment (AS-CMA step)	$d_i$, $\epsilon_i$, $t_i$ as above
3	Measurement with multiplicative noise	$y_i = f_{meas}(\mathbf{x}_i, t_i)$
4	Sorting and selection	Sort $\mathbf{x}_i$ by $y_i$, apply CMA-ES update rules
5	Landscape parameter updates	Update $y_{avg}$, re-fit $k_{avg}$

Initialization requires initial guesses for $\hat{y}_{min}$, $\hat{y}_{max}$, and a noise model $E(t)$, but sensitivity analyses reveal minor impact from imperfect initial values; subsequent online updates rapidly adapt to empirical landscape features.

4. Hyperparameters, Sensitivity, and Practical Considerations

AS-CMA introduces three state variables—$d_{max}$ (domain diameter), $y_{avg}$ (average fitness), and $k_{avg}$ (local slope)—and one metaparameter: $\beta$, the required signal-to-noise ratio for candidate sorting. All other CMA-ES hyperparameters (e.g., population size, selection weights) remain as standard. The noise model $E(t)$ is empirically determined in advance. Empirical studies reveal robust performance with $\beta = 1.3$ across cost landscapes and strong insensitivity to initial guess choices for $\hat{y}_{min}$, $\hat{y}_{max}$. Moderate errors in the noise model $E(t)$ have only modest impacts on convergence [Appendix A1, (Martin et al., 14 Jan 2026)]. This suggests that AS-CMA introduces negligible additional tuning burden while delivering significant performance gains.

5. Empirical Evaluation and Comparative Performance

Benchmarking was performed on four simulated cost landscapes—empirical exoskeleton metabolic-cost ("4D Ankle"), and shifted versions of 4D Rosenbrock, 4D Levy, and 20D Sphere functions—using human-derived multiplicative noise models: $t \in [0.5, 5.5]$ minutes yields $\epsilon \in [34.2\%, 0.4\%]$, with +3% baseline.

AS-CMA achieved fine convergence on 98% of runs (coarse/fine criteria defined as staying below 20%/5% of global minimum), matching or exceeding reliability of best static-tuned CMA-ES and requiring no parameter readjustment across landscapes. Relative to best static sample-time CMA-ES, AS-CMA converged 24–65% faster, incurred 29–76% less evaluation cost, and maintained higher rank-ordering fidelity throughout optimization. Bayesian optimization with static sampling excelled on simple landscapes, outperforming AS-CMA in efficiency by 66–71% on Ankle and 20D Sphere, but was slower and less reliable ($\leq 90\%$ convergence, 5–15$\times$ slower) on complex landscapes (Rosenbrock, Levy). KL-KG CMA-ES, a recent dynamic-resampling strategy, improved static CMA-ES on easy landscapes but was otherwise 3–6$\times$ slower and 2–4$\times$ less cost-efficient than AS-CMA (Martin et al., 14 Jan 2026).

6. Real-World Application: Robotic Exoskeleton Optimization

AS-CMA was deployed in an experimental setting optimizing assistance parameters for a belt-driven ankle exoskeleton. The optimization vector specified peak torque, peak time, rise time, and fall time. Evaluation times per candidate were allowed in $[0.5, 5.5]$ minutes. Early optimization phases allocated minimal sampling ($\approx 0.5$–1 min) for exploration; as optimization progressed and candidate solutions clustered, sampling durations increased automatically, up to 5 minutes. The final policy yielded a 42% energy reduction, closely matching prior experimental expectations (≈39%). Notably, no retuning of $\beta$ or trial-duration bounds was necessary. This demonstrates AS-CMA’s capacity for online speed-precision balancing and robust adaptation to practical experimental constraints.

7. Implementation Considerations and Extensions

AS-CMA can be incorporated into existing CMA-ES codebases by tracking three state variables and introducing metaparameter $\beta$ and a pre-measured noise model $E(t)$. No modifications to the core CMA-ES adaptation equations or selection scheme are required. Open-source reference code is available (RussellMMartin/AS-CMA-ES), facilitating further extensions, such as early stopping policies or hybridization with Bayesian surrogate approaches. A plausible implication is that such extensions could further improve resource allocation in highly stochastic or multi-objective optimization settings, specifically where cost landscapes exhibit substantial local variance or when evaluation noise models are non-stationary (Martin et al., 14 Jan 2026).