CMA-ES with Two-Point Step-Size Adaptation (0805.0231v4)

Abstract: We combine a refined version of two-point step-size adaptation with the covariance matrix adaptation evolution strategy (CMA-ES). Additionally, we suggest polished formulae for the learning rate of the covariance matrix and the recombination weights. In contrast to cumulative step-size adaptation or to the 1/5-th success rule, the refined two-point adaptation (TPA) does not rely on any internal model of optimality. In contrast to conventional self-adaptation, the TPA will achieve a better target step-size in particular with large populations. The disadvantage of TPA is that it relies on two additional objective function

• The paper introduces a two-point step-size adaptation method integrated with CMA-ES, enhancing robustness without relying on predefined optimality models.
• It revises recombination weights and learning rates within CMA-ES, optimizing performance in large population and high noise scenarios.
• Experimental results show the approach prevents extreme step-size reductions, offering a practical alternative to traditional cumulative step-size adaptation.

CMA-ES with Two-Point Step-Size Adaptation

The 2008 paper by Nikolaus Hansen introduces a novel approach integrating a refined version of the two-point step-size adaptation (TPA) with the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). This research aims to address some limitations of the conventional cumulative step-size adaptation (CSA) used in evolutionary algorithms, particularly in scenarios involving large population sizes or high noise levels.

Key Contributions

The main contribution of the paper is the integration of TPA with CMA-ES. This method contrasts with traditional CSA by not relying on predefined internal models of optimality. One significant advantage of TPA is its robustness to changes in conditions, such as population size.

1. Step-Size Adaptation: TPA offers an alternative strategy by employing a straightforward method of evaluating two different step lengths for mean displacement and selecting the more effective one. This approach does not depend on internal models such as the 1/5th success rule.
2. Algorithm Revision: The paper also proposes modifications to the recombination weights and learning rate formulas within CMA-ES, optimizing its performance across diverse scenarios.

Numerical and Theoretical Insights

Experimental results indicate that TPA's performance can match CSA across various conditions, though neither consistently outperforms the other drastically. The choice between them may depend heavily on specific objectives and contexts. Notably, TPA avoids target step-size reduction to zero under high noise conditions, a situation where CSA struggles.

Implications and Future Directions

The practical implications of this research suggest that TPA could serve as a viable alternative to CSA where conventional step-size control mechanisms may falter. This flexibility can be particularly beneficial in optimization tasks with high-dimensional search spaces or in environments with significant noise.

Theoretically, the introduction of TPA opens up further research avenues. Particularly, exploring the trade-offs between computation complexity and optimization accuracy, given that TPA requires additional objective function evaluations per iteration.

Future work could focus on extensive empirical studies to delineate scenarios where TPA has decisive advantages over CSA. Investigating the combination of TPA with other evolutionary strategies, or its integration into hybrid algorithms, could further bolster its applicability and performance.

In conclusion, this paper presents TPA as a promising adaptation mechanism for CMA-ES, providing a foundation for future innovation in evolutionary optimization strategies.