Weight Adaptation for Improving Parallel Performance of Adaptive Stochastic Natural Gradient

Abstract: Probabilistic model-based evolutionary algorithms are promising for black-box optimization. Specifically, the adaptive stochastic natural gradient (ASNG) adaptively updates its learning rate, a typical hyperparameter in probabilistic model-based evolutionary algorithms, thereby realizing efficient and robust optimization. Although weight parameters are common hyperparameters, with the increasing demand for parallel evaluation of time-consuming tasks, it remains unclear how to set suitable weights for larger population sizes. In this paper, we propose Weight Adaptation ASNG (WA-ASNG), which incorporates a weight adaptation mechanism into ASNG. We calculated the estimated signal of the update direction from the accumulations of the natural gradient. Then, to maximize the signal, WA-ASNG adaptively updates its weight parameters by a gradient ascent over the optimization. While the learning rate adaptation plays a role in satisfying a sufficient condition for monotonic improvement of the expected objective value, the mechanism of weight adaptation is intended to maximize this improvement. The experimental results demonstrate that WA-ASNG outperforms PBIL and ASNG across various settings with population sizes ranging from 25 to 100 for binary optimization problems. Furthermore, WA-ASNG can perform efficiently in the presence of strong noise. Our code is available at https://github.com/shiralab/WA-ASNG .

• The paper introduces WA-ASNG, which integrates a gradient-based weight adaptation mechanism to maximize the natural gradient signal in parallel optimization.
• It employs dual adaptation layers combining learning rate and weight updates, leading to faster convergence and reduced function evaluations on binary tasks.
• Experimental results show WA-ASNG’s superior robustness and efficiency over fixed-weight ASNG variants, especially under noisy conditions and higher parallelism.

Weight Adaptation for Enhanced Parallel Performance in Adaptive Stochastic Natural Gradient

Overview and Motivation

The paper "Weight Adaptation for Improving Parallel Performance of Adaptive Stochastic Natural Gradient" (2606.19861) addresses a fundamental challenge in probabilistic model-based evolutionary algorithms (PMBEAs) when applied to black-box optimization tasks, specifically in discrete binary domains. While adaptive learning rate mechanisms such as those in ASNG provide robustness and efficient optimization, the determination and adaptation of sample ranking weights are less well-understood, especially under increased parallelism with larger population sizes.

To solve this, the authors propose Weight Adaptation ASNG (WA-ASNG), which introduces a gradient-based mechanism for dynamically updating the ranking weights in ASNG. The adaptation aims to maximize the signal of the estimated natural gradient, thus increasing the efficiency of parallel search and enhancing robustness—even under strong noise environments.

Theoretical Framework

WA-ASNG extends the information geometric optimization (IGO) framework, leveraging its stochastic natural gradient ascent on exponential family parameterizations. The algorithm aims to maximize the expectation of the objective function, with updates performed along the estimated natural gradient direction. The learning rate adaptation in ASNG is designed to satisfy sufficient monotonicity conditions, derived from theoretical bounds on KL-divergence between updated and current distributions (as formalized in Theorem 4 of [asngnas]).

Weight adaptation in WA-ASNG is motivated by the insight that the improvement bound is proportional to the signal strength $\|E[G_\theta^t]\|^2_{\mathbf{F}(\bm{\theta}^t)}$. Thus, the WA-ASNG mechanism employs gradient ascent over the weight vector $\bm{w}$ to maximize this signal, using cumulative statistics of the natural gradient.

Algorithmic Enhancements

WA-ASNG incorporates two adaptation layers:

• Learning rate adaptation: Inherited directly from ASNG, adjusting $\delta_\theta$ based on signal-to-noise ratio statistics.
• Weight adaptation: Every $L$ iterations (with $L$ proportional to $N^{1.3}/\lambda_\theta$), the algorithm performs $E$ epochs of gradient ascent to update $\bm{w}$, maximizing the signal estimated from archived natural gradients. The weight vector is then sorted and the learning rate for weight adaptation ($\eta_w$) decayed for stability.

The initial weights are set according to ASNG default (a ternary scheme), but are adaptively updated based on the optimization progress.

Experimental Evaluation

WA-ASNG is benchmarked on three canonical binary optimization functions: ne (one-max), in (binary valued), and eading (leading ones). Population sizes ($\lambda_\theta$) range from 25 to 100, with dimensionality $\bm{w}$0 up to 500. Comparative baselines include PBIL and standard ASNG with various fixed and normalized weight schemes.

WA-ASNG consistently delivers superior search efficiency on complex landscapes (in, eading) and maintains high learning rates during optimization, as evidenced by its rapid convergence to the optimal probability vector and fewer function evaluations required.

Figure 1: WA-ASNG versus ASNG and PBIL: median and interquartile ranges of evaluation count highlight WA-ASNG's superior efficiency across population sizes and problem instances.

WA-ASNG demonstrates robust adaptation of the mean probability vector, achieving faster convergence on in and eading, as shown in the transitions of probability vector statistics.

Figure 2: Probability vector mean transitions reveal WA-ASNG's accelerated progression toward optimality compared to ASNG and PBIL.

Learning rate dynamics further confirm that WA-ASNG sustains higher $\bm{w}$1 values, facilitating aggressive yet stable updates.

Figure 3: Learning rate $\bm{w}$2 dynamics in WA-ASNG versus ASNG on in and eading for $\bm{w}$3, $\bm{w}$4, showcasing WA-ASNG's higher sustained rates.

Element-wise analysis of the probability vector in sequential optimization is performed, with WA-ASNG excelling for larger populations ($\bm{w}$5) but struggling in smaller population settings ($\bm{w}$6) on eading, due to erratic signal maximization and subsequent probability vector fluctuation.

Figure 4: Sequential and fluctuating probability vector transitions on eading; WA-ASNG excels with $\bm{w}$7 but underperforms for $\bm{w}$8.

Comparative Analysis with Fixed Weight Schemes

Variants of ASNG equipped with different fixed weights (linear, CMA-ES, CMA-ES normalized) indicate that optimal weight configuration is problem-dependent. WA-ASNG's dynamic adaptation provides robust performance across families, approaching the efficacy of CMA-ES normalized weights on in, and outperforming all fixed schemes under noise.

Figure 5: SP1 (evaluations per success) comparison demonstrates WA-ASNG's robust efficiency versus ASNG variants with different weights.

Learning rate trajectories across these variants reaffirm that the choice of weighting influences update magnitude and convergence speed.

Figure 6: Learning rate trajectories for different ASNG variants and WA-ASNG; linear weights yield high $\bm{w}$9 on ne.

Adapted weight vectors in WA-ASNG converge toward the CMA-ES normalized pattern on in, evidencing the algorithm's capacity to infer effective weights via signal maximization.

Figure 7: WA-ASNG's weight adaptation trajectory compared to CMA-ES normalized weights; adaptive convergence toward optimal fixed schemes.

Under additive Gaussian noise, WA-ASNG demonstrates marked robustness, maintaining high success rates and low SP1 values compared to fixed-weight ASNG variants, which deteriorate sharply with increasing noise variance.

Figure 8: Robustness to Gaussian noise: WA-ASNG exhibits highest resilience and success rates; fixed-weight variants falter as noise increases.

Implications and Future Directions

The introduction of adaptive weight mechanisms in WA-ASNG offers significant practical advantages for parallelized evolutionary search, particularly in high-throughput, noisy, or unknown landscapes. The method's robustness and efficiency in signal-driven adaptation position it as a promising direction for PMBEA development.

Theoretical implications include potential transferability to other exponential family models (e.g., categorical distributions, Gaussian PMBEAs), and the prospect of extending weight adaptation mechanisms to continuous domains and deceptive functions. This may enable broad improvements in adaptive evolutionary optimization without manual hyperparameter tuning.

Future research should address:

• Evaluation on broader problem sets, including deceptive and categorical landscapes.
• Transfer and integration of weight adaptation mechanisms in continuous PMBEAs (e.g., CMA-ES, NES variants).
• Investigation of convergence guarantees, signal maximization properties, and interactions between learning rate and weight adaptation dynamics.

Conclusion

WA-ASNG advances adaptive stochastic natural gradient methods by integrating an online weight adaptation mechanism, maximizing update direction signal and elevating parallel search efficiency. Empirical results substantiate improvements over PBIL and classical ASNG variants in binary optimization, including strong robustness under noise. The methodology lays groundwork for further exploration of adaptive mechanisms in evolutionary algorithms, with implications for scalability and automatic parameter tuning in both discrete and continuous domains.