Capability preservation under Evolution Strategies

Determine whether Evolution Strategies preserve the pretrained capabilities of large language models during post-training, beyond the specific tasks and evaluation settings examined in the paper.

Background

The paper studies Evolution Strategies (ES) as a memory-efficient post-training method for LLMs and compares their effects with gradient-based methods such as Group Relative Policy Optimization. A key practical concern is catastrophic forgetting: post-training may improve reasoning on the target task while degrading capabilities acquired during pretraining.

Prior work had reported substantial parameter drift and catastrophic forgetting under ES, but the paper argues that the available evidence is limited to particular tasks and small training sets. Although the experiments presented here generally find that held-out capabilities are preserved, the authors explicitly leave the broader question of whether capability preservation holds for ES unresolved.