Optimal budget allocation and timing for GEPA’s system-aware crossover (Merge)

Determine the optimal allocation of rollout budget between reflective prompt mutation and the system-aware crossover strategy Merge within GEPA, and ascertain when Merge should be invoked during optimization to maximize performance and generalization across different language models and tasks.

Background

GEPA+Merge extends the base GEPA optimizer with a system-aware crossover (Merge) that combines complementary modules from different optimization lineages. The authors report that GEPA+Merge can provide additional gains over GEPA in some settings (e.g., GPT-4.1 Mini), but may degrade performance in others (e.g., Qwen3 8B), suggesting sensitivity to how optimization budget is split and when Merge is applied.

They attribute discrepancies to allocation of rollouts between mutation and crossover and to the timing of Merge invocation. The authors explicitly state that identifying the optimal budget allocation and invocation strategy needs further paper, motivating a formal investigation of these hyperparameters and conditions.