EvoGM: Learning to Merge LLMs via Evolutionary Generative Optimization

Abstract: Evolutionary model merging provides a powerful framework for the automated, training-free composition of LLMs through parameter-space search. However, existing methods predominantly rely on stochastic, hand-crafted operators that overlook the underlying performance landscape of the coefficient space. We propose Evolutionary Generative Merging (EvoGM), a framework that transcends manual heuristics by employing learnable generative modeling to optimize merging coefficients. Specifically, EvoGM features a dual-generator architecture with cycle-consistent learning to adaptively sample and refine promising merging candidates. By constructing winner-loser pairs from historical search trajectories, our framework effectively captures high-performance parameter distributions and maximizes data efficiency. This generative process is seamlessly integrated into a multi-round evolutionary pipeline, where elite merged models iteratively serve as new expert foundations. Extensive experiments across diverse benchmarks demonstrate that EvoGM significantly outperforms state-of-the-art baselines, exhibiting robust performance on both seen and unseen tasks. Code and data are available at https://github.com/JiangTao97/evogm.

• The paper introduces EvoGM as a novel framework that reformulates model merging into an adaptive search for optimal coefficients.
• It employs a dual-generator architecture with cycle consistency and winner-loser pairing to effectively navigate and refine the parameter space.
• EvoGM outperforms static baselines with improved test accuracy and convergence on both seen and unseen benchmarks.

Evolutionary Generative Model Merging for LLMs: The EvoGM Framework

Motivation and Problem Formulation

LLM adaptation through full-parameter fine-tuning is increasingly infeasible under resource and data constraints. Model merging offers a paradigm for integrating the capabilities of multiple fine-tuned expert models into a unified model, obviating retraining costs and facilitating scalable capability composition. Prior merging approaches rely predominantly on heuristic or stochastic parameter aggregation, often failing to adaptively navigate the underlying performance landscape of the merging coefficient space and subsequently suffering from inefficient or suboptimal results.

EvoGM reformulates model merging as an adaptive search problem over merging coefficients: given $N$ task-specific experts $\{\boldsymbol{\theta}_i\}_{i=1}^N$, each fine-tuned from a shared pretrained base $\boldsymbol{\theta}_{pre}$, EvoGM parameterizes the merged model as $$\boldsymbol{\theta}_{pre} + \sum_{i=1}^{N} \lambda_i (\boldsymbol{\theta}_i - \boldsymbol{\theta}_{pre})$$. The core challenge is efficient search for optimal coefficients $\boldsymbol{\lambda}^*$ that maximize performance on a validation set, constrained to the low-dimensional space $\mathbb{R}^N$.

EvoGM Methodology

EvoGM introduces learnable generative optimization to evolutionary merging. The framework utilizes a dual-generator architecture, cycle-consistent constraints, and winner-loser paired data from historical search to optimize the merging coefficients.

Figure 1: Schematic of EvoGM, depicting adaptive population initialization, winner-loser pairing, cycle-consistent dual-generator training, generative evolution, and evolving expert bases.

1. Population Initialization

EvoGM initializes a diverse population of merging coefficients via three strategies: average merging, one-hot expert selection, and uniform random sampling. This ensures coverage of the coefficient space, providing seed trajectories required for generative learning.

2. Winner-Loser Pairing

Historical search trajectories ($\mathcal{H}$) are partitioned into winners ($\mathcal{H}^+$) and losers ($\mathcal{H}^-$) according to validation performance. Pairwise Cartesian products between winner and loser sets furnish training data for learning generative transitions.

3. Dual-Generator Training with Cycle Consistency

A forward generator ($G_{-\rightarrow+}$) and backward generator ($\{\boldsymbol{\theta}_i\}_{i=1}^N$0), both MLPs, are trained to respectively map losers to winners and winners to losers. Cycle consistency ensures reversibility and structural preservation in coefficient space. Optimization-guided loss explicitly drives the generators toward the centroid of winners, steering the search toward high-performing regions.

4. Generative Evolution and Selection

The trained forward generator transforms the population to yield new candidate merging coefficients, which are evaluated on the validation set. High-performing candidates are propagated, and the search history is continually updated.

5. Evolving Expert Basis

Periodic basis shifts are implemented via selecting elite candidates as new expert models, recomputing task vectors with respect to the base model, and refining the coefficient space. This hierarchical mechanism allows EvoGM to escape local optima and progressively refine mergeability.

Experimental Results

Single-Task and Multi-Task Merging

EvoGM is empirically validated on two distinct settings: merging on seen tasks (expert models explicitly fine-tuned for the evaluation task) and unseen tasks (generalization to tasks not encountered during fine-tuning). Benchmarks include GLUE, MMLU, MMLU-Pro, HellaSwag, GSM8K, Knowledge Crosswords, NLGraph, TruthfulQA, and AbstainQA.

Figure 2: EvoGM surpasses baseline methods on nearly all single-task unseen benchmarks using Qwen2.5-1.5B experts.

Figure 3: Radar plots comparing EvoGM and baselines across Validation/Test sets in multi-task merging for major benchmarks.

EvoGM consistently exhibits superior task transfer, attaining the highest test accuracy on 5 of 8 single-task benchmarks and delivering a 66% relative improvement in AbstainQA for multi-task configuration. Notably, static model soup and joint multi-task fine-tuning baselines underperform single best experts, underscoring inter-task interference and the necessity for adaptive merging.

Convergence Behavior

Figure 4: EvoGM achieves faster, higher convergence in multi-task merging, with pronounced gains post basis shift.

EvoGM achieves improved optimization efficiency and robustness compared to evolutionary baselines like Model Swarm and PSO-Merging. Basis shifts broaden exploration, prevent premature convergence, and enhance solution quality.

Ablation Studies

Figure 5: Each component—dual-generator, multi-round updates, cycle-consistency loss—is critical for optimal performance on complex benchmarks.

Removal of the multi-round mechanism or cycle-consistency loss results in substantial performance degradation, particularly on tasks with reasoning or highly divergent domains. The dual-generator architecture contributes significantly to optimal coefficient discovery across benchmarks.

Scaling and Robustness Analysis

Figure 6: Merging quality and performance scale robustly with the number of integrated expert models in both validation and test settings.

EvoGM maintains robustness and performance improvements across ensemble sizes up to ten experts and demonstrated superior performance as ensemble size increases, validating scalability and efficacy.

Implications and Future Directions

EvoGM advances model merging by shifting from stochastic, unstructured search to data-driven, generative optimization. By leveraging search history and comparative signals, EvoGM achieves superior efficiency, robustness, and generalization for both seen and unseen task compositions. The approach is highly parallelizable and memory-efficient, facilitating deployment on large LLMs and PEFT-style scenarios.

Practically, EvoGM enables flexible model composition without retraining overhead, promoting sustainable model development, knowledge transfer, and customized AI system construction. Theoretically, this bridging of generative optimization and population-based search offers new avenues for performance-driven neural parameter space exploration.

Future directions include extension to heterogeneous architectures, cross-modal merging, and multi-objective Pareto optimization for preference-aware adaptation. EvoGM provides a foundation for preference-tunable, multi-domain compositionality in model design, with implications for federated learning, continual learning, and distributed adaptation.

Conclusion

EvoGM demonstrates that learnable generative evolutionary optimization is an effective and scalable solution for merging LLMs, outperforming state-of-the-art baselines across tasks and domains. Its dual-generator architecture and basis shift mechanism enable robust navigation of the merging coefficient space, making it a promising framework for future adaptive model composition and cross-task generalization in AI.