PSO-Merging: Merging Models Based on Particle Swarm Optimization (2508.19839v1)

Abstract: Model merging has emerged as an efficient strategy for constructing multitask models by integrating the strengths of multiple available expert models, thereby reducing the need to fine-tune a pre-trained model for all the tasks from scratch. Existing data-independent methods struggle with performance limitations due to the lack of data-driven guidance. Data-driven approaches also face key challenges: gradient-based methods are computationally expensive, limiting their practicality for merging large expert models, whereas existing gradient-free methods often fail to achieve satisfactory results within a limited number of optimization steps. To address these limitations, this paper introduces PSO-Merging, a novel data-driven merging method based on the Particle Swarm Optimization (PSO). In this approach, we initialize the particle swarm with a pre-trained model, expert models, and sparsified expert models. We then perform multiple iterations, with the final global best particle serving as the merged model. Experimental results on different LLMs show that PSO-Merging generally outperforms baseline merging methods, offering a more efficient and scalable solution for model merging.

• The paper introduces a novel gradient-free approach using PSO to merge fine-tuned models into a single multitask LLM.
• It leverages pre-trained, sparsified experts and iterative velocity updates to balance exploration and exploitation efficiently.
• Experimental results demonstrate superior multitask scores, rapid convergence, and significant memory efficiency over gradient-based methods.

PSO-Merging: Merging Models Based on Particle Swarm Optimization

Introduction

PSO-Merging introduces a data-driven, gradient-free approach for merging multiple fine-tuned LLM experts into a single multitask model, leveraging Particle Swarm Optimization (PSO) in the parameter space. The method addresses the limitations of both data-independent merging (which lacks task-specific adaptation) and gradient-based data-driven approaches (which are computationally prohibitive for large models). By initializing the PSO swarm with pre-trained, fine-tuned, and sparsified expert models, PSO-Merging efficiently searches for an optimal parameter combination that maximizes multitask performance.

Figure 1: Overview of PSO-Merging, showing initialization with pre-trained, fine-tuned, and sparsified experts, and iterative update cycles in parameter space.

Methodology

Problem Formulation

Given a set of tasks $T = \{\tau_1, \ldots, \tau_n\}$ and corresponding fine-tuned experts $$\{\boldsymbol{\theta}_1, \ldots, \boldsymbol{\theta}_n\}$$ derived from a base model $\boldsymbol{\theta}_0$, the objective is to merge these experts into a single model $\boldsymbol{\theta}_{\mathrm{merged}}$ that performs well across all tasks.

PSO-Merging Algorithm

Initialization

• The swarm is initialized with:
  • The pre-trained base model
  • All fine-tuned expert models
  • Sparsified versions of each expert (using Bernoulli masking on parameter deltas)
• Sparsification mitigates parameter conflicts and increases the diversity of initial solutions.

Iterative Updates

• Each particle (model) is evaluated on a small optimization set, with fitness defined as the average score across all tasks.
• Particle velocities are updated using the canonical PSO formula:

$$\boldsymbol{v}_t^{(i)} = w \cdot \boldsymbol{v}_t^{(i-1)} + c_1 r_1 (\boldsymbol{\theta}_{\mathrm{gbest}}^{(i-1)} - \boldsymbol{\theta}_t^{(i-1)}) + c_2 r_2 (\boldsymbol{\theta}_{t,\mathrm{pbest}}^{(i-1)} - \boldsymbol{\theta}_t^{(i-1)})$$

• Positions are updated as:

$$\boldsymbol{\theta}_t^{(i)} = \boldsymbol{\theta}_t^{(i-1)} + \boldsymbol{v}_t^{(i)}$$

• After $S$ steps, the global best particle is selected as the merged model.

Intuitive Justification

• The update rule is a data-guided linear combination of the current, personal best, and global best solutions, with momentum for exploration.
• Sparsification in the initial swarm enables the search to avoid regions of high parameter conflict.

Experimental Results

PSO-Merging was evaluated on Flan-T5-Base, Llama-2-13B, Llama-3-8B, and Mistral-7B-v0.3, merging up to four experts per base model. The method was compared against a comprehensive set of baselines, including Task Arithmetic, DARE-Linear, TIES-Merging, DELLA-Merging, RankMean, Evo (CMA-ES), Adamerging, Fisher-Merging, and RegMean.

• On Flan-T5-Base, PSO-Merging achieved the highest average multitask score (81.24), outperforming all baselines, with a notable improvement on MNLI.
• On Llama-2-13B, Llama-3-8B, and Mistral-7B-v0.3, PSO-Merging consistently yielded the best average scores, with substantial gains in instruction-following and mathematical reasoning tasks.

  Figure 2: Score trajectories for all particles under different momentum coefficients $w$, illustrating rapid convergence and the effect of initialization.

• Increasing the number of particles (via sparsification and inclusion of the base model) improved final performance.
• PSO-Merging demonstrated rapid convergence, typically within 5–10 steps, and was robust to the choice of $w$ (optimal at $w=0.2$).

Analysis

Hyperparameter Sensitivity

• The momentum coefficient $w$ is critical: $w=0.2$ balances exploration and exploitation, enabling all particles to converge to high scores.
• Larger particle swarms (enabled by sparsification) facilitate better exploration and higher final scores.

Efficiency

• PSO-Merging is significantly more memory-efficient than gradient-based methods (Adamerging, Fisher-Merging, RegMean), requiring only inference-time memory for each particle.
• For three 7B models, PSO-Merging requires 14GB, compared to 28–42GB for gradient-based approaches.

Scalability

• The method scales to merging four or more experts, maintaining superior performance and convergence speed.

Practical Implications

PSO-Merging provides a scalable, efficient solution for constructing multitask LLMs from existing fine-tuned experts, without the need for full retraining or expensive gradient computations. The approach is particularly suited for scenarios with limited task-specific data and large model sizes, where traditional gradient-based merging is infeasible.

• The method is applicable to any scenario where multiple experts on the same base architecture are available.
• The sparsification mechanism is essential for mitigating parameter conflicts and should be incorporated in practical deployments.
• The optimization set can be small, enabling rapid prototyping and deployment.

Limitations and Future Directions

• The current method assumes all experts are derived from the same base model. Extending PSO-Merging to heterogeneous architectures remains an open challenge.
• Further research is needed to adapt the approach for cross-architecture or cross-family model merging.

Conclusion

PSO-Merging leverages swarm intelligence and data-driven fitness evaluation to efficiently merge multiple LLM experts into a single multitask model. The method achieves superior multitask performance, rapid convergence, and high memory efficiency compared to existing baselines. Its practical utility is evident in scenarios requiring the consolidation of diverse expert capabilities without retraining. Future work should explore its extension to heterogeneous model architectures and broader application domains.