Adaptive Iterative Model Merging

• The paper introduces a continual learning framework that treats model merging as a dynamic multi-round process, balancing stability against plasticity.
• It employs explicit learning and forgetting signals to decide optimal merge intervals and integrate new and past task knowledge.
• Empirical results demonstrate enhanced overall performance, reduced catastrophic forgetting, and improved forward/backward transfer on multiple benchmarks.

Adaptive Iterative Model Merging (AimMerging) is a continual learning framework for LLMs in which model merging is treated as a dynamic, multi-round process rather than a single post-training operation. It was introduced to address the stability–plasticity trade-off in sequential learning, especially the observation that existing model merging-based continual learning methods suffer from suboptimal number of merges and merging frequency. AimMerging monitors the training trajectory through explicit learning and forgetting signals, uses a training trajectory-guided merge controller to decide merge timing and frequency, and performs rehearsal-based knowledge fusion to combine new-task and past-task task vectors into a single updated model (Feng et al., 22 Sep 2025).

1. Problem formulation and conceptual scope

AimMerging is formulated in a continual learning setting for LLMs exposed to a task sequence

$\{\mathcal{T}_1,\dots,\mathcal{T}_K\},$

where task $k$ has dataset

$$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$

The ideal objective is

$$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$

but training is sequential, so catastrophic forgetting arises when optimization on $\mathcal{T}_k$ harms performance on earlier tasks. The central tension is the standard stability–plasticity trade-off: stability requires preserving knowledge from past tasks and avoiding catastrophic forgetting, whereas plasticity requires learning new tasks well and enabling transfer (Feng et al., 22 Sep 2025).

The framework assumes a replay-based continual learning regime. For each past task $\mathcal{T}_i$, a small memory buffer $\mathcal{M}_i$ is stored, with the default setup using $2\%$ of the task training set. At task $k$, the model trains with current task data $\mathcal{D}_k$ and memory $k$0. AimMerging is implemented in a parameter-efficient fine-tuning setting, mainly with LoRA adapters on top of a frozen base LLM. If $k$1 denotes the task-$k$2 model parameters, the task vector is

$k$3

Model merging methods define a merge function $k$4 and produce a unified model, for example

$k$5

Within this framing, AimMerging departs from single-round, static merging and from fixed-schedule multi-round merging by making the merge schedule itself trajectory-dependent (Feng et al., 22 Sep 2025).

In the broader model-merging literature, “adaptive iterative model merging” also refers to a wider family of procedures that repeatedly update a merged model using task vectors or quadratic proxy objectives. Related work has treated iterative merging as alternating tuning-and-merging, adaptive projective gradient descent, spectral regularization of noisy inverse problems, or sequential orthogonal projection of incoming task vectors (Zhou et al., 2024, Wei et al., 2 Jan 2025, Wei et al., 5 Jun 2026, Tang et al., 16 Jan 2025). AimMerging is more specific: it is a continual learning framework for LLMs in which the adaptive signal comes from the training trajectory and replay memory rather than from purely data-free weight-space geometry (Feng et al., 22 Sep 2025).

2. Training trajectory signals and the merge controller

AimMerging’s controller is built from two explicit signals: a learning signal that tracks parameter change, and a forgetting signal that tracks replay loss. Suppose the $k$6-th merge during task $k$7 occurs at iteration $k$8, with interval $k$9 since the previous merge. The task vector between merges is

$$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$0

The learning signal is defined as

$$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$1

namely the average absolute parameter change per step over the preceding interval. Large $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$2 indicates active learning, while small $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$3 indicates slow convergence. To reduce noise, AimMerging maintains a history

$$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$4

and uses a sliding window of length $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$5 to compare upward and downward trends (Feng et al., 22 Sep 2025).

The forgetting signal is defined through the historical loss on replay data. At each iteration, a batch from $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$6 is sampled and evaluated without gradient updates, yielding historical loss $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$7. During the first $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$8 steps after the last merge, an average historical loss $$\mathcal{D}_k = \{(x_i^k, y_i^k)\}_{i=1}^{N_k}, \quad x_i^k\in\mathcal{X}_k,\, y_i^k\in\mathcal{Y}_k.$$9 is estimated, and the threshold for the upcoming interval is set to

$$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$0

with default $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$1. If at any later step before the next scheduled merge

$$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$2

the forgetting signal is activated and the counter is incremented: $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$3 If $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$4 reaches $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$5, with default $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$6, an early merge is triggered (Feng et al., 22 Sep 2025).

The merge controller first sets a nominal interval $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$7 from learning-signal trends. If upward trends dominate, the method interprets the phase as rapid learning and shortens the interval: $$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$8 If the trends are balanced, it keeps

$$\mathcal{L} = \mathbb{E}_{(x,y)\sim \bigcup_{k=1}^K \mathcal{D}_k}[-\log p_\Theta(y\mid x)],$$9

If downward trends dominate, the phase is treated as slow convergence and the interval is lengthened: $\mathcal{T}_k$0 The default hyperparameters are $\mathcal{T}_k$1, $\mathcal{T}_k$2, $\mathcal{T}_k$3, $\mathcal{T}_k$4, with $\mathcal{T}_k$5, $\mathcal{T}_k$6 when $\mathcal{T}_k$7, and $\mathcal{T}_k$8, $\mathcal{T}_k$9 otherwise. A cold-start phase uses fixed $\mathcal{T}_i$0 for the first $\mathcal{T}_i$1 merges (Feng et al., 22 Sep 2025).

The actual interval $\mathcal{T}_i$2 is then determined by combining the nominal schedule with the forgetting signal. If forgetting activates repeatedly before the nominal merge time, the method triggers an early merge, so $\mathcal{T}_i$3. If forgetting does not activate by the time the nominal interval is reached, the merge can be deferred to allow more new-knowledge acquisition, up to the hard cap

$\mathcal{T}_i$4

This policy is intended to support aggressive merging under high forgetting risk and conservative merging when forgetting is low (Feng et al., 22 Sep 2025).

3. Rehearsal-based knowledge fusion and full algorithmic pipeline

When a merge is triggered at iteration $\mathcal{T}_i$5, AimMerging constructs two task vectors. The new-task vector over the realized interval $\mathcal{T}_i$6 is

$\mathcal{T}_i$7

To construct a historical-task vector, the model is temporarily fine-tuned on replay memory $\mathcal{T}_i$8 for $\mathcal{T}_i$9 steps, producing $\mathcal{M}_i$0, and then

$\mathcal{M}_i$1

The first vector captures accumulated updates from the current training interval, while the second encodes parameter changes that favor historical tasks only (Feng et al., 22 Sep 2025).

The merge weights are derived from the learning and forgetting signals. The proportion assigned to new knowledge is

$\mathcal{M}_i$2

where $\mathcal{M}_i$3 is the number of upward learning-signal trends inside the sliding window. The proportion assigned to past knowledge is

$\mathcal{M}_i$4

These are normalized into fusion weights

$\mathcal{M}_i$5

so that $\mathcal{M}_i$6. The resulting fusion update is

$\mathcal{M}_i$7

This is a global parameter-wise fusion rather than a masking-based rule. In practice, the method operates on LoRA parameters or adapters rather than on the full backbone weights (Feng et al., 22 Sep 2025).

The full training loop reorganizes each task into repeated merge cycles. Starting from $\mathcal{M}_i$8, AimMerging trains on batches from $\mathcal{M}_i$9 plus memory $2\%$0, but only the new-task batch is used for gradient updates; the replay batch contributes to loss monitoring when computing the forgetting signal. Whenever the elapsed steps reach the current interval bound, or the forgetting counter reaches threshold, or the hard cap $2\%$1 is reached, the merge routine is executed, the controller state is updated, and training resumes. After the task ends, a replay memory $2\%$2 is stored and the final merged LoRA state becomes the initialization for the next task (Feng et al., 22 Sep 2025).

This pipeline can be read as a continual-learning specialization of a more general iterative-merging template. In the broader literature, ATM formulates iterative merging as alternating tuning and merging, with updates of the form

$2\%$3

where task vectors approximate multi-task gradients in early training epochs (Zhou et al., 2024). AimMerging differs in that it does not merge on a fixed outer-loop schedule and does not rely on a single averaging coefficient; it uses trajectory-dependent timing and signal-derived fusion coefficients instead (Feng et al., 22 Sep 2025).

4. Evaluation methodology and empirical behavior

AimMerging is evaluated on three continual learning benchmarks. The Standard CL benchmark contains five classification tasks: AG News, Amazon Reviews, Yelp Reviews, DBpedia, and Yahoo Answers. The Long Sequence benchmark contains fifteen classification tasks: the five Standard tasks, four GLUE tasks, five SuperGLUE tasks, and IMDB. The SuperNI benchmark contains fifteen generation and classification tasks from Super-NaturalInstructions, including dialogue generation, information extraction, question answering, summarization, and sentiment analysis. Across these settings, experiments use T5-large, Qwen3-1.7B, LLaMA2-7B, and LLaMA2-13B, all in a LoRA-based PEFT regime with frozen backbone weights (Feng et al., 22 Sep 2025).

The main continual-learning metrics are Overall Performance,

$2\%$4

Backward Transfer,

$2\%$5

and Forward Transfer,

$2\%$6

Here $2\%$7 denotes performance on task $2\%$8 after training through task $2\%$9, and $k$0 denotes performance when training task $k$1 alone (Feng et al., 22 Sep 2025).

On T5-large, AimMerging improves transfer and forgetting relative to Recurrent-KIF across all three benchmarks. On Standard CL, OP is 78.1 versus 78.4, FWT is $k$2 versus $k$3, and BWT is $k$4 versus $k$5. On Long Sequence, OP is 77.9 versus 77.8, FWT is $k$6 versus $k$7, and BWT is $k$8 versus $k$9. On SuperNI, OP is 44.3 versus 43.3, FWT is 2.2 versus 0.4, and BWT is $\mathcal{D}_k$0 versus $\mathcal{D}_k$1. Averaged across benchmarks, FWT improves from $\mathcal{D}_k$2 to $\mathcal{D}_k$3, approximately an 80% relative improvement, and BWT improves from $\mathcal{D}_k$4 to $\mathcal{D}_k$5, approximately a 59% relative improvement (Feng et al., 22 Sep 2025).

Scaling experiments on the Long Sequence benchmark show that the gains persist from T5-large through Qwen3-1.7B, LLaMA2-7B, and LLaMA2-13B. For LLaMA2-7B, the reported example is FWT 79.3 versus 78.2 and BWT $\mathcal{D}_k$6 versus $\mathcal{D}_k$7. The paper also reports catastrophic forgetting curves: on Long Sequence, the first-task performance drops by roughly 32% for vanilla replay, roughly 10% for Recurrent-KIF, and only roughly 4% for AimMerging (Feng et al., 22 Sep 2025).

Ablation studies identify both learning and forgetting signals as necessary. On SuperNI, the full method reports OP 45.1, FWT 1.3, and BWT $\mathcal{D}_k$8. Removing the learning signal yields OP 43.9, FWT 0.5, and BWT $\mathcal{D}_k$9; removing the forgetting signal yields OP 44.3, FWT 0.8, and BWT $k$00. Replacing trajectory-derived weights with manually set global merging weights gives OP 44.2, FWT 0.7, and BWT $k$01, while using importance-based fine-grained merging gives OP 44.9, FWT 1.2, and BWT $k$02. This indicates that the trajectory-derived weighting mechanism is slightly stronger than importance-based weights and avoids their computational overhead (Feng et al., 22 Sep 2025).

Memory and efficiency measurements also show a concrete trade-off. With Qwen3-1.7B, training time per epoch is 3.3 minutes for LoRAReplay, 3.4 for TaSL, 4.9 for Recurrent-KIF, and 4.4 for AimMerging. The method is therefore about $k$03 slower than single-round baselines but faster than fixed multi-round merging. On a 19-task mixed Standard CL + SuperNI sequence, AimMerging reports OP 48.3, FWT $k$04, and BWT $k$05, compared with Recurrent-KIF at OP 46.4, FWT $k$06, and BWT $k$07, indicating improved behavior under stronger distribution shifts (Feng et al., 22 Sep 2025).

5. Broader theoretical lenses on iterative model merging

The term “adaptive iterative model merging” is used more broadly in adjacent literature than in the continual-learning formulation of AimMerging. One important line of work treats task vectors as approximate gradients and merging as repeated optimization. ATM shows that, in a single-epoch setting under gradient descent, task vectors are mathematically equivalent to gradients obtained in a multi-task setting, and it proposes alternating tuning and merging as an iterative multi-task procedure (Zhou et al., 2024). DOGE, in turn, models multi-task model merging as a constrained optimization problem over task vectors and solves it via adaptive projective gradient descent in a shared subspace, with layer-wise merging coefficients interpreted as task-aware learning rates (Wei et al., 2 Jan 2025). This suggests a conceptual link: AimMerging’s trajectory-guided cycles can be interpreted as a continual-learning instance of a wider class of iterative, gradient-like merge procedures.

A second line of work reinterprets iterative merging through second-order or spectral geometry. ACE-Merging derives a closed-form data-free merge

$k$08

where $k$09 are covariance proxies inferred from parameter differences, and explicitly positions itself relative to adaptive iterative methods by arguing that covariance and curvature modeling can guide or replace heuristic update schedules (Xu et al., 3 Mar 2026). Closed-form spectral regularization work then shows that iterative solvers for quadratic interference minimization behave primarily as implicit spectral regularizers of ill-posed normal equations, and proposes SWUDI and SWUDI-A as one-shot spectral filters that match or outperform iterative loops (Wei et al., 5 Jun 2026). These results suggest that AimMerging-style controllers could potentially benefit from explicit spectral or covariance diagnostics rather than only trajectory counts and replay losses.

A third line extends iterative merging to continual or sequential settings without replay. A projection-based continual merging method processes models sequentially by projecting new parameter updates onto subspaces orthogonal to existing merged updates and uses an adaptive scaling mechanism to maintain stable parameter distances, with constant memory complexity in the number of models (Tang et al., 16 Jan 2025). BECAME, from the continual-learning literature, derives a closed-form adaptive merge coefficient from a Bayesian formulation,

$k$10

to interpolate between a stable projection-based solution and a more plastic unconstrained solution (Li et al., 3 Apr 2025). This suggests that the dynamic weighting rule in AimMerging could in principle be complemented by curvature- or Fisher-informed weighting in future designs.

More recent work also treats optimization trajectories themselves as mergeable objects. TPMM-DPO constructs a reference model for iterative DPO by learning fusion weights over a sequence of policy models generated during alignment, rather than using only the immediately previous checkpoint. Its merged reference takes the form

$k$11

with $k$12 learned from preference data (Fu et al., 22 May 2026). That trajectory-aware formulation is not AimMerging, but it sharpens a useful interpretation: in iterative systems, adaptation can target not only model parameters within a cycle, but also the distribution of influence across the entire training trajectory.

6. Limitations, misconceptions, and future directions

AimMerging is sometimes conflated with the broader data-free model-merging literature, but its defining characteristics are different. It is not a one-shot merger of frozen experts, and it is not data-free: it depends on replay memory both for loss monitoring and for constructing the historical-task vector through short memory-only fine-tuning (Feng et al., 22 Sep 2025). This means that its performance claims are tied to a replay-based continual-learning setting rather than to the fully data-free merging regime studied by ACE-Merging, SWUDI, DOGE, or sequential projection-based merging (Xu et al., 3 Mar 2026, Wei et al., 5 Jun 2026, Wei et al., 2 Jan 2025, Tang et al., 16 Jan 2025).

The authors identify two main limitations. First, the signal design is narrow: the current controller uses parameter changes and historical loss as proxies for learning and forgetting, while it remains open whether gradient norms, Hessian approximations, or layer-wise activation drift would better characterize model state and catastrophic forgetting. Second, the controller is semi-heuristic: interval bounds and thresholds such as $k$13, $k$14, $k$15, $k$16, and $k$17 are hand-designed rather than learned (Feng et al., 22 Sep 2025).

Additional practical constraints follow from the same design. The method depends on replay memory; a memory-free variant is not developed. It introduces extra training time because each merge involves both controller logic and rehearsal-based fusion, even if it remains faster than fixed multi-round merging in the reported setting. The paper also notes that scaling to ultra-large models may present engineering challenges, although experiments already reach 13B-parameter backbones (Feng et al., 22 Sep 2025).

The most direct future directions remain internal to the AimMerging formulation: fully automated or bi-level optimization frameworks for learning merge policies and thresholds, memory-free or retrieval-augmented variants where forgetting is estimated by other proxies, and combinations with more sophisticated PEFT mechanisms such as AdaLoRA, BiLoRA, or module routing (Feng et al., 22 Sep 2025). Broader work on iterative merging suggests additional possibilities. Spectral filtering results imply that a controller could adapt merge timing from per-layer eigenspectra rather than only from loss excursions (Wei et al., 5 Jun 2026). Covariance-aware closed-form methods suggest that task-vector statistics may provide data-free curvature surrogates for setting merge weights (Xu et al., 3 Mar 2026). Trajectory-aware preference-guided merging suggests that entire sequences of intermediate models can themselves be fused to suppress late-stage over-optimization (Fu et al., 22 May 2026). Taken together, these directions indicate that AimMerging is best understood not as a fixed algorithmic recipe, but as one explicit continual-learning instantiation of a broader program: adapt the merge operator, the merge schedule, and the merge weights to the evolving geometry of training.