Controllable Model Merging

• Controllable model merging is a technique that fuses pretrained models using explicit control parameters, ensuring continuous variation in output behavior.
• It employs methods such as linear interpolation, orthogonal delta merging, and dynamic gating to balance trade-offs in accuracy, memory, and regulatory compliance.
• The approach supports reversible and scalable integration, enabling efficient adaptation and fine-grained prioritization of multiple tasks or domains.

Controllable model merging refers to algorithmic frameworks and methodologies that enable explicit, user-driven manipulation of the trade-offs and task priorities when fusing multiple trained neural networks into a single, unified model. Unlike naïve parameter averaging or unstructured merging approaches, controllable model merging introduces interpretable, continuous control parameters—such as mixing weights, gate values, or preference vectors—that allow practitioners to tune model behavior along axes such as accuracy, memory footprint, backward compatibility, interpretability, and even regulatory compliance. This topic is central to the deployment of complex machine learning systems that must evolve, accommodate new capabilities, or satisfy domain-specific requirements without catastrophic forgetting or destructive interference among constituent task expertise. Contemporary research exhibits a spectrum of techniques encompassing direct parameter space interpolations, modular delta projections, optimization-based fusion, routing and gating mechanisms, and preference-aware closed-form solutions.

1. Conceptual Foundations and Formal Definition

Controllable model merging consists of techniques that combine two or more pre-trained or fine-tuned models into a single monolithic or modular artifact whose inference-time behavior can be continuously varied using a small number of explicit control parameters. The primary desiderata include:

• Continuity: Smooth variation of model outputs as control parameters (e.g., mixing coefficients α, preference vectors p) are adjusted.
• Interpretability: The controls (such as α or β for per-task or per-layer importance) have clear correspondence to specific subtask or capability prioritizations.
• Scalability: Efficiently merging large numbers of tasks or domains without quadratic resource blowup.
• Efficiency: Minimal computational overhead, ideally requiring only lightweight optimization or linear algebra, not full retraining (Ruan et al., 12 Mar 2025, Khan et al., 28 Jul 2025).
• Reversibility/Composability: The ability to undo, amplify, or subtract individual task contributions (e.g., via orthogonal delta projections).
• Privacy Alignment: Merge operations performed solely in weight or update space, minimizing leakage of training data or labels.

Mathematically, the canonical form for merging is:

• For models $\theta^1, ..., \theta^K$ (possibly relative to a base $\theta^0$), and merge weights $\alpha \in \mathbb{R}^K$ (often constrained:

$$\theta_{\rm merged} = \theta^0 + \sum_{k=1}^K \alpha_k (\theta^k - \theta^0)$$

More sophisticated frameworks introduce per-layer, per-chunk, or data-dependent controls, as in modular delta merging (Khan et al., 28 Jul 2025), dynamic gating (Lu et al., 17 Jun 2024), or closed-form representations (Wu et al., 14 Nov 2025).

2. Methodological Landscape: Categories and Mechanisms

A taxonomy of controllable model merging methods encompasses multiple axes of innovation, as summarized below (Ruan et al., 12 Mar 2025, Khan et al., 28 Jul 2025, Lu et al., 17 Jun 2024, Wu et al., 14 Nov 2025, Zhang et al., 30 Sep 2025):

Category	Control Parameters	Example Methods
Linear Interpolation	Global α, {αₖ}	Model Soup, Task Arithmetic, DARE
Layer-/Chunk-wise	{α_ℓ}, {βₖ^ℓ}, {γ}	Layer-wise fusion, Expert Merging++
Orthogonal/Modular	{αₖ}, projections	MDM-OC, TWIN-Merging
Gating/Routing	Data-dependent wₖ(x)	Twin-Merging, Mixture-of-Experts
Optimization-based	α optimized by gradient	AdaMerging, Fisher-weighted averaging
Preference-to-merge	Preference vector p	ReACT (closed-form), Bone Soup

• Orthogonal Delta Merging: Modular Delta Merging with Orthogonal Constraints (MDM-OC) maintains an explicit, interference-free representation by projecting each task delta ($\Delta_i$) into mutually orthogonal subspaces relative to a base checkpoint, allowing exact addition or removal of task contributions (Khan et al., 28 Jul 2025).
• Dynamic Gating/Router Networks: Twin-Merging constructs modular experts with shared and exclusive knowledge, fusing them at inference via a data-dependent router that outputs soft mixture weights conditioned on the input, facilitating instance-level specialization (Lu et al., 17 Jun 2024).
• Layer- and Chunk-wise Coefficients: Expert Merging and its importance-guided extension (Expert Merging++) learn a minimal set of per-layer (or per-chunk) coefficients using only unlabeled calibration data, enabling per-domain trade-offs and focusing control resources on salient network regions (Zhang et al., 30 Sep 2025).
• Optimization-Based Fusion: Methods such as AdaMerging, regression mean, and Fisher-weighted averaging optimize mixing coefficients on a held-out validation set, optionally enforcing regularization or sparsity (Ruan et al., 12 Mar 2025, Wu et al., 14 Nov 2025).
• Representation-Space Correction: ReACT computes a single-step, closed-form optimal linear transformation of the model's final representation to align with user preferences, bypassing the need for iterative search or training as task count scales (Wu et al., 14 Nov 2025).
• Multi-Objective Generation: Bone Soup decouples backbone training from merging by generating a basis of Pareto-optimal models via multi-objective RL, then merges them using a user-supplied preference vector transformed by a circulant mapping matrix (Xie et al., 15 Feb 2025).

3. Knobs and Controllability: Parameter Design and Use

Explicit controls (“knobs”) in model merging frameworks enable practitioners to choose trade-off points on the Pareto frontier of multitask performance:

• Merge Weights ($\alpha$): Directly scale the influence of each expert or delta; may be global, layerwise, or task-specific (Ruan et al., 12 Mar 2025, Khan et al., 28 Jul 2025).
• Task or Domain Preferences ($\mathbf{p}, \beta_k$): Vector-valued priorities that weight loss terms, projection operators, or fusion coefficients. In ReACT, varying p realizes any convex combination of per-task performance (Wu et al., 14 Nov 2025); in Bone Soup, user intent is mapped to merging coefficients via an invertible design matrix (Xie et al., 15 Feb 2025).
• Pruning/Masking Thresholds ($\tau, p$): Sparsify merged updates to resolve conflicts or reduce memory/compute (Ruan et al., 12 Mar 2025).
• Layer-/Chunk-Allocation: Number of coefficients allocated per layer or submodule as a function of learned importance or gradient signal strength (Zhang et al., 30 Sep 2025).
• Gating/Router Outputs: Data-dependent weights steering the model to emphasize certain experts or behaviors at runtime, supporting task mixture or domain adaptation (Lu et al., 17 Jun 2024).
• Dynamic Adjustment and Unmerging: For reversible frameworks (e.g., MDM-OC), setting αₖ=0 removes a task’s contribution without perturbing others, supporting regulatory use cases such as GDPR compliance (Khan et al., 28 Jul 2025).

4. Algorithmic Procedures and Computational Properties

Algorithmic sketches and complexity analyses highlight common operational themes and scaling behaviors:

• Orthogonalization and Delta Storage: MDM-OC projects deltas into a null-space using Gram–Schmidt, storing only K compressed deltas (O(K·d)); subsequent merges or unmerges are reduced to linear algebra (Khan et al., 28 Jul 2025).
• Router Training and Dynamic Fusion: Twin-Merging and similar methods only train a lightweight router on a small validation set, with frozen expert parameters. At inference, the router assembles expert-exclusive components “on the fly” (Lu et al., 17 Jun 2024).
• Closed-Form Preference Queries: ReACT’s linear map Wₚ for a user-supplied p can be computed in O(T·D²) time, enabling instantaneous generation of Pareto-optimal models as preferences shift (Wu et al., 14 Nov 2025).
• Multi-Fidelity Optimization Loops: Automated merge frameworks conduct rapid, budgeted validation sweeps in the space of possible merge strategies, using surrogates and successive halving to minimize expensive full validation (Su et al., 6 Feb 2025).
• Parallelizable Knowledge Distillation: DMM trains a style-promptable student model against multiple diffusion-based teachers via distributed score matching and feature imitation, with style control implemented by learned codebook embeddings (Song et al., 16 Apr 2025).
• Alignment-Based Coefficient Learning: Expert Merging++ optimizes per-layer/chunk coefficients to minimize hidden-state and logit alignment losses, regularized for coefficient stability, requiring only a handful of unlabeled samples per task (Zhang et al., 30 Sep 2025).

5. Empirical Results, Evaluation Protocols, and Benchmarking

Quantitative research consistently demonstrates the effectiveness of controllable merging frameworks, comparing them on:

• Performance Interpolation Curves: Plotting task accuracy vs. control parameter (e.g., α) reveals convex, monotonic, or even phase-transition behaviors. Empirically, merged models often trace accuracy–efficiency frontiers that strictly Pareto-dominate one or both parents (Lan et al., 26 Sep 2025).
• Unmerging Fidelity: In MDM-OC, removing a task’s delta incurs only a marginal accuracy drop (UAD: 1.8% vision, 2.3% language) and fast recovery (12.4 s), versus higher drop and latency in prior methods (Khan et al., 28 Jul 2025).
• Adaptability/Domain Prioritization: In Twin-Merging and Expert Merging++, adjusting router biases or task weights β_k results in explicit, interpretable shifts in sub-task behavior. For MLLMs, importance-guided chunking yields 0.3–1.5pp improvements even over multitask mixture supervision (Zhang et al., 30 Sep 2025, Lu et al., 17 Jun 2024).
• Storage and Compute: MDM-OC achieves near-linear O(K) memory growth versus quadratic scaling in memory-based replay, and achieves orders-of-magnitude cost savings over retraining (Khan et al., 28 Jul 2025).
• Preference Alignment: ReACT surpasses other Pareto-optimal merging strategies in accuracy and hypervolume by 5–10 points under various test cases, and delivers 36×–208× better speed on adapting to new user preferences (Wu et al., 14 Nov 2025).
• Score Distillation and Style Control: DMM reconstructs per-teacher FIDt to within 3–5% of the teacher upper bound, and enables smooth, continuous style interpolation by embedding mixing (Song et al., 16 Apr 2025).
• Multi-objective Trade-offs: Bone Soup achieves full controllability and the best hypervolume/inner-product metrics under diverse user preferences for multi-reward RL, outperforming “Rewarded Soup” and adaptive policy interpolation (Xie et al., 15 Feb 2025).

6. Applications, Limitations, and Future Directions

Practical deployments of controllable model merging span continual learning, compliance (e.g., data “unlearning”), multi-domain or multi-modal fusion, dynamic adaptation to novel distributions, and memory-efficient LLM/MLLM deployment. Key limitations and opportunities include:

• Assumptions of Linear Interpolation: Many approaches hinge on the efficacy of linear or affine interpolation in parameter or representation space, which may break down with highly non-linear update patterns (Wu et al., 14 Nov 2025).
• Conflict Detection and Interference: Without orthogonalization or explicit gating, parameter overlap can cause destructive interference; modular delta, chunking, and dynamic routing directly address this (Khan et al., 28 Jul 2025, Zhang et al., 30 Sep 2025).
• Calibration Data Requirements: Closed-form techniques and coefficient learning often require small calibration sets, typically unlabeled; zero-shot or unsupervised proxies are promising directions (Wu et al., 14 Nov 2025, Zhang et al., 30 Sep 2025).
• Higher-Order and Nonlinear Fusion: Current merging is largely linear or piecewise linear (with some attention to SVD/PCA, block-level corrections); nonlinear correction methods and calibration-free merges remain open challenges (Wu et al., 14 Nov 2025, Khan et al., 28 Jul 2025).
• Automated Configuration and User Interaction: Multi-fidelity search frameworks accelerate configuration selection, but adaptive, interaction-driven tuning (e.g., in edge deployments or privacy-sensitive domains) is an open avenue (Su et al., 6 Feb 2025).
• Extensions to Very Large Models and Architectures: While merging is demonstrated on transformer LLM/MLLMs and diffusion models, scalable adaptation to future model architectures and even decentralized or federated settings is under exploration.

In summary, controllable model merging constitutes a robust, mathematically grounded approach for the compositional integration and dynamic adaptation of deep networks. By exposing explicit, interpretable controls and leveraging orthogonality, modularization, and preference-aware optimization, state-of-the-art frameworks deliver scalable, reversible, and fine-grained trade-offs in complex AI deployments (Khan et al., 28 Jul 2025, Lu et al., 17 Jun 2024, Song et al., 16 Apr 2025, Wu et al., 14 Nov 2025, Zhang et al., 30 Sep 2025, Xie et al., 15 Feb 2025, Su et al., 6 Feb 2025, Ruan et al., 12 Mar 2025).