Compositional Fine-Tuning (CompFT)

• Compositional Fine-Tuning (CompFT) is a framework that enables modular and composable adaptation of pre-trained models by fine-tuning isolated task-specific modules.
• It leverages theoretical constructs like a second-order Taylor expansion and Fisher-informed penalties to maintain composability within the pre-training basin.
• CompFT has practical applications in language and vision models, yielding improved accuracy and generalization for complex, composite tasks.

Compositional Fine-Tuning (CompFT) refers to a set of methodologies and theoretical frameworks for fine-tuning pre-trained models in a manner that enables modular, recombinable specialization, facilitating the construction of multi-task or highly adaptive models with minimal interference between learned skills. CompFT approaches aim to induce and leverage compositional properties in neural models, enabling the combination of task-specific adaptations (“modules”) to solve new, complex, or composite tasks efficiently. This paradigm is supported by advances in neural theory, curriculum construction, optimization, and practical instantiation across vision, language, and multimodal domains (Porrello et al., 2024, Wu et al., 30 Apr 2025, Hayati et al., 2024, Bursztyn et al., 2022, Zhou, 13 Mar 2025, Peleg et al., 30 May 2025, Yu et al., 2021).

1. Theoretical Foundations of Compositional Fine-Tuning

A general theoretical basis for CompFT in non-linear neural networks is provided by a second-order Taylor expansion of the loss function around the pre-trained initialization θ₀. Consider a model $f(x;\theta)$ with twice-differentiable loss $\ell(\theta|x,y)$. The central result is the quadratic local surrogate: $$\ell(\theta) \approx \ell(\theta_0) + \frac{1}{2} (\theta-\theta_0)^T H_0 (\theta-\theta_0)$$ where $H_0$ is the Hessian at $\theta_0$ under the assumption of local optimality. If fine-tuned task-specific modules (parameter updates $\Delta_t$) remain within the “pre-training basin”—the region where the cubic term is negligible—then convex combinations of these modules $\theta_P = \theta_0 + \sum_t w_t \Delta_t$ guarantee low loss under the quadratic approximation. Jensen's inequality further ensures that the composed model achieves loss no greater than the ensemble average of modules if each achieves low individual quadratic loss (Porrello et al., 2024).

This composability breaks down if modules venture outside the basin, i.e., when the third-order residue is no longer small. Consequently, CompFT frameworks emphasize strong regularization or architectural constraints that tether updates close to θ₀, maximizing composable capacity.

2. Canonical Algorithms for CompFT

The core practical algorithms that instantiate the theoretical insights of CompFT in non-linear models include:

Individual Task Arithmetic (ITA):

Each task t learns its own displacement $\Delta_t$ by optimizing its loss on its dataset plus a Fisher-informed quadratic penalty: $$\min_{\Delta_t} \mathbb{E}_{(x,y)\sim D_t}[\ell(\theta_0 + \Delta_t|x,y)] + \frac{\alpha}{2} \|\Delta_t\|^2_{F}$$ where $F$ denotes the (diagonal) pre-training Fisher Information matrix, enforcing a constraint analogous to Elastic Weight Consolidation centered at θ₀. Modules are trained independently; composition at inference is performed by averaging their displacements.

Incremental Ensemble Learning (IEL):

IEL directly trains the composed model $\theta_P$ with a diversity penalty that aligns the Fisher-normed task vectors, minimizing the multi-module gap: $$\min_{\Delta_t} \mathbb{E}_{(x,y)\sim D_t}[\ell(\theta_P|x,y)] + \lambda \hat{\Omega}(\Delta_1 \dots \Delta_t)$$ where $\hat{\Omega}$ penalizes pairwise divergence between modules in the Fisher metric. IEL thus encourages not only composability but also representation alignment between the modules (Porrello et al., 2024).

Complementary approaches in the representation space, such as Compositional Subspace Representation Fine-tuning (CS-ReFT), define learnable, orthogonal skill subspaces in the hidden state, with a lightweight router performing dynamic, input-dependent composition, thereby improving parameter efficiency and further isolating task-specific knowledge (Zhou, 13 Mar 2025).

3. CompFT in Language and Vision–LLMs

CompFT methods extend beyond supervised classifiers to both LLMs and multimodal (vision–language) models, relying on compositional task structure at the data and curriculum level.

Chain-of-Instructions (CoI) Tuning:

CoI tuning teaches LLMs to carry out multi-step tasks by exposing them to explicit sequences of instructions where output of one subtask becomes input to the next, and the model is supervised on the full chain of outputs. CoI assemblages are semi-automatically constructed by filtering composable single-step tasks and generating prompt–intermediate–final answer tuples, with maximum-likelihood supervision over each substep (Hayati et al., 2024). Training on a mix of varying chain lengths (e.g., CoI₁₂₃) yields substantial gains on multi-step generalization and out-of-domain composite tasks.

Data-Centric Visual Compositionality (COMPACT):

COMPACT addresses the compositional gap in multimodal LLMs by curating datasets with controlled "compositional complexity," determined by the number $k$ of atomic visual skills required per task instance. Balanced sampling over $k=1,2,3$ (atomic to moderately complex) and careful coverage of all atomic skill combinations yields training curricula that induce strong generalization to high-complexity (e.g., $k\geq 4$) tasks, far outperforming baseline visual instruction tuning both in efficiency (≤10% data) and accuracy on complex visual-language queries (Wu et al., 30 Apr 2025).

Compositionality-aware CLIP Fine-tuning (CLIC):

CLIC introduces compositionally structured batches for CLIP by concatenating images and mixing-related, hard-positive and hard-negative captions in the contrastive loss, training the model to better resolve lexical and semantic composition. CLIC applies losses over hard negatives (synthetic cross-capability swaps) and ensures invariance to syntax by enforcing consistency across reordered caption pairs. This results in marked improvements on lexical and semantic composition benchmarks (e.g., SugarCrepe++ ITT/TOT), surpassing earlier approaches without the retrieval tradeoff observed in competing methods (Peleg et al., 30 May 2025).

4. Empirical Findings and Evaluation

Experimental studies across domains consistently demonstrate that CompFT strategies yield enhanced multi-task and compositional generalization, outperforming standard (and often larger) models or naive fine-tuning:

• Image and Vision Benchmarks: ITA and IEL in conjunction with low-rank adapters (LoRA) provide 5–10 percentage point improvements in final accuracy and ≤5% final forgetting on standard continual learning benchmarks (Split CIFAR-100, ImageNet-R) compared to PEFT approaches such as L2P, CODA, and TMC (Porrello et al., 2024).
• LLMs and Multi-step Language Tasks: CoI-tuned LMs achieve a >2× boost in ROUGE-L on multi-step tasks compared to single-instruction or chain-of-thought fine-tuning, and CoI₁₂₃ Mistral achieves a 3× improvement over the base model on BIG-Bench Hard (Hayati et al., 2024). Compositional curricula for smaller LMs rival the performance of chain-of-thought prompting on much larger models (Bursztyn et al., 2022).
• Multimodal Curriculum: COMPACT-trained MLLMs yield 83%–94% relative improvements for $k \geq 4$ composite vision-language benchmarks, with balanced curricula necessary for optimal results. Omitting atomic capabilities or skewing $k$-distribution degrades performance, affirming the necessity of explicit compositional coverage (Wu et al., 30 Apr 2025).
• Compositionality in Representation: Fine-tuned models display superior capability for specialization (reweighting module averages) and “unlearning” (subtracting a module) not possible in monolithic continual fine-tuning (Porrello et al., 2024). CS-ReFT demonstrates nearly complete elimination of cross-skill interference compared to LoRA, with state-of-the-art win rates (93.94%) and only 0.0098% parameter overhead (Zhou, 13 Mar 2025).
• Controlled Generalization and Retrieval: CLIC fine-tuned CLIP models obtain the best-known ITT scores on SugarCrepe++, and uniquely improve rather than degrade retrieval accuracy (e.g., COCO text→image R@5 +5.4% over baseline), disproving that compositional improvements must trade off with downstream retrieval (Peleg et al., 30 May 2025).

5. Scope, Limitations, and Best Practices

Empirical and controlled analyses have revealed both the power and boundary conditions of CompFT:

• Basin Limitation: Compositionality by parameter averaging is limited to regions close to θ₀. Exceeding this pre-training basin (large $\|\Delta_t\|$ updates) destroys quadratic approximability and thus composability (Porrello et al., 2024).
• Dataset and Objective Design: Effectiveness of CompFT depends on semantically rich, structurally varied curricula that enforce genuine compositional reasoning—datasets with superficial lexical/structural cues (e.g., PAWS-QQP) fail to induce deep composition (Yu et al., 2021).
• Representation vs. Weight Composition: Approaches like CS-ReFT that operate on hidden-state representations, rather than weight deltas, provide better task isolation and parameter efficiency; weight-space methods may still leave interference channels (Zhou, 13 Mar 2025).
• Automation Challenges: Task decomposition has often relied on manual design, limiting scalability; automated task graph induction and demonstration remain open research challenges (Bursztyn et al., 2022).
• Scaling and Generalization: CompFT’s robust benefits extend across LLM architectures and modalities, but further scaling to more diverse and deeply nested task structures, as well as interpretability of modular routers, remain active points for investigation (Wu et al., 30 Apr 2025, Zhou, 13 Mar 2025).

6. Perspectives and Future Research Directions

Ongoing work in CompFT is converging on several promising avenues:

• Automating Task Decomposition: Integrating unsupervised decomposition or question-graph induction into the CompFT pipeline promises greater scalability, especially for language and multimodal tasks (Bursztyn et al., 2022).
• Curriculum and Loss Innovation: Developing contrastive or auxiliary decompositional objectives, explicit compositionality loss, and harder negative curricula are identified as key to advancing robust phrase-level composition, as is careful control of surface cues and phrase variability (Yu et al., 2021).
• Extending to Unified and Generative Models: Adapting data-centric and compositional fine-tuning (e.g., concatenation, routed subspaces) to unimodal and multimodal generative architectures beyond contrastive models is a primary research aim (Wu et al., 30 Apr 2025, Peleg et al., 30 May 2025).
• Specialization, Unlearning, and Transfer: CompFT’s intrinsic capability for dynamic specialization (reweighting/disabling modules), zero-shot unlearning (module removal), and structured transfer learning offers tools for privacy-preserving incremental and continual learning pipelines (Porrello et al., 2024).

In summary, CompFT frameworks are reshaping approaches to adaptability, multi-task learning, and scalability in advanced neural models by formalizing and operationalizing compositionality through both algorithmic and data-centric innovations. The paradigm informs not only more efficient and robust learning in supervised and continual learning but also the foundational design of data and objectives for complex reasoning tasks across modalities (Porrello et al., 2024, Wu et al., 30 Apr 2025, Hayati et al., 2024, Bursztyn et al., 2022, Zhou, 13 Mar 2025, Peleg et al., 30 May 2025, Yu et al., 2021).