---
title: Two-Stage Fine-Tuning
url: https://www.emergentmind.com/topics/two-stage-fine-tuning
type: topic
---

# Two-Stage Fine-Tuning

Two-stage fine-tuning is a versatile training paradigm in modern machine learning, involving the explicit sequential decomposition of model adaptation into two distinct optimization phases. This approach has arisen independently across numerous domains—including large language models (LLMs), diffusion-based video editing, multilingual reasoning, class-imbalanced classification, neural architecture search, knowledge distillation, and parameter estimation—each leveraging the structure of two-stage fine-tuning to address complex adaptation, representation, or generalization challenges. The defining characteristic is the strict separation of learning objectives or modules between the two stages, avoiding interference and optimizing synergies that are otherwise degraded under single-stage or simultaneous adaptation.

## 1. Formulation and Core Principles

Two-stage fine-tuning is characterized by the consecutive execution of two training or adaptation phases, with each stage serving a targeted role in the model’s overall adaptation. The first stage usually introduces new knowledge, alignment, or structure under strong constraints (e.g., parameter or modular scope, data type, or loss type), while the second stage performs specialized or task-specific adaptation, often with different trainable parameters, objectives, data regimes, or regularization strengths. Crucially, parameters or modules tuned in Stage I may be frozen in Stage II, and the two optimization steps are not interleaved.

### Generalized Schematic

Let $\theta$ denote all trainable parameters of a pre-trained model, and let $\mathcal{L}_1$, $\mathcal{L}_2$ be stage-specific loss functions. The workflow is:
1. **Stage I**: Optimize $\min_{\theta_1} \mathcal{L}_1(\theta_1; D_1)$, where $\theta_1 \subseteq \theta$ (often a strict subset: e.g., normalization scales, prompt vectors, head layers).
2. **Stage II**: With $\theta_1$ fixed, optimize $\min_{\theta_2} \mathcal{L}_2(\theta_2; D_2)$ where $\theta_2 \cap \theta_1 = \emptyset$ or, in some cases, $\theta_2 \supseteq \theta_1$ with a different focus.

Distinct roles for $\mathcal{L}_1$ and $\mathcal{L}_2$ are essential: for instance, $\mathcal{L}_1$ may be a knowledge-injection loss, a reweighted loss for rare class amplification, or a distillation objective, while $\mathcal{L}_2$ is often a downstream task objective or another loss more attuned to the target evaluation metric.

## 2. Domain-Specific Instantiations

The two-stage paradigm has been instantiated in a wide range of domains with distinct module, data, and loss decompositions, including but not limited to:

| Domain                    | Stage I Objective / Module        | Stage II Objective / Module            |
|---------------------------|-----------------------------------|----------------------------------------|
| Video diffusion/editing   | Norm tuning for temporal alignment| Spatial adapters for per-frame fidelity|
| LLM instruction-tuning    | Broad medical knowledge injection | MCQ exam adaptation                    |
| Multilingual LLMs         | Language alignment via code-switch | English-only instruction tuning        |
| Vision/NER                | Architectural mutation (NAS)      | Fine-tune mutated weights              |
| Embodied AI               | Standard RL/BC adaptation         | Gradient noise, batch/sample reduction |
| Graph-to-text             | Wikipedia graph-text warmup        | Targeted graph-to-text fine-tuning     |
| Class-imbalance learning  | Head-only, weighted loss          | Full model, standard loss              |
| Model Fusion/Selection    | BO hyperparam/search trajectory   | BO-based model fusion (Pareto-optimal) |
| Multimodal retrieval-gen  | RL for filtering irrelevant docs  | RL for explainable QA and retrieval    |
| Model distillation        | Distill pre-training              | Distill fine-tuning                    |

Domain-specific designs exploit the decoupling property to mitigate mutual interference, unlock parameter efficiency, and yield representations more suited for generalization, specialization, or robustness ([2505.07057], [2409.05732], [2412.12499], [2211.00635], [2207.10858], [2411.06710], [2512.17194], [2503.14523], [2503.22672], [2302.09779], [2412.20707], [2512.03976], [2105.08021], [2409.06980], [2504.04480]).

## 3. Mathematical and Optimization Structures

The two-stage framework leverages mathematically distinct loss landscapes, parameter subsets, or modules at each stage.

- **Parameter isolation/freeze**: Explicit parameter scopes per stage (e.g., normalize-and-adapt, head-then-body, prompt-then-model).
- **Customized loss functions**: Stage I may use Huber loss on residuals ([2505.07057]), margin-based reweighting ([2207.10858]), language modeling ([2512.03976]), or custom regularizers (BO/fusion, [2411.06710]); Stage II typically employs task-specific CE or structured RL (MMRAG, [2512.17194]).
- **Batch and schedule modulation**: Learning rate decay, noisy gradient injection (smaller batch, reduced sample size) in Stage II for regularization and better generalization ([2307.11343]).
- **Adapter, prompt, or side-branch modules**: Parameter-efficient adapters (LoRA, PEFT), soft prompts, or alignment layers are developed and frozen or replaced between stages ([2412.12499], [2409.05732], [2505.07057]).

Sample decomposition for LLMs, biomedical tuning ([2409.05732]):
\[
\mathcal{L}_{\text{Stage 1}} = -\sum_{t} \log P_\theta(y_t \mid x, y_{<t}) \qquad
\mathcal{L}_{\text{Stage 2}} = -\sum_{i=1}^4 q_i\log p_i
\]
with different LoRA ranks and learning rates.

## 4. Motivations and Theoretical Rationale

Common motivations for two-stage fine-tuning include:

- **Decoupling mutually adverse adaptations**: In video editing, temporal norm tuning and spatial detail enhancement conflict if trained jointly; separating them allows each to reach optimality without degradation ([2505.07057]).
- **Mitigating overfitting and improving generalization**: Stage II with more gradient noise or targeted replay (e.g., reweighted batches) combats sharp minima and catastrophic forgetting ([2307.11343], [2410.05802], [2207.10858]).
- **Boosted adaptation for under-represented modalities or languages**: Dedicated alignment or pre-adaptation stages allow low-resource languages or classes to benefit from richer pretraining ([2412.12499], [2512.03976], [2207.10858]).
- **Efficient and modular parameter utilization**: PEFT, soft prompts, adapters: only a small subset of parameters is trained in each stage, reducing memory and time cost ([2409.05732], [2505.07057], [2412.20707], [2412.12499]).
- **Improved multitask and out-of-distribution (OOD) resilience**: By priming models on general or partially known information, two-stage approaches enhance OOD capabilities and context transfer ([2410.05802], [2512.03976], [2211.00635]).

## 5. Empirical Findings and Impact

Empirical studies across application domains consistently report that two-stage fine-tuning offers:

- **Performance gains** versus conventional or single-stage fine-tuning, especially on minority-domain or minority-class metrics, and in cross-lingual or OOD scenarios ([2409.05732], [2505.07057], [2412.12499], [2503.14523], [2503.22672], [2411.06710]).
- **Reduction in overfitting and better retention of pretrained knowledge or skills**, as seen in class balance ([2207.10858]), knowledge replay ([2410.05802]), or modularity ([2211.00635]).
- **Parameter- and compute-efficiency**, often requiring less than 1% of full model updates per stage, and enabling single-GPU or limited-resource adaptation where appropriate ([2409.05732], [2505.07057], [2412.20707], [2409.06980]).
- **Established best practices** for task freezing/unfreezing, adaptive reweighting, selection of checkpointing strategies, and robust model fusion ([2411.06710], [2503.14523], [2202.08604]).

## 6. Ablations, Limitations, and Practical Guidance

Ablation studies and practical guidelines highlight several key points:

- **Sequentiality is critical**: Merged or joint training of both stages leads to mutual degradation or suboptimal trade-offs between objectives ([2505.07057], [2412.12499], [2207.10858]).
- **Right module selection per stage**: PEFT modules (e.g., LoRA), soft prompts, adapters, or layer freezing are often more effective than full-model updates, but their optimal scope may be task or domain dependent ([2409.05732], [2211.00635], [2412.20707]).
- **Loss surfaces and metric misalignment**: In domains such as language model fusion, the metric of interest may be poorly aligned with the task loss, which two-stage BO fusion overcomes ([2411.06710]).
- **Scaling and transfer extension**: Two-stage systems scale favorably to larger models or multilingual/low-resource regimes with appropriate adjustments ([2412.12499], [2512.03976]).
- **Limitations**: Some schemes rely on the quality of pretrained features, the availability of high-resource data for the first stage, synthetic data for OOD adaptation, or assume simulator accessibility ([2504.04480], [2512.03976]). There is sometimes sensitivity to module design and the proportion of replay or noise-injection samples.

## 7. Representative Algorithms and Quantitative Benchmarks

Distinct algorithmic blueprints and their measured benefits include:

- **DAPE for Video Editing**: Norm-tuning (+0.20 CLIP-frame, +0.06 CLIP-text), adapter tuning for fidelity; two-stage decoupling achieves best composite metrics ([2505.07057]).
- **Medical LLMs**: Stage I injection of 200k multilingual QA pairs; Stage II MCQ tuning; +3-17% accuracy over single-stage ([2409.05732]).
- **ManiSkill2 Embodied Policy**: Batch/sample noise injection in Stage II yields 3–15% absolute test gain ([2307.11343]).
- **LLM Generalization (ProMoT)**: Soft-prompt + model two-stage reduces format overfitting; up to +4.74 normalized average over conventional fine-tuning ([2211.00635]).
- **Class-Imbalanced Text**: Head reweighting then full fine-tuning: up to +0.0161 F1 gain on tails, +0.0133 micro-F1 overall ([2207.10858]).
- **Model Fusion (BOMF)**: Two-stage BO + fusion finds Pareto-optimal model averages, outperforming SWA by ≈ +1 pt GLUE, +0.5–1 BLEU/ROUGE ([2411.06710]).
- **SDF-TopoNet**: Two-stage SDF regression then topological loss, yielding ≈200–400% improvements in both Dice and clDice over prior persistent-homology segmentation ([2503.14523]).

## References

- DAPE: Dual-Stage Parameter-Efficient Fine-Tuning for Consistent Video Editing with Diffusion Models [2505.07057]
- Towards Democratizing Multilingual Large Language Models For Medicine Through A Two-Stage Instruction Fine-tuning Approach [2409.05732]
- LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning [2412.12499]
- Two-stage LLM Fine-tuning with Less Specialization and More Generalization [2211.00635]
- Two-Stage Fine-Tuning: A Novel Strategy for Learning Class-Imbalanced Data [2207.10858]
- Model Fusion through Bayesian Optimization in Language Model Fine-Tuning [2411.06710]
- MMRAG-RFT: Two-stage Reinforcement Fine-tuning for Explainable Multi-modal Retrieval-augmented Generation [2512.17194]
- SDF-TopoNet: A Two-Stage Framework for Tubular Structure Segmentation via SDF Pre-training and Topology-Aware Fine-Tuning [2503.14523]
- Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers [2503.22672]
- Incremental Few-Shot Object Detection via Simple Fine-Tuning Approach [2302.09779]
- Metadata-Enhanced Speech Emotion Recognition: Augmented Residual Integration and Co-Attention in Two-Stage Fine-Tuning [2412.20707]
- Adapting Large Language Models to Low-Resource Tibetan: A Two-Stage Continual and Supervised Fine-Tuning Study [2512.03976]
- Stage-wise Fine-tuning for Graph-to-Text Generation [2105.08021]
- PanAdapter: Two-Stage Fine-Tuning with Spatial-Spectral Priors Injecting for Pansharpening [2409.06980]
- Fine Tuning a Data-Driven Estimator [2504.04480]
- Two-stage architectural fine-tuning with neural architecture search using early-stopping in image classification [2202.08604]

These primary sources provide implementation blueprints and empirical validation contexts for two-stage fine-tuning across modalities, tasks, and adaptation challenges.

Source: https://www.emergentmind.com/topics/two-stage-fine-tuning