---
title: Phased Instruction Fine-Tuning (Phased IFT)
url: https://www.emergentmind.com/topics/phased-instruction-fine-tuning-phased-ift
type: topic
---

# Phased Instruction Fine-Tuning (Phased IFT)

Phased Instruction Fine-Tuning (Phased IFT) is a training paradigm for improving instruction-following ability and domain adaptation in large language models (LLMs). It decomposes fine-tuning into ordered stages based on instructional difficulty or task specificity. Unlike one-off supervised instruction fine-tuning, Phased IFT either (1) partitions general-purpose instruction–response corpora by difficulty and fine-tunes sequentially from easy to hard [2406.04371], or (2) adapts a model to a specialized domain (e.g., medicine) by first injecting domain-general knowledge and then specializing for a target sub-task [2409.05732]. This phased approach reduces interference across heterogeneous instructions, enhances progressive alignment, and yields improved empirical results compared to single-stage fine-tuning.

## 1. Motivation and Theoretical Foundations

Phased IFT is motivated by observed deficiencies in conventionally fine-tuned LLMs: task interference, slow convergence on complex instructions, and suboptimal alignment when training on heterogeneous task mixtures [2406.04371]. The Progressive Alignment Hypothesis posits that aligning next-token generations with complex human intent requires a curriculum—beginning with easy tasks and gradually increasing difficulty. In medical domain adaptation, direct fine-tuning on specialized tasks without broad domain knowledge results in limited reasoning and terminology competence; continual pre-training is often intractable due to resource demands [2409.05732]. Phased IFT addresses these by controlling the distributional shift across fine-tuning phases, reducing negative transfer and facilitating stepwise knowledge accumulation.

## 2. Formalism and Core Algorithm

For general instruction adherence, Phased IFT proceeds as follows [2406.04371]:

- Let $D = \{(x_i, y_i)\}_{i=1}^N$ be the instruction–response dataset.
- Compute difficulty scores $s(x_i, y_i)$ for each sample via a proxy (e.g. GPT-4, scale 1–5).
- Partition $D$ into $K$ subsets $\{D_k\}_{k=1}^K$ with non-overlapping difficulty thresholds: 
  $D_k = \left\{(x_i, y_i) : \tau_{k-1} \leq s(x_i, y_i) < \tau_k\right\}$.
- Sequentially fine-tune on each $D_k$ for epochs $E_k$, updating parameters via
  $$\mathcal{L}_k(\theta) = \sum_{(x, y) \in D_k} -\log p_{\theta}(y|x)$$

For domain adaptation (e.g. medicine), Phased IFT comprises two stages [2409.05732]:

1. **General Knowledge Injection**: Train on a broad, multilingual QA corpus, maximizing the log-likelihood of correct responses in instruction format.
2. **Task-Specific Specialization**: Fine-tune on target-task data (e.g., medical multiple-choice), aligning output to specific question formats and concise rationales.

Adapters (LoRA/DoRA/QLoRA) are used for efficient parameter injection, merging weights across phases.

## 3. Dataset Construction and Phase Partitioning

### Difficulty-based Phasing (General Instruction)
- Datasets: Alpaca 52K and Alpaca-cleaned 52K [2406.04371].
- Difficulty scores via GPT-4 API, batch-evaluated.
- Example phase thresholds: $K=3$, $\tau_1=1.5$, $\tau_2=3.5$.
- Phase sizes: $|D_1|\approx30.3$K, $|D_2|\approx14.7$K, $|D_3|\approx6.8$K.

### Domain-based Phasing (Medical Adaptation)
- Stage-1 Data (MMed-IFT): $\sim$300K QA samples spanning English, Chinese, Japanese, Korean, French, Spanish.
- Composition: English ($\sim$135K), Chinese ($\sim$80K), other languages ($\sim$20K each).
- Formats: short-answer QA; MCQ with rationale.
- Stage-2 Data (MMed-IFT-MC): $\sim$50K MCQs derived from MMedBench, KorMedMCQA, formatted to emphasize rationale and answer index.
- Train/test splits conform to original benchmarks [2409.05732].

| Phase or Dataset  | Scale   | Content                      | Purpose                    |
|-------------------|---------|------------------------------|----------------------------|
| D₁ / MMed-IFT     | 30–300K | Easy/General QA              | Bootstrap instruction/domain knowledge |
| D₂ / MMed-IFT-MC  | 15–50K  | Medium/Task-specific MCQ     | Specialize reasoning and output format |

## 4. Training Objectives, Losses, and Hyperparameters

Phased IFT employs a standard causal-LM objective within each phase. For phase $k$:
- Input: $x_k$ (instruction plus optional input)
- Target: $y_k$ (expected output, e.g., answer, rationale)
- Loss: $\mathcal{L}_k(\theta) = -\sum_{(x, y) \in D_k} \log p_\theta(y|x)$

Parameter-efficient fine-tuning methods (LoRA/DoRA/QLoRA adapters) enable compute-light updates:
- Stage 1: LoRA rank=32, $\alpha$=16, batch size=1, gradient accumulation=4, 2 epochs, LR=$5 \times 10^{-5}$.
- Stage 2: QLoRA rank=16, $\alpha$=8, batch size as above, 2 epochs, LR=$2 \times 10^{-5}$.
- Hardware: single RTX 4090 (25 GB) achieves full training [2409.05732].
- No phase mixing or curriculum beyond ordered phasing; dropout and cosine decay LR schedules applied [2406.04371].

High-level pseudocode for the two-stage medical Phased IFT:
```python
# Stage 1: General Knowledge Injection
M1 = ApplyLoRA(M0, config=stage1)
for epoch in 1..2:
    for batch x, y in D1:
        loss = -log p(y|x; M1)
        backprop(loss, only phi1's weights)
merge phi1 into base -> M_merg

# Stage 2: Task-Specific MC Fine-Tuning
M2 = ApplyLoRA(M_merg, config=stage2)
for epoch in 1..2:
    for batch x, y in D2:
        loss = -log p(y|x; M2)
        backprop(loss, only phi2's weights)
Return M_final = M_merg ⊕ phi2
```

## 5. Empirical Outcomes and Ablation Findings

### General Instruction Adherence
Phased IFT yields significant “win-rate” improvements over one-off fine-tuning across models and datasets [2406.04371]:

| Base Model    | Avg. Win-Rate (Alpaca) | Avg. Win-Rate (Alpaca-cleaned) |
|---------------|-----------------------|-----------------------------|
| Llama-2 7B    | +7.26%                | +3.53%                      |
| Mistral-7B    | +6.30%                | —                           |
| Llama-2 13B   | +7.35%                | +6.49%                      |
| Llama-2 70B   | +2.82%                | +7.59%                      |
| Llama-3 8B    | +7.64%                | +3.97%                      |
| Llama-3 70B   | +5.23%                | —                           |

Gradual progression from easy to hard phases yields monotonically increasing win-rates; random segmentation produces negative gains (e.g., Llama-2 7B: −0.42%), establishing the necessity of ordered difficulty [2406.04371]. Permuting phase order confirms optimal performance only when the easiest phase precedes harder phases.

### Domain Adaptation (Medicine)
Substantial accuracy improvements are realized versus single-stage fine-tuning [2409.05732]:

| Method              | USMLE-1 | USMLE-2 | USMLE-3 | MedQA-4 | MedMCQA |
|---------------------|---------|---------|---------|---------|---------|
| Llama3-8B (base)    | 46.8    | 43.1    | 44.3    | 53.7    | 49.3    |
| Stage-2 only        | 55.3    | 52.3    | 57.4    | 59.7    | 53.6    |
| Two-Stage IFT       | 56.4    | 52.3    | 67.2    | 62.4    | 57.0    |

Cross-lingual knowledge transfer is demonstrated; phase 1 English injection yields +12–23 pt boosts on non-English medical benchmarks. The two-stage approach achieves performance on par with or slightly below proprietary, closed-source continual-pretrain+IFT methods.

## 6. Strengths, Limitations, and Future Directions

**Strengths:**
- General and scalable: applies to both instruction adherence (via difficulty partitioning) and domain adaptation.
- Effective for open-source models and datasets.
- Computationally efficient, especially with parameter-efficient fine-tuning.

**Limitations:**
- Manual selection of phase boundaries ($\tau_k$) introduces human bias.
- Dependency on GPT-4 for difficulty scoring incurs API costs.
- Fixed phase granularity ($K$ typically set to 3).
- No automatic phase partitioning or multi-objective balancing (e.g., difficulty + diversity).

**Future Directions:**
- *Automatic thresholding*: Develop cluster or density-based segmentation for phase boundaries.
- *Cheaper proxies*: Explore fine-tuned difficulty predictors to replace GPT-4.
- *Adaptive phasing*: Dynamically select number and scope of phases.
- *Multi-axis curricula*: Integrate additional measures (e.g., domain coverage) with instructional difficulty.

A plausible implication is that Phased IFT provides a versatile blueprint for adapting LLMs to domains under resource constraints, yielding empirical and theoretical advantages over single-phase alternatives. The methodology is robust, empirically validated, and extensible to new domains and models.

Source: https://www.emergentmind.com/topics/phased-instruction-fine-tuning-phased-ift