SFT Depth Ladders: Multi-Axis Fine-Tuning Insights

Updated 5 July 2026

SFT Depth Ladders is a framework that interprets supervised fine-tuning across multiple axes—including training, architectural, capability, and stage depths—to reveal nuanced performance dynamics.
Empirical findings show that minimal high-quality SFT can unlock higher reasoning tiers while excessive SFT may reduce model plasticity and hinder subsequent RL adaptation.
Explicit constructions using task vectors, side networks, and layer-specific tuning strategies enable targeted improvements and expose critical trade-offs in model alignment and reasoning capabilities.

“SFT Depth Ladders” can be read as an umbrella description for several depth-structured views of supervised fine-tuning in recent LLM and VLM research. In this literature, “depth” is not confined to network depth: it also denotes progression along training time, dataset scale, capability tiers, and staged post-training pipelines. Across these works, SFT is repeatedly shown to be non-uniform across those axes: small amounts of high-quality SFT can unlock specific reasoning tiers, mid-network layers carry a disproportionate share of alignment-relevant change, excessive SFT can reduce entropy and plasticity before RL, and explicit composition methods can turn separately trained checkpoints into controllable capability ladders (Sun et al., 16 Apr 2025, Zhao et al., 12 Apr 2026, Liu et al., 7 Jun 2026, Yuan et al., 1 May 2026).

1. Depth as a multi-axis concept

Across the cited papers, “SFT Depth Ladders” is best understood as a family of related constructs rather than a single canonical formalism. One axis is training depth, where checkpoints differ only by SFT duration or SFT dataset size. A second is architectural depth, where analyses ask which transformer layers are stable, plastic, or alignment-critical. A third is capability depth, where reasoning benchmarks organize problems into progressively harder tiers that become accessible only after additional SFT. A fourth is stage depth, where SFT, RL, RLVR, critique, and test-time composition are assigned distinct roles in a larger post-training stack (Liu et al., 7 Jun 2026, Zhao et al., 12 Apr 2026, Sun et al., 16 Apr 2025, He et al., 3 Jun 2026).

Depth axis	Operationalization in the literature	Representative finding
Training depth	SFT epochs, checkpoints, or dataset size	Over-trained SFT can reduce RL plasticity (Liu et al., 7 Jun 2026)
Architectural depth	Early, middle, and late transformer layers	Middle layers $20\%-80\%$ are stable; final layers are sensitive (Zhao et al., 12 Apr 2026)
Capability depth	Tiered benchmark difficulty	Hard accuracy grows roughly logarithmically with SFT size and plateaus around $65\%$ on AIME24 (Sun et al., 16 Apr 2025)
Stage depth	SFT, RL, RLVR, critique, or test-time synthesis	Stage-specific data sets improve SLM reasoning (He et al., 3 Jun 2026)
Composition depth	Coefficients on task vectors or side networks	DoTS defines a 2D capability plane with SFT and RLVR coefficients (Yuan et al., 1 May 2026)

This synthesis suggests that the term is most useful when it preserves the distinction between these axes. Confusion often arises when “deeper SFT” is taken to mean both “more epochs” and “more reasoning ability,” although several papers show that those quantities need not be monotone.

2. Capability ladders induced by SFT data and training duration

One line of work makes the ladder metaphor literal by sorting reasoning problems into tiers. On AIME24, questions are partitioned into Easy, Medium, Hard, and Extremely Hard (Exh). The base Qwen2.5-32B-Instruct already solves the Easy tier to a substantial extent, but the Medium tier is largely unlocked by minimal R1-style SFT: approximately $500$– $1\text{K}$ long R1-style trajectories are sufficient for Medium accuracy to reach about $90\%$ . Hard-tier accuracy then grows only roughly logarithmically with SFT size and plateaus around $65\%$ , while Exh-level questions remain essentially unsolved by any SFT depth studied in the paper (Sun et al., 16 Apr 2025).

The same paper localizes the transition mechanism. The Easy→Medium jump is tied to adoption of an R1 reasoning style: extended chain-of-thought with self-reflection and explicit verification. Medium-tier performance is largely category-invariant once that style is learned, which suggests that the relevant ladder rung is structural rather than topical. By contrast, the Medium→Hard transition is dominated by step-wise instability and computational burden. In the paper’s subquestion analyses, overall success is well approximated by a product of per-step success probabilities, so long chains suffer multiplicative error accumulation. This is why larger SFT datasets help, but only with diminishing returns (Sun et al., 16 Apr 2025).

A second capability-oriented study defines a different quantity, the procedural- $\Delta$ : $\Delta = \mathrm{pass\_rate}(\mathrm{CU}) - \mathrm{pass\_rate}(\mathrm{BL}),$ where BL is a baseline solver prompt and CU injects a matching procedural skill block. The SFT-attributable lift is

$\Delta\text{-lift} = \Delta_{\text{post-SFT}} - \Delta_{\text{pre-SFT}}.$

Across Qwen3.5 dense 0.8B, 2B, and 4B, the procedural- $\Delta$ lift is roughly uniform: $65\%$ 0, $65\%$ 1, and $65\%$ 2, while the pre-SFT trajectory is W-shaped: $65\%$ 3, $65\%$ 4, and $65\%$ 5, with Claude Haiku 4.5 at $65\%$ 6 (Strozzi, 12 May 2026).

That W-shape matters for ladder interpretation. It implies that a similar absolute SFT contribution can play very different roles depending on the initial regime: at 0.8B it rescues a negative procedural regime to near-neutral, at 2B it boosts an already positive regime, and at 4B it flips a slightly negative regime to a positive one. This suggests that SFT depth should not be read off post-SFT scores alone. The same nominal “rung” can correspond to repair, incremental refinement, or recovery from procedural overhead, depending on where the base model begins (Strozzi, 12 May 2026).

3. Layer-wise depth ladders inside the model

A separate literature studies depth literally, at the level of transformer layers. A comprehensive layer-wise analysis across OLMo2 $65\%$ 7– $65\%$ 8 and Mistral-7B finds a consistent pattern: early layers remain close to the base model, middle layers $65\%$ 9 are stable and high-rank, and final layers are highly sensitive. Base–SFT CKA and cosine similarity remain high in early layers, while final layers show sharp representational divergence, large mean shift, a J-shaped rise in weight-change norms, and an information bottleneck marked by effective-rank collapse and spectral-norm growth (Zhao et al., 12 Apr 2026).

That paper further shows that instruction-following signals emerge abruptly near the top of the stack in probing experiments, but the most effective adaptation locus is not the extreme top. In layer-swapping experiments, replacing middle blocks can preserve or even improve downstream performance, whereas replacing early or late blocks is consistently more harmful. This motivates Mid-Block Efficient Tuning, which applies LoRA only to middle segments. Empirically, it outperforms standard LoRA by up to $500$0 on GSM8K for OLMo2-7B while reducing parameter overhead, supporting the claim that effective alignment is architecturally localized rather than uniformly distributed (Zhao et al., 12 Apr 2026).

A large-scale study of more than $500$1 SFT models reaches a closely related conclusion through a different methodology. There, absolute weight-delta magnitude rises toward upper layers, but the correlation between layer-wise change and downstream improvement peaks around normalized layer position $500$2. Mid-layer weight changes correlate most strongly with performance gains, model-to-model similarity of SFT-induced changes is also highest in the middle, and intrinsic dimensionality increases sharply after SFT beginning near the same depth (Harada et al., 17 Jun 2025).

In VLMs, the architectural ladder can be made explicit. On Qwen3-VL-2B, standard LoRA-only SFT lowers reasoning average from $500$3 to $500$4, a “reasoning tax.” Adding fixed cross-depth aggregation raises reasoning average to $500$5, and the input-adaptive, modality-aware IADA mechanism raises it to $500$6, with perception average $500$7 and all-benchmark average $500$8, using only $500$9 additional parameters. This suggests that depth ladders may be understood not only as where SFT changes the model, but also as explicit routes that preserve or restore access to intermediate representations after SFT (Ren et al., 27 Mar 2026).

4. SFT-to-RL handoff, plasticity loss, and rank inversion

One of the strongest uses of the ladder metaphor concerns the SFT→RL transition. In math and agentic settings, excessive SFT can reduce model plasticity, defined as the ability of an SFT-initialized policy to be effectively reshaped by subsequent RL. In EvoLM-4B, a moderate SFT checkpoint at 2 epochs (“ModSFT”) and an over-trained checkpoint at 32 epochs (“OverSFT”) are trained on the same data with the same optimizer and learning rate. OverSFT has near-zero training loss ( $1\text{K}$ 0), token entropy $1\text{K}$ 1, and only a marginal Pass@1 improvement over ModSFT, but RL gains shrink markedly: average performance rises from $1\text{K}$ 2 to $1\text{K}$ 3 for OverSFT, versus $1\text{K}$ 4 to $1\text{K}$ 5 for ModSFT (Liu et al., 7 Jun 2026).

The mechanistic picture is coherent across parameter space, output space, and RL dynamics. OverSFT shows larger, spikier drift from the base model, sharper parameter landscapes, near-deterministic token distributions, and smaller actual parameter updates during GRPO despite larger gradient norms. To counter this, the paper proposes Rejuvenation, combining base-anchored model fusion

$1\text{K}$ 6

with attribution-guided neuron reset. This restores entropy on over-confident tokens while preserving much of the SFT prior, and improves RL performance on both in-domain and out-of-distribution tasks (Liu et al., 7 Jun 2026).

A coding-focused study sharpens the failure mode further. For Qwen2.5-Coder-3B under binary-reward GRPO with group size $1\text{K}$ 7, deeper SFT checkpoints have higher pre-RL pass@1 but worse RL outcomes. Pre-RL pass@1 rises across SFT depths, but peak GRPO pass@10 falls from $1\text{K}$ 8 to $1\text{K}$ 9, and pre-RL entropy is positively associated with the GRPO outcome with $90\%$ 0. The central analytic quantity is the expected within-group advantage variance,

$90\%$ 1

where $90\%$ 2 is pass@1 at the rollout temperature. For $90\%$ 3, the paper gives $90\%$ 4; below this threshold, most groups have identical rewards and provide no group-relative signal (Aphale et al., 16 Jun 2026).

This produces rank inversion: the SFT checkpoint that looks best by pre-RL pass@1 can be the worst initializer for GRPO because SFT has compressed the rollout distribution. The paper’s two-stage diagnostic therefore replaces the usual “pick the best pass@1 checkpoint” heuristic with pre-RL entropy triage plus an early GRPO entropy monitor. It also reports that simple KL-to-reference regularization and label smoothing do not rescue the collapsed checkpoint in that setting, suggesting that the failure is not a trivial hyperparameter artifact (Aphale et al., 16 Jun 2026).

The same non-additivity appears in reasoning VLMs. Long-CoT SFT improves the hardest multimodal questions through structured reasoning but degrades easier ones through overthinking and verbosity; RL improves all difficulty levels more uniformly and preserves brevity. Yet two-staged, interleaved, progressive, mixed-data, and model-merging strategies fail to yield additive gains, producing trade-offs instead. This “synergy dilemma” shows that climbing the SFT ladder and then the RL ladder is not, in general, commutative (Chen et al., 10 Jul 2025).

5. Explicit ladder construction: task vectors, side nets, and stage-specific datasets

Several papers move from analysis to construction. In “Decouple before Integration: Test-time Synthesis of SFT and RLVR Task Vectors,” SFT and RLVR are represented as task vectors

$90\%$ 5

and a merged model is written as

$90\%$ 6

The paper identifies three structural obstacles to direct integration: a $90\%$ 7 magnitude disparity, $90\%$ 8 sign interference, and heterogeneous module-wise update distributions. DoTS addresses these by selective sparsification, norm-preserving rescaling, and Bayesian optimization of coefficients on a small unlabeled adaptation set. This turns SFT and RLVR into a 2D capability plane in which the SFT axis corresponds to breadth and the RLVR axis to reasoning depth (Yuan et al., 1 May 2026).

The same paper makes the ladder interpretation explicit: once sparse task vectors are available, one can trace paths $90\%$ 9 through coefficient space. Fixed coefficients already produce distinct capability profiles, and optimized DoTS reaches average $65\%$ 0 on Qwen2.5-Math-7B, surpassing SFT-only $65\%$ 1, RL-only $65\%$ 2, and naive dense merging $65\%$ 3, while using only $65\%$ 4 of the compute of LUFFY (Yuan et al., 1 May 2026).

Another explicit ladder is architectural rather than arithmetic. “Ladder Up, Memory Down” revisits Ladder Side Tuning, where a frozen $65\%$ 5-layer backbone is coupled to a small trainable side network through layer-to-layer cross-connections. The method matches QLoRA’s compute scaling slope while cutting peak memory by $65\%$ 6, enabling 7B-parameter fine-tuning on a single 12 GB consumer GPU with 2k-token contexts. Its depth-extended variant, xLadder, increases effective depth via cross-connections and shortens chain-of-thought at fixed parameter count, showing a direct depth-for-tokens trade-off (Zheng et al., 16 Dec 2025).

A third construction is stage-specific rather than parameter-space-based. In “Learning What to Learn,” the SFT→RL pipeline for small LLMs is organized into an Acquisition Set for SFT, a Consolidation Set for RL, and a Recycled Set for the next SFT stage. Difficulty is estimated by empirical correctness rate $65\%$ 7, medium and hard examples are separated, hard traces are transformed by a Bridge mechanism using per-step importance, jumpiness, and local difficulty, and all-zero-reward RL failures are converted into diagnostic, repair, and new-trace supervision through Critique Fine-Tuning. On Qwen2.5-0.5B and Llama3.2-1B, this stage-specific ladder consistently improves over representative SFT, distillation, and RL baselines across five reasoning benchmarks (He et al., 3 Jun 2026).

6. Evaluation artifacts, misconceptions, and open directions

Several recurring misconceptions are explicitly challenged. First, more SFT is not always better. On AIME24, more SFT data helps Hard-tier reasoning only logarithmically and does not solve Exh-tier questions (Sun et al., 16 Apr 2025). In SFT→RL handoffs, more SFT can reduce plasticity and harm RL (Liu et al., 7 Jun 2026, Aphale et al., 16 Jun 2026). In reasoning VLMs, more long CoT can improve difficult cases while degrading easier ones (Chen et al., 10 Jul 2025).

Second, higher superficial similarity between SFT data and test tasks is not the dominant predictor of SFT quality. A large controlled study finds that perplexity consistently predicts SFT effectiveness, often surpassing superficial similarity between training data and benchmark (Harada et al., 17 Jun 2025). This is consistent with the broader picture that SFT works best when it can reuse and reorganize existing abstractions rather than force disruptive changes.

Third, evaluation pipelines can themselves create fake ladders. In the procedural-skill study, $65\%$ 8 of the holdout originally used a deterministic ANSWER-line extractor that under-counted free-form conclusions. LLM-only re-judging revealed systematic bias against the curated procedural condition, overturned earlier “format-only at 0.8B” and “shrinking SFT at 4B” narratives, and was corroborated by GPT-5.4 re-judging with Cohen’s $65\%$ 9 and agreement $\Delta$ 0 across all seven configurations (Strozzi, 12 May 2026).

Open problems follow directly from these results. Several papers call for richer layer-wise or module-wise coefficient schemes rather than global mixing coefficients (Yuan et al., 1 May 2026), larger-capacity tests such as Qwen3.5-8B and 14B for the W-shaped procedural hypothesis (Strozzi, 12 May 2026), and adaptive schemes that infer stable versus sensitive layers before tuning (Zhao et al., 12 Apr 2026). More generally, the literature suggests that future SFT depth ladders will likely be conditional rather than global: depth chosen by difficulty, stage, layer band, or inference-time routing, rather than a single uniform increase in SFT duration or CoT length.

In that sense, SFT Depth Ladders name a broader shift in post-training methodology. SFT is no longer treated as a single homogeneous operation. It is increasingly analyzed as a structured progression across capability tiers, a localized intervention in middle or upper-middle layers, a potentially hazardous precursor to RL if pushed too far, and a component that can be recomposed or recycled through task vectors, side networks, or stage-specific datasets. The common thread is that useful SFT is depth-sensitive, and that both performance gains and failure modes become legible only when that depth structure is made explicit.