Reasoning-Guided Parameter-Efficient Fine-tuning

• Reasoning-guided parameter-efficient fine-tuning is defined as methods that partition data and trainable parameters based on reasoning demands rather than uniform updates.
• Techniques like LoRA-PAR and CRFT use selective activation and targeted representation intervention to optimize internal reasoning processes.
• These approaches boost reasoning benchmarks and data efficiency by dynamically allocating trainable resources and structuring supervision around reasoning requirements.

Searching arXiv for the cited works to ground the article in current literature. arXiv search: LoRA-PAR (Huang et al., 28 Jul 2025), SGFT (Bi et al., 2024), SIBO (Wen et al., 2024), CRFT (Huang et al., 14 Jul 2025), HEFT (Hill, 11 Sep 2025), LR-LoRA (Garg et al., 3 Jun 2026), SMoA (Liu et al., 12 Jan 2026), MoELoRA (Luo et al., 2024), review (Balne et al., 2024), PEFT vs FFT theory (Liu et al., 28 May 2025), plus application papers (Mansha, 6 Oct 2025) and (Guo et al., 24 May 2026). Reasoning-guided parameter-efficient fine-tuning denotes a family of adaptation strategies in which the PEFT budget is organized around reasoning structure rather than treated as a uniform update over a mixed training distribution. In the current literature, this idea appears most directly in methods that partition data and trainable subspaces by reasoning demand, or that intervene on internal states identified as causally important for chain-of-thought computation. It also appears more broadly in reasoning-guided supervision paradigms that use efficient backends without introducing a new adapter architecture. Around these direct formulations lies an adjacent literature on PEFT mechanisms that improve reasoning benchmarks through better capacity allocation, expert specialization, or robustness, together with review and theory papers that delimit where standard PEFT still falls short of full fine-tuning (Huang et al., 28 Jul 2025, Huang et al., 14 Jul 2025, Bi et al., 2024, Liu et al., 28 May 2025).

1. Conceptual scope and boundaries

The central distinction in this area is between reasoning-guided PEFT and reasoning-targeted PEFT. In the direct sense, a method is reasoning-guided when reasoning requirements determine how training data are split, how trainable parameters are allocated, or which internal computations are edited. LoRA-PAR is explicit on this point: its novelty is that adaptation is organized around reasoning requirements, with data and LoRA parameters partitioned by System 1 or System 2 demands, while the backbone remains frozen and LoRA is attached at Q/K/V/Gate/Up/Down positions (Huang et al., 28 Jul 2025). CRFT is similarly direct, because it identifies “critical representations” through information-flow analysis and edits only those hidden states while freezing the base model (Huang et al., 14 Jul 2025).

A second category consists of reasoning-guided fine-tuning paradigms that are not strict PEFT algorithms in the narrow adapter-design sense. SGFT belongs here. It argues that small LLMs can be trained to emit Solution Guidance, a compact decomposition of a problem into semantic and logical subgoals, and it implements this with LISA as an efficient backend rather than a new adapter parameterization (Bi et al., 2024).

A third category is adjacent rather than direct. SIBO, HEFT, LR-LoRA, SMoA, and MoELoRA all improve reasoning-task performance, but they do so through anti-over-smoothing residual injection, hierarchical composition of PEFT spaces, learnable effective rank, high-rank structured modulation, or contrastive expert specialization, not through explicit reasoning traces, rationale-conditioned routing, or reasoning-specific data partitioning (Wen et al., 2024, Hill, 11 Sep 2025, Garg et al., 3 Jun 2026, Liu et al., 12 Jan 2026, Luo et al., 2024). This distinction matters because the current literature repeatedly shows that strong reasoning results do not, by themselves, imply that a PEFT method is reasoning-guided in the methodological sense.

2. Reasoning as supervision and data organization

One major line of work uses reasoning structure to organize the training corpus itself. LoRA-PAR begins by classifying examples into

$$D_1 \quad \text{(System 1)}, \qquad D_2 \quad \text{(System 2)}.$$

The split is produced by a multi-model role-playing and voting procedure in which stronger teacher LLMs are prompted to act as the target model and judge whether a query is a fast, single-step problem or a multi-step reasoning problem. On GSM8K with LLaMA2 7B + LoRA + SFT, this classification choice materially affects downstream performance: role play + voting $(n=5)$ reaches 27.60, compared with 25.32 for QwQ without role play, 26.23 for QwQ with role play, 26.84 for DeepSeek-R1 with role play, and 25.85 for random partitioning (Huang et al., 28 Jul 2025). The direct implication is that reasoning-aware data partition is itself a trainable resource, not merely a preprocessing convenience.

SGFT uses a different form of reasoning guidance. Instead of supervising full Chain-of-Thought traces, it supervises Solution Guidance, a lighter artifact that captures problem understanding and decomposition while omitting concrete calculations. The SG pipeline uses GPT-4o as teacher, starts from 7 GSM8K training questions, and expands to datasets of 1,000, 2,000, and 3,000 SG examples. The actual SG training set is redefined as

$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$

because explicit “no calculations” prompting was found necessary to prevent drift into full derivations and final-answer generation (Bi et al., 2024). The paper’s strongest practical claim is data efficiency: SGFT uses 3,000 SG samples against 30,000 CoT samples for the baseline, and it further states that even 1,000 SG samples outperform 30,000 CoT examples. This suggests that the form of the reasoning signal may matter as much as its quantity.

Domain-specific reasoning PEFT extends the same logic to corpus construction. Geo-Expert does not fine-tune on generic geology text; it builds a reasoning-oriented instruction corpus from five canonical textbooks through chapter-aware recursive chunking, domain-tree-based question generation, and chain-of-thought answer construction, yielding 11,518 high-quality instruction pairs (Guo et al., 24 May 2026). Here, the reasoning guidance lies in the synthesis pipeline: the supervision signal is designed to encode geological deduction rather than surface terminology.

3. Reasoning-aware parameter selection and internal intervention

The most explicit parameter-space formulation is LoRA-PAR’s reasoning-guided partition of the trainable adapter subspace. After splitting data into $D_1$ and $D_2$, it scores each LoRA parameter $\phi_j$ with a second-order Taylor approximation of masked output loss:

$$I(\phi_j) = \left| g_j \,\phi_j - \tfrac{1}{2}\,\hat{F}_{jj}\,\phi_j^2 \right|,$$

with

$$g_j = \frac{\partial L}{\partial \phi_j}, \qquad \hat{F}_{jj} \approx \frac{1}{N} \sum_{k=1}^N \left( \frac{\partial L_k}{\partial \phi_j} \right)^2.$$

The retained top-importance sets $S_1$ and $S_2$ induce three logical regions,

$(n=5)$0

and training is sequential: Stage 1 performs SFT on $(n=5)$1, activating $(n=5)$2 and an $(n=5)$3 fraction of the shared subset; Stage 2 performs RL with GRPO on $(n=5)$4, activating $(n=5)$5 and a $(n=5)$6 fraction of the shared subset (Huang et al., 28 Jul 2025). The paper is explicit that this is training-time selective activation and freezing, not inference-time routing.

CRFT moves the reasoning-guided principle from parameter space to representation space. It defines critical representations as hidden states whose perturbation can change correctness, then identifies them through information-flow analysis. Self-referential filtering targets representations that retain important information:

$(n=5)$7

while multi-referential filtering targets representations that regulate many downstream states:

$(n=5)$8

The intervention itself is a low-rank edit applied only to selected states:

$(n=5)$9

In the main experiments, the rank is $G'=\{(\text{prompt}+q_i, d_i)\}^{N},$0, the default number of intervention representations per layer is 14, and the backbone is frozen throughout (Huang et al., 14 Jul 2025). Relative to native ReFT, the claim is not simply greater efficiency, but better alignment between intervention sites and the internal reasoning path.

HEFT is adjacent but instructive. It composes two PEFT spaces sequentially—first LoRA in weight space, then LoReFT in representation space—on BoolQ. Its contribution is a coarse-to-fine hierarchy rather than explicit reasoning supervision, but it demonstrates that heterogeneous PEFT composition can be staged rather than joint, and that weight-space alignment followed by representation-space refinement can improve reasoning-task efficiency (Hill, 11 Sep 2025).

4. Capacity allocation, expert specialization, and expressivity

A substantial adjacent literature argues that reasoning performance under PEFT is often limited by capacity allocation rather than by the absence of explicit reasoning traces. SIBO is the clearest example of a general-purpose PEFT booster whose gains are especially visible on reasoning benchmarks. It injects an initial residual,

$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$1

into PEFT module inputs to mitigate over-smoothing, adds no trainable parameters, and yields large reasoning-task gains. On GPT-J arithmetic, Adapter improves from 33.8 to 39.1 and LoRA from 37.5 to 42.6; on GPT-J commonsense reasoning, LoRA rises from 50.2 to 62.0 average (Wen et al., 2024). This is not reasoning-guided supervision, but it is strong evidence that preserving token discriminability matters disproportionately on reasoning-heavy tasks.

MoELoRA makes a related point through conditional specialization. It replaces a single LoRA branch with $G'=\{(\text{prompt}+q_i, d_i)\}^{N},$2 rank-$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$3 LoRA experts and Top-2 token-wise routing, then regularizes expert outputs with load balancing and contrastive loss. With the same 18.9M trainable parameters as LoRA, it reports an average math-reasoning gain of 4.2 points and an average commonsense gain of 1.0 point over LoRA (Luo et al., 2024). The guidance signal here is contrastive expert specialization rather than reasoning supervision, but the empirical pattern suggests that heterogeneous subskills are useful on reasoning datasets.

LR-LoRA and SMoA push the same issue into the structure of the update itself. LR-LoRA replaces a fixed-rank LoRA update with

$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$4

and uses stable rank only as a diagnostic, not as a loss term (Garg et al., 3 Jun 2026). Across the eight-task commonsense suite, it is best in all six backbone/regime combinations at $G'=\{(\text{prompt}+q_i, d_i)\}^{N},$5; for example, LLaMA3-8B on the 170k regime improves from 85.24 with LoRA to 88.22 with LR-LoRA (Garg et al., 3 Jun 2026). SMoA instead uses structured high-rank modulation over singular subspaces of pretrained weights and reaches 72.14% on GSM8K, compared with 65.89% for LoRA, 66.12% for DoRA, and 67.89% for MoRA (Liu et al., 12 Jan 2026). These results do not make the methods reasoning-guided in the strict sense, but they strongly suggest that reasoning tasks are unusually sensitive to how PEFT capacity is distributed across modules, subspaces, and experts.

5. Empirical landscape across benchmarks and domains

The most direct reasoning-guided PEFT evidence currently comes from LoRA-PAR and CRFT. On GSM8K, the proposed PiSSA$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$6 variant in LoRA-PAR reaches 41.85 after two epochs, compared with 33.59 for vanilla PiSSA and 37.45 for PiSSA+RL; proposed LoRA$G'=\{(\text{prompt}+q_i, d_i)\}^{N},$7 reaches 34.57, compared with 31.86 for vanilla LoRA and 32.83 for OLoRA (Huang et al., 28 Jul 2025). The same framework also improves MMLU and HumanEval, while using only about 40% of full LoRA parameters for each system when $G'=\{(\text{prompt}+q_i, d_i)\}^{N},$8 is around $G'=\{(\text{prompt}+q_i, d_i)\}^{N},$9–$D_1$0 in the QKVGUD configuration. Its ablations show that all three ingredients matter: role-play voting for data partition, importance-based parameter selection, and the sequential SFT$D_1$1RL schedule with shared parameters.

CRFT shows a different but complementary pattern. On GSM8K with LLaMA-2-7B, the base model scores 14.6, ReFT scores 29.0, and the best CRFT variant reaches 32.8; the trainable-parameter fractions are 0.031% for ReFT, 0.016% for CRFT, 0.103% for LoRA $D_1$2, and 0.826% for LoRA $D_1$3 (Huang et al., 14 Jul 2025). On LLaMA-3-8B, the best CRFT result reaches 71.0 on GSM8K, and on Mistral-7B it reaches 48.2. The perturbation studies are especially important: adding Gaussian noise to the top 5 SAF-selected critical representations degrades accuracy far faster than noising the last 5 representations, which supports the claim that CRFT is editing states with disproportionate causal relevance to final correctness.

SGFT occupies a different point in the landscape because it is modular and cross-model. Its strongest result comes from cross-model collaborative inference: Qwen2-7B_SG + ChatGLM3-6B achieves 48.3 on GSM8K, 57.8 on SVAMP, 72.9 on MultiArith, 79.8 on StrategyQA, and 75.7 on CommonsenseQA (Bi et al., 2024). The same-model comparisons are also large: for example, ChatGLM3-6B rises from 27.4 on GSM8K to 43.7 with ChatGLM3-6B_SG + ChatGLM3-6B. This is not evidence for a new PEFT adapter, but it is evidence that the right reasoning artifact can dominate raw trace quantity in small-model reasoning adaptation.

Domain applications further show that reasoning-oriented PEFT is not confined to generic math or commonsense benchmarks. Geo-Expert fine-tunes Qwen3-8B, Qwen3-32B, and Gemma-3-27B with LoRA on a reasoning-oriented geological instruction corpus and evaluates on Geo-Eval. The reported average Geo-Eval scores are 4.60 $D_1$4 6.27 for Qwen3-8B, 5.00 $D_1$5 6.82 for Qwen3-32B, and 5.16 $D_1$6 6.59 for Gemma-3-27B, with $D_1$7 on the 387-question benchmark (Guo et al., 24 May 2026). By contrast, the medical QLoRA case study is more cautious: it fine-tunes LLaMA-3.2-3B Instruct on medical chain-of-thought data under Kaggle-style 15–16 GB memory constraints and reports identical baseline and fine-tuned ROUGE-L $D_1$8, with qualitative increases in visible intermediate reasoning steps rather than a measured accuracy gain (Mansha, 6 Oct 2025). The contrast between these two application papers is instructive: reasoning-oriented PEFT can produce substantial domain-specific gains, but evaluation quality remains decisive.

6. Limits, misconceptions, and open problems

A recurrent misconception is that any PEFT method that improves reasoning benchmarks is therefore reasoning-guided. The current literature does not support that equivalence. SIBO, MoELoRA, LR-LoRA, SMoA, and HEFT improve reasoning-task outcomes, but their mechanisms are anti-over-smoothing, MoE-style specialization, adaptive effective rank, high-rank structured modulation, and heterogeneous PEFT composition, not explicit reasoning-process supervision or reasoning-aware routing (Wen et al., 2024, Luo et al., 2024, Garg et al., 3 Jun 2026, Liu et al., 12 Jan 2026, Hill, 11 Sep 2025). The stricter reasoning-guided category is still relatively small.

The direct methods also have clear limitations. LoRA-PAR acknowledges that multiple external teacher LLMs increase annotation overhead, that the System 1 / System 2 split is coarse-grained, and that experiments are limited to LLaMA2 7B; exact role-play prompts and exact RL reward/objective equations are omitted in the extracted text (Huang et al., 28 Jul 2025). SGFT depends on GPT-4o both to invent the SG schema and to generate and filter SG data, and it does not investigate multi-path reasoning or self-consistency over multiple SG candidates (Bi et al., 2024). CRFT relies on attention- and saliency-based proxies rather than a full causal decomposition of transformer computation, cannot easily target representations with negative rather than merely large impact, and requires access to hidden states and internals of open-weight models (Huang et al., 14 Jul 2025).

Theory and large-scale empirical comparison make these caveats sharper. The theoretical comparison between PEFT and full fine-tuning argues that PEFT is a strict subset of FFT, gives upper bounds on output movement under PEFT, and links PEFT to greater perturbation sensitivity; empirically, on LLaMA2-7B, FFT beats LoRA by 7.31% on GSM8K and also leads on MT-Bench averages (Liu et al., 28 May 2025). The instruction-tuning study reaches a related conclusion from a different angle: only LoRA and adapters get close to full fine-tuning under ideal settings, but both still lag on complex reasoning, coding, and long-form generation; on TÜLU evaluation, LoRA scores 29.1 on GSM against 37.0 for full fine-tuning, and both LoRA and adapters score 19.7 on Codex-Eval against 33.9 for full fine-tuning (He, 2024). The broader review literature points to the same heterogeneity: LoReFT is strongest on commonsense reasoning with 80.2% average on LLaMA-7B and 83.3% on LLaMA-13B using only 0.031% and 0.025% trainable parameters, but it is weaker on arithmetic reasoning than LoRA, at 42.6 and 49.6 versus 46.9 and 51.1 (Balne et al., 2024).

Taken together, the literature suggests that the field has moved beyond a single question—whether PEFT can support reasoning at all—and toward a more structured one: which parts of the reasoning process should determine data selection, capacity allocation, intervention location, and optimization regime. The strongest current evidence favors methods that make reasoning demand an organizing principle for at least one of those decisions. The main unresolved issue is generality: most existing results are proof-of-concept demonstrations tied to specific backbones, benchmarks, or supervision pipelines, rather than standardized, architecture-agnostic frameworks.