Critical Representation Fine-Tuning

• CRFT is a parameter-efficient methodology that fine-tunes a select subset of hidden-state vectors to enhance reasoning in large language models.
• It employs information-flow analysis using attention and saliency metrics to identify and modify the most influential neuronal positions.
• Empirical results show that CRFT improves arithmetic and commonsense benchmarks with far fewer trainable parameters compared to methods like LoRA.

Critical Representation Fine-Tuning (CRFT) is a parameter-efficient fine-tuning methodology for LLMs that focuses on low-rank subspace optimization of strategically selected “critical” hidden-state vectors, rather than updating model weights or prompt parameters. In contrast to standard methods that edit fixed representation positions with uncertain output influence, CRFT identifies and modifies only the most influential hidden-state vectors (“critical representations”) via information-flow analysis. This targeted optimization enables substantial improvements in complex chain-of-thought (CoT) reasoning tasks while using orders of magnitude fewer trainable parameters, as validated across arithmetic and commonsense reasoning benchmarks on LLaMA and Mistral model families (Huang et al., 14 Jul 2025).

1. Definition and Motivation

CRFT departs from conventional Parameter-Efficient Fine-Tuning (PEFT) approaches by pinpointing a minimal subset of hidden representations—specific neuron vectors within transformer layers—that exert maximal impact on reasoning outputs. Formally, for representation vectors $h^{(l)}_i \in \mathbb R^d$ at layer $l$, CRFT selects a mask $\mathcal M \subset \lbrace (l, i)\rbrace$, and only these are edited via a learned low-rank subspace projection. This selective focus contrasts with Representation Fine-tuning (ReFT), which edits fixed slots (typically the first or last $k$ vectors of each layer), regardless of their downstream importance, often leading to suboptimal or even degraded performance in multi-step reasoning tasks.

Critical representations are defined as hidden vectors that either (a) aggregate significant information from previous layers or (b) distribute information extensively to subsequent positions. Empirically, minor perturbations at such positions frequently result in drastic changes to model outputs, demonstrating their centrality to reasoning outcomes (Huang et al., 14 Jul 2025).

2. Identification of Critical Representations

CRFT employs information-flow analysis leveraging both attention and saliency metrics to score and select critical representations. Let $\mathrm{Info}^{(l)}(i, j)$ denote the influence from position $j$ in layer $l$ to position $i$ in layer $l+1$. Two complementary filtering criteria are utilized:

2.1 Self-Referential Filtering (Retention): A vector $h^{(l)}_i$ is flagged as critical if it evidences substantial self-attention in the next layer, i.e., if the attention score

$l$0

where $l$1 is a threshold. Saliency-weighted variants—using $l$2—further prioritize representations most influential for loss minimization.

2.2 Multi-Referential Filtering (Distribution): A vector is classified as critical if it disperses substantial attention to many downstream positions, quantified by the average attention weight

$l$3

where $l$4 is a distribution threshold. Saliency-weighted analogues are also employed.

2.3. Selection Protocol: Multiple strategies (Self-Referential, Multi-Referential, and their saliency-weighted counterparts) are combined, retaining the union of top-$l$5 candidates per layer based on these scores. The procedure ensures that the set $l$6 captures positions of demonstrably high causal impact on reasoning chains.

3. Fine-Tuning Protocol

Critical Representation Fine-Tuning freezes the entire base model ($l$7), learning only a compact set of low-rank updates for the representations in $l$8. Each critical vector $l$9 receives an update $\mathcal M \subset \lbrace (l, i)\rbrace$0 constrained to a rank-$\mathcal M \subset \lbrace (l, i)\rbrace$1 subspace via

$\mathcal M \subset \lbrace (l, i)\rbrace$2

with $\mathcal M \subset \lbrace (l, i)\rbrace$3, $\mathcal M \subset \lbrace (l, i)\rbrace$4, $\mathcal M \subset \lbrace (l, i)\rbrace$5. Thus,

$\mathcal M \subset \lbrace (l, i)\rbrace$6

Model training minimizes the supervised cross-entropy loss $\mathcal M \subset \lbrace (l, i)\rbrace$7 on chain-of-thought exemplars, with an optional nuclear-norm regularization on updates:

$\mathcal M \subset \lbrace (l, i)\rbrace$8

where $\mathcal M \subset \lbrace (l, i)\rbrace$9 collects all critical $k$0, and $k$1 is usually set to zero in practice.

Default hyperparameters include: AdamW optimizer, learning rate $k$2, rank $k$3, and intervention at $k$4 vectors per layer. Only $k$5 (and any additional offsets for Q/K/V projections) are updated.

4. Empirical Results and Benchmarks

Experiments on LLaMA-2-7B, LLaMA-2-13B, LLaMA-3-8B, and Mistral-7B evaluated CRFT on eight datasets: GSM8K, AQuA, MAWPS, SVAMP (arithmetic) and BoolQ, SocialIQA, WinoGrande, OpenBookQA (commonsense). The following table summarizes key accuracy results on GSM8K with LLaMA-2-7B:

Method	#Trainable %	Accuracy
None	–	14.6%
LoRA ($k$6)	0.83%	38.5%
ReFT ($k$7)	0.03%	29.0%
CRFT	0.016%	32.8%

CRFT yields an 18.2 percentage point improvement over the unfine-tuned model and a 3.8 pp gain over ReFT—while employing less than one-tenth the trainable parameter budget of LoRA and half that of ReFT.

In low-resource scenarios, CRFT demonstrates marked advantage. For GSM8K one-shot training, CRFT achieves 32.6% accuracy versus the 16.2% baseline (Δ = +16.4 pp). Across all arithmetic and commonsense benchmarks, accuracy gains range from 2–10 percentage points (Huang et al., 14 Jul 2025).

5. Ablation Studies and Analysis

Ablation experiments illuminate CRFT’s sensitivity to subspace rank ($k$8), number of critical slots per layer ($k$9), and intervention layer depth:

• Subspace Rank: $\mathrm{Info}^{(l)}(i, j)$0 is typically sufficient; $\mathrm{Info}^{(l)}(i, j)$1 yields a ∼2 pp drop, while $\mathrm{Info}^{(l)}(i, j)$2 improves <1 pp.
• Vectors per Layer ($\mathrm{Info}^{(l)}(i, j)$3): Table 2 shows performance peaks at $\mathrm{Info}^{(l)}(i, j)$4 (30.3%), with diminished returns for larger $\mathrm{Info}^{(l)}(i, j)$5.
• Layer Selection: Editing the lower half of layers (0–15) achieves nearly the full accuracy benefit, reflecting the widespread propagation of early layer features.
• Criticality Validity: Injecting Gaussian noise ($\mathrm{Info}^{(l)}(i, j)$6) into the top-5 scored critical slots produces a 14 pp performance drop, versus just 2 pp for bottom-5, empirically confirming the critical selection protocol’s effectiveness.

6. Implementation Guidelines and Computational Considerations

Recommended practices include:

• Subspace dimension $\mathrm{Info}^{(l)}(i, j)$7 to $\mathrm{Info}^{(l)}(i, j)$8 optimizes trade-off between trainable variable count and performance.
• Information-flow analysis should use attention maps and loss gradients to obtain scoring metrics (attention saliency); thresholds $\mathrm{Info}^{(l)}(i, j)$9 select around 14 vectors per layer, adjustable based on resource constraints.
• Inference overhead is negligible, requiring only a low-rank adjustment per selected critical slot.
• Example computational budget: full GSM8K training on LLaMA-2-7B completes in approximately 4 hours on one A100 GPU (Huang et al., 14 Jul 2025).

7. Practical Implications and Extensions

CRFT establishes a viable path for efficient, scalable advanced reasoning specialization with minimal resource footprint. Its targeted focus on high-leverage representations, validated through information flow and saliency, ensures superior gains per parameter over weight- or prompt-based PEFT approaches. A plausible implication is that subsequent model specialization, interpretability work, and modular adaptation for new reasoning domains may benefit from refinement of the CRFT pipeline, particularly with enhanced or domain-specific criteria for identifying critical representations. The method’s explicit representation-level intervention also suggests new avenues for disentangling model behavior and for hybridizing with other modular PEFT strategies (Huang et al., 14 Jul 2025).