Reasoning Instruction Finetuning

• Reasoning Instruction Finetuning (RIF) is a process that refines large reasoning models to adhere strictly to detailed, multi-step instructions.
• It employs methods like multi-turn feedback and synthetic data supervised finetuning to significantly improve the Instruction Following Score (IFS).
• RIF enhances model transparency and reliability in safety-critical tasks while highlighting a trade-off between reasoning fidelity and overall accuracy.

Reasoning Instruction Finetuning (RIF) refers to the specialized fine-tuning procedures for large reasoning models (LRMs) so that their entire reasoning process—not just the final answer—faithfully adheres to explicit user instructions. This includes internal alignment with constraints such as formatting, language, length, and style throughout multi-step reasoning traces. RIF plays a crucial role by improving model transparency, controllability, and safety: ensuring every step in reasoning conforms to user-provided requirements, thereby reducing risks of shortcut solutions, hallucination, or reward hacking in intermediate steps (Kwon et al., 17 Oct 2025).

1. Rationale and Importance

Instruction adherence in LRMs has usually been assessed only by inspecting the main output, ignoring whether intermediate reasoning steps follow user specifications. The fidelity of reasoning traces is especially important in safety-critical applications (e.g., legal analysis, scientific computation), where an opaque or misaligned reasoning process undermines trust. Empirical evidence shows that even state-of-the-art open-source reasoning models such as GPT-OSS, Qwen3, and DeepSeek-R1 exhibit substantial failures in reasoning instruction adherence: the highest reasoning Instruction Following Score (IFS) remains below 0.25 (i.e., fewer than 25% of reasoning traces comply with instructions), despite high correctness in final answers. Furthermore, as task difficulty increases, adherence tends to degrade, and a strong positive correlation is observed between accuracy and reasoning IFS values.

2. Benchmarking and Metrics

Systematic evaluation of RIF uses benchmarks such as ReasonIF, which include diverse instruction prompt categories—multilingual settings, formatting requirements, length and output control—and present instructions together with reasoning prompts. The quantitative metric for reasoning instruction adherence is the Instruction Following Score (IFS), defined as:

$$IFS = \frac{1}{n} \sum_{i=1}^n g_\text{inst-checker}\left(x_i^{(\text{inst})}, \hat{y}_i\right)$$

where $g_\text{inst-checker}$ outputs 1 if the reasoning trace $\hat{y}_i$ for instance $i$ follows the instruction $x_i^{(\text{inst})}$ and 0 otherwise.

Experiments show that IFS for reasoning traces is much lower than for main answers, indicating the models’ intermediate reasoning often fails to comply even when final answers are correctly produced.

3. Methodologies for RIF

Current RIF research explores multiple methodologies to close the reasoning instruction adherence gap:

1. Multi-turn Reasoning: The model completes an initial reasoning trace, receives explicit feedback about instruction violations, and is prompted to revise its reasoning until requirements are met. This iterated feedback loop has been shown to boost IFS by an average of 16.6% across several leading open-source LRMs.
2. Synthetic Data Supervised Finetuning: Synthetic prompt–reasoning–answer triples generated either via rule-based transformation or LLM calls are used to train models on high-fidelity reasoning traces that scrupulously follow instructions. In practice, supervised finetuning of GPT-OSS-20B with synthetic reasoning data improved its reasoning IFS from 0.11 to 0.27—highlighting the approach’s efficacy. Further increases are possible (e.g., reaching 0.44 IFS), though possibly at the cost of a slight accuracy decrease, indicative of minor overfitting.

4. Characteristics and Limitations

RIF-trained models are more likely to produce reasoning traces that conform to multi-constraint instructions, even for previously challenging requirements such as “uppercase only” or strict “JSON formatting.” Nonetheless, several limitations are observed:

• Substantial room remains for improving adherence, with best IFS scores still below 30%.
• As the complexity of both task and instructions increases, reasoning instruction fidelity diminishes markedly.
• Overfitting risks exist: excessive exposure to synthetic data or repeated multi-turn feedback can privileged instruction adherence at the expense of raw reasoning accuracy.

Tables presented in the paper illustrate benchmark distributions and model performance before and after RIF, clearly indicating a trade-off between fidelity and accuracy for higher levels of intervention.

5. Practical Implications and Future Research

RIF is critical for achieving controllable and transparent LRMs, especially in real-world deployments where end-to-end reliability and comprehensible reasoning are prerequisites. Synthetic data finetuning offers a cost-effective method for teaching stepwise instruction adherence, while multi-turn feedback presents a pathway for iterative refinement, potentially enabling deeper self-reflection in agentic reasoning models.

Future directions suggest expanding RIF to multi-instruction scenarios and agentic systems; developing robust finetuning methods that avoid overfitting; and devising competence-aware RIF strategies that balance improved reasoning fidelity with maintenance of task accuracy. The quantification and enhancement of stepwise instruction adherence are expected to be central in the next phase of safe, controllable reasoning AI.

6. Quantitative Assessment and Reporting

The ReasonIF paper uses explicit table-based and metric reporting to validate improvements. A representative table compares pre- and post-RIF IFS and accuracy for GPT-OSS-20B:

Model	Reasoning IFS Before	Reasoning IFS After	Accuracy Before	Accuracy After
GPT-OSS-20B	0.11	0.27	...	...

Such empirical evidence affirms the measurable gains provided by RIF, while highlighting remaining gaps.

7. Open Questions and Outlook

Open research questions concern the best strategies for balancing instruction fidelity and overall accuracy, especially in multi-instruction or highly complex settings. There is ongoing investigation into whether more sophisticated multi-turn feedback and data generation approaches can increase adherence without penalty to performance. Integrating reasoning instruction fidelity into agentic and interactive model evaluation is expected to be central to future developments.

Reasoning Instruction Finetuning thus establishes a crucial mechanism for endowing LRMs with stepwise controllability, transparency, and adherence to user requirements during complex multi-step tasks, directly addressing several of the most pressing challenges for reliable AI in practice (Kwon et al., 17 Oct 2025).