ThinkLogit: Inference-Time Reasoning Transfer

• ThinkLogit is a decoding-time method that adds a token-level logit guidance signal from a small reasoning model to a large frozen model, promoting long chain-of-thought behavior.
• It employs a warm-up schedule to stabilize the guidance, ensuring that the target model steers its generation trajectory towards self-corrective and reflective outputs.
• The enhanced variant, ThinkLogit-DPO, aligns the small reasoning guider via direct preference optimization, achieving significant accuracy improvements on math reasoning benchmarks.

Searching arXiv for the ThinkLogit papers and closely related decoding-time reasoning-transfer work. ThinkLogit is a decoding-time method for eliciting long chain-of-thought reasoning in a frozen LLM by adding a logit-space guidance signal derived from a much smaller reasoning model. In its canonical formulation, a large target model $L$ is steered by the difference between a small reasoning guider $S$ and its non-reasoning base $S_0$, so that the target inherits a token-level bias toward long reasoning without any weight updates to the target itself. A strengthened variant, ThinkLogit-DPO, first aligns the small guider with Direct Preference Optimization and then applies the same logit arithmetic during decoding (Zhang et al., 10 Oct 2025).

1. Problem formulation and conceptual basis

ThinkLogit addresses a specific tension in contemporary reasoning systems. Large reasoning models can produce long chain-of-thought traces with planning, backtracking, reflection, and self-correction, but such behavior is commonly associated with costly post-training such as reinforcement learning or large-scale distillation. ThinkLogit targets the regime in which a large model is already competent on the task but does not reliably exhibit long-CoT behavior, while a much smaller model has already acquired that behavior through SFT, RFT, or related post-training.

The central idea is to transfer not a completed reasoning trace, nor a prompt demonstration, but a distributional correction in logit space. The paper interprets the difference between a small reasoning model and its base model as a token-level “reasoning increment.” Adding that increment to the large target’s logits changes the generation trajectory itself, step by step, rather than reranking finished outputs or sampling many completions and voting among them.

This framing also clarifies what ThinkLogit is not. It is not standard chain-of-thought prompting, because prompting does not alter the model’s next-token distribution. It is not self-consistency, because it does not sample many full reasoning paths and aggregate answers afterward. It is not large-model distillation, because the large target remains frozen. And it is not merely a verbosity intervention: the paper explicitly argues that long reasoning is not equivalent to generating more tokens, and that decoding must be altered in a way that specifically favors long-CoT behavior (Zhang et al., 10 Oct 2025).

2. Logit arithmetic and decoding dynamics

At decoding step $t+1$, for a prefix $\mathbf z_{1:t}=z_1,\dots,z_t$, a model $f$ produces next-token pre-softmax logits

$$\mathbf{\ell}_{t+1}^{(f)} = f(\mathbf z_{1:t}) \in \mathbb R^{|\mathcal V|}.$$

ThinkLogit assumes three models: the frozen large target $L$, the small base model $S_0$, and the small reasoning guider $S$. The fused logits are

$S$0

The term $S$1 is the mechanism’s core object. In the paper’s interpretation, it is the shift that turns a plain small model into a long-reasoning one. The target contributes knowledge and broad competence, while the small-model difference contributes a reasoning-style direction in logit space. The decoding distribution is therefore changed online, token by token.

A practical complication is that applying the guidance from the first generated token caused repetition and instability. ThinkLogit therefore uses a warm-up schedule: $S$2 The reported default is $S$3 and $S$4. In the July 2025 version, the practical decoding setup also uses temperature $S$5, top-$S$6 $S$7, and max length $S$8 (Zhang et al., 17 Jul 2025).

This warm-up is not an incidental detail. The paper treats it as a stabilization mechanism: too little warm-up, especially $S$9, yields repetitive low-quality generations, while too much warm-up causes the output to revert toward the shorter reasoning style of the target.

3. ThinkLogit-DPO and guider alignment

ThinkLogit-DPO preserves the same decoding rule but replaces the raw guider with a preference-optimized one. The motivation is distribution mismatch. A small model may possess long-CoT behavior yet still disagree with the target’s strengths or introduce characteristic errors. The paper therefore aligns the guider with preference pairs mined jointly from the large target and the small guider.

Two preference-pair types are constructed. Type-1 pairs take a large-model correct short CoT and a small-model incorrect long CoT, preferring the former. Type-2 pairs take a small-model correct long CoT and a large-model incorrect short CoT, preferring the former. The first type preserves the target’s correctness; the second teaches the guider to repair the target’s reasoning failures.

For a preference pair $S_0$0, the DPO term is

$S_0$1

with implicit reward

$S_0$2

The mixed training objective is

$S_0$3

with default

$S_0$4

The July 2025 version reports DPO training with 10K preference pairs sampled from 55,183 total pairs, batch size 32, 1 epoch, learning rate $S_0$5, AdamW, cosine schedule with 0.1 warmup, $S_0$6, cutoff length 8192, and about 4 hours per run. In ablations, mixed dual-source preferences outperform guider-only self-pairs, and DPO outperforms SFT-based guider alignment on the same correct completions (Zhang et al., 17 Jul 2025).

4. Empirical results, evaluation settings, and cross-model transfer

The empirical record is reported in two closely related versions of the work. In the July 2025 preprint, the main evaluation covers four mathematical datasets—AIME2024, AIME2025, AMC23, and MATH-hard—and reports pass@1 and pass@8. With Qwen2.5-32B as target and R1-Distill-Qwen-1.5B as guider, baseline average pass@1 is 32.7, ThinkLogit reaches 41.2, and ThinkLogit-DPO reaches 42.2, corresponding to relative improvements of 26% and 29% over the target baseline. In the same setup, ThinkLogit-DPO reaches pass@8 65.6, which is 1.9 points below s1.1-32B’s 67.5 (Zhang et al., 17 Jul 2025).

The October 2025 version broadens evaluation to five reasoning benchmarks—AIME 2024, AIME 2025, AMC 23, MATH Level 5 subset, and GPQA Diamond—and reports Avg@8, computed from 8 completions per problem at temperature 0.6 and max length 8192. In that setting, the target Qwen2.5-32B averages 32.3, ThinkLogit 40.2, and ThinkLogit-DPO 41.7, corresponding to relative improvements of 24.5% and 29.1%. The paper emphasizes that the guider is 21× smaller than the target (Zhang et al., 10 Oct 2025).

Several further experimental patterns are central to the method’s characterization.

First, ThinkLogit remains effective when guider and target come from different model families. Because logit arithmetic requires aligned vocabularies, the authors use a one-time minimum edit distance mapping between tokenizers. With that mapping, a Qwen-based guider can steer Llama-3.3-70B-Instruct, and the paper reports improvements on AMC23 together with visibly longer and more self-corrective traces.

Second, the method is presented as orthogonal to post-training. Guiders improved through supervised distillation or reinforcement learning can be plugged in directly. The July 2025 version reports that using One-Shot-RLVR-1.5B as guider raises the Qwen2.5-32B average pass@1 from 32.7 to 36.8, a 13% relative improvement. The October 2025 version also discusses RL-trained guiders such as DeepScaleR-1.5B-Preview, with the same general interpretation: the large frozen model can inherit a small model’s RL-acquired reasoning style through logit fusion (Zhang et al., 10 Oct 2025).

Third, the paper claims cross-scale reuse of the learned guidance signal: preference signals learned with a 32B Qwen target transfer to Qwen2.5-72B, and the DPO-aligned guider continues to outperform vanilla ThinkLogit on the larger target.

5. Ablations, failure modes, and computational trade-offs

The ablation studies are used to distinguish ThinkLogit from simpler decoding heuristics. A central negative result is that budget forcing—for example, replacing EOS with “Wait”—produces much longer outputs but lower accuracy. The paper treats this as evidence that the gain comes from the quality of the guidance signal rather than output length alone. A second negative result is that naive SFT on distilled long-CoT traces from the small guider lengthens outputs but does not improve answer accuracy in the large target. A third is that SFT-based guider alignment does not match DPO-based alignment.

Warm-up and guidance strength are also sensitive hyperparameters. Without warm-up, repetitive generations appear and accuracy drops; the July 2025 version gives the example that on AIME2025, pass@1 falls from 19.2 to 17.9. Guidance strength is similarly non-monotone: the paper reports that $S_0$7 is the best setting among the tested values, while $S_0$8 and $S_0$9 both hurt performance (Zhang et al., 17 Jul 2025).

The method’s computational profile is favorable relative to post-training a 32B model, but not negligible. In the October 2025 version, ThinkLogit requires the large target plus two small models, $t+1$0 and $t+1$1, during decoding. In the reported prototype setup this increases total parameters by about 1.1× relative to the target alone and reduces throughput by about 25%. The method also often produces substantially longer generations, which increases latency further.

The paper also lists several domain and deployment limitations. Its evaluation is confined to math and science reasoning; it does not establish robustness on coding, planning, or tool-use tasks. ThinkLogit-DPO uses a fixed offline preference dataset and cannot adapt online to new failure modes or distribution drift. The authors note that online RL could address that limitation but would reintroduce optimization complexity.

6. Interpretation and place in the reasoning-transfer landscape

ThinkLogit changes the unit of reasoning transfer from examples to logit-space directions. In this respect its significance is conceptual as well as practical. The method suggests that a useful part of long-reasoning behavior can be represented as a small-model logit delta and then superposed onto a much larger target at inference time. The target’s parametric knowledge is preserved, while the guider supplies a structured bias toward backtracking, self-correction, and extended deliberation.

A common misconception is that ThinkLogit is merely a method for making outputs longer. The paper explicitly rejects that interpretation. Longer outputs can be induced by simpler interventions, but those interventions do not replicate the accuracy gains. The relevant mechanism is token-level steering toward long-CoT behavior, not length alone.

A second misconception is that the method is a lightweight form of large-model fine-tuning. The large model is not updated. The only training, in ThinkLogit-DPO, is applied to the small guider. This separation is important to the claimed use case: large-model training may be infeasible, whereas small-model alignment remains comparatively cheap. The October 2025 version therefore presents ThinkLogit as a practical path to unlock long reasoning in large-scale models “without costly post-training,” and the July 2025 version frames it as an inference-time alternative to directly post-training the 32B target (Zhang et al., 10 Oct 2025).

In that sense, ThinkLogit occupies a specific niche in the broader decoding and reasoning literature: it is a training-free method for the target model, a distribution-level rather than prompt-level transfer mechanism, and a modular interface through which advances in small-model SFT or RL can be reused immediately by larger frozen models.