JointThinking: Calibrated Reasoning in LLMs

• The paper introduces JointThinking, a dual-mode paradigm that uses explicit reasoning and intuitive pattern recognition to calibrate responses in large language models.
• It utilizes a consistency check between Thinking and Nothinking answers to decide if a second Thinking pass is needed, balancing efficiency and accuracy.
• Experiments reveal that JointThinking reduces error rates and outperforms repeated reasoning approaches, demonstrating effective compute allocation across benchmarks.

Thinking with Nothinking Calibration, or JointThinking, is an in-context learning paradigm for reasoning LLMs in which the same model is prompted to produce two first-pass responses in parallel—one in Thinking mode and one in Nothinking mode—and a second round of Thinking is invoked only when the two answers are inconsistent. In the formulation introduced in “Thinking with Nothinking Calibration: A New In-Context Learning Paradigm in Reasoning LLMs” (Wu et al., 5 Aug 2025), JointThinking is a no-training meta-reasoning procedure designed for reasoning LLMs whose inference already contains an explicit thinking phase and a final-solution phase. Its central claim is that structural thinking diversity between full chain-of-thought and direct answering can be used as a calibration signal: agreement permits early acceptance of the Thinking answer, whereas disagreement triggers additional reasoning only where it is likely to matter.

1. Conceptual basis and motivation

Reasoning LLMs are described as models trained, typically via reinforcement learning, to produce explicit, structured, multi-step reasoning traces before final answers. The paper places JointThinking against several existing inference-time strategies for such models. Few-shot chain-of-thought prompting is reported to provide no gain over plain Thinking on MATH500 with R1-7B as the number of exemplars increases. Thinking twice is described as unstable: it can improve or fail to improve over single Thinking, and can induce overthinking by revisiting already-correct answers. Majority voting / Best-of-N is strong, especially on smaller models, but is compute-inefficient because it samples multiple full reasoning traces. Training-based adaptive methods such as AdaptThink require additional RL training and may overfit to the distribution of training tasks (Wu et al., 5 Aug 2025).

The conceptual motivation of JointThinking is that Thinking and Nothinking are complementary rather than merely longer and shorter versions of the same behavior. The paper characterizes Nothinking as relying on pattern recognition and “System 1”-like intuition, while Thinking is associated with explicit “System 2”-like reasoning. Across GSM8K, MATH500, AIME24, and AMC23, there are cases where Thinking is wrong but Nothinking is correct, and vice versa, and this complementary pattern is reported to hold across difficulty levels. This suggests that disagreement between the two modes is informative in a way that repeated Thinking is not (Wu et al., 5 Aug 2025).

In this sense, JointThinking belongs to a broader line of work on adaptive reasoning depth. Training-based frameworks such as AdaptThink teach models to choose Thinking or NoThinking based on problem difficulty (Zhang et al., 19 May 2025), and OThink-R1 learns a single-model fast/slow switch by classifying reasoning trajectories as redundant or essential (Zhang et al., 3 Jun 2025). JointThinking differs in being an ICL-only mechanism: no parameter update is required, and calibration emerges from prompt-level orchestration of two modes already exposed by the reasoning model (Wu et al., 5 Aug 2025).

2. Formal framework and inference procedure

JointThinking operates on a question $q$ and a reasoning model $M$. The paper defines two first-stage responses:

$$r_{\text{thinking}} = M(q + \langle \text{BOT} \rangle)$$

and

$$r_{\text{nothinking}} = M(q + \langle \text{BOT} \rangle + s_{\text{skip}} + \langle \text{EOT} \rangle),$$

where $s_{\text{skip}}$ is the fixed skip string “Okay, I think I have finished thinking.” The answer extractor $A(\cdot)$ parses the final answer, typically from boxed{}. A consistency check is then defined as

$$\text{Con}(q) = \begin{cases} 1, & \text{if } A(r_t) = A(r_n) \ 0, & \text{otherwise}. \end{cases}$$

If the two first-pass answers are identical, JointThinking returns the Thinking answer. If they differ, the model is prompted to perform a second Thinking pass conditioned on the two candidate answers. The final decision rule is

$$R(q) = \begin{cases} A(r_t), & \text{if } \text{Con}(q) = 1 \ \text{ST}(q, A(r_t), A(r_n)), & \text{if } \text{Con}(q) = 0. \end{cases}$$

Algorithmically, the procedure is: generate a Thinking response, generate a Nothinking response, extract both answers, test equality, and invoke a second Thinking pass only on inconsistent cases (Wu et al., 5 Aug 2025).

This procedure differs from “Thinking twice” in a precise way. Thinking twice always re-enters the reasoning loop after a first Thinking pass. JointThinking performs the second pass only when cross-mode inconsistency is observed. The calibration signal is therefore not “I should think again because I have already thought once,” but rather “I should think again because structurally different first-pass policies disagree.”

3. Prompt construction and calibration mechanism

The paper gives concrete prompt templates. In Thinking mode, the prompt ends with:

$M$0

In Nothinking mode, the prompt inserts an immediate stop string inside the thinking block:

$M$1

For Qwen3 models, the paper follows the official \think / \nothink setting for the first stage, but the same functional distinction remains: a full reasoning trajectory versus a skipped explicit thinking phase (Wu et al., 5 Aug 2025).

The second Thinking stage uses both candidate answers inside the > block. The core instruction is:

$M$2

A central prompt-design result is that this instruction works best when placed after <think>, as part of the internal reasoning trace, rather than before <think> as a standard external instruction. The paper reports that placing the instruction before <think> yields smaller gains or can hurt performance, whereas placing it after <think> is consistently best (Wu et al., 5 Aug 2025). This is presented as evidence that current RLLMs are more responsive to guidance integrated into the reasoning channel than to guidance placed outside it.

The calibration analysis is centered on the Error Rate over the subset of examples where the two first-pass answers are consistent. Let $\mathcal{C}$ be the set of consistent cases. Then

$$\text{ER} = \frac{\left|\{ q \in \mathcal{C} : A(r_{\text{thinking}}) \neq \text{GT}(q) \}\right|}{|\mathcal{C}|}.$$

This ER measures the fraction of confidently wrong cases: examples where both modes agree and the answer is still incorrect. Lower ER means that consistency is a more reliable signal for accepting the first-stage result. For R1-7B, the paper reports that Thinking–Nothinking calibration yields lower ER than Thinking–Thinking calibration on all four evaluated datasets: GSM8K, MATH500, AIME24, and AMC23 (Wu et al., 5 Aug 2025). This is the paper’s main evidence that structural diversity between the two modes provides a stronger calibration signal than repeated use of the same mode.

4. Empirical performance and scaling behavior

The experiments use six RLLMs: DeepSeek-R1-Distill-Qwen at 1.5B, 7B, 14B, and 32B, plus Qwen3 reasoning models at 8B and 14B, evaluated on GSM8K, MATH500, AIME24, and AMC23. Across these datasets, JointThinking is reported to outperform few-shot CoT, Thinking twice, and majority voting on average, while remaining a pure ICL method (Wu et al., 5 Aug 2025).

The average accuracies reported in Table 1 illustrate the pattern. For R1-1.5B, JointThinking reaches 68.61, compared with 62.40 for Thinking, 65.71 for Majority Voting, and 65.50 for Thinking Twice. For R1-7B, JointThinking reaches 79.64, compared with 77.44 for Thinking and 78.87 for Majority Voting. For R1-14B, JointThinking reaches 85.73, exceeding 82.99 for Thinking and 83.99 for Majority Voting. For R1-32B, JointThinking reaches 87.90, above 84.23 for Thinking and 86.80 for Thinking Twice. For Qwen3-8B and Qwen3-14B, JointThinking reaches 87.57 and 89.15, respectively, and is again the best average method in the reported comparisons (Wu et al., 5 Aug 2025).

The paper also reports benchmark-level peaks: 96.29% on GSM8K with R1-32B, 92.59% on MATH500 with Qwen3-14B, 71.67% on AIME24 with Qwen3-14B, and 96.88% on AMC23 with Qwen3-14B. A second experimental axis compares JointThinking with the training-based method AdaptThink on in-distribution tasks (GSM8K, MATH500, AIME24) and out-of-distribution tasks (MMLU-Pro, GPQA). The reported pattern is that JointThinking is competitive on in-distribution math tasks and significantly outperforms AdaptThink on OOD tasks, for example on R1-1.5B and R1-7B for both MMLU-Pro and GPQA (Wu et al., 5 Aug 2025).

A further result concerns compute allocation. Optional second thinking—triggered only on disagreement—outperforms always running a second pass while also using less compute. The paper states that on GSM8K with R1-32B, only about 6% of questions trigger second thinking, yet JointThinking still outperforms the naive “always rethink” strategy. The interpretation offered in the paper is that always thinking again can corrupt initially correct answers, whereas the consistency check protects against this form of overthinking (Wu et al., 5 Aug 2025).

The scaling analysis focuses on disagreement cases. The paper defines an “ideal” second-thinking upper bound that would always choose the correct candidate whenever exactly one of the two first-pass answers is correct. It then compares this with actual second-thinking performance. As model size increases from 1.5B to 32B, the gap between actual and ideal second-thinking performance narrows on GSM8K and MATH500; on GSM8K, R1-32B surpasses the ideal bound by 1.14%, which is attributed to second thinking occasionally recovering a new correct answer even when both initial answers are wrong (Wu et al., 5 Aug 2025). This suggests that second thinking is not merely a verifier over two candidates but can function as a genuine corrective re-reasoning mechanism.

5. Relation to adjacent approaches and common misconceptions

A frequent misconception is that JointThinking is simply a variant of repeated CoT. The paper argues against this by directly comparing Thinking–Thinking and Thinking–Nothinking calibration. The latter yields consistently lower ER, indicating that the gain does not come from “more of the same reasoning,” but from cross-mode diversity (Wu et al., 5 Aug 2025).

Another misconception is that standard few-shot CoT remains the natural ICL baseline for RLLMs. The paper reports that for R1-7B on MATH500, increasing the number of few-shot CoT examples yields no gain over plain Thinking, which the authors interpret as evidence that RL-trained reasoning models already internalize a thinking policy and may not benefit from additional CoT demonstrations in the same way that standard LLMs do (Wu et al., 5 Aug 2025).

JointThinking also sits within a broader research landscape on adaptive reasoning. AdaptThink learns a policy that chooses NoThinking while maintaining performance through a constrained RL objective and importance sampling across modes (Zhang et al., 19 May 2025). OThink-R1 constructs a supervised dataset in which redundant reasoning is pruned to empty <think> while essential reasoning is preserved, and trains a single model against both slow-thinking and fast-thinking references (Zhang et al., 3 Jun 2025). THOUGHTTERMINATOR, by contrast, is a training-free black-box decoding method that predicts per-instance budgets from difficulty and injects interrupt messages to terminate overthinking (Pu et al., 17 Apr 2025). These approaches all seek calibrated allocation of reasoning effort, but JointThinking is distinctive in using parallel first-pass dual-mode inference plus selective second Thinking as its calibration primitive.

The paper’s findings also connect to later work on hybrid thinking controllability. “Demystifying Hybrid Thinking” reports that current hybrid models achieve only partial mode separation, with reasoning leakage into no-think mode, and identifies larger data scale, more no-think data, and a two-phase strategy as important for better controllability (Wang et al., 14 Oct 2025). Path-Lock Expert addresses the same problem architecturally by separating think and no-think MLP pathways and reports large reductions in no-think reflective tokens while preserving think-mode performance (Wang et al., 29 Apr 2026). These results suggest that JointThinking’s calibration signal is strongest when the two modes are genuinely distinct; where no-think leakage is severe, the benefits of structural diversity may be attenuated.

6. Limitations and future directions

The paper identifies several limitations. First, experiments are limited to models up to 32B because of hardware and context constraints. Second, the operational mode set is restricted to Thinking and Nothinking, although the framework is described as conceptually extensible to richer mode families such as programmatic reasoning or different system personas. Third, the consistency check is based on exact answer equality, which is effective for math benchmarks with short canonical answers but does not directly address open-ended tasks; the paper explicitly points to semantic equivalence mechanisms as a future requirement for broader deployment (Wu et al., 5 Aug 2025).

A fourth limitation is the existence of Both Incorrect ≡ cases, where Thinking and Nothinking produce the same wrong answer. These cases satisfy the consistency check and therefore do not trigger second thinking; they define a lower bound on achievable calibration error within the present scheme. A fifth limitation concerns instruction following: the prompt-placement ablation shows that guidance before <think> can be ignored or mishandled, whereas guidance inside the reasoning channel is much more effective. This is presented as evidence that RL training with simplistic prompts may have weakened the sensitivity of current RLLMs to ordinary external instructions (Wu et al., 5 Aug 2025).

Subsequent research sharpens these concerns. “Don’t Think Twice! Over-Reasoning Impairs Confidence Calibration” reports that increasing reasoning budgets can systematically worsen confidence calibration, even when modest additional reasoning initially helps, and argues that information access rather than reasoning depth is often the bottleneck in knowledge-intensive settings (Lacombe et al., 20 Aug 2025). “When Calibration Rankings Reverse” further shows that raw global calibration metrics can be confounded by accuracy differences and advocates accuracy-controlled evaluation when comparing thinking and non-thinking modes (Yang et al., 29 Jun 2026). A plausible implication is that future JointThinking systems may need not only better mode orchestration, but also separate calibration layers for answer correctness, confidence estimation, and evidence access.

Within that broader context, JointThinking’s main contribution is to establish a prompt-level template for selective second reasoning based on cross-mode disagreement. It frames Thinking and Nothinking as complementary inference channels, treats consistency as a calibration observable, and shows that a substantial part of adaptive reasoning can be realized without additional training by exploiting the structural heterogeneity already present in reasoning LLMs (Wu et al., 5 Aug 2025).