---
title: 'JointThinking: Calibrated Reasoning in LLMs'
url: https://www.emergentmind.com/topics/thinking-with-nothinking-calibration-jointthinking
type: topic
---

# JointThinking: Calibrated Reasoning in LLMs

Thinking with Nothinking Calibration, or **JointThinking**, is an in-context learning paradigm for reasoning large language models in which the same model is prompted to produce two first-pass responses in parallel—one in **Thinking mode** and one in **Nothinking mode**—and a **second round of Thinking** is invoked only when the two answers are inconsistent. In the formulation introduced in “Thinking with Nothinking Calibration: A New In-Context Learning Paradigm in Reasoning Large Language Models” [2508.03363], JointThinking is a no-training meta-reasoning procedure designed for reasoning large language models whose inference already contains an explicit thinking phase and a final-solution phase. Its central claim is that **structural thinking diversity** between full chain-of-thought and direct answering can be used as a calibration signal: agreement permits early acceptance of the Thinking answer, whereas disagreement triggers additional reasoning only where it is likely to matter.

## 1. Conceptual basis and motivation

Reasoning large language models are described as models trained, typically via reinforcement learning, to produce explicit, structured, multi-step reasoning traces before final answers. The paper places JointThinking against several existing inference-time strategies for such models. **Few-shot chain-of-thought prompting** is reported to provide **no gain** over plain Thinking on MATH500 with R1-7B as the number of exemplars increases. **Thinking twice** is described as unstable: it can improve or fail to improve over single Thinking, and can induce **overthinking** by revisiting already-correct answers. **Majority voting / Best-of-N** is strong, especially on smaller models, but is **compute-inefficient** because it samples multiple full reasoning traces. Training-based adaptive methods such as AdaptThink require additional RL training and may overfit to the distribution of training tasks [2508.03363].

The conceptual motivation of JointThinking is that Thinking and Nothinking are **complementary** rather than merely longer and shorter versions of the same behavior. The paper characterizes Nothinking as relying on pattern recognition and “System 1”-like intuition, while Thinking is associated with explicit “System 2”-like reasoning. Across GSM8K, MATH500, AIME24, and AMC23, there are cases where Thinking is wrong but Nothinking is correct, and vice versa, and this complementary pattern is reported to hold across difficulty levels. This suggests that disagreement between the two modes is informative in a way that repeated Thinking is not [2508.03363].

In this sense, JointThinking belongs to a broader line of work on adaptive reasoning depth. Training-based frameworks such as AdaptThink teach models to choose Thinking or NoThinking based on problem difficulty [2505.13417], and OThink-R1 learns a single-model fast/slow switch by classifying reasoning trajectories as redundant or essential [2506.02397]. JointThinking differs in being an **ICL-only** mechanism: no parameter update is required, and calibration emerges from prompt-level orchestration of two modes already exposed by the reasoning model [2508.03363].

## 2. Formal framework and inference procedure

JointThinking operates on a question \(q\) and a reasoning model \(M\). The paper defines two first-stage responses:

\[
r_{\text{thinking}} = M(q + \langle \text{BOT} \rangle)
\]

and

\[
r_{\text{nothinking}} = M(q + \langle \text{BOT} \rangle + s_{\text{skip}} + \langle \text{EOT} \rangle),
\]

where \(s_{\text{skip}}\) is the fixed skip string “Okay, I think I have finished thinking.” The answer extractor \(A(\cdot)\) parses the final answer, typically from `boxed{}`. A consistency check is then defined as

\[
\text{Con}(q) =
\begin{cases}
1, & \text{if } A(r_t) = A(r_n) \\
0, & \text{otherwise}.
\end{cases}
\]

If the two first-pass answers are identical, JointThinking returns the Thinking answer. If they differ, the model is prompted to perform a second Thinking pass conditioned on the two candidate answers. The final decision rule is

\[
R(q) =
\begin{cases}
A(r_t), & \text{if } \text{Con}(q) = 1 \\
\text{ST}(q, A(r_t), A(r_n)), & \text{if } \text{Con}(q) = 0.
\end{cases}
\]

Algorithmically, the procedure is: generate a Thinking response, generate a Nothinking response, extract both answers, test equality, and invoke a second Thinking pass only on inconsistent cases [2508.03363].

This procedure differs from “Thinking twice” in a precise way. Thinking twice always re-enters the reasoning loop after a first Thinking pass. JointThinking performs the second pass only when **cross-mode inconsistency** is observed. The calibration signal is therefore not “I should think again because I have already thought once,” but rather “I should think again because structurally different first-pass policies disagree.”

## 3. Prompt construction and calibration mechanism

The paper gives concrete prompt templates. In **Thinking mode**, the prompt ends with:

```text
Problem: {problem}
Please put your final answer in the format boxed{}
<|Assistant|> Answer:<think>
```

In **Nothinking mode**, the prompt inserts an immediate stop string inside the thinking block:

```text
Problem: {problem}
Please put your final answer in the format boxed{}
<|Assistant|> Answer:<think> Okay I have finished thinking. </think>
```

For Qwen3 models, the paper follows the official `\think` / `\nothink` setting for the first stage, but the same functional distinction remains: a full reasoning trajectory versus a skipped explicit thinking phase [2508.03363].

The second Thinking stage uses both candidate answers inside the `<think>` block. The core instruction is:

```text
I think there are two candidate answers {thinking_answer} and {nothinking_answer} for this question. One of them is correct or both are wrong. I need to first verify them. If both are wrong, I need to rethink step by step and avoid making the same mistake.
```

A central prompt-design result is that this instruction works best when placed **after `<think>`**, as part of the internal reasoning trace, rather than before `<think>` as a standard external instruction. The paper reports that placing the instruction before `<think>` yields smaller gains or can hurt performance, whereas placing it after `<think>` is consistently best [2508.03363]. This is presented as evidence that current RLLMs are more responsive to guidance integrated into the reasoning channel than to guidance placed outside it.

The calibration analysis is centered on the **Error Rate** over the subset of examples where the two first-pass answers are consistent. Let \(\mathcal{C}\) be the set of consistent cases. Then

\[
\text{ER} = \frac{\left|\{ q \in \mathcal{C} : A(r_{\text{thinking}}) \neq \text{GT}(q) \}\right|}{|\mathcal{C}|}.
\]

This ER measures the fraction of **confidently wrong** cases: examples where both modes agree and the answer is still incorrect. Lower ER means that consistency is a more reliable signal for accepting the first-stage result. For R1-7B, the paper reports that **Thinking–Nothinking** calibration yields lower ER than **Thinking–Thinking** calibration on all four evaluated datasets: GSM8K, MATH500, AIME24, and AMC23 [2508.03363]. This is the paper’s main evidence that structural diversity between the two modes provides a stronger calibration signal than repeated use of the same mode.

## 4. Empirical performance and scaling behavior

The experiments use six RLLMs: DeepSeek-R1-Distill-Qwen at **1.5B**, **7B**, **14B**, and **32B**, plus Qwen3 reasoning models at **8B** and **14B**, evaluated on GSM8K, MATH500, AIME24, and AMC23. Across these datasets, JointThinking is reported to outperform few-shot CoT, Thinking twice, and majority voting on average, while remaining a pure ICL method [2508.03363].

The average accuracies reported in Table 1 illustrate the pattern. For **R1-1.5B**, JointThinking reaches **68.61**, compared with **62.40** for Thinking, **65.71** for Majority Voting, and **65.50** for Thinking Twice. For **R1-7B**, JointThinking reaches **79.64**, compared with **77.44** for Thinking and **78.87** for Majority Voting. For **R1-14B**, JointThinking reaches **85.73**, exceeding **82.99** for Thinking and **83.99** for Majority Voting. For **R1-32B**, JointThinking reaches **87.90**, above **84.23** for Thinking and **86.80** for Thinking Twice. For **Qwen3-8B** and **Qwen3-14B**, JointThinking reaches **87.57** and **89.15**, respectively, and is again the best average method in the reported comparisons [2508.03363].

The paper also reports benchmark-level peaks: **96.29%** on GSM8K with R1-32B, **92.59%** on MATH500 with Qwen3-14B, **71.67%** on AIME24 with Qwen3-14B, and **96.88%** on AMC23 with Qwen3-14B. A second experimental axis compares JointThinking with the training-based method AdaptThink on **in-distribution** tasks (GSM8K, MATH500, AIME24) and **out-of-distribution** tasks (MMLU-Pro, GPQA). The reported pattern is that JointThinking is competitive on in-distribution math tasks and **significantly outperforms** AdaptThink on OOD tasks, for example on R1-1.5B and R1-7B for both MMLU-Pro and GPQA [2508.03363].

A further result concerns compute allocation. Optional second thinking—triggered only on disagreement—outperforms always running a second pass while also using less compute. The paper states that on GSM8K with R1-32B, only about **6%** of questions trigger second thinking, yet JointThinking still outperforms the naive “always rethink” strategy. The interpretation offered in the paper is that always thinking again can corrupt initially correct answers, whereas the consistency check protects against this form of overthinking [2508.03363].

The scaling analysis focuses on disagreement cases. The paper defines an “ideal” second-thinking upper bound that would always choose the correct candidate whenever exactly one of the two first-pass answers is correct. It then compares this with actual second-thinking performance. As model size increases from **1.5B** to **32B**, the gap between actual and ideal second-thinking performance narrows on GSM8K and MATH500; on GSM8K, **R1-32B surpasses the ideal bound by 1.14%**, which is attributed to second thinking occasionally recovering a new correct answer even when both initial answers are wrong [2508.03363]. This suggests that second thinking is not merely a verifier over two candidates but can function as a genuine corrective re-reasoning mechanism.

## 5. Relation to adjacent approaches and common misconceptions

A frequent misconception is that JointThinking is simply a variant of repeated CoT. The paper argues against this by directly comparing **Thinking–Thinking** and **Thinking–Nothinking** calibration. The latter yields consistently lower ER, indicating that the gain does not come from “more of the same reasoning,” but from **cross-mode diversity** [2508.03363].

Another misconception is that standard few-shot CoT remains the natural ICL baseline for RLLMs. The paper reports that for R1-7B on MATH500, increasing the number of few-shot CoT examples yields **no gain** over plain Thinking, which the authors interpret as evidence that RL-trained reasoning models already internalize a thinking policy and may not benefit from additional CoT demonstrations in the same way that standard LLMs do [2508.03363].

JointThinking also sits within a broader research landscape on adaptive reasoning. AdaptThink learns a policy that chooses NoThinking while maintaining performance through a constrained RL objective and importance sampling across modes [2505.13417]. OThink-R1 constructs a supervised dataset in which redundant reasoning is pruned to empty `<think></think>` while essential reasoning is preserved, and trains a single model against both slow-thinking and fast-thinking references [2506.02397]. THOUGHTTERMINATOR, by contrast, is a training-free black-box decoding method that predicts per-instance budgets from difficulty and injects interrupt messages to terminate overthinking [2504.13367]. These approaches all seek calibrated allocation of reasoning effort, but JointThinking is distinctive in using **parallel first-pass dual-mode inference plus selective second Thinking** as its calibration primitive.

The paper’s findings also connect to later work on hybrid thinking controllability. “Demystifying Hybrid Thinking” reports that current hybrid models achieve only **partial mode separation**, with reasoning leakage into no-think mode, and identifies larger data scale, more no-think data, and a two-phase strategy as important for better controllability [2510.12680]. Path-Lock Expert addresses the same problem architecturally by separating think and no-think MLP pathways and reports large reductions in no-think reflective tokens while preserving think-mode performance [2604.27201]. These results suggest that JointThinking’s calibration signal is strongest when the two modes are genuinely distinct; where no-think leakage is severe, the benefits of structural diversity may be attenuated.

## 6. Limitations and future directions

The paper identifies several limitations. First, experiments are limited to models up to **32B** because of hardware and context constraints. Second, the operational mode set is restricted to **Thinking** and **Nothinking**, although the framework is described as conceptually extensible to richer mode families such as programmatic reasoning or different system personas. Third, the consistency check is based on **exact answer equality**, which is effective for math benchmarks with short canonical answers but does not directly address open-ended tasks; the paper explicitly points to semantic equivalence mechanisms as a future requirement for broader deployment [2508.03363].

A fourth limitation is the existence of **Both Incorrect ≡** cases, where Thinking and Nothinking produce the same wrong answer. These cases satisfy the consistency check and therefore do not trigger second thinking; they define a lower bound on achievable calibration error within the present scheme. A fifth limitation concerns instruction following: the prompt-placement ablation shows that guidance before `<think>` can be ignored or mishandled, whereas guidance inside the reasoning channel is much more effective. This is presented as evidence that RL training with simplistic prompts may have weakened the sensitivity of current RLLMs to ordinary external instructions [2508.03363].

Subsequent research sharpens these concerns. “Don’t Think Twice! Over-Reasoning Impairs Confidence Calibration” reports that increasing reasoning budgets can systematically worsen confidence calibration, even when modest additional reasoning initially helps, and argues that information access rather than reasoning depth is often the bottleneck in knowledge-intensive settings [2508.15050]. “When Calibration Rankings Reverse” further shows that raw global calibration metrics can be confounded by accuracy differences and advocates accuracy-controlled evaluation when comparing thinking and non-thinking modes [2606.30814]. A plausible implication is that future JointThinking systems may need not only better mode orchestration, but also separate calibration layers for answer correctness, confidence estimation, and evidence access.

Within that broader context, JointThinking’s main contribution is to establish a prompt-level template for **selective second reasoning based on cross-mode disagreement**. It frames Thinking and Nothinking as complementary inference channels, treats consistency as a calibration observable, and shows that a substantial part of adaptive reasoning can be realized without additional training by exploiting the structural heterogeneity already present in reasoning large language models [2508.03363].

Source: https://www.emergentmind.com/topics/thinking-with-nothinking-calibration-jointthinking