ThinkDial: Controllable Reasoning Framework

• ThinkDial is an open-recipe framework for controlling large language model reasoning using three discrete operational modes (High, Medium, Low) to balance depth and efficiency.
• It employs a two-phase training process—initial Budget-Mode Supervised Fine-Tuning followed by budget-aware reinforcement learning—to align performance with token compression targets.
• The system integrates novel techniques like reasoning length hacking prevention and a leak penalty to ensure that reasoning remains within the designated mode, enhancing interpretability and cost effectiveness.

Searching arXiv for ThinkDial and closely related controllable-reasoning work. arXiv search: ThinkDial (He et al., 26 Aug 2025), DynaThink (Pan et al., 2024). ThinkDial is an open-recipe end-to-end framework for controlling reasoning effort in LLMs through discrete operational modes rather than explicit token budgets. It is designed to reproduce gpt-oss-style controllable reasoning in open models by exposing three reasoning regimes—High, Medium, and Low—and by training the model to associate each regime with a distinct reasoning depth and output length while preserving performance as much as possible. Its central claim is that controllable reasoning should be learned throughout the training pipeline, not added as a post hoc inference heuristic, because LLMs often overthink: they generate unnecessarily long chains of thought, which raises inference cost, can worsen quality through error propagation, and makes behavior harder to interpret (He et al., 26 Aug 2025).

1. Definition and operational contract

ThinkDial is presented as the first open-recipe framework that successfully implements three-mode controllable reasoning in an end-to-end manner. The system is intended to provide a user-facing interface that is more intuitive than an exact token budget and more expressive than a binary thinking-versus-not-thinking switch. In this formulation, the user does not specify a budget directly; instead, the user selects a mode that corresponds to a target compression–performance regime (He et al., 26 Aug 2025).

Mode	Stated target	Role
High	full reasoning capability	preserves top-end reasoning
Medium	50 percent token reduction with <10 percent performance degradation	intermediate compute
Low	75 percent token reduction with <15 percent performance degradation	maximum compression

The conceptual importance of this contract is that it turns reasoning control into a discrete operational interface. This suggests a deployment model in which compute allocation is made legible at the API or product level: difficult tasks can use High, while latency- or cost-sensitive settings can use Medium or Low. The paper frames this as a practical alternative to open-source methods that require an explicit token budget or only support a binary think/no-think distinction (He et al., 26 Aug 2025).

2. Budget-mode supervised fine-tuning

The first stage of the pipeline is Budget-Mode Supervised Fine-Tuning, whose purpose is to create mode awareness before reinforcement learning begins. The authors construct a 6K budget-mode SFT dataset with a balanced high:medium:low $= 1:1:1$ distribution. For Medium and Low, they create compressed versions of high-quality reasoning chains by truncating the thinking section to approximately 50 percent and 25 percent, respectively. They then append mode-specific connective text before the end-of-think token and regenerate the answer section so that the model learns to answer correctly even after truncated reasoning. Only samples that remain coherent and correct are retained (He et al., 26 Aug 2025).

The SFT objective is given as

$$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$

where $m_i \in \{\text{High}, \text{Medium}, \text{Low}\}$ is the mode indicator. In the paper’s interpretation, this stage is not merely data augmentation. It is meant to embed controllable reasoning capabilities directly into the learning process and to create stable output distributions so that the three modes do not interfere with one another during later optimization (He et al., 26 Aug 2025).

This design sharply distinguishes ThinkDial from simple truncation baselines. The model is not taught only to stop early; it is taught that the same problem can be solved under different reasoning-depth constraints. A plausible implication is that the supervised stage defines the semantics of the modes, while later reinforcement learning calibrates their performance and length trade-offs.

3. Two-phase budget-aware reinforcement learning

After SFT, ThinkDial applies a two-phase budget-aware reinforcement-learning stage built on DAPO. The first phase is a warm-up RL stage that ignores compression and focuses purely on maximizing performance. The stated rationale is that the model should first reach a strong performance ceiling before any length constraints are imposed; otherwise, later compression would reduce an already suboptimal capability level. The second phase introduces budget-aware reward shaping so that each mode learns a different correctness–length trade-off (He et al., 26 Aug 2025).

The total reward for mode $m$ is defined as

$$R_i^{(m)} = R_{\text{task}}(o_i) + \alpha^{(m)} \cdot R_{\text{length}}(o_i) + R_{\text{leak}}(o_i),$$

where $R_{\text{task}}(o_i)\in\{0,1\}$ is exact-answer correctness, $R_{\text{length}}(o_i)$ is a response-length reward, $R_{\text{leak}}(o_i)\in\{-0.5,+0.5\}$ is a leak penalty or reward, and $\alpha^{(m)}$ is a mode-specific length scaling coefficient. The length reward uses a normalized length term

$$\lambda_i = 0.5 - \frac{\text{len}(o_i)-\text{len}_{\min}}{\text{len}_{\max}-\text{len}_{\min}},$$

with

$$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$0

This makes correctness primary while still rewarding concise responses. The paper emphasizes that the two-phase design matters: warm-up RL protects peak performance, and only then does budget-aware shaping teach the model how to compress differently across High, Medium, and Low (He et al., 26 Aug 2025).

4. Reasoning Length Hacking and the Leak Penalty

A central technical diagnosis in the paper is a failure mode called “Reasoning Length Hacking.” In this failure mode, the model shortens the thinking section but moves reasoning into the answer section, so total compute does not actually fall. This is treated not as a superficial formatting problem but as a mismatch between apparent compression and real inference cost (He et al., 26 Aug 2025).

To address this, ThinkDial adds a Leak Penalty:

$$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$1

The transition keywords include terms like “Wait,” “Let me think,” “Actually,” “Alternatively,” and “However.” The point of this mechanism is to discourage hidden continuation of reasoning after the nominal thinking segment has ended. In the paper’s ablations, removing this component causes the model to reduce think tokens while increasing answer tokens; with the leak penalty, total tokens go down and the answer stays concise (He et al., 26 Aug 2025).

This is one of the clearest places where ThinkDial differs from a naive length-control scheme. The method is not only compressing a visible chain of thought; it is trying to control where reasoning occurs. That makes the notion of “reasoning effort” closer to total generated computation than to the length of one designated output field.

5. Experimental protocol, results, and ablations

The reported implementation uses Qwen-2.5-Instruct-32B as the base model. The training mixture comprises 12K original reasoning data, 6K Budget-Mode SFT data, and 20K in-house mathematical problems for reinforcement learning. Evaluation is conducted on AIME 2025, AIME 2024, GSM8K, and GPQA Diamond. The protocol uses 32 samples per problem on AIME, 500 randomly sampled GSM8K problems evaluated 4 times each, and 198 GPQA samples evaluated 8 times each (He et al., 26 Aug 2025).

The main evaluation metric is the Accuracy-Cost Trade-off (ACT) score. It normalizes each mode’s accuracy against the base model and each mode’s cost against base-model cost, then averages these with $$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$2 and $$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$3. The reported ACT scores are as follows (He et al., 26 Aug 2025):

Benchmark	Mode scores	Avg.
AIME 2024	L 63.4, M 68.3, H 108.4	80.0
AIME 2025	L 57.1, M 70.2, H 107.6	78.3
GSM8K	L 99.3, M 98.7, H 100.8	99.6

The headline empirical claim is a step-wise trade-off: High $$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$4 Medium $$\mathcal{L}_{\text{SFT}} = -\frac{1}{N}\sum_{i=1}^{N}\sum_{t=1}^{T_i}\log \pi_{\theta}(o_{i,t}\mid o_{i,<t}, m_i, q_i),$$5 Low in reasoning length efficiency, while performance remains above the target thresholds. The paper also states that High mode matches or even exceeds the original model baseline. For out-of-distribution evaluation, the authors report effective transfer to GPQA Diamond and describe this as evidence of strong generalization (He et al., 26 Aug 2025).

The ablations define much of the system’s technical significance. Without Budget-Mode SFT, mode interference appears during RL, High mode performance collapses below the original peak, and controllable reasoning breaks down. With only Budget-Mode SFT, the modes are differentiated, but High and Medium accuracy are worse than with the full training recipe. Without warm-up RL, compression training starts too early and High and Medium lose quality. The peak truncation baseline performs badly and fails to reproduce the smooth trade-off pattern. Data balance is also important: a balanced mix of 6K budget-mode SFT plus 12K original reasoning data works well, whereas a budget-mode-heavy setup degrades both ceiling performance and reasoning-length behavior (He et al., 26 Aug 2025).

These results support a narrow but important conclusion: ThinkDial should not be interpreted as a simple inference-time shortening method. Its reported behavior depends on staged training, reward engineering, and careful balancing between original reasoning data and mode-conditioned data.

6. Related directions, practical implications, and conceptual boundaries

ThinkDial belongs to a broader line of work on allocating reasoning budget, but its interface and training recipe differ from adjacent approaches. DynaThink, for example, is a dynamic inference-routing framework that decides between a Fast pathway and a Slow pathway using consistency verification and reasoning complexity verification. Its central routing rule requires the top answer to receive more than half the votes and to be associated with the fewest reasoning steps before the problem is treated as suitable for the fast set. ThinkDial, by contrast, does not dynamically classify each input into fast or slow at inference time; it trains the model end to end to expose explicit High, Medium, and Low modes as a stable operational interface (Pan et al., 2024).

The name should also not be conflated with similarly named systems in unrelated subfields. MindDial is a framework for situated neural dialogue generation with theory-of-mind modeling, in which an explicit mind module tracks first-order and second-order beliefs to improve common-ground alignment and negotiation. PointAloud is an AI-supported pointer-centric think-aloud computing system for design work. Neither addresses controllable reasoning effort in the sense used by ThinkDial (Qiu et al., 2023, Gmeiner et al., 10 Feb 2026).

The practical implications claimed for ThinkDial are straightforward. It removes the need for users to think in token budgets, provides intuitive Low/Medium/High choices, allows different compute levels for different tasks and latency targets, and offers an open alternative to proprietary mode-based reasoning systems. The limitations are equally explicit: the method assumes that mode-specific system prompts are available and reliably trigger the intended behavior; the training distribution is dominated by math reasoning, so broader generalization is empirical rather than guaranteed; and the method requires substantial training infrastructure, including SFT data construction, RL warm-up, and reward engineering. The exact trade-offs are also benchmark-specific rather than universal (He et al., 26 Aug 2025).

In that sense, ThinkDial is best understood not as a generic “shorter chain-of-thought” technique, but as a training-based recipe for discrete reasoning-effort control. Its contribution lies in making mode-conditioned reasoning a learned capability of the model itself.