Zero-Step Thinking in Inference Models

• Zero-Step Thinking is an inference approach that minimizes visible multi-step reasoning by encoding internal computations into hidden tokens or direct responses.
• It leverages techniques like hidden latent reasoning, step-0 mode selection, and direct-answer generation to balance efficiency with effective problem solving.
• Empirical analyses show that while reducing token usage and latency, zero-step methods may experience trade-offs in accuracy on challenging tasks.

Searching arXiv for papers on zero-step thinking, hidden thinking, and adaptive reasoning. Zero-step thinking is a family of inference regimes in which a model produces little or no visible chain-of-thought while still attempting to preserve reasoning quality. The term is not used uniformly. In some work it denotes direct-answer generation with no explicit reasoning segment; in others it denotes deciding at step $0$ whether to think at all; and in latent-reasoning systems it denotes moving multi-step reasoning into hidden states so that only compact placeholders or the final answer remain visible (Shen et al., 31 Jan 2025, Tan et al., 22 Oct 2025, Zhang et al., 19 May 2025). Across these usages, the common objective is to reduce the token, latency, and memory overhead of explicit reasoning without giving up the benefits that chain-of-thought supervision or internal deliberation can provide.

1. Terminological scope and major variants

The phrase “zero-step thinking” names a broader research direction rather than a single mechanism. The literature distinguishes direct-answer prompting, hidden or latent reasoning, adaptive no-thinking modes, and step-$0$ mode selection.

Usage	Operational meaning	Representative work
Hidden latent reasoning	Multi-step CoT is encoded in hidden states or compact thinking tokens rather than natural language	"Efficient Reasoning with Hidden Thinking" (Shen et al., 31 Jan 2025)
Step-$0$ mode selection	Decide Thinking vs NoThinking before generating genuine reasoning, using fake thoughts	"The Zero-Step Thinking" (Tan et al., 22 Oct 2025)
No-thinking direct response	Set reasoning budget to $0$ or close the thinking segment immediately	"Think in Blocks" (Zhu et al., 21 Aug 2025); "AdaptThink" (Zhang et al., 19 May 2025); "Reasoning Models Can Be Effective Without Thinking" (Ma et al., 14 Apr 2025)
Minimal-prompt direct answering	Omit explicit “step-by-step” prompting and answer directly	"Think or Step-by-Step? UnZIPping the Black Box in Zero-Shot Prompts" (Sadr et al., 5 Feb 2025)

This multiplicity matters because different papers optimize different objects. Some optimize visible token count while preserving internal multi-step computation; some optimize the decision of whether to invoke explicit reasoning; some treat zero-step as a prompt-level baseline against which chain-of-thought is measured. A plausible implication is that comparisons across papers are meaningful only after specifying whether “zero-step” refers to hidden reasoning, absent reasoning text, or a control policy over reasoning depth.

2. Hidden and latent forms of zero-step reasoning

One major line of work treats zero-step thinking as an interface property rather than a cognitive one: the model may still perform multi-stage reasoning, but the reasoning is hidden in latent state. Heima is the clearest formulation. It replaces each textual CoT stage with a single token $<CoT>_{(k)}$, trains a multimodal encoder to emit $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$, and uses progressive encoding so that hidden-space reasoning gradually replaces explicit text. The hidden state $H_{<CoT>_{(k)}^{(i)}}$ carries the compressed reasoning for that stage, and a separate decoder can reconstruct stagewise textual CoT from that hidden state when interpretability is needed (Shen et al., 31 Jan 2025).

LightThinker pursues a related objective with a different mechanism. It segments long reasoning traces into subsequences $S_i$, inserts cache tokens $C^{(i)}$ and continuation tokens $[o]^{(i)}$, and applies specialized attention masks so that future reasoning depends only on $0$0 and the compressed cache history rather than the raw earlier thought tokens. The original thought chain can then be discarded from the active context. Its Dependency metric formalizes compression as the total amount of historical context that generated tokens depend on, rather than only peak context length (Zhang et al., 21 Feb 2025).

ThinkLess takes a training-free route. In DeepSeek-style > ... </think> models, it inserts the terminator token immediately after <think>, collapsing the visible reasoning span to <think></think>, then relies on short post-regulation instructions to preserve output format. The paper argues from attention analysis and hidden-state similarity that answer tokens focus primarily on the reasoning terminator token and that much of explicit CoT is redundant once information has migrated forward under causal masking (Li et al., 21 May 2025).

These systems separate visible reasoning from internal reasoning in different ways, but they converge on the same claim: the absence of natural-language CoT does not imply the absence of intermediate computation. This suggests that, in this part of the literature, zero-step thinking is better understood as hidden-thought inference than as literal one-step cognition.

3. Direct-answer and no-thinking modes

A second line of work uses “zero-step” more literally: the model is prompted or trained to bypass explicit reasoning and go directly to the answer. In "Think in Blocks," the reasoning budget is made explicit through <thought_segments> n_{\hat y} </thought_segments>. Zero-step thinking corresponds to $0$1: no reasoning blocks, no <continue_think> markers, and a direct response. The framework trains this behavior through a three-stage pipeline of SFT, DPO, and RL, with a block-cap decoding algorithm that can force or restrict the allowed reasoning depth at inference time (Zhu et al., 21 Aug 2025).

AdaptThink adopts the same broad objective for DeepSeek-R1-Distill-Qwen models but realizes the mode decision through the first token after <think>. Thinking mode begins with a normal thought token; NoThinking is the case $0$2, so the output becomes an empty <think> segment followed by the final solution. Its RL objective explicitly rewards NoThinking through an indicator term $0$3 while constraining performance relative to a reference policy (Zhang et al., 19 May 2025).

"Reasoning Models Can Be Effective Without Thinking" uses a simpler prompting intervention. It pre-fills the thinking box with a dummy sentence such as “Okay, I think I have finished thinking.” and immediately closes it, then asks the model to continue with a compact solution and final answer. This NoThinking mode is paired with parallel scaling and answer aggregation, so that many short direct responses compete with one long explicit reasoning trajectory (Ma et al., 14 Apr 2025).

These works treat explicit thought tokens as optional runtime overhead. Their central claim is not that chain-of-thought supervision is useless, but that inference can often omit visible chain-of-thought while retaining much of the capability induced by CoT training.

4. Step-$0$4 selection, adaptive budgets, and minimal-step control

"The Zero-Step Thinking: An Empirical Study of Mode Selection as Harder Early Exit in Reasoning Models" formalizes zero-step thinking as a control problem. Classical early exit decides when to stop after seeing partial actual thoughts $0$5; zero-step mode selection instead applies the same exit function only once, at $0$6, using the question $0$7 and a fixed fake-thought template $0$8. The paper evaluates nine baselines—Thinking, NoThinking, Pre-judge, FlashThink, PromptConf, Dynasor-CoT, ProbeConf, DEER, and Entropy—on GSM8K, MATH-500, AIME 2025, and GPQA-Diamond, using Acc, Tok, NR, ROC-AUC, ECE, and Brier score. Its main conclusion is that mode selection is a harder variant of early exit: prompt-based approaches often fail under minimal information, internal-state methods are generally better, but stability remains poor across tasks and model scales (Tan et al., 22 Oct 2025).

Adaptive compute methods address the same problem from the opposite direction: not by estimating the right mode from a fake step, but by learning a budgeted policy. "Think in Blocks" predicts an integer reasoning budget before reasoning begins and can be run in auto mode or override mode with bounds cap_low and cap_high; cap_low=cap_high=0 forces no-think, while larger caps permit shallow or deep reasoning (Zhu et al., 21 Aug 2025). AdaptThink similarly learns a difficulty-sensitive policy over Thinking and NoThinking, using importance-sampled PPO so that both modes remain available during training (Zhang et al., 19 May 2025).

NoWait sits slightly outside the zero-step definition but is directly relevant to minimal-step control. It suppresses reflection tokens such as “Wait,” “Hmm,” “Alternatively,” “Check,” and “Verify” at decoding time by setting their logits to a large negative value, thereby shortening R1-style CoT trajectories without retraining. The paper argues that these markers often trigger redundant branches rather than essential computation, so removing them pushes reasoning toward a shorter, more linear trace (Wang et al., 10 Jun 2025).

Taken together, these papers frame zero-step thinking as a resource-allocation problem. The core question is not only whether explicit CoT should exist, but when to allocate it, how much to allocate, and whether step-$0$9 signals are strong enough to support that decision.

5. Empirical efficiency–accuracy trade-offs

The empirical record is heterogeneous but consistent on one point: visible reasoning can often be reduced dramatically. Heima reports six multimodal benchmarks—MMStar, MMBench v1.1, MMVet, MathVista, AI2D, and HallusionBench—with average accuracy $0$0 for Heima versus $0$1 for LLaVA-CoT and $0$2 for the plain Llama3.2-11B-Vision model, while using only about $0$3–$0$4 tokens per sample instead of roughly $0$5–$0$6. On MMStar it uses $0$7 tokens instead of $0$8, and the paper states that token usage can be reduced to as little as $0$9 of the original CoT; on MMBench, Heima reaches $0$0 versus $0$1 for LLaVA-CoT (Shen et al., 31 Jan 2025).

"Think in Blocks" reports a long-CoT baseline with mean length $0$2 tokens and overall accuracy $0$3, versus an SFT $0$4 DPO $0$5 GRPO block-structured model with mean length $0$6 and overall accuracy $0$7. Easy accuracy improves from $0$8 to $0$9, while difficult accuracy remains $<CoT>_{(k)}$0. However, forcing zero-step for all instances is too aggressive: under the ≤0 cap, length drops to $<CoT>_{(k)}$1 tokens but overall accuracy falls to $<CoT>_{(k)}$2 (Zhu et al., 21 Aug 2025).

ThinkLess shows that near-zero visible CoT can work even without retraining. For Qwen2.5-7B, the Distill baseline has average Top-1 $<CoT>_{(k)}$3, $<CoT>_{(k)}$4 tokens, and $<CoT>_{(k)}$5 seconds, while ThinkLess reaches $<CoT>_{(k)}$6, $<CoT>_{(k)}$7 tokens, and $<CoT>_{(k)}$8 seconds. For Qwen2.5-14B, Top-1 changes from $<CoT>_{(k)}$9 to $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$0 while tokens fall from $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$1 to $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$2; for LLaMA 3.1-8B, Top-1 changes from $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$3 to $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$4 while tokens fall from $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$5 to $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$6 (Li et al., 21 May 2025).

AdaptThink reports that on three math datasets it reduces the average response length of DeepSeek-R1-Distill-Qwen-1.5B by $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$7 and improves accuracy by $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$8. For the 7B model, average accuracy improves by $\{<CoT>_{(k)}\}_{k=1}^{K_i}, Y_a^{(i)}$9 points while response length drops by $H_{<CoT>_{(k)}^{(i)}}$0. On GSM8K, the 7B policy chooses NoThinking for $H_{<CoT>_{(k)}^{(i)}}$1 of examples; on AIME 2024 it chooses NoThinking for only $H_{<CoT>_{(k)}^{(i)}}$2, reflecting a strong difficulty gradient (Zhang et al., 19 May 2025).

"Reasoning Models Can Be Effective Without Thinking" shows the strongest case for direct-answer inference under tight budgets. It reports $H_{<CoT>_{(k)}^{(i)}}$3 versus $H_{<CoT>_{(k)}^{(i)}}$4 on AMC 23 with $H_{<CoT>_{(k)}^{(i)}}$5 tokens for NoThinking versus Thinking, and on theorem proving it reports MiniF2F pass@1 $H_{<CoT>_{(k)}^{(i)}}$6 for NoThinking versus $H_{<CoT>_{(k)}^{(i)}}$7 for full Thinking at latency $H_{<CoT>_{(k)}^{(i)}}$8 versus $H_{<CoT>_{(k)}^{(i)}}$9, and ProofNet pass@1 $S_i$0 versus $S_i$1 at latency $S_i$2 versus $S_i$3 (Ma et al., 14 Apr 2025).

NoWait targets a milder compression regime. Its abstract reports chain-of-thought length reductions of $S_i$4–$S_i$5 across five R1-style model series on ten textual, visual, and video reasoning benchmarks, without compromising utility (Wang et al., 10 Jun 2025).

The aggregate pattern is clear. When the objective is average-case efficiency, many tasks tolerate hidden reasoning, shallow reasoning, or direct answers. When the objective is worst-case robustness on hard instances, fully explicit CoT still often retains an edge.

6. Theoretical interpretations, misconceptions, and unresolved problems

A central misconception is that zero-step thinking means “no reasoning.” Several papers argue otherwise. Heima and LightThinker encode multi-step reasoning into hidden states or compressed cache tokens; ThinkLess closes the visible thinking span but claims that reasoning persists implicitly; AdaptThink and Think in Blocks learn when to expose or suppress visible CoT rather than whether the model is capable of reasoning at all (Shen et al., 31 Jan 2025, Zhang et al., 21 Feb 2025, Li et al., 21 May 2025, Zhang et al., 19 May 2025).

A second misconception is that more visible reasoning is always better. The locality-of-experience account gives a sharper condition. "Why think step by step? Reasoning emerges from the locality of experience" proves a reasoning gap for autoregressive models trained on local observations from a chain-structured probabilistic model: direct prediction for nonadjacent variables collapses toward a uniform baseline, while reasoning through intermediate variables strictly reduces bias. The same paper also shows the opposite regime: when training data is fully observed or dependencies are weak, direct zero-step answering is sufficient and step-by-step reasoning adds little beyond variance (Prystawski et al., 2023).

Prompt-level evidence points in the same direction. "Think or Step-by-Step? UnZIPping the Black Box in Zero-Shot Prompts" finds that both “think” and “step-by-step” can have high ZIP scores, but “step-by-step” is more consistently important across models and tasks, especially on math and algebra. This suggests that zero-step and step-by-step prompting are not merely stylistic variants; they trigger different operating modes, and the effective lexical trigger depends on model family and task distribution (Sadr et al., 5 Feb 2025).

Verification results also sharpen the boundary. "Zero-Shot Verification-guided Chain of Thoughts" shows that direct-answer Base prompts are far weaker than COT-like prompts on GSM8K, while COT STEP provides a machine-parsable step structure that enables zero-shot verification. Yet verifier-guided search brings only modest gains over plain COT and little advantage over self-consistency, especially outside math, where verifier accuracy is poor (Chowdhury et al., 21 Jan 2025).

Across the literature, the unresolved problems are consistent. Step-$S_i$6 mode selection remains unstable under minimal information; zero-block or forced no-think policies often under-think on hard tasks; hidden-state reconstruction is semantically faithful but not exact; predicted and actual block counts can diverge; large models may restart implicit reasoning in the answer segment even under NoThinking templates; and distilled models can be more brittle than RL-trained models when reflection markers are suppressed (Tan et al., 22 Oct 2025, Zhu et al., 21 Aug 2025, Shen et al., 31 Jan 2025, Wang et al., 10 Jun 2025). This suggests that zero-step thinking is best viewed not as the elimination of reasoning, but as the search for architectures and policies that relocate, compress, or selectively expose reasoning under strict compute budgets.