Continuous Chain-of-Thought

• Continuous Chain-of-Thought is a neural reasoning paradigm that represents each inference step as a dense, real-valued vector, enabling dynamic, parallel processing.
• It achieves significant efficiency by facilitating multimodal fusion and parallel updates through the superposition of multiple reasoning traces.
• Empirical findings demonstrate improvements in accuracy, speed, and robustness over traditional tokenized chain-of-thought methods across diverse benchmarks.

Continuous Chain-of-Thought (CoT) is a paradigm for internal reasoning in neural models that represents each step of a multi-step inference process as a dense, real-valued vector in latent space, rather than as a tokenized natural-language sequence. This approach enables parallel search, dynamic cross-modal alignment, and compact reasoning while mitigating the inefficiencies and sequential bottlenecks of traditional discrete CoT prompting. Continuous CoT methods have been developed for both unimodal and multimodal contexts, offering strong empirical gains in accuracy, inference speed, and robustness, and are theoretically grounded in their ability to maintain superpositions of multiple reasoning traces.

1. Conceptual Foundations and Motivations

Traditional discrete CoT prompting, where a model emits each intermediate reasoning step as a sequence of tokens, is effective for language-based tasks but entails significant verbosity, high memory consumption, and limited alignment between modalities in multimodal settings. Continuous CoT, as operationalized in frameworks such as MCOUT, CODI, MARCOS, and CoT2, replaces these explicit token sequences with real-valued thought vectors that evolve throughout the reasoning process before any output is finally decoded (Pham et al., 18 Aug 2025, Zhu et al., 18 May 2025, Gozeten et al., 29 May 2025, Shen et al., 28 Feb 2025, Liu et al., 29 Sep 2025).

The continuous paradigm confers several principal advantages:

• Dynamic Multimodal Fusion: Reasoning in a shared latent space allows fine-grained, cross-modal information exchange (e.g., between vision and language), with each latent step able to directly attend to and integrate diverse signals (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).
• Parallel and Efficient Exploration: Continuous vectors enable the simultaneous representation and evolution of multiple reasoning traces (superposition), reducing the need for exhaustive autoregressive token generation (Zhu et al., 18 May 2025, Gozeten et al., 29 May 2025, Zhu et al., 27 Sep 2025).
• Reflective and Abstract Reasoning: The latent state can be iteratively refined, mimicking human “mental sketches” that are updated internally, rather than verbalized at every substep (Pham et al., 18 Aug 2025, Liu et al., 29 Sep 2025).

2. Mathematical Formalisms and Model Architectures

Continuous CoT architectures formalize the reasoning process as a sequence or chain of hidden latent states:

• Let $x$ be the input (text, image, or both), and $h_t \in \mathbb{R}^d$ the latent thought at step $t$.
• The initial state $h_0$ is obtained via encoding and possibly interleaving representations from different modalities (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).
• The update rule for each step is generally of the form:

$$h_{t+1} = h_t + \alpha \cdot \Delta(h_t, \text{context})$$

where $\Delta(\cdot)$ may be (for multimodal models) a cross-modal attention mechanism, as in MCOUT-Multi or CoCoVa, or a simple state reuse as in MCOUT-Base (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).

• In methods such as CoT2, the latent vector at each step is an explicit convex superposition over all possible token embeddings, permitting the model to represent all possible discrete traces in parallel (Gozeten et al., 29 May 2025, Zhu et al., 18 May 2025).
• Variations include Markovian approaches, where each step depends only on the current compressed state (e.g., Markov Chain of Thought (Yang et al., 2024) and MARCOS (Liu et al., 29 Sep 2025)), as well as Jacobi-style parallel updates for further efficiency (Wu et al., 23 Jun 2025).

Training objectives typically combine losses over auxiliary intermediate steps (to encourage useful intermediate reasoning), the final answer, and possibly explicit alignment to discrete CoT teacher signals via distillation (as in CODI (Shen et al., 28 Feb 2025)). Variational frameworks are also used to enable stochastic exploration during latent reasoning steps (Liu et al., 29 Sep 2025).

3. Parallelism, Superposition, and Efficiency

A central theoretical result underpinning continuous CoT is the ability to encode and propagate superpositions of multiple reasoning traces, in contrast to the sequential, locally committed development of discrete CoT (Zhu et al., 18 May 2025, Gozeten et al., 29 May 2025, Zhu et al., 27 Sep 2025). In directed graph reachability, for example, a two-layer transformer equipped with continuous CoT requires only $D$ steps (graph diameter) to solve the task, leveraging a latent superposition to execute a parallel breadth-first search. In contrast, discrete CoT must enumerate each path sequentially, requiring $\mathcal{O}(n^2)$ steps for $n$-node graphs (Zhu et al., 18 May 2025).

Correspondingly, in tasks such as subset-sum or symbolic reasoning, a one-layer transformer using continuous CoT can represent all $2^n$ partial sums in parallel by encoding each as a set of orthogonal latent components, reading out the optimum in a single step (Gozeten et al., 29 May 2025).

This parallelism underlies profound efficiency gains:

• Inference compression: CODI achieves a 3.1x reduction in chain length and nearly 3x speedup over explicit CoT while matching discrete-CoT accuracy on GSM8k (Shen et al., 28 Feb 2025).
• Parallel updates: PCCoT accelerates both training and inference by up to 50% through simultaneous Jacobi updates of latent thought tokens (Wu et al., 23 Jun 2025).
• Non-autoregressive “thinking”: MARCOS’s latent Markov chain allows for step-wise, rather than token-wise, randomness and emission, yielding 15.7x acceleration (Liu et al., 29 Sep 2025).
• Self-consistency via superposition: In CoT2, a single continuous chain matches or exceeds the performance of large $h_t \in \mathbb{R}^d$0-shot discrete ensembles in one pass, due to internal self-consistency (Gozeten et al., 29 May 2025).

4. Extensions to Multimodal and Visual Reasoning

Continuous CoT is particularly advantageous for multimodal models, as intermediate representations can blend high-dimensional, modality-specific semantics without bottlenecking via linguistic tokenization:

• Vision-LLMs (VLMs): MCOUT, CoCoVa, and CoVT frameworks demonstrate accuracy gains up to 8.23% and substantial BLEU score improvements on established VQA and science-visual reasoning benchmarks through iterative refinement of latent multimodal thought vectors (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025, Qin et al., 24 Nov 2025).
• Numerical Visual CoT: NV-CoT expands the action space for image grounding from discrete “textified” box coordinates to continuous Euclidean polices, greatly enhancing precision and enabling gradient-based reinforcement learning in visual reasoning (Zhao et al., 27 Feb 2026).
• Continuous Visual Tokens: CoVT trains VLMs to emit compact sets of continuous latent tokens encoding segmentation, depth, edge, and semantic features, improving dense perception and interpretability while maintaining text performance (Qin et al., 24 Nov 2025).

The integration of latent cross-modal alignment, dynamic region selection via learned attention maps, and multi-task objectives (contrastive, reconstruction, and language modeling losses) anchors these continuous representations in both semantic and perceptual domains (Ma et al., 4 Nov 2025, Qin et al., 24 Nov 2025).

5. Empirical Performance, Stability, and Limitations

Continuous CoT frameworks consistently achieve, or surpass, explicit discrete CoT on a range of reasoning benchmarks:

• MCOUT-Base outperforms 1B-param VLM baselines by +8.23% (MMMU), +4.79% (ScienceQA), and +3.98% (MMStar) (Pham et al., 18 Aug 2025).
• CODI, CoT2, and MARCOS match or exceed discrete CoT on GSM8K and other math or logic tasks, but with compressed sequences and increased inference speed (Shen et al., 28 Feb 2025, Gozeten et al., 29 May 2025, Liu et al., 29 Sep 2025).
• CoVT and CoCoVa yield +3–16% gains on a wide array of visual and vision-language tasks, with increased interpretability of reasoning via latent token probing (Qin et al., 24 Nov 2025, Ma et al., 4 Nov 2025).

Ablations indicate that parallelism (PCCoT), explicit full-sequence alignment (SynAdapt), and curriculum or distillation from discrete CoT are critical for optimal performance (Wu et al., 23 Jun 2025, Wang et al., 1 Aug 2025, Shen et al., 28 Feb 2025). However, several limitations persist:

• Interpretability: Intermediate latent steps lack human-readable semantics, making error attribution and downstream control more challenging than with discrete CoT (Pham et al., 18 Aug 2025, Li et al., 9 Feb 2026).
• Modality collapse: Over-aggressive cross-modal fusion may lead to low-variance, non-informative latent states (modality collapse), requiring contrastive or reconstruction regularization (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).
• Training overhead: Auxiliary stepwise losses and iterative updates increase resource consumption, demanding careful balancing of iteration counts and loss coefficients (Pham et al., 18 Aug 2025, Wu et al., 23 Jun 2025).
• Representational commitment: Empirical causal analysis reveals that output-level commitment to an answer often occurs earlier in the latent trajectory than representational (hidden-state) commitment, introducing volatility that may require stability-aware objectives (Li et al., 9 Feb 2026).

6. Theoretical and Causal Insights

The expressivity of continuous CoT expands the representational capacity of shallow neural architectures. Formal results demonstrate that superposition in latent vectors allows for:

• Efficient parallel search: One vector can encode and expand all frontiers in BFS simultaneously, requiring only as many continuous steps as the inherent task depth, not the size of the tree (Zhu et al., 18 May 2025, Gozeten et al., 29 May 2025).
• Gradient-based emergence of superposition: Training dynamics in two-layer transformers naturally converge to maintain bounded index-matching logits, balancing exploration (multiple traces) and exploitation (local structure), as empirically validated and theoretically characterized (Zhu et al., 27 Sep 2025).
• Causal structure of reasoning: Viewing latent CoT as a structural causal model (SCM) reveals the specific functional roles of each step, the existence of skip connections or non-local routing (unlike chain-local discrete CoT), and the gap between early output prediction and late hidden-state stabilization (Li et al., 9 Feb 2026).

Superpositional representations enable latent-step budgets to serve as complex, staged interfaces—rather than homogeneous “depth”—and motivate mode-conditional and stability-aware training objectives.

7. Future Directions and Practical Implications

Research suggests several avenues for advancing continuous CoT:

• Multimodal and cross-domain generalization: Extending the latent reasoning interface to audio, video, and other modalities, with joint pretraining of latent reasoning loops on large-scale multimodal corpora (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).
• Adaptive and explainable decoding: Learning to dynamically adapt the number of reasoning steps per instance and developing probes to map latent thoughts to interpretable, human-friendly formats (Pham et al., 18 Aug 2025, Ma et al., 4 Nov 2025).
• Policy optimization in continuous space: Utilizing fine-grained step-level control and RL objective gradients over continuous latent trajectories for more efficient exploration, leveraging group relative policy optimization and structured stochasticity (Gozeten et al., 29 May 2025, Zhao et al., 27 Feb 2026, Liu et al., 29 Sep 2025).
• Latent compositionality and tool integration: Discovering and integrating new types of latent tokens or continuous “tools” for specialized reasoning (e.g., geometric, symbolic, procedural) (Qin et al., 24 Nov 2025, Ma et al., 4 Nov 2025).

Current practice demonstrates that continuous CoT paradigms enable large language and multimodal models to reason more efficiently, more robustly, and with greater parallelism and cross-modal alignment than their discrete-predecessor counterparts. These advances set the stage for the next generation of scalable, interpretable, and multimodal neural reasoning systems.