Latent Thought Flow

• Latent Thought Flow (LTF) is a generalization of chain-of-thought that uses continuous hidden states to refine internal beliefs before token generation.
• It encompasses various architectures—from per-token hidden state recursion to ordered latent planning slots—each designed to optimize the internal reasoning process.
• Recent methods leverage inference-time optimization and curriculum learning to balance certainty and exploration, improving performance across language and dialogue tasks.

Latent Thought Flow (LTF) denotes a family of reasoning schemes in which intermediate computation is carried by continuous latent states rather than explicit rationale tokens. Across recent work, these states appear as reused hidden states before each next token, soft mixtures of vocabulary embeddings updated while listening, ordered latent planning slots between perception and generation, autoregressive continuous thought slots with tractable likelihood, or loop trajectories refined before emission. The shared pattern is a temporally ordered internal trajectory—per token, per block, per dialogue frame, or per loop—that refines beliefs before committing to surface symbols, thereby decoupling reasoning from verbalization and opening distinct training, inference-time, and communication interfaces (Zeng et al., 27 Sep 2025, Wu et al., 18 Mar 2026, Tu et al., 4 Jun 2026, Wang et al., 29 Jan 2026).

1. Conceptual scope and relation to chain-of-thought

LTF is best understood as a generalization of chain-of-thought (CoT) from discrete token space to continuous hidden space. CoT scales reasoning by emitting explicit natural-language steps after or during generation. LTF instead scales internal computation by inserting or iterating latent states that need not be verbalized. In PonderLM-2, this is framed as extending the generative process for each token by appending latent thoughts in the sequence dimension, which the authors call “horizontal scaling” of computation; in FLAIR, latent reasoning runs concurrently with speech perception; in PLaT, reasoning is reformulated as a deterministic trajectory of latent planning states while a separate decoder verbalizes only when needed (Zeng et al., 27 Sep 2025, Wu et al., 18 Mar 2026, Wang et al., 29 Jan 2026).

This family includes both per-token and per-sequence formulations. Some methods treat every token position as having its own internal micro-trajectory, such as $$\mathbf{h}_i^0 \to \mathbf{h}_i^1 \to \dots \to \mathbf{h}_i^K$$, before predicting $x_{i+1}$. Others construct explicit latent blocks or thought slots between prompt and answer, or maintain loop states $h_j^{(t)}$ before emitting token $y_j$. A plausible implication is that LTF is less a single architecture than a design principle: intermediate reasoning becomes an optimizable latent process, while token generation becomes one possible readout of that process (Zeng et al., 27 Sep 2025, Tu et al., 4 Jun 2026, Jonathan et al., 11 Feb 2026).

A recurring motivation is that discrete CoT couples reasoning to communication. Several papers state this explicitly. PLaT describes its objective as decoupling reasoning from verbalization; the inference-time rethinking framework factorizes reasoning into a continuous latent thought vector, “what to reason about,” and a decoder, “how to reason”; FLAIR argues that explicit post-hoc “thinking” mechanisms do not match spoken dialogue, where listening and internal cognition are concurrent (Wang et al., 29 Jan 2026, Kong et al., 6 Feb 2026, Wu et al., 18 Mar 2026).

2. Representational forms and dynamical mechanisms

Recent LTF systems instantiate latent thoughts in several distinct but related forms.

Form	Representative papers	Core mechanism
Per-token hidden-state recursion	PonderLM-2 (Zeng et al., 27 Sep 2025)	The latent thought is the last hidden state $\mathbf{h}_t$, reused as input with the same positional encoding
Soft latent embeddings	FLAIR (Wu et al., 18 Mar 2026); LT-Tuning (Liu et al., 10 Feb 2026)	Latent thought is built from logits as $\operatorname{Softmax}(\cdot)E$ or fused with contextual hidden state
Ordered latent planning states	PLaT (Wang et al., 29 Jan 2026); SignThought (Jiang et al., 16 Apr 2026)	Latent planning slots or thought chains mediate between input evidence and verbal output
Block or slot-based continuous thoughts	LaDiR (Kang et al., 6 Oct 2025); NF-CoT (Tu et al., 4 Jun 2026)	Reasoning uses latent thought blocks or continuous thought slots before answer decoding
Looped or continuous-depth trajectories	RLTT on LoopLMs (Jonathan et al., 11 Feb 2026); LFT (Wu et al., 20 May 2025)	Internal loop states or continuous transport across layers refine computation before emission

In PonderLM-2, the latent thought for position $t$ is exactly the last-layer hidden state $\mathbf{h}_t$. Training uses Jacobi iteration over interleaved sequences,

$$\mathbf{S}^k = [\mathbf{e}(x_1), \mathbf{h}_1^k, \mathbf{e}(x_2), \mathbf{h}_2^k, \dots, \mathbf{e}(x_T), \mathbf{h}_T^k],$$

so each token position acquires a short latent trajectory in continuous space before the next token is predicted. The latent thought shares the same positional encoding as its corresponding token, so the method deepens computation at a slot rather than advancing the positional clock (Zeng et al., 27 Sep 2025).

FLAIR instantiates LTF in streaming spoken dialogue with a dual-mode decoder. During listening, explicit <SIL> placeholders are replaced by latent thought embeddings

$$\hat{Z}_t = \operatorname{Softmax}(y^{\text{txt}}_t)E,$$

and the decoder input is

$x_{i+1}$0

where $x_{i+1}$1 selects latent thinking versus explicit speaking. This yields a causal latent variable process that runs while the user is still speaking, without an additional post-hoc reasoning stage (Wu et al., 18 Mar 2026).

LT-Tuning uses a related but more explicitly stabilized construction. At <thinking> positions, the latent embedding is not a fixed learned vector but a Context-Prediction-Fusion of contextual hidden state and predictive semantic guidance: $x_{i+1}$2 The predictive component $x_{i+1}$3 is a probability-weighted mixture of vocabulary embeddings, while $x_{i+1}$4 preserves problem-specific context. This is presented as a response to feature collapse and distribution mismatch when raw hidden states are recurrently reused as embeddings (Liu et al., 10 Feb 2026).

Other papers broaden the representational repertoire. SignThought defines an ordered chain $x_{i+1}$5 of latent thoughts between sign-video evidence and text, with causal self-attention over thought slots and routed cross-attention to visual segments, followed by a plan-then-ground decoder (Jiang et al., 16 Apr 2026). PLaT defines latent planning states $x_{i+1}$6, aggregates them with exponential moving averages $x_{i+1}$7, and uses $x_{i+1}$8 as the planning state to be verbalized when needed (Wang et al., 29 Jan 2026). LaDiR encodes reasoning sentences into latent thought blocks and then evolves them by flow matching in latent space, while NF-CoT inserts continuous thought slots $x_{i+1}$9 into the causal stream and models them with autoregressive Gaussian factors inside the same LLM backbone (Kang et al., 6 Oct 2025, Tu et al., 4 Jun 2026).

An adjacent continuous-depth formulation appears in the Latent Flow Transformer, which replaces a contiguous block of transformer layers by a learned transport operator

$h_j^{(t)}$0

trained by flow matching or Flow Walking. This is not primarily a latent-reasoning interface, but it turns a discrete chunk of transformer computation into a continuous latent trajectory and therefore supplies a useful continuous-time analogue to LTF (Wu et al., 20 May 2025).

3. Learning latent thought flows

The training objectives for LTF systems vary, but nearly all preserve a standard task-level loss while redirecting intermediate computation into latent space. PonderLM-2 keeps next-token cross-entropy as the sole supervision signal and applies it at positions corresponding to final latent thoughts $h_j^{(t)}$1, with no auxiliary losses on the thoughts themselves. To avoid overfitting to a fixed thought depth, it samples the number of Jacobi iterations $h_j^{(t)}$2 from $h_j^{(t)}$3 for each training instance. SignThought likewise learns its latent thought chain without direct thought supervision: the chain is optimized only through sentence-level cross-entropy, supplemented by a monotonicity loss and a contiguity loss on the routing matrix $h_j^{(t)}$4 so that different thoughts align with ordered, contiguous video segments (Zeng et al., 27 Sep 2025, Jiang et al., 16 Apr 2026).

A second group of methods adopts explicit latent-variable objectives. FLAIR introduces a conditional ELBO with a non-causal Global-aware Expert $h_j^{(t)}$5 that produces latent label embeddings $h_j^{(t)}$6, while the causal spoken-dialogue LLM learns a prior $h_j^{(t)}$7. Its loss combines conditional reconstruction $h_j^{(t)}$8, variational regularization $h_j^{(t)}$9, and timing prediction $y_j$0,

$y_j$1

with $y_j$2 and $y_j$3. NF-CoT uses a different latent-variable stack: a frozen VAE encoder compresses explicit CoT traces to $y_j$4, a shallow invertible flow maps them to $y_j$5, and a unified supervised loss combines latent likelihood and answer cross-entropy (Wu et al., 18 Mar 2026, Tu et al., 4 Jun 2026).

Curriculum and progressive internalization are recurrent themes. LT-Tuning trains in three stages: explicit CoT warm-up, dynamic latent token generation at low-confidence positions, and Context-Prediction-Fusion refinement. SCOUT, which introduces Flow CoT, uses progressive distillation rather than explicit CoT labels for each latent stage: with $y_j$6 latent iterations, it aligns iteration 1 to Qwen2.5-1.5B, iteration 2 to Qwen2.5-3B, and iteration 3 to Qwen2.5-7B, while a cross-attention retrospective module injects previous latent states without overwriting the original input-grounded computation (Liu et al., 10 Feb 2026, Li et al., 30 May 2025).

Theoretical work supplies a stronger rationale for curriculum. One analysis of latent CoT proves that curriculum learning is theoretically necessary because direct training provably fails due to distributional mismatch. In that account, latent reasoning trades off decisional certainty, exploration, and execution stability, and curriculum is the mechanism that moves a model from explicit high-certainty trajectories to usable low-certainty latent ones without collapsing into biased latent policies (Zou et al., 1 Feb 2026).

4. Inference-time optimization, control, and communication

A major development in LTF is the shift from static latent computation to explicit inference-time control. LTPO treats intermediate latent thought vectors as dynamic parameters optimized per instance. Given appended [THINK] tokens with embeddings $y_j$7, it samples perturbations from a Gaussian policy,

$y_j$8

evaluates an intrinsic confidence-based reward from the frozen LLM’s own output distributions, and updates

$y_j$9

The base model parameters remain frozen; only the latent thought vectors are optimized at test time (Ye et al., 5 Oct 2025).

Inference-Time Rethinking pushes this idea further by alternating between candidate-trace generation and latent re-estimation. It factorizes reasoning as $\mathbf{h}_t$0, initializes $\mathbf{h}_t$1 from the question alone, then iterates: generate a trace $\mathbf{h}_t$2 from the decoder conditioned on $\mathbf{h}_t$3, and optimize the ELBO for $\mathbf{h}_t$4 to obtain a revised latent $\mathbf{h}_t$5. The paper proves that this Gibbs-style alternation is a form of coordinate-ascent variational inference and yields monotonic ELBO improvement. In the article’s own terms, the resulting sequence $\mathbf{h}_t$6 is a trajectory in latent thought space (Kong et al., 6 Feb 2026).

Other systems introduce structured control rather than direct latent optimization. NF-CoT preserves left-to-right autoregression and exact likelihood by sampling continuous thought slots $\mathbf{h}_t$7 from Gaussian factors predicted by an NF head, then supports direct policy-gradient optimization in latent reasoning space because $\mathbf{h}_t$8 is tractable (Tu et al., 4 Jun 2026). RLTT modifies reinforcement learning for looped LLMs by distributing reward across the full latent reasoning trajectory,

$\mathbf{h}_t$9

instead of assigning credit only to the final loop (Jonathan et al., 11 Feb 2026). PLaT uses lazy decoding to decide when to stop latent planning and start verbalization: it decodes only the first token from the current planning state $\operatorname{Softmax}(\cdot)E$0, and if that token is not $\operatorname{Softmax}(\cdot)E$1, latent planning continues (Wang et al., 29 Jan 2026).

LTF also extends beyond single-agent reasoning. In multiagent thought communication, latent thoughts $\operatorname{Softmax}(\cdot)E$2 generate all agents’ model states $\operatorname{Softmax}(\cdot)E$3, and identifiability theorems show that both shared and private latent thoughts between agents, as well as the global structure of thought sharing, can be recovered under a sparse nonparametric latent-variable model. A plausible implication is that LTF can serve not only as an internal reasoning substrate but also as a communication substrate more direct than token exchange (Zheng et al., 23 Oct 2025).

5. Empirical regimes and representative results

The empirical record for LTF spans language modeling, downstream NLP, speech dialogue, sign-language translation, code generation, mathematical reasoning, and looped latent computation.

Domain	Representative result	Paper
Language modeling and general NLP	PonderLM-2-Pythia-1.4B vs Pythia-2.8B: avg 0-shot 58.5 vs 57.3; avg 5-shot 59.5 vs 57.6	(Zeng et al., 27 Sep 2025)
Full-duplex speech dialogue	FLAIR w/ thk vs w/o thk: LlamaQ 78.0 vs 73.0; MMSU 56.2 vs 50.2; turn-taking latency 0.33–0.43s	(Wu et al., 18 Mar 2026)
Gloss-free sign language translation	LC-HKSLT: BLEU-4 21.15, ROUGE 47.87; with additional in-domain pretraining 30.22 and 60.01	(Jiang et al., 16 Apr 2026)
Code generation	NF-CoT Unified vs base: average pass@1 68.8 vs 55.8; total time 325.6s vs LaDiR 625.3s	(Tu et al., 4 Jun 2026)
Test-time math reasoning	Qwen-2.5-7B-Instruct with LTPO on AIME2024/2025: 16.67 and 13.33; SoftCoT 0.00 and 0.00	(Ye et al., 5 Oct 2025)
Looped latent reasoning	RLTT over GRPO: +14.4% on MATH-500, +16.6% on AIME24, +10.0% on BeyondAIME	(Jonathan et al., 11 Feb 2026)

Additional results show that these gains are not confined to a single formulation. In PonderLM-2’s LLaMA comparison, the 1.4B latent-thought model under $\operatorname{Softmax}(\cdot)E$4 inference FLOPs achieves the best Pile PPL 7.89, Lambada OpenAI PPL 7.39, Wikitext PPL 16.99, Lambada Standard PPL 12.20, avg 0-shot accuracy 52.3, and avg 5-shot accuracy 51.9, outperforming Looped, Pause, Ponder, and a scaled-up LLaMA-2.8B under matched or larger FLOP budgets (Zeng et al., 27 Sep 2025). LT-Tuning reports average reasoning accuracies of 36.4% on Llama-3.2-1B, 52.4% on Llama-3.2-3B, and 68.8% on Llama-3.1-8B, rising to 70.3% with an adapter; the same paper shows large degradations when Stage 2, Stage 3, or test-time latent use are removed (Liu et al., 10 Feb 2026).

In latent planning, PLaT exhibits a different profile: greedy accuracy is lower than explicit CoT baselines, but Pass@128 on GSM8K reaches 74.2%, exceeding Coconut’s 66.7% and CODI’s 70.1%. The paper presents this as a precision–diversity trade-off, where latent planning broadens the solution manifold even when single-path decoding remains weaker (Wang et al., 29 Jan 2026). SCOUT reports smaller but consistent finetuning gains: across eight benchmarks, the best SCOUT configuration improves the average score from 37.21 for SFT to 39.03, described as up to 1.8% gains under fine-tuning, with qualitative evidence that later iterations refine both belief formation and explanation granularity (Li et al., 30 May 2025).

LaDiR broadens the empirical picture by combining VAE thought blocks with latent diffusion. It is reported to improve accuracy, diversity, and interpretability over autoregressive, diffusion-based, and latent reasoning baselines, and on Countdown tasks it outperforms autoregressive LLaMA 8B SFT by $\operatorname{Softmax}(\cdot)E$5 points in CD-4 Pass@1 and by $\operatorname{Softmax}(\cdot)E$6 points in CD-5 while also increasing the number of unique valid solutions among 100 samples (Kang et al., 6 Oct 2025). The inference-time rethinking framework shows a complementary result: a 0.2B model trained from scratch on GSM8K reaches 31.54% on GSM8K, 51.50% on SVAMP, and 68.00% on MultiArith after 30 rethinking iterations, and the abstract states that it surpasses baselines with 10 to 15 times more parameters, including a 3B counterpart (Kong et al., 6 Feb 2026).

6. Interpretation, controversies, and open problems

The strongest interpretability analyses treat latent thought trajectories as causal objects rather than opaque vectors. One study models latent steps $\operatorname{Softmax}(\cdot)E$7 as variables in a structural causal model, uses step-wise $\operatorname{Softmax}(\cdot)E$8-interventions such as $\operatorname{Softmax}(\cdot)E$9, and measures flip rates, early-decoding sufficiency $t$0, and an influence matrix $t$1. Its findings are that latent-step budgets behave less like homogeneous extra depth and more like staged functionality with non-local routing, and that there is a persistent gap between early output bias and late representational commitment (Li et al., 9 Feb 2026).

A second line of work emphasizes fundamental trade-offs. In “Capabilities and Fundamental Limits of Latent Chain-of-Thought,” the key control variable is the Symbolic Index,

$t$2

which quantifies decisional commitment. The paper proves an Exploration-Execution Trade-off: high certainty enables precise execution but inhibits exploration, while low certainty facilitates search but causes error accumulation. The empirical illustration is stark: Latent CoT reaches 97.0% on ProsQA yet only 34.1% on GSM8K, while curriculum learning is argued to be theoretically necessary because direct training provably fails due to distributional mismatch (Zou et al., 1 Feb 2026).

A more direct controversy concerns superposition. “The Illusion of Superposition?” finds that only models trained from scratch exhibit signs of using superposition. In the training-free regime, Soft Thinking’s convex combinations of token embeddings collapse so quickly that soft and argmax runs have cosine similarity around $t$3. In the fine-tuned regime, a Coconut model on ProsQA reaches 99.0% with latent tokens but 96.6% without them, and entity-level probing shows that the target entity already dominates at the first latent step, consistent with shortcutting rather than maintained multi-hypothesis reasoning. In contrast, a 2-layer from-scratch Coconut model on ProsQA falls from 94.5% with latent steps to 13.8% without them, and its intermediate latent states retain substantial mass on both correct-next and wrong-neighbor entities (Rizvi-Martel et al., 7 Apr 2026).

Several limitations recur across the literature. Many systems fix the number of latent steps or thought slots uniformly across inputs; PonderLM-2 explicitly notes that it does not provide adaptive per-token halting, and one latent step doubles inference FLOPs (Zeng et al., 27 Sep 2025). FLAIR and SignThought both learn latent thoughts without explicit semantic control over their content, which preserves flexibility but leaves interpretation underdetermined (Wu et al., 18 Mar 2026, Jiang et al., 16 Apr 2026). A plausible synthesis is that future LTF work will move toward adaptive compute allocation, mode-conditional and stability-aware analysis, richer latent structures, and broader communication interfaces, including multiagent latent thought sharing that is not reducible to natural-language exchange (Li et al., 9 Feb 2026, Zheng et al., 23 Oct 2025).