Latent Thought Flow: Efficient Latent Reasoning in Large Language Models

Abstract: LLMs increasingly rely on intermediate reasoning, yet explicit Chain-of-Thought (CoT) suffers from a linguistic space bottleneck: each thought must be decoded into tokens, causing high inference overhead. Latent reasoning moves deliberation into continuous space, but existing methods mostly learn deterministic or reward-maximizing paths, lacking a principled way to allocate probability across trajectories with different correctness and costs. We propose Latent Thought Flow (LTF), which models reasoning as variable-length continuous trajectories and trains a sampler to match a reward-induced posterior over answer quality and computation cost. We instantiate this with a continuous GFlowNet using stochastic latent transitions. To handle sparse answer supervision, we introduce an Entropy-Weighted Subtrajectory Balance objective for intermediate rewards and a reference-prior regularizer to anchor exploration. Experiments under finetuning and transfer learning settings show that LTF outperforms explicit CoT and latent reasoning baselines, improving accuracy by 9.5% while reducing reasoning length by 27.2% on average compared with strong latent reasoning baselines.

• The paper introduces a novel latent reasoning framework that models continuous trajectories via GFlowNets, optimizing accuracy and efficiency.
• It leverages entropy-weighted subtrajectory balance and reference-prior regularization to enhance exploration and prevent mode collapse.
• Empirical evaluations show up to a 12.9% accuracy improvement and a 34.5% reduction in latent step count compared to strong baseline methods.

Latent Thought Flow: Efficient Latent Reasoning in LLMs

Introduction

Latent Thought Flow (LTF) (2606.16222) provides a principled framework for latent reasoning in LLMs, reconciling the need for intermediate computation with the infeasibility of explicit reasoning token generation due to efficiency, redundancy, and linguistic bottlenecks. Prior explicit approaches such as Chain-of-Thought (CoT) and related methods incur substantial inference costs by externalizing every intermediate step. Contemporary latent reasoning methods shift reasoning into internal states but largely optimize point estimates—single, deterministic trajectories—underreward or compression objectives, which neglect the distributional nature of reasoning and the trade-off between correctness and computational effort.

LTF models latent reasoning as a distribution over variable-length continuous trajectories in LLM hidden states, training a sampler so that path probabilities are proportional to a reward aggregating answer quality and the cost of computation. This is formalized via continuous Generative Flow Networks (GFlowNets), enabling a stochastic generative process that allocates probability mass over the manifold of reasoning paths according to their utility.

Figure 1: The LTF framework integrates a trainable LoRA module and latent reasoning head, sampling latent thought trajectories as continuous flows with a GFlowNet objective.

Methodological Innovations

Variable-Length Latent Trajectories and Reward Structure

LTF operates over sequences of adaptive length, with each latent state sampled from a learned Gaussian, and termination determined dynamically via a trainable head. For training, a differentiable reward $\mathcal{R}_{x,y}(\tau)$ is introduced, combining a task-specific verification metric and normalized log-likelihood of the answer, penalized exponentially by latent reasoning length. This utility drives the posterior $p^*(\tau|x, y)\propto \mathcal{R}_{x,y}(\tau)$, yielding a target for amortized variational inference over latent thought flows.

Continuous GFlowNet Training

Unlike RL or maximum likelihood, GFlowNets enable learning a generator whose sampled trajectories are distributed according to the unnormalized reward. LTF extends GFlowNet theory to continuous, high-dimensional latent spaces, modeling reasoning as a flow over trajectory prefixes with possible termination at every prefix. Forward transitions use conditional Gaussian densities, and flow conservation is enforced over subtrajectories via a Subtrajectory Balance objective.

Entropy-Weighted Subtrajectory Balance

LTF proposes an entropy-weighted variant of the GFlowNet SubTB objective, which adaptively reweights subtrajectory consistency constraints according to their entropy, enhancing credit assignment for diverse latent states and mitigating mode collapse.

Reference-Prior Regularization

Early in training, exploration can stray into degenerate regions of latent space. LTF employs a reference prior regularizer, using a teacher-provided rationale (when available) as guidance. A branch predicts these reference latents; their log-likelihood under the current policy is maximized, then this regularization is annealed away as reward optimization becomes reliable.

Empirical Evaluation and Results

Experiments span LLaMA and DeepSeek-R1 backbones ranging from 1B to 8B, evaluated on GSM8K, ASDiv, DU, and out-of-distribution math datasets. Baselines include explicit (CoT, Assist-CoT), hybrid (SoftCoT++), and latent (Coconut, CoLaR, ReGuLaR) reasoning. Metrics are answer accuracy and reasoning length (latent step count).

Key results include:

• LTF achieves up to 12.9% absolute accuracy improvement and a 34.5% reduction in latent step count over strong latent baselines in finetuning scenarios.
• Under extreme compression (reasoning budget = 1), LTF maintains superior accuracy (improvement of 2–4% over ReGuLaR) on challenging numerical reasoning datasets.
• On transfer learning benchmarks, LTF yields consistent gains (1.8–2.4% absolute) in accuracy and produces shorter latent trajectories compared to ReGuLaR and CoLaR.
• Ablations demonstrate that entropy weighting and increased rollout sample diversity during training further enhance accuracy–efficiency tradeoffs, and that GFlowNet-based training outperforms both RL-style and simpler balance-based objectives.

Theoretical and Practical Implications

LTF substantiates that reasoning in LLMs should be modeled as sampling from a reward-proportional distribution over latent computational paths, not as trajectory compression, distillation, or reward maximization along point paths. By explicitly encoding the accuracy–efficiency trade-off in the utility, LTF guides LLMs to allocate computational resources dynamically—longer/thorough reasoning for harder queries, and minimal computation for simpler cases—thus directly addressing the reasoning scalability bottleneck for deployed LLMs.

Practically, LTF reduces inference costs as the need for explicit stepwise token generation vanishes; only final answers are decoded, and reasoning remains purely latent. The framework's modularity (LoRA head and Gaussian latent sampling atop fixed backbones) makes it compatible with large-scale models and potentially extensible to adaptation scenarios or multimodal reasoning, given the correct latent interface.

Prospects for Future Research

LTF's architectural and objective orthogonality to the underlying backbone invites investigation of transfer to vision-language or speech models, higher-order latent variable structure, hierarchical reasoning, and joint training with explicit-consistency constraints for tasks requiring rationale interpretability. The observed effective entropy regime also motivates further theoretical analysis into the geometry and informativeness of learned reasoning manifolds, particularly in high-compression regimes and few-shot generalization.

Conclusion

LTF offers a rigorous, distributional approach for latent reasoning in LLMs via reward-proportional continuous GFlowNet training, enabling concise, adaptive, high-utility computation in the hidden space. The proposed entropy-aware objectives and reference regularizers improve both exploration and convergence, resulting in substantial gains over state-of-the-art baselines in accuracy and efficiency. Future extensions can further push the reasoning accuracy–efficiency frontier across modalities and tasks.