Vertical Reasoning and Latent State Propagation

• Vertical reasoning is a paradigm where multi-step inference occurs through iterative latent state propagation, bypassing explicit token chains.
• It employs methods such as continuous state transitions and dynamic termination to enhance computational efficiency and diversify solution exploration.
• Modern frameworks integrate hierarchical latent models to improve interpretability and optimize bandwidth usage during reasoning processes.

Vertical reasoning and latent state propagation form the core of a new paradigm in machine reasoning, wherein large models (language, vision-language, or embodied systems) perform multi-step inference entirely within continuous or mixed latent spaces, decoupling internal computation from explicit, discrete verbalization. This approach is motivated by computational efficiency, bandwidth maximization, and the need for richer, more scalable exploration of solution spaces compared to explicit chain-of-thought (CoT) in discrete token spaces. Modern frameworks implement vertical reasoning as a developmental hierarchy or a dynamic, multi-step trajectory traversed by the model’s hidden states, leveraging sophisticated propagation, planning, and compression mechanisms. These methods undergird advances in latent reasoning for mathematical, visual, and embodied tasks, and provide novel opportunities for interpretability, control, and efficient search.

1. Conceptual Foundations and Motivations

Vertical reasoning refers to multi-step inference carried out through the iterated propagation of internal, continuous or hybrid latent states ("hidden" or "thought" vectors) within a deep network. Unlike explicit CoT, which elongates token context with natural language rationales, vertical reasoning compresses these traces into compact, recursive updates of the model’s hidden activations. The principal motivations are as follows:

• Computational efficiency: Latent state propagation reduces the computational overhead and context-window expansion caused by explicit rationales (Wang et al., 29 Jan 2026, Wang et al., 11 Jan 2026, Amos et al., 9 Feb 2026).
• Bandwidth exploitation: Continuous representations in high-dimensional latent spaces carry much greater information density than discrete tokens, vastly increasing expressive bandwidth (Zhu et al., 8 Jul 2025).
• Reasoning diversity and scalability: By supporting non-collapsing, high-entropy distributions of alternatives, latent vertical reasoning avoids the “collapse” characteristic of greedy token generation, thus enabling exploration of broader solution spaces (Wang et al., 29 Jan 2026, Li et al., 9 Feb 2026).
• On-demand interpretability: Separation of reasoning and verbalization allows intermediate states to be decoded into text or probed for interpretability without interrupting the internal reasoning process (Wang et al., 29 Jan 2026).

2. Formalization and Mechanisms of Latent State Propagation

The mechanistic and mathematical formulation of vertical reasoning in latent spaces varies by framework, but several core elements recur:

• State Transition: Reasoning is modeled as a deterministic or stochastic trajectory through a sequence of latent states $\{z_t\}$, often analogous to a Markov decision process or drift-diffusion stochastic dynamical system (Wu et al., 10 Jul 2025, Carson et al., 4 Jun 2025).
• Update Equations: State updates typically involve a sequence of transformations or recursions:

$z_{t+1} = f(z_t,\,x;\,\theta)$

or in the planning formalism (Wang et al., 29 Jan 2026),

$$\tilde{\mathbf s}_{k,i} = \phi_{\mathrm{H2L}}\Bigl(\mathcal{M}([\cdots])_{-1}\Bigr)$$

• Aggregation and EMA: Smoothing and memory are maintained via mechanisms such as Exponential Moving Average (EMA), which integrate micro-step updates over successive reasoning iterations (Wang et al., 29 Jan 2026).
• Explicit Regimes and Switching: Some models use regime-switching dynamics (e.g., SLDS models) that capture discrete reasoning phases and latent switches associated with decomposition, synthesis, exploration, and failure (Carson et al., 4 Jun 2025).

In hybrid or hierarchical models, multiple “vertical” levels or modules (e.g., textual, visual, action) may be stacked, each propagating latent reasoning states specialized for their modality (Bai et al., 1 Feb 2026).

3. Key Architectural Paradigms

A range of vertical reasoning architectures have emerged:

Framework (Paper)	Core Mechanism	Propagation Strategy
PLaT (Wang et al., 29 Jan 2026)	Planner–Decoder separation	Micro-step EMA, dynamic termination
Laser (Wang et al., 11 Jan 2026)	Global-to-local (DWAL) latent chain	Dynamic window/soft superposition
CTRLS (Wu et al., 10 Jul 2025)	MDP with distributional RL	Latent state RL, Dirichlet actions
System-1.5 (Wang et al., 25 May 2025)	Depth/step shortcuts in Transformer	Early-exit routers, latent copying
Thinking States (Amos et al., 9 Feb 2026)	Interleaved "thinking" embeddings	Chunked-inject, recurrent token/embedding compression
LSTR (Wang et al., 2 Feb 2026)	Sparse transcoder updates	Residual skip, sparse innovation
LT-Tuning (Liu et al., 10 Feb 2026)	Context-Prediction Fusion	Confidence-based latent insertion

These architectures implement vertical reasoning via recurrent application of network layers, auxiliary adapters, or dedicated latent-transcoder modules. Verticality is further manifested in stacking of latent modules by cognitive level (textual, visual, action), as in LaRA-VLA (Bai et al., 1 Feb 2026).

4. Dynamic Reasoning Length, Diversity, and Interpretability

• Dynamic termination: Unlike fixed-step models, dynamic termination strategies (e.g., lazy decoding in PLaT) enable the model to decide when to halt reasoning based on confidence or answer detection, optimizing compute for task complexity (Wang et al., 29 Jan 2026, Liu et al., 10 Feb 2026).
• Diversity scaling: High-entropy latent state propagation and distributional objectives allow for superposition of multiple reasoning paths, enabling scalable exploration (as quantified by Pass@k metrics), and avoiding early commitment (Wang et al., 29 Jan 2026, Wang et al., 11 Jan 2026).
• On-demand verbalization: The decoder, decoupled from the planner, can be invoked to interpret any intermediate latent state without disrupting ongoing computation, providing a mechanism for transparency and traceability (Wang et al., 29 Jan 2026).
• Interpretability and probes: Causal, logit-lens, and attention-probe analyses allow localization of functional sub-trajectories within the latent reasoning chain, identification of causally effective steps, and the detection of non-local routing or shortcutting (Liang et al., 31 Jan 2026, Li et al., 9 Feb 2026).

5. Causal Structure, Shortcut Mechanisms, and Diagnostic Insights

Recent empirical studies model latent step propagation as a structural causal process, enabling the identification of bottlenecks, non-local routing, and instability in reasoning trajectories:

• SCM modeling: By formulating the latent chain as an SCM, researchers analyze how interventions at each step affect downstream states and final outputs, quantifying how influence and necessity spread or concentrate (Li et al., 9 Feb 2026).
• Non-local routing and collapse: Latent steps do not always maintain uniform functionality; some induce direct skips or late-fusion shortcuts, especially under fixed latent step budgets or in highly compressible tasks (Liang et al., 31 Jan 2026, Li et al., 9 Feb 2026).
• Instability metrics: Flip rates and stepwise causal effect metrics reveal mid-trajectory leverage points, and differences between representational and output-level commitment (Li et al., 9 Feb 2026).
• Regularized training: Mode-conditional and stability-aware regularizers are advocated to prevent premature collapse, encourage maintenance of competing hypotheses, and structure semantic bottlenecks (Li et al., 9 Feb 2026).

6. Empirical Performance, Scalability, and Tradeoffs

• Efficiency: Vertical latent reasoning models reduce inference tokens by 90% or more relative to explicit CoT, achieving corresponding decreases in latency and compute requirements (Wang et al., 11 Jan 2026, Liu et al., 10 Feb 2026, Bai et al., 1 Feb 2026).
• Accuracy/diversity trade-off: While greedy accuracy may underperform strong explicit-CoT baselines, vertical latent methods deliver higher Pass@k (diversity) and can maintain robust performance under greater breadth of sampling or out-of-distribution generalization (Wang et al., 29 Jan 2026, Wu et al., 10 Jul 2025).
• Compression and interpretability: Sparse transcoder architectures such as LSTR provide active, controllable semantic resolution, merging interpretability with competitive accuracy (Wang et al., 2 Feb 2026).
• Modality and task generalization: Vertical latent propagation supports transfer across vision-language, math, RL, and planning domains, with explicit mechanisms to preserve cross-task robustness and compositionality (Wang et al., 11 Jan 2026, Bai et al., 1 Feb 2026).

7. Future Directions, Limitations, and Open Challenges

• Adaptive latent budgeting: Scaling vertical reasoning to arbitrarily long or deeply recursive tasks requires dynamic allocation of latent steps, and mechanisms for adjusting budget based on task complexity (Liang et al., 31 Jan 2026).
• Bottleneck and non-locality control: Explicit constraints or auxiliary losses may be necessary to prevent collapse or shortcutting, and to enforce algorithmic faithfulness in latent rollout (Liang et al., 31 Jan 2026, Li et al., 9 Feb 2026).
• Multimodal and hierarchical stacks: Stacking vertical reasoning modules across modalities (text, vision, action) expands the applicability of latent state propagation for VLA and embodied agents, but introduces new integration challenges (Bai et al., 1 Feb 2026).
• Interpretability and white-box operators: Sparse and structured latent operators offer a path to more interpretable reasoning, but balancing expressivity, transparency, and efficiency remains open (Wang et al., 2 Feb 2026).
• Training stability and mode maintenance: Ensuring that vertical reasoning maintains necessary diversity, avoids semantic collapse, and properly allocates causal leverage is a focus for continued objective design (Wang et al., 11 Jan 2026, Li et al., 9 Feb 2026, Liu et al., 10 Feb 2026).

In summary, vertical reasoning and latent state propagation advance the field beyond explicit, token-based stepwise reasoning, enabling scalable, high-bandwidth, and dynamically adaptive multi-step inference. These innovations, grounded in principled architectures, training objectives, and diagnostic frameworks, underpin much of the recent progress in efficient, interpretable, and robust model reasoning (Wang et al., 29 Jan 2026, Wang et al., 11 Jan 2026, Li et al., 9 Feb 2026, Zhu et al., 8 Jul 2025, Liu et al., 10 Feb 2026).