LaST$_{0}$: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Model

Abstract: Vision-Language-Action (VLA) models have recently demonstrated strong generalization capabilities in robotic manipulation. Some existing VLA approaches attempt to improve action accuracy by explicitly generating linguistic reasoning traces or future visual observations before action execution. However, explicit reasoning typically incurs non-negligible inference latency, which constrains the temporal resolution required for robotic manipulation. Moreover, such reasoning is confined to the linguistic space, imposing a representational bottleneck that struggles to faithfully capture ineffable physical attributes. To mitigate these limitations, we propose LaST$_0$, a framework that enables efficient reasoning before acting through a Latent Spatio-Temporal Chain-of-Thought (CoT), capturing fine-grained physical and robotic dynamics that are often difficult to verbalize. Specifically, we introduce a token-efficient latent CoT space that models future visual dynamics, 3D structural information, and robot proprioceptive states, and further extends these representations across time to enable temporally consistent implicit reasoning trajectories. Furthermore, LaST$_0$ adopts a dual-system architecture implemented via a Mixture-of-Transformers design, where a reasoning expert conducts low-frequency latent inference and an acting expert generates high-frequency actions conditioned on robotics-oriented latent representations. To facilitate coordination, LaST$_0$ is trained with heterogeneous operation frequencies, enabling adaptive switching between reasoning and action inference rates during deployment. Across ten simulated and six real-world manipulation tasks, LaST$_0$ improves mean success rates by 8% and 13% over prior VLA methods, respectively, while achieving substantially faster inference. Project website: https://sites.google.com/view/last0

• The paper introduces LaST₀, a model that builds its chain-of-thought in a compact latent space to enable fast, temporally consistent reasoning for robotic manipulation.
• It employs a mixture-of-transformers dual-system to decouple high-level reasoning and low-level acting, achieving 15.4 Hz inference and an 82% success rate in simulated tasks.
• Experimental results demonstrate robust performance in both simulated and real-world settings, with improved task completion and reduced latency compared to explicit CoT approaches.

Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Models

Introduction and Motivation

This paper introduces LaST$_{0}$, a Vision-Language-Action (VLA) model that integrates reason-before-act capabilities for robot manipulation by means of a Latent Spatio-Temporal Chain-of-Thought (CoT). The authors identify two key bottlenecks in existing explicit CoT VLA frameworks: significant inference latency stemming from explicit autoregressive reasoning (text/image tokens), and a restrictive representational scope—language reasoning is insufficient for implicit, ineffable physical characteristics and fine-grained dynamics intrinsic to robotic manipulation. LaST$_{0}$ addresses both constraints by operating in a compact multimodal latent space, supporting fast, physically grounded, and temporally consistent reasoning required for closed-loop control.

Figure 1: Overview of the LaST$_{0}$ framework, illustrating the dual-system architecture that performs latent spatio-temporal reasoning and high-frequency control through Mixture-of-Transformers.

Methodology

Latent Spatio-Temporal CoT

Instead of relying on explicit CoT representations in language or images, LaST$_{0}$ constructs its CoT in a compressed continuous latent space spanning three modalities: future visual frames (RGB), 3D point clouds, and proprioceptive robot states. For each future step within a control horizon, features from these modalities are pooled into interleaved tokens, capturing both semantic and geometric structure with internal state information. This design enables the policy to support fine-grained, temporally coherent implicit reasoning unattainable by discrete representations.

Mixture-of-Transformers Dual-System Policy

LaST$_{0}$ utilizes a Mixture-of-Transformers architecture to decouple high-level, low-frequency "reasoning expert" computation from high-frequency "acting expert" control. Both experts operate within the same 24-layer DeepSeek-LLM 1B transformer backbone but utilize separate parameter sets for all non-embedding layers; a global shared self-attention context fuses information for coordinated control.

• Reasoning expert: Activated at sparse keyframes, autoregressively synthesizes a sequence of latent tokens (the latent CoT) from instructions and low-rate observations.
• Acting expert: Runs at every control step, generating robot actions conditional on high-frequency observations and the most recently cached latent CoT.

This separation allows fast action updates without incurring the overhead of repeated reasoning rollout. By storing and attending to Key-Value states from the reasoning expert in a cache, the actor retrieves the latest latent conditions with $O(1)$ latency.

Figure 2: Asynchronous frequency coordination and dual-expert operation for long-horizon manipulation with periodic latent knowledge updates.

Training Paradigm

The dual system is jointly pretrained on 400K trajectories from large-scale multi-task robotic datasets (Open-X-Embodiment, DROID, RoboMIND, among others), bridging robot-specific low-level data and generic multimodal foundation knowledge. Downstream, the latent reasoning expert is first optimized to fit auxiliary-encoder-derived future latent states via an autoregressive, cosine similarity-based regression loss. Following this, the acting expert is fine-tuned with the reasoning expert weights frozen utilizing a flow matching loss, trained on action prediction under varying reasoning-action frequency ratios.

Experimental Results

Benchmarking and Ablations

LaST$_{0}$ is evaluated on 10 RLBench simulated manipulation benchmarks and six real-world robotic tasks (single and dual arm). Key outcomes include:

• Simulated tasks: LaST$_{0}$ achieves a mean 82% success rate, surpassing the best previous method by 8 percentage points, with the largest gains on tasks requiring precise object-robot interaction and long-horizon stability. Its inference runs at 15.4 Hz—over $14\times$ faster than explicit CoT approaches, closing the gap to pure-reactive methods without sacrificing reasoning quality.
• Real-world tasks: In single- and dual-arm setups, LaST$_{0}$ attains 72% mean success, outperforming strong vision-language-action and explicit CoT baselines by 13–33 points. On long-horizon execution, success rate degrades much more slowly over consecutive trials, delivering nearly $5\times$ higher task completion at the third execution.

Attention visualizations confirm that LaST$_{0}$ attends more sharply to the semantic-spatial relationships between actor and manipulated objects than either non-CoT or explicit CoT baselines.

Figure 3: Last-layer attention heatmaps across model variants, highlighting LaST$_{0}$’s focused spatial-temporal reasoning and manipulation-object coupling.

Figure 4: Ablation study on latent modality selection, token allocation, temporal horizon, and reasoning-action collaboration frequency—demonstrating the contribution of each component to performance.

Ablations demonstrate:

• Each latent modality contributes significantly, but maximal effect is seen with multimodal integration;
• Minimal token allocations suffice, with diminishing returns for greater token budgets;
• Increasing latent temporal horizon directly boosts success until a plateau, with the fast-slow system design amortizing added compute overhead;
• Joint frequency training improves robustness, making the system agnostic to frequency mismatches at deployment.

Real-World Deployment

LaST$_{0}$ exhibits robust, adaptive behaviors in practical single- and dual-arm settings, including tasks involving contact, occlusion handling, and high-precision placements. Notably, the model preserves coherent latent task representations across changing environment states and action cycles, enabling stable action plans over extended temporal windows.

Figure 5: Details of single-arm real-robot setup and diverse manipulation assets as used in the real-world evaluations.

Figure 6: Task execution sequences for single-arm and dual-arm LaST$_{0}$ deployments, illustrating the smoothness and accuracy of manipulation under latent CoT guidance.

Figure 7: Dual-arm real-robot system overview, showing synchronized perception and actuation pipelines for bimanual coordination.

Implications and Future Directions

LaST$_{0}$ contributes a significant advancement in VLA system design for robotics by enabling efficient, compact, physically-grounded internal reasoning tightly coupled to real-time control. Its mixture-of-transformers dual-system architecture sets a practical blueprint for fast, robust decision making with scalable model sizes and task domains.

The theoretical implications extend to the role of latent reasoning in generalist robotics; moving reasoning out of explicit language/image spaces and into a compressed physical latent manifold not only improves efficiency but also aligns better with the ineffable aspects of sensorimotor planning. This decoupling also provides a template for hierarchical VLA architectures in heterogeneous tasks and multi-embodiment scenarios.

Future directions include:

• Expanding the structure of latent spatio-temporal reasoning spaces (e.g., introducing richer abstraction hierarchies, incorporating more modalities like tactile input);
• Scaling both model capacity and data scope to enable robust open-world manipulation;
• Joint or RL-based optimization of reasoning and action experts, further closing the gap between internal simulation and closed-loop execution, and enabling task adaptation with delayed rewards and complex physical causality.

Conclusion

LaST$_{0}$ demonstrates that a reason-before-act paradigm, realized via token-efficient latent spatio-temporal representations and a mixture-of-transformers fast-slow dual system, confers decisive advantages in both action quality and inference efficiency for robotic VLA models. The model establishes a new state of the art in both simulated and real-world benchmarks, and suggests a practical architecture for scalable, generalist robotic intelligence that tightly integrates high-level deliberation with reactive, low-latency execution (2601.05248).