LaST-R1: Reinforcing Action via Adaptive Physical Latent Reasoning for VLA Models

Abstract: Vision-Language-Action (VLA) models have increasingly incorporated reasoning mechanisms for complex robotic manipulation. However, existing approaches share a critical limitation: whether employing explicit linguistic reasoning that suffers from latency and discretization, or utilizing more expressive continuous latent reasoning, they are predominantly confined to static imitation learning that limits adaptability and generalization. While online reinforcement learning (RL) has been introduced to VLAs to enable trial-and-error exploration, current methods exclusively optimize the vanilla action space, bypassing the underlying physical reasoning process. In this paper, we present \textbf{LaST-R1}, a unified VLA framework that integrates latent Chain-of-Thought (CoT) reasoning over physical dynamics prior to action execution, along with a tailored RL post-training paradigm. Specifically, we propose \textbf{Latent-to-Action Policy Optimization (LAPO)}, a novel RL algorithm that jointly optimizes the latent reasoning process and the action generation. By bridging reasoning and control, LAPO improves the representation of physical world modeling and enhances robustness in interactive environments. Furthermore, an \textbf{adaptive latent CoT mechanism} is introduced to allow the policy to dynamically adjust its reasoning horizon based on environment complexity. Extensive experiments show that LaST-R1 achieves a near-perfect 99.8\% average success rate on the LIBERO benchmark with only one-shot supervised warm-up, significantly improving convergence speed and performance over prior state-of-the-art methods. In real-world deployments, LAPO post-training yields up to a 44\% improvement over the initial warm-up policy across four complex tasks, including both single-arm and dual-arm settings. Finally, LaST-R1 demonstrates strong generalization across simulated and real-world environments.

• The paper introduces LaST-R1, which jointly optimizes latent reasoning and action policies using the novel LAPO framework for enhanced performance.
• It employs an adaptive latent Chain-of-Thought mechanism to dynamically adjust reasoning length, balancing structured planning with reactive control.
• Experimental results show superior convergence, high success rates on LIBERO benchmarks, and significant improvements in real-world robotic manipulation.

LaST-R1: Reinforcing Action via Adaptive Physical Latent Reasoning for VLA Models

Motivation and Problem Statement

Current Vision-Language-Action (VLA) models for robotic manipulation benefit from recent advances in vision-LLMs and structured reasoning mechanisms (e.g., explicit linguistic or latent CoT). However, existing frameworks predominantly optimize action sequences either via imitation learning—suffering from poor generalization and lack of closed-loop adaptation—or, more recently, via reinforcement learning (RL), but only in the action space, neglecting the model's latent reasoning states. This omission limits robustness, generalization, and sample efficiency, especially in long-horizon or out-of-distribution (OOD) tasks where powerful reasoning is needed to bridge semantic perception and robust control.

The "LaST-R1" framework (2604.28192) addresses this gap by unifying adaptive latent Chain-of-Thought reasoning with physical action generation and introducing a RL paradigm—Latent-to-Action Policy Optimization (LAPO)—that jointly optimizes both reasoning and action policies. The result is a VLA model that dynamically adapts its internal cognitive horizon, achieves superior convergence, and generalizes robustly across both simulated and real-world domains.

Figure 1: LaST-R1 integrates adaptive latent reasoning with action execution, achieving improved convergence and generalization over vanilla RL baselines.

Model Architecture and Latent Reasoning Formalism

LaST-R1 adopts a multimodal transformer backbone initialized from Qwen3-VL-4B, with visual features encoded by SigLIP2-Large and semantic priors by DINOv3-based global representations. The model processes visual observations and language instructions, autoregressively generates structured latent reasoning tokens, and then decodes action token sequences for chunked execution. The latent tokens are constructed via a top-k selection of salient dimensions from the DINOv3 <CLS> token, providing a semantically dense anchor for reasoning. Action tokens are normalized, discretized, and mapped to and from continuous SE(3) action spaces using parameter-free tokenizers.

Figure 2: Overview of LaST-R1, highlighting the integration of physically grounded visual latents, latent CoT reasoning, and closed-loop RL-based post-training.

To enable efficient policy learning under RL, the architecture includes a value head for advantage estimation, supporting joint optimization of latents and actions.

The inference pipeline is orchestrated using a hybrid causal-bidirectional attention mask, enabling autoregressive latent reasoning followed by parallel, globally-informed action decoding for execution acceleration.

Latent-to-Action Policy Optimization (LAPO)

A core novelty is the LAPO framework, which unifies latent reasoning and action generation under the same RL policy gradient objective. At each decision step, the policy samples latent tokens autoregressively, emits a dynamic <latent_end> transition token when sufficient reasoning is detected, and then decodes an action chunk in parallel.

During RL rollout, the joint likelihood ratio for the entire decision step includes contributions from both the sequence of latents (modeled via an isotropic Gaussian distance between sampled and current latents) and the action tokens (via log-probabilities over the action chunk). The total LAPO loss is the sum of separate, clipped surrogate objectives for the two decision spaces, augmented with value and transition-specific losses.

Joint optimization ensures that environmental rewards inform not just the observable actions but also the structure and efficiency of latent reasoning, permitting the model to internalize more robust and transferable physical world models.

Adaptive Latent Chain-of-Thought Mechanism

LaST-R1 addresses the computational and cognitive inefficiency of static-length latent reasoning by introducing an adaptive reasoning mechanism. The model dynamically determines the optimal reasoning horizon per task instance by sampling the emission of the <latent_end> token among several candidate positions, balancing structured planning and fast reactive control.

The reasoning length selection is directly optimized via RL—sampling over candidate positions during training for exploration, and adopting a confidence-based early-exit policy (using softmax probability thresholds) during inference. This adaptivity is explicitly regularized via a standalone transition-specific policy loss.

This mechanism ensures efficient cognitive compute allocation: the model can "think longer" for complex, multi-stage tasks and "exit early" for simpler cases, resulting in both improved sample efficiency and real-time operational performance.

Figure 3: Ablations on latent representation, CoT reasoning length, and adaptive latent termination demonstrate the performance benefits of each design.

Experimental Results

LIBERO Simulation Benchmark

On all four suites of the LIBERO benchmark (Spatial, Object, Goal, Long), LaST-R1 achieves a 99.8% average success rate, outperforming both SFT and RL-based state-of-the-art (SOTA) baselines—including action-only RL methods such as $\pi_{RL}$ and SimpleVLA-RL—even under a one-shot SFT warm-up. Notably, LaST-R1 excels on the challenging LIBERO-Long suite, with an absolute gain of 5.4% over the nearest RL competitor.

In learning curve analysis, LaST-R1 converges faster and achieves higher asymptotic performance post-RL training compared to action-only baselines, especially under low-data or long-horizon conditions, directly confirming the practical utility of latent reasoning space optimization.

Figure 4: Online RL learning curves on LIBERO, evidencing faster convergence and higher asymptotic success rates for LaST-R1 with LAPO vs action-only PPO.

Ablations reveal that: DINOv3-based latent tokens yield up to 3% higher SR vs alternative compressions (convolutions, Q-Former, pooling); longer reasoning horizons increase task accuracy, with diminishing returns past 8 tokens; and adaptive reasoning termination (with 4 candidate emission positions) provides the strongest balance between accuracy and efficiency.

Real-World Robotic Manipulation

In physical deployments on four diverse manipulation tasks (single-arm and dual-arm), LaST-R1 post-RL optimization increases average performance from 52.5% to 93.75% success rate, with up to 44% improvements relative to the SFT-only policy. Real-world tasks tested include contact-rich coordination (e.g., bottle cap opening, wiping with dual arms), under severe OOD conditions such as novel objects, background distractors, and dynamic lighting.

Figure 5: Real-world execution trajectories confirm precise, robust, temporally efficient control across diverse manipulation tasks.

Figure 6: Policy robustness in highly diverse, cluttered visual environments; LaST-R1 maintains task success under severe OOD visual disturbances.

Generalization and Execution Analysis

Comprehensive OOD analysis across LIBERO suites demonstrates that, unlike classical action-only RL—which rapidly overfits and degrades on unseen configurations—LaST-R1 exhibits monotonic generalization improvements, extracting transferable semantics of spatial structure and task dynamics.

Empirically, LaST-R1 post-training reduces the average trajectory steps below even those found in expert demonstrations, confirming its ability to optimize not just for task completion but also temporal and computational efficiency.

Figure 7: LaST-R1 achieves superior temporal efficiency, frequently surpassing human/expert demonstration step counts after RL post-training.

Adaptive reasoning length analysis shows that post-LAPO, the model predominantly selects minimal reasoning steps for easier tasks and allocates longer horizons for difficult ones, validating the efficiency and cognitive flexibility of the approach.

Theoretical and Practical Implications

LaST-R1 demonstrates that explicit, reward-driven optimization of latent reasoning states (not just action sequences) endows VLA policies with improved physical grounding, abstraction, and robustness, especially notable in OOD generalization and sample efficiency. The success of the adaptive reasoning horizon further motivates research into dynamic cognitive resource allocation within embodied AI, adding a layer of self-regulatory computation previously unavailable in static architectures.

This architecture advances both theoretical understanding of RL in structured latent spaces and deployment of scalable VLA models in real-world manipulation, moving closer to policies that can self-organize their reasoning and execution to novel tasks with minimal demonstration and interaction data.

Conclusion

LaST-R1 establishes a new standard for closed-loop physical reasoning in VLA models, coupling adaptive latent Chain-of-Thought with RL-based joint optimization of cognitive and action policies. Its empirical gains highlight the necessity of internal state optimization and reasoning flexibility for robust, sample-efficient, and generalizable robotic manipulation. This work opens pathways for further integration of dynamic, adaptive reasoning paradigms in embodied AI, especially in domains requiring compositionality, zero-shot generalization, and robust execution under real-world uncertainties.