VLASH: Real-Time VLAs via Future-State-Aware Asynchronous Inference (2512.01031v1)

Abstract: Vision-Language-Action models (VLAs) are becoming increasingly capable across diverse robotic tasks. However, their real-world deployment remains slow and inefficient: demonstration videos are often sped up by 5-10x to appear smooth, with noticeable action stalls and delayed reactions to environmental changes. Asynchronous inference offers a promising solution to achieve continuous and low-latency control by enabling robots to execute actions and perform inference simultaneously. However, because the robot and environment continue to evolve during inference, a temporal misalignment arises between the prediction and execution intervals. This leads to significant action instability, while existing methods either degrade accuracy or introduce runtime overhead to mitigate it. We propose VLASH, a general asynchronous inference framework for VLAs that delivers smooth, accurate, and fast reaction control without additional overhead or architectural changes. VLASH estimates the future execution-time state by rolling the robot state forward with the previously generated action chunk, thereby bridging the gap between prediction and execution. Experiments show that VLASH achieves up to 2.03x speedup and reduces reaction latency by up to 17.4x compared to synchronous inference while fully preserving the original accuracy. Moreover, it empowers VLAs to handle fast-reaction, high-precision tasks such as playing ping-pong and playing whack-a-mole, where traditional synchronous inference fails. Code is available at https://github.com/mit-han-lab/vlash

• The paper introduces future-state aware asynchronous inference for VLAs by conditioning predictions on the future execution state, reducing latency and improving control stability.
• It leverages state rollforward, fine-tuning augmentation, and action quantization to enhance data efficiency and accelerate robotic actions.
• Experimental results show up to a 30.5% accuracy improvement over naïve asynchronous methods and up to 2.03× speedup in both simulated and real-world scenarios.

VLASH: Real-Time Vision-Language-Action via Future-State-Aware Asynchronous Inference

Motivation and Background

Vision-Language-Action models (VLAs) have demonstrated substantial progress in robotic generalization and task performance, with architectures like $\pi_{0.5}$, Gemini, and Gr00t excelling across diverse, real-world tasks. Despite this, deployed VLA systems often rely on synchronous inference paradigms that enforce sequential model inference and action execution. This design induces significant action latency, visible as stalls or delayed reactions—even in demonstration videos, fluidity is achieved by artificially speeding up footage. The resulting limitation is critical for tasks demanding dynamic, continuous interactions, where both perceiving and acting in real time are essential.

Recent proposals advocate asynchronous inference as an alternative: the robot executes current actions while inferring the next chunk, seeking to overlap action execution and model inference. However, asynchronous inference introduces a temporal misalignment between when actions are predicted (based on current states) and when they are actually executed after a delay induced by inference ($\Delta$). This prediction-execution misalignment degrades control stability and accuracy.

Figure 1: Prediction-execution misalignment in asynchronous inference; actions predicted for $[t, t+K)$ are executed on $[t + \Delta, t + \Delta + K)$, inducing unstable behavior if uncorrected.

VLASH Framework

VLASH addresses this misalignment by introducing future-state awareness to the robotic policy. Instead of conditioning the policy on the current robot state $s_t$, VLASH conditions inference on the state at execution time $s_{t+\Delta}$, computed by rolling forward the known robot state through the already-issued actions. This approach ensures that proposed actions are aligned to the execution-time robot configuration, significantly improving stability and responsiveness.

The core implementation is distinguished by:

• State Rollforward: Efficient computation of the execution-time state using available proprioceptive data and the deterministic action chunk scheduled during inference.
• Fine-Tuning Augmentation: Modified data curation during fine-tuning, employing temporal offsets such that a shared observation $o_t$ is paired with future robot states $s_{t+\delta}$ and corresponding actions. This forces the VLA to attend to state input, counteracting typical model underutilization of state data and increasing compatibility across hardware with varying inference delays.

  Figure 2: VLASH contrasts with synchronous and naïve asynchronous inference; VLASH achieves aligned, smooth actions by rolling robot state forward to the point of execution.

Efficient Training and Action Quantization

VLASH exploits shared observation sequence packing during fine-tuning, increasing data efficiency by enabling multiple offset branches from a single observation, dramatically reducing compute requirements per training sample. Block-sparse attention masks ensure offset independence while maximizing observation reuse.

Figure 3: Attention pattern for offset-based efficient fine-tuning; shared observation tokens are reused across multiple offset branches, improving training efficiency.

To further accelerate real-world robotic actions, VLASH introduces action quantization, grouping micro-actions into macro-actions for coarser motion granularity. This reduces execution steps, offering significant speedup with minimal impact on task completion for most benchmarks.

Figure 4: Action quantization: fine-grained actions (gray) are grouped into macro-actions (black), yielding accelerated execution.

Experimental Evaluation

VLASH is rigorously validated on both simulated and physical robot platforms:

• Kinetix (Dynamic Simulation): VLASH matches synchronous inference upper bounds for success rate even as execution horizon and inference delay grow, outperforming Real-Time Chunking (RTC) and naïve async baselines. For delay $\Delta=4$, VLASH delivers a $30.5\%$ accuracy improvement over naïve async.

  Figure 5: VLASH maintains high success rate under increasing execution horizons $K$ and inference delays $\Delta$ on Kinetix benchmark.

• LIBERO (Generalization): VLASH is evaluated with both $\pi_{0.5}$ and SmolVLA across four sub-benchmarks, demonstrating up to $2.03\times$ speedup and $17.4\times$ reduction in perception-to-action reaction latency over synchronous inference, with competitive or superior success rates. Notably, VLASH retains synchronous performance under zero delay, evidencing no sacrifice in baseline efficiency.
• Real-World Manipulation: On Galaxea R1 Lite and LeRobot SO-101, VLASH yields smoother, faster, and more accurate manipulation across multiple tasks. Action quantization further increases execution speed with only minor decreases in average score percentage, demonstrating control over the speed-accuracy trade-off.

  Figure 6: VLASH improves both score percentages and task completion times across three real-world robotic manipulation tasks compared to synchronous and naïve async baselines.

Implications and Future Developments

VLASH demonstrates that future-state aware asynchronous inference enables VLAs to execute highly dynamic, physically interactive tasks that are simply infeasible with synchronous protocols. The framework requires no changes to model architecture, thus serving as a drop-in solution for current and future VLA models. This has practical ramifications for deploying VLAs in scenarios demanding real-time perception and low-latency reaction, such as human-robot interaction, robotic sports, and fast-paced industrial automation.

From a theoretical perspective, VLASH substantiates the necessity of compensating for prediction-execution misalignment, a core challenge in embodied learning with compute-constrained policies. The fine-tuning augmentation technique potentially generalizes to other temporal tasks, and future directions may address even more complex environmental rollforward for non-deterministic settings, or further compression acceleration via adaptive quantization strategies.

Conclusion

VLASH introduces a principled framework for enabling smooth, accurate, and fast-reaction asynchronous inference in Vision-Language-Action models via future-state awareness and efficient fine-tuning. It consistently achieves superior or matched performance and significant latency reduction across both simulated and real-world benchmarks, expanding the feasible deployment domains for VLA-driven robotics. VLASH stands as a critical milestone towards scalable, responsive, and practical embodied AI systems.

(2512.01031)