FASTER: Fast Action Sampling for Immediate Reaction

• The paper introduces a Horizon-Aware Schedule (HAS) that reallocates denoising steps, reducing Time to First Action (TTFA) to nearly one AE step beyond feature extraction.
• FASTER minimizes latency by streaming actions immediately as they complete, achieving up to 3.1× faster TTFA compared to constant scheduling methods.
• Empirical results show robust performance improvements in dynamic tasks like table tennis and manipulation, even on hardware with constrained computational resources.

Fast Action Sampling for ImmediaTE Reaction (FASTER) is a scheduling and inference approach designed to minimize reaction time in real-world deployment of vision-language-action (VLA) models, particularly those using flow-based trajectory chunking. FASTER achieves immediacy by adaptively allocating denoising computation across actions, compressing the latency for near-term control steps, and streaming actions to client systems as soon as they are available. This leads to significant improvements in reaction times for robotic systems, with demonstrated benefits in dynamic manipulation tasks such as table tennis (Lu et al., 19 Mar 2026).

1. Reaction Latency in Chunked VLA Inference

When VLA models output action sequences in fixed-length "chunks," real-time performance is constrained by both inference latency and the length of the execution horizon before a new action chunk is computed. Let $\Delta t_{\text{ctrl}}$ denote the robot’s control period, $\tau_{\text{lat}}$ the end-to-end inference latency, $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$ the discretized model delay, and $s$ the chunk (horizon) length. The time to react to a sudden external event is uniformly distributed: $$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$ with expected value: $$\mathbb{E}[T_\text{react}] = d\,\Delta t_{\text{ctrl}} + \frac{1}{2}s\,\Delta t_{\text{ctrl}}$$ Consequently, smaller $s$ and lower $\tau_{\text{lat}}$ (hence $d$) are critical to achieving low-latency responsiveness.

Time to First Action (TTFA) quantifies the interval from request to the first available action. For flow-matching samplers with $N$ integration steps for $\tau_{\text{lat}}$0-dimensional outputs and VLM feature extraction latency $\tau_{\text{lat}}$1, vanilla approaches result in: $\tau_{\text{lat}}$2 where $\tau_{\text{lat}}$3 is the per-step action expert runtime.

2. Inefficiencies of Constant Scheduling in Flow-Based VLAs

Standard flow-based VLA models generate $\tau_{\text{lat}}$4 steps of a trajectory by discretizing and integrating a velocity field from $\tau_{\text{lat}}$5 for $\tau_{\text{lat}}$6 steps, using a shared $\tau_{\text{lat}}$7 for all action indices. This forces completion of all $\tau_{\text{lat}}$8 denoising steps before any action becomes available, regardless of the ease of predicting near-term actions. As a result, the inference pipeline imposes a hard lower bound on TTFA: $\tau_{\text{lat}}$9 Even though the first action is typically the easiest to infer (most causally conditioned), spending the entire denoising budget on it is unnecessary, but forced by the rigid constant schedule. This creates an avoidable bottleneck, especially detrimental to tasks requiring low-latency reaction (Lu et al., 19 Mar 2026).

3. Horizon-Aware Scheduling in FASTER

FASTER introduces a Horizon-Aware Schedule (HAS) that reallocates the denoising budget non-uniformly across actions within the chunk, prioritizing immediate actions at the start of the output sequence. Each action index $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$0 (for $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$1) is associated with a unique "hit time" $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$2: $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$3 where $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$4 ensures the first action is denoised after the first AE step and $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$5 modulates decay.

On each global AE iteration $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$6 (with $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$7), the timestep for action $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$8 is: $$d = \lfloor\tau_{\text{lat}}/\Delta t_{\text{ctrl}}\rfloor$$9 Each action becomes eligible for streaming as soon as $s$0. Early actions are typically complete after the earliest steps; later actions receive more computational budget as inference proceeds.

During fine-tuning, HAS is applied with probability $s$1 to maintain compatibility with pre-trained weights, preventing the collapse of early-action learning.

4. FASTER Inference Pipeline and Streaming

FASTER consists of two synergistic system-level advances:

A. Single-Step Immediate Reaction

• The system executes a one-time VLM feature extraction.
• Initializes the action chunk, then iteratively applies AE updates using HAS-defined schedules.
• As soon as any action $s$2 (especially $s$3) becomes ready, it is streamed to the robotic controller, with possible early stopping if all relevant actions for the chunk have completed.
• The result is TTFA reduced to one AE step beyond feature extraction: $s$4

B. Streaming Client–Server Pipeline

• On the server, AE denoising and action streaming are interleaved, allowing immediate communication of finished actions.
• On the client, actions are inserted into a buffer and consumed at the control frequency.
• Parallelism permits the system to denoise future actions while current ones are executing, effectively hiding communication latency and compressing reaction time windows.

This streaming architecture eliminates the need to wait for an entire trajectory chunk before initiating movement, enabling real-time responsiveness even on consumer-grade hardware.

5. Quantitative Outcomes and Practical Deployments

Empirical results substantiate substantial gains from FASTER. The following table summarizes TTFA, minimal execution horizon $s$5, and expected reaction time $s$6 in milliseconds for flow-based VLA models ($s$7, X-VLA) on two GPU hardware classes:

Method	TTFA (ms)	$s$8	$s$9 (ms)
$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$0 Async (4090)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$1	3	130.0
FASTER (4090)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$2	3	112.1
$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$3 Async (4060)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$4	10	470.0
FASTER (4060)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$5	8	371.9
X-VLA Async (4090)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$6	4	180.4
FASTER (4090)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$7	2	78.1
X-VLA Async (4060)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$8	12	599.5
FASTER (4060)	$$T_{\text{react}} \sim \mathcal{U}(d\,\Delta t_{\text{ctrl}}, d\,\Delta t_{\text{ctrl}} + s\,\Delta t_{\text{ctrl}})$$9	6	229.2

FASTER consistently reduces TTFA by factors ranging from 1.3× to 3.1×, with expected reaction times decreasing by 1.2× to 2.6×, surpassing constant-schedule baselines. The advantage widens on resource-constrained hardware.

In dynamic table tennis experiments, FASTER achieved up to 93% ball return rate (mean score ≈0.93) on RTX 4090, while standard asynchronous methods reached only ≈0.65. On the less powerful RTX 4060, FASTER maintained ≈0.88 mean score against <0.5 for baselines. This demonstrates FASTER's ability to initiate racket swings promptly for correct posture at contact, while traditional pipelines produced late and suboptimal responses.

On manipulation tasks such as Pick-Beverage and Fold-Towel, FASTER achieved similar or improved completion rates and reduced episode durations by 20–30%. In simulated LIBERO and CALVIN benchmarks, long-horizon trajectory quality was essentially preserved, with success rate losses below 1%.

6. Broader Connections and Extensions

FASTER experimentally exposes constant-denoising schedules as the limiting factor for real-time flow-based action models, pinpointing computational and system-level early-exit opportunities to shrink the reaction time envelope (Lu et al., 19 Mar 2026). The temporal rescheduling logic implicitly parallels fast resampling/termination mechanisms in hierarchical RL, such as emergency action termination (EAT), which also tackle rapid plan update problems by injecting per-step reassessment (Bortkiewicz et al., 2022). However, FASTER's methods are distinguished by their focus on continuous control, index-wise ODE integration schedules, and immediate streaming.

From a sampling-theoretical perspective, FASTER's principle of immediate selection/acceptance on arrival is closely related to "stream sampling with immediate decision," which proves distributional correctness, unbiasedness, and optimality for streaming domains (Panahbehagh et al., 2021). A plausible implication is that the scheduling paradigm can generalize to other sequential decision tasks requiring fast and irrevocable on-the-fly commitments.

7. Concluding Remarks

FASTER provides a systematic, deployable solution to the reaction-time bottlenecks encountered in real-world VLA deployments using flow-based chunking. By dynamically rescheduling denoising across the chunk horizon, compressing the TTFA to the theoretical minimum, and streaming immediate actions to execution hardware, it delivers a significant step toward true real-time vision-language-action control, even under edge compute constraints, with no architectural modification or additional model distillation required (Lu et al., 19 Mar 2026).