StreamVLA: Dual-System VLA Architecture

• StreamVLA is a dual-system vision-language-action architecture that decouples high-level reasoning from continuous control for efficient robotic manipulation.
• It employs a novel Lock-and-Gated mechanism to trigger multimodal planning only at sub-task boundaries, minimizing redundant inference.
• The design achieves a 48% reduction in latency (128 ms/step) and superior performance on long-horizon tasks compared to larger models.

StreamVLA is a dual-system, parameter-efficient Vision-Language-Action (VLA) architecture designed to achieve robust, low-latency long-horizon robotic manipulation by decoupling high-level multimodal reasoning (planning) from continuous control, using a self-gated hierarchical approach to minimize redundant inference. It unifies task decomposition, goal imagination, and high-frequency action generation within a single transformer backbone, leveraging a novel "Lock-and-Gated" mechanism that conditionally triggers expensive multimodal reasoning only at sub-task boundaries, and otherwise relies on a time-invariant completion-state anchor to optimize steady-state action execution (Wu et al., 1 Feb 2026).

1. Architectural Design and Computational Flow

StreamVLA organizes its computation within a dual-system hierarchy inspired by cognitive control theory. The architecture comprises:

• System 2 ("Slow Thinking", Sparse Inference):
  • Sub-task head $\mathcal{H}_{\rm sub}$—a causal LLM (LM) for textual task decomposition.
  • Imagination head $\mathcal{H}_{\rm img}$—an Infinity-style autoregressive model for visual completion state prediction.
• System 1 ("Fast Action", Dense Inference):
  • Flow-matching action head $\mathcal{H}_{\rm act}$, outputting $K$-step action chunks conditioned on a composite embedding (current sensory, proprioceptive, and locked intent/goal imagery).
• Shared Transformer Backbone $\mathcal{T}$, ingesting multi-view visual encodings $E_v(O_t)$, proprioceptive inputs $E_p(p_t)$, and latent instruction $E_\ell(\mathcal{I})$ to produce $h_t$:

$$h_t = \mathcal{T}\bigl[E_v(O_t)\Vert E_p(p_t)\Vert E_\ell(\mathcal{I})\bigr]$$

During each control step, computation proceeds as follows:

+--------------------------------------+ | Shared Backbone 𝒯(LayerNorm+Heads) | | E_v(O_t), E_p(p_t), E_ℓ(𝓘) → h_t | +--------------------------------------+ ↙ ↓ ↘ System2: Gating System1: ℋ_sub, ℋ_img Module 𝒢 ℋ_act (Flow-Match)

System 2 is only invoked upon detection of sub-task transition; otherwise, System 1 operates recurrently, conditioned on "locked" goal and intent (Wu et al., 1 Feb 2026).

2. Lock-and-Gated Mechanism

The Lock-and-Gated mechanism is central to computational efficiency and goal stability. At each timestep, a discrepancy score $d_t\in[0,1]$ quantifies the current observation’s divergence from the locked completion state:

$$d_t = \sigma\left(\mathrm{MLP}\left(\mathrm{GlobalPool}(\mathrm{CrossAttn}(Q=o_t^{\mathrm{head}},\,KV=\hat o_{\mathrm{locked}}^{\mathrm{future}},\,\mathrm{cond}=E_\ell(s_{\mathrm{locked}})))\right)\right)$$

with gating variable

$$g_t = \begin{cases} 1, & d_t \le \tau \quad \text{(trigger re-planning)}\ 0, & d_t > \tau \quad \text{(keep current plan)} \end{cases}$$

where $\tau = 0.5$ is held-out validated.

• If $g_t = 1$: Enter Full Mode; invoke System 2 to (re-)generate sub-task $s_t$, and completion state $S_c$ (locking both until the next transition).
• If $g_t = 0$: Enter Skip Mode; System 1 acts using cached $s_{\mathrm{locked}}$, $S_c$.

Inferential pseudocode:

Initialize t←0; force g0←1 for each timestep t: h_t ← Backbone(O_t, p_t, 𝓘) if g_{t-1}=1: s_t ← ℋ_sub(h_t) S_c_t ← ℋ_img(h_t, s_t) s_locked, S_c_locked ← s_t, S_c_t Evaluate d_t via 𝒢(O_t^{head}, S_c_locked, s_locked) if d_t ≤ τ: g_t←1 else g_t←0 a_t ← ℋ_act(h_t, s_locked, S_c_locked)

This structure ensures that both textual plan and goal state remain invariant during sub-task execution, only updating at genuine task boundaries (Wu et al., 1 Feb 2026).

3. Task Decomposition, Goal Imagination, and Anchoring

Sub-task boundaries are detected and leveraged via explicit language modeling, while sub-goal anchoring is realized through a predicted completion state image, not a generic future frame:

• Sub-task LM loss:

$$\mathcal{L}_{\mathrm{sub}} = -\,\sum_{i=1}^{|s|}\log p_\theta\left(s_i \mid h_t,s_{<i}\right)$$

• Image-head loss (bitwise AR):

$$\mathcal{L}_{\mathrm{img}} = -\,\sum_{b=1}^{B}\log p_\theta\left(q_b \mid h_t,s_t,q_{<b}\right)$$

Ground-truth sub-task intervals and completion images are derived from demonstration traces using semi-automatic labeling or simulation rules.

Time-invariant goal anchoring means that, once $S_c$ (completion state) is locked, it is held constant for all steps of the sub-task. This ensures that policy output

$\pi_\theta(\mathbf{a}_t \mid h_t, S_c)$

remains robust to execution rate variations within the sub-task, and

$$\pi_\theta(\mathbf{a}_t \mid h_t, S_c) \approx \pi_\theta(\mathbf{a}_{t'} \mid h_{t'}, S_c)$$

for $t, t'$ inside the same segment, modulo current observation. The completion state is a semantic anchor (e.g., "drawer closed") rather than a timestamped prediction, enhancing sub-goal stability under real-world disturbances (Wu et al., 1 Feb 2026).

4. Conditional Flow Matching for Continuous Control

Action generation in StreamVLA uses Conditional Flow Matching (CFM) to enable efficient, non-autoregressive chunked action rollout:

• Forward (diffusion) process: $\mathbf{a}_1$ is expert action, $\mathbf{a}_0 \sim \mathcal{N}(0, I)$.
• Score prediction: At diffusion time $u \in [0,1]$, the model synthesizes $\phi_u(\mathbf{a}_0)$ and predicts $v_u(\phi_u(\mathbf{a}_0) | C_t)$.
• Loss:

$$\mathcal{L}_{\mathrm{act}} = \mathbb{E}_{u,\mathbf{a}_0,\mathbf{a}_1}\left\| v_u(\phi_u(\mathbf{a}_0)\mid C_t)- (\mathbf{a}_1 - \mathbf{a}_0)\right\|^2$$

At inference, the action head executes in closed-form (ODE solver-based) without the need for autoregressive decoding, unlocking substantial speed gains.

Due to gating, System 2 is bypassed in 72% of steps, with the "fast path" head producing up to $K$ continuous actions per activation, capitalizing on long stretches of unperturbed execution (Wu et al., 1 Feb 2026).

5. Training Procedure and Multi-Task Optimization

Training StreamVLA employs a multi-task loss:

$$\mathcal{L}_{\mathrm{total}} = \mathcal{L}_{\mathrm{act}} + \lambda_{\mathrm{sub}}\mathcal{L}_{\mathrm{sub}} + \lambda_{\mathrm{img}}\mathcal{L}_{\mathrm{img}} + \lambda_{\mathrm{gate}}\mathcal{L}_{\mathrm{gate}}$$

with $\mathcal{L}_{\mathrm{gate}}$ as the binary cross-entropy (BCE) between $d_t$ and ground-truth transition labels. Empirical scaling:

• $\lambda_{\mathrm{sub}} = 0.1$
• $\lambda_{\mathrm{img}} = 0.1$
• $\lambda_{\mathrm{gate}} = 1.0$

A two-stage curriculum is used:

1. Freeze backbone and action head; optimize the sub-task and imagination heads.
2. Joint fine-tuning of all modules.

Sub-task boundaries and completion images are derived from demonstration mining, ensuring the gating and decomposition network's supervision is well-aligned with policy execution (Wu et al., 1 Feb 2026).

6. Latency, Computational Efficiency, and Empirical Results

The gated reasoning mechanism yields significant latency reductions and robust empirical performance:

• Autoregressive heads skipped: 72% of all control steps.
• Latency: Full-reasoning baseline ≈ 244 ms/step; StreamVLA ≈ 128 ms/step (48% reduction).
• FLOPs savings: Proportional to reduction in AR decoding, since backbone and fast head remain always active.

Benchmark Results:

• LIBERO (long-horizon manipulation): Spatial 99.2%, Object 99.4%, Goal 98.6%, Long 96.6%; overall avg. 98.5%. Surpasses 7B-param baselines by ∼1.4% using only 3B parameters.
• RoboTwin 2.0: Easy 71.3% (vs. 62.7%); Hard 37.2% (vs. 26%).
• Real-world (AgileX Piper):
  • Spelling: 90%
  • Insertion: 70%
  • Interference Spelling: 55%
  • (vs. next best methods 40–45%, 35%, 10–15%)

Natural recovery: When human perturbations occur and $d_t$ spikes, $g_t \rightarrow 1$ triggers System 2, resulting in re-planned intent and recovery without explicit hand-coded interventions.

7. Contextual Significance and Comparison

StreamVLA's approach is distinct from other streaming vision-language architectures (e.g., StreamingVLM (Xu et al., 10 Oct 2025), StarStream (Zhang et al., 19 Aug 2025)) in two principal respects:

1. Dual-system gating tied to semantic task structure, rather than uniform streaming over dense sensory streams.
2. Explicit use of completion-state goal imagination, yielding time-invariant sub-goal anchors that decouple planning and control, in contrast to traditional rolling-window attention or continuous vision-language token streaming.

A plausible implication is that time-invariant semantic anchoring and conditional flow-matching can generalize to other domains requiring temporally-extended, goal-directed behavior with minimal reasoning overhead.

Summary Table: StreamVLA Key Metrics and Features

Feature	Value/Description	Source
Skipped reasoning steps	72%	(Wu et al., 1 Feb 2026)
Inference latency	128 ms/step (vs. 244 ms baseline)	(Wu et al., 1 Feb 2026)
LIBERO success rate	98.5% (avg. across tasks)	(Wu et al., 1 Feb 2026)
Parameters	3B (vs. 7B baselines)	(Wu et al., 1 Feb 2026)
Empirical advantage	+1.4% over best baseline (LIBERO)	(Wu et al., 1 Feb 2026)
Recovery on real interference	Natural/reset-free, via gating	(Wu et al., 1 Feb 2026)

StreamVLA demonstrates that self-gated, completion-state-anchored hierarchical VLA models can achieve SOTA long-horizon manipulation performance with sharply reduced computation by selectively invoking high-level reasoning only when sub-task transitions or disturbances are detected, representing a notable advance in efficient multimodal robotic policy design (Wu et al., 1 Feb 2026).