Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Abstract: Hierarchical Vision-Language-Action (VLA) models have rapidly become a dominant paradigm for robotic manipulation. It typically comprising a Vision-Language backbone for perception and understanding, together with a generative policy for action generation. However, its performance is increasingly bottlenecked by the action generation proceess. (i) Low inference efficiency. A pronounced distributional gap between isotropic noise priors and target action distributions, which increases denoising steps and the incidence of infeasible samples. (ii) Poor robustness. Existing policies condition solely on the current observation, neglecting the constraint of history sequence and thus lacking awareness of task progress and temporal consistency. To address these issues, we introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM). GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path and reducing the umber of function evaluations (NFE). LCM dynamically models executed action sequence to infer task progress and injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory. Across three simulation benchmarks, OptimusVLA consistently outperforms strong baselines: it achieves 98.6% average success rate on LIBERO, improves over pi_0 by 13.5% on CALVIN, and attains 38% average success rate on RoboTwin 2.0 Hard. In Real-World evaluation, OptimusVLA ranks best on Generalization and Long-horizon suites, surpassing pi_0 by 42.9% and 52.4%, respectively, while delivering 2.9x inference speedup.

• The paper introduces OptimusVLA, a 3.6B-parameter vision-language-action model that combines Global Prior Memory for retrieving task-level trajectory priors with Local Consistency Memory for action-history constraints.
• The model achieves state-of-the-art results across LIBERO, CALVIN, RoboTwin 2.0, and real-world bimanual tasks, including 98.6% LIBERO success, 85.0% real-world generalization success, and a 2.9× inference speedup.
• The results show that memory coverage is critical: retrieved priors reduce flow-policy function evaluations and improve efficiency, while weak coverage or overly deterministic retrieval can limit performance on novel and heavily randomized tasks.

OptimusVLA is a dual-memory augmented Vision-Language-Action (VLA) framework for robotic manipulation that targets two bottlenecks of hierarchical VLA models: the inefficiency and infeasibility risk of generative action policies, and the lack of temporal robustness caused by Markovian conditioning on only the current observation. Built on a $\pi_{0.5}$-style hierarchical backbone with a flow-matching policy, the framework introduces a Global Prior Memory (GPM) that replaces isotropic Gaussian noise with retrieved task-level priors, and a Local Consistency Memory (LCM) that injects action-history-based consistency constraints. The model totals 3.6B parameters and reports state-of-the-art results on LIBERO, CALVIN, RoboTwin 2.0, and real-world bimanual tasks, alongside a 2.9× inference speedup.

Motivation and problem formulation

The authors identify two limitations of contemporary hierarchical VLAs. First, inefficient action generation: diffusion or flow-matching policies transport an isotropic Gaussian prior to structured action distributions, a large cross-domain transformation requiring many function evaluations (NFEs) and frequently initializing generation in kinematically invalid regions. A naive fix—starting directly from an action prior—collapses diversity, so the paper frames prior initialization as a memory-driven retrieval problem instead. Second, poor temporal robustness: policies conditioned solely on $O_t$ cannot distinguish visually similar task phases (e.g., an unopened versus just-closed drawer), and concatenating long observation histories inflates latency while misaligning with single-frame pre-training distributions; existing working-memory designs require a full VLM forward pass per memory update.

The underlying generative machinery is Conditional Flow Matching (CFM) over an optimal-transport path $x_t = (1-t)x_0 + t x_1$ with constant target velocity $u_t = x_1 - x_0$. OptimusVLA keeps Gaussian noise during policy pre-training but substitutes a retrieved prior as the ODE source at inference, shortening the transport path without altering the pre-training paradigm.

Global Prior Memory

GPM comprises three components. A Prior Head (a two-layer MLP) projects mean-pooled VLM tokens into a normalized retrieval token $z_{re}$, trained with an InfoNCE objective using a Task-Pair Batch Sampler guaranteeing in-batch positives. A Memory Bank stores $M$ key-value pairs of task embeddings and full trajectories, indexed with FAISS for cosine-similarity retrieval of the top-$k$ neighbors. Retrieved trajectories are aligned to execution progress via a progress scalar $\rho_t$ and sliding windows, then moment-matched into a diagonal Gaussian prior $\mathcal{N}(\mu, \mathrm{diag}(\mathrm{Var}))$ with softmax weights over similarity scores. A Prior-Aware Sampler adapts both the injected noise scale $\lambda$ and the NFE schedule $O_t$0 to the global retrieval confidence $O_t$1: high-confidence retrievals reduce noise and denoising steps, while low-confidence scenarios gracefully fall back toward higher noise and more compute, preserving exploration on novel tasks. A session-level cache performs retrieval once per episode, keeping online overhead negligible.

Local Consistency Memory

LCM is a lightweight working memory operating purely on action sequences rather than observations. A Consistency Layer applies self-attention over the previous action chunk $O_t$2 to capture intra-chunk dependencies, and a Mamba-based Dynamic Awareness Module maintains an internal state across chunks to predict an additive consistency bias $O_t$3. This bias is added to the GPM initialization before the flow policy runs. Training regresses $O_t$4 toward the residual between the ground-truth chunk and the prior mean, $O_t$5, with a cold-start masking strategy for episode beginnings. Because LCM never invokes the VLM backbone, it adds progress awareness at negligible cost—a deliberate contrast to observation-centric memory designs such as MemoryVLA.

Simulation results

On LIBERO (500 rollouts per suite), OptimusVLA achieves a 98.6% average success rate, exceeding OpenVLA-OFT (97.1%), MemoryVLA (96.7%), and $O_t$6 (96.9%), with 99.6/99.8/98.4/96.4 on Spatial/Object/Goal/Long respectively. On CALVIN ABC→D it reaches an average completion length of 4.45, improving over reproduced $O_t$7 by 13.5% and surpassing VPP (4.29). On RoboTwin 2.0 Hard (16 tasks, 100 rollouts each), it attains 38% average success versus 29% for reproduced $O_t$8 and 25% for $O_t$9, ranking first overall; the appendix reports 30% average over all 16 tasks. Notably, absolute success rates remain low even for the best method under Hard domain randomization (e.g., 4% on Place Bread Skillet), indicating the benchmark's difficulty is not resolved by prior retrieval alone. The claimed mechanism is supported: GPM reduces NFEs from 10.0 ($x_t = (1-t)x_0 + t x_1$0) to 3.2, and ablations show removing GPM costs 3.8% on CALVIN and 9.4% on real-world generalization tasks, while removing LCM costs 1.7% on LIBERO-Long. Memory-size ablations show performance scales with bank richness (96.4% LIBERO-Long at Num=6500, k=8 versus 92.6% at k=1), confirming that overly deterministic priors hurt.

Real-world evaluation and efficiency

On a Galaxea R1 Lite bimanual platform, OptimusVLA achieves 85.0% average success on Generalization Tasks (varying lighting and scenes, 50 rollouts each) and 64.0% on Long-horizon Tasks (25 rollouts each), outperforming $x_t = (1-t)x_0 + t x_1$1 by 42.9% and 52.4% respectively—the largest reported margins in the paper. Efficiency gains are substantial: 6.5× faster inference and 3.1× fewer NFEs than $x_t = (1-t)x_0 + t x_1$2 on LIBERO, and 2.9× speedup in the real world. Training also converges faster: 97.6% on LIBERO-Goal within 18,000 steps versus roughly 26,000 steps for $x_t = (1-t)x_0 + t x_1$3 from identical initial weights. Qualitative analysis attributes this to initialization near the target manifold and to LCM resolving phase ambiguity in visually similar states.

Limitations and open questions

The paper concedes two principal constraints. GPM's effectiveness depends on memory-bank coverage: when a task departs substantially from stored trajectories, retrieved priors can be misleading and bias the policy toward suboptimal behavior—an issue directly relevant to the modest RoboTwin Hard numbers. LCM operates on fixed-length action chunks and local coherence, which may be insufficient for very long horizons, multi-stage dependencies, or delayed effects. The authors leave open adaptive memory mechanisms with online consolidation, forgetting, and uncertainty-aware retrieval, as well as end-to-end joint training of GPM, LCM, and the flow policy (the current three-stage pipeline freezes modules sequentially). An additional unstated dependency is that retrieval quality rests on the assumption that semantically similar tasks share similar action distributions, which may fail for multi-modal solution spaces.

Conclusion

OptimusVLA demonstrates that replacing the Gaussian source distribution of flow-matching policies with retrieved, confidence-adapted task priors, combined with a lightweight action-history consistency module, yields consistent gains in success rate, generalization, and inference efficiency across three simulators and real-world bimanual manipulation. The strongest evidence lies in the long-horizon and generalization suites, where temporal consistency and semantic anchoring matter most; the residual gap on heavily domain-randomized benchmarks makes clear that memory coverage remains the binding constraint on this approach.