$μ$VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models

Abstract: Vision-language-action (VLA) models predict chunks of future actions from the current observation, an assumption that fails under partial observability, where decisions depend on information no longer visible. Existing memory-augmented VLAs simultaneously introduce recurrence, retrieval, compression modules, auxiliary objectives, hierarchical memory, or task-specific architectural changes, so the contribution of recurrence itself remains entangled with surrounding machinery. We present a controlled isolation study of recurrence in a strong pretrained VLA backbone. Our formulation augments the transformer with a small set of learnable memory tokens carried across timesteps and updated through self-attention, trained end to end with truncated backpropagation through time, with no auxiliary losses and no architectural changes. We instantiate this as $μ$VLA, a family of OpenVLA-OFT variants parameterized by memory width m, TBPTT length K, and the memory update rule (cross-step gradients or a detached EMA), so that recurrence is the only varying factor. On MIKASA-Robo, $μ$VLA improves average success rate on five training tasks from 0.42 to 0.84 at the strongest setting and reaches 0.23 on held-out tasks with the same memory structure versus 0.07 for the memoryless baseline. On tasks requiring different memory structure, performance remains near baseline. On LIBERO, the strongest recurrent variant achieves 96.2% average success, indicating no regression under full observability. We interpret these results as a calibration of the capability envelope of minimal in-backbone recurrence, identifying the regime in which it is sufficient and the regime where additional memory structure is required. Demos and videos can be found in https://avanturist322.github.io/mu-vla/.

• The paper demonstrates that adding a small recurrent memory token bank to a pretrained VLA transformer significantly improves performance in partially observable manipulation tasks.
• It employs truncated backpropagation through time with attention masking and a round-robin episodic dataloader to isolate pure recurrence effects.
• Experimental results on benchmarks like MIKASA-Robo show dramatic gains in cue-recall tasks while preserving performance in fully observable regimes.

$\mu$VLA: Isolated Recurrent Memory in Vision-Language-Action Models for Partially Observable Manipulation

Introduction and Motivation

Vision-language-action (VLA) models, such as OpenVLA, OpenVLA-OFT, RT-1, and RT-2, have become the dominant paradigm for generalist robot control: they cast multimodal robotic perception and actuation as next-token prediction over fused image, language, and proprioception sequences. Standard VLAs operate under a unifying assumption: the current observation (optionally with a small history window) is a sufficient statistic for action selection. However, in partially observable scenarios—where critical information is transient, occluded, or set early in the episode—a Markovian policy is fundamentally limited.

While recent work integrates various memory mechanisms (recurrent modules, external retrieval, chunked-KV caches, auxiliary objectives, hierarchical state) into VLAs, these systems typically combine several enhancements simultaneously. As a result, the singular contribution of recurrence becomes inseparable from entangled components. The paper “$μ$VLA: On Recurrent Memory for Partially Observable Manipulation in VLA Models” (2606.12497) introduces a tightly controlled isolation study to measure the effects of pure in-backbone recurrence, free from other architectural confounders.

Method: Minimal Recurrent Augmentation to a Pretrained VLA

$μ$VLA is formulated by inserting a small, learnable bank of memory tokens into a pretrained VLA backbone (OpenVLA-OFT). These tokens are positioned just after vision and proprioception tokens and before language and action tokens in the model input sequence. At every environment timestep, memory tokens from the previous step are carried forward, allowing information to flow recurrently via the transformer’s self-attention.

The recurrence is trained end-to-end using truncated backpropagation through time (TBPTT) over windows of $K$ steps, with no auxiliary losses, retrieval modules, compression, or architectural changes. Key practical interventions include:

• Attention-mask guard: Memory tokens are restricted from attending to action token outputs, preventing trivial action-copying as a degenerate solution.
• Round-robin episodic dataloader: Ensures stepwise temporal continuity per-episode in each batch slot, preserving the order required for recurrence.
• Receding-horizon inference: Rather than open-loop chunk rollout, the policy is queried with every environment step, keeping state updates synchronized with the control loop.
• EMA alternative: An exponential moving average (EMA) update rule, detached from the gradient graph, serves as a non-learned recurrent baseline.

The model family is parameterized by memory width ($m$), TBPTT truncation length ($K$), and memory update rule (full TBPTT vs. EMA). This strict control enables attribution of observed effects solely to recurrence.

Figure 1: TBPTT training of $\mu$VLA—memory tokens are recurrently unrolled, with gradients flowing through $K$ steps and detached at window boundaries.

Empirical Evaluation

Benchmarks

• MIKASA-Robo: A 23-task benchmark specifically designed for memory-dependent tabletop manipulation under partial observability, with tasks categorized as cue-recall, occlusion-tracking, and sequential/predictive memory.
• LIBERO: A fully observable control suite to ensure the recurrence channel does not cause regression in Markovian settings.

Main Experimental Questions:

1. Fine-tuning recurrence: Can recurrence be introduced to a memoryless pretrained VLA via fine-tuning alone?
2. Cross-step gradient flow: Does TBPTT length influence the quality of learned recurrence?
3. Robustness under memory-semantic distribution shift: Is the learned recurrent state transferable to tasks with different or unseen memory requirements?
4. Impact on MDP/fully observable regimes: Does the recurrence degrade performance where memory is unnecessary?

Results

Effect of Recurrence on Partial Observability

Augmenting OpenVLA-OFT with $m=64$ recurrent memory tokens and training with TBPTT ($K=2$) raises the five-task MIKASA-Robo training average from $μ$0 (reference) to $μ$1 (Table~\ref{tab:main} in the paper). The effect is concentrated in cue-recall tasks, e.g., RememberColor5 improves from $μ$2 to $μ$3.

Figure 2: Memory dynamics on RememberColor5 ($μ$4): sharp state transitions correspond to latent phase changes in the environment logic.

TBPTT Truncation Analysis

A short TBPTT window ($μ$5) yields substantially higher cue-recall performance compared to $μ$6 (removes cross-step gradients) or $μ$7 (overly diffuse signal). A detached EMA update provides only modest gains, tracking $μ$8 but trailing $μ$9 on required memory tasks.

Figure 3: Chunk-length sweep on key MIKASA-Robo tasks; receding-horizon (chunk=1) maximizes memory gains, with rapid collapse when chunking is increased.

Generalization

On held-out tasks with memory semantics matched to the training set, $μ$0 lifts SR from $μ$1 (memoryless baseline) to $μ$2; on novel semantics (such as tasks with unseen memory requirements), recurrence provides only weak benefits. Generalization to longer delay or OOD-cue tasks confirms the learned memory is robust within the training regime, but degrades outside its support.

Figure 4: Generalization probes on RememberColor5. $μ$3 dominates in-distribution phase-length, but its performance declines sharply when the delay exceeds the training window.

Causal Probing and Ablations

• Memory noise/freeze interventions: Blanking the recurrent state or freezing it to its initial value at test time collapses cue-recall performance, verifying that decisions depend on dynamic state encoding, not memorized visual/trajectory templates.
• Chunk-length and phase-length sweeps: Recurrent memory is effective only under closely synchronized memory-update cadence; open-loop chunked inference or tasks requiring very long retention exceed the capacity of minimal recurrence.

Figure 5: Memory intervention on the five MIKASA-Robo training tasks.

Fully Observable Tasks (LIBERO)

On LIBERO, the strongest $μ$4VLA configuration matches or slightly exceeds the memoryless OpenVLA-OFT baseline ($μ$5 success), with no evidence of regression. The EMA variant trails on some suites, and using open-loop chunk inference severely degrades long-horizon performance due to train-test mismatch.

Mechanistic Analysis

Visualization of memory token norms, cosine dynamics, and attention rollouts (memory-to-vision pathways) shows that the learned recurrent state undergoes sharp, interpretable reorganizations at physically meaningful event boundaries (e.g., cue disappearance, action phase transitions). Attention overlays confirm that, before occlusion or masking, the memory tokens tightly attend to critical regions (e.g., color lamp or cue grid), then diffuse afterwards—mirroring state consolidation.

Figure 6: Memory-to-vision attention rollout on RememberColor5 ($μ$6); attention peaks on the cue grid when visible, matching the observed memory write event.

Theoretical Implications

The results recalibrate expectations for in-backbone recurrence in large VLA models:

• Fine-tuning suffices: Introducing recurrence during fine-tuning alone produces large gains in partial observability, with no need to re-pretrain the backbone.
• Minimal state, qualitative shift: Even a small recurrent state (a handful of tokens) fundamentally extends the model’s control regime, enabling tasks that are intractable for feedforward or limited-context models.
• Strong, but bounded envelope: Benefits do not transfer across semantic task boundaries or extremely long horizons—pointing to the need for more structured memory or hybrid approaches (e.g., episodic retrieval, hierarchical state).

Open Issues and Future Directions

• Scaling recurrent memory: Increasing memory bandwidth ($μ$7) or truncation length ($μ$8) beyond small values may support longer retention and more abstract representation, with computational and stability trade-offs.
• Integration with structured memory: Hybridizing pure recurrence with external memory modules, retrieval policies, or hierarchical composition could bridge the generalization gaps seen in out-of-distribution partial observability.
• Mechanistic interpretability: The observed alignment of memory reorganization with latent task phases raises the question of whether learned recurrent-key events can act as cheap, learned keyframe detectors—eliminating the need for explicit query-based selection as used in retrieval-augmented designs.
• Benchmarks: The work further motivates suite design (like MIKASA-Robo) that enables fine-grained probing of memory and partial observability, a crucial step for progress in real-world long-horizon robotics.

Conclusion

This work demonstrates that minimal, isolated recurrence—implemented as a learned memory token bank within a pretrained VLA transformer—enables substantial improvements on memory-intensive, partially observable manipulation tasks. The protocol’s careful isolation of recurrence effects provides a new calibration point for vision-language-action research: recurrence per se is already powerful, and future designs must quantify which enhancements (retrieval, external storage, hierarchical planning) provide additional value beyond this strong baseline. Recurrence, when introduced in a controlled and interpretable manner, expands the operational regime of VLA models and sets a clear target for future research in embodied memory policy learning.

Figure 7: Memory dynamics on RememberShapeAndColor3x3 ($μ$9); two-phase write patterns correspond to sequential cue exposure and action execution.