---
title: 'VPWEM: Visuomotor Memory Architecture'
url: https://www.emergentmind.com/topics/vpwem
type: topic
---

# VPWEM: Visuomotor Memory Architecture

Searching arXiv for the exact VPWEM paper and nearby usage.
VPWEM denotes **“Visuomotor Policy with Working and Episodic Memory”**, a framework for imitation-learned robotic control designed for **non-Markovian** tasks in which the current observation does not contain all information needed for optimal action selection. The framework combines a fixed-size sliding window of recent observations as **working memory** with a compact learned summary of older observations as **episodic memory**, and instantiates this design on **diffusion policies** so that action generation conditions on both short-term and episode-wide information while maintaining nearly constant memory and computation per step [2603.04910].

## 1. Definition and problem setting

VPWEM is formulated in a POMDP,
\[
\mathcal{M}=(\mathcal{S}, \mathcal{A}, \mathcal{P}, \mathcal{R}, \Omega, \mathcal{O}),
\]
where the policy should in principle depend on the full history
\[
\pi_\theta(a_t \mid h_t), \qquad h_t = o_{\le t} = (o_0,\dots,o_t).
\]
In practice, most visuomotor policies use only a truncated recent history,
\[
\pi_\theta(a_t \mid o_{t-L:t}),
\]
which is often sufficient for nearly Markovian manipulation tasks but fails when task success depends on hidden past events, latent goals, or temporally distant cues [2603.04910].

The framework is motivated by two limitations of standard visuomotor and diffusion policies. First, they typically condition on a single frame or a very short history such as 2, 6, or 10 frames. Second, simply enlarging the context window is computationally costly because Transformer attention over length \(L\) scales as \(O(L^2)\), and longer histories can encourage reliance on nuisance correlations and “copycat” behavior. VPWEM addresses this by retaining explicit recent observations and compressing the rest of the episode into a fixed-size learned memory [2603.04910].

This design places VPWEM in the class of **non-Markovian visuomotor policies** rather than standard short-context behavior cloning systems. A plausible implication is that the framework is intended less as a replacement for local reactive control than as a memory augmentation for policies deployed in partially observable manipulation settings.

## 2. Memory structure: working memory and episodic memory

The central architectural distinction in VPWEM is between **working memory** and **episodic memory**. Working memory is a fixed-size sliding window over the most recent observation tokens,
\[
w_t \triangleq o_{t-L:t} = \mathrm{concat}(o_{t-L+1},\dots,o_t)\in \mathbb{R}^{L\times D}.
\]
This preserves fine-grained local temporal information for immediate control [2603.04910].

Episodic memory is a fixed number of learned memory tokens summarizing all older observations outside that window. Older observations,
\[
o_{\le t-L} = \mathrm{concat}(o_0,o_1,\dots,o_{t-L}),
\]
are not stored verbatim. Instead, when an observation token \(o_\tau\) leaves the working-memory window, it is augmented with positional information,
\[
f_\tau = o_\tau + PE(\tau)\in\mathbb{R}^D,
\]
and inserted into an observation cache,
\[
\mathcal{C}_f \leftarrow \mathcal{M}(\mathcal{C}_f \cup f_\tau),
\]
where \(\mathcal{M}\) is the cache management policy and the cache has maximum size \(S\). The resulting episodic memory output is
\[
e_\tau \in \mathbb{R}^{M\times D}.
\]
The policy then conditions action generation on both \(w_t\) and \(e_\tau\) [2603.04910].

At the observation level, VPWEM processes multi-modal inputs including RGBD observations from \(N_c\) cameras and low-dimensional proprioceptive states. Camera observations are encoded into visual features \(o_{I,t}\in\mathbb{R}^{D_I}\), proprioception into \(o_{P,t}\in\mathbb{R}^{D_P}\), and these are concatenated and passed through an MLP to form a frame token \(o_t\in\mathbb{R}^D\) [2603.04910].

This separation of memory roles suggests a specific inductive bias: recent observations are treated as high-fidelity control context, whereas distant history is treated as content to be compressed for long-horizon retrieval.

## 3. Contextual memory compressor

The technical core of VPWEM is a **Transformer-based contextual memory compressor** that recursively converts out-of-window observations into a fixed number of episodic memory tokens. The compressor uses trainable query tokens \(q\in\mathbb{R}^{M\times D}\), an observation cache \(\mathcal{C}_f\), and one summary-token cache \(\mathcal{C}_{q,n}\) for each compressor block [2603.04910].

The initial query state is
\[
q_{1,\tau} = q.
\]
The observation cache and summary cache are concatenated as
\[
\bar{\mathcal{C}_f} = \mathrm{concat}(\mathcal{C}_f)\in\mathbb{R}^{S\times D},
\]
\[
\bar{\mathcal{C}_{q} = \mathrm{concat}(\mathcal{C}_{q})\in\mathbb{R}^{S\times M\times D}.
\]
The compressor then applies, for each layer \(n\),
\[
x_{1,n} = q_{n,\tau} + attn(q_{n,\tau}Q_s,\ \bar{\mathcal{C}_{q,n}K_s,\ \bar{\mathcal{C}_{q,n}V_s),
\]
\[
x_{2,n} = x_{1,n} + attn(x_{1,n}Q_c,\ \bar{\mathcal{C}_{f}K_c,\ \bar{\mathcal{C}_{f}V_c),
\]
\[
q_{n+1,\tau} = x_{2,n} + MLP(x_{2,n}),
\]
and outputs
\[
e_{\tau} = MLP(q_{N,\tau}).
\]
The operator \(attn(Q,K,V)\) refers to standard Transformer attention [2603.04910].

The first attention stage performs self-attention over cached summary tokens; the second performs cross-attention over cached historical observations. Operationally, the compressor updates online as tokens leave the working-memory window: the token is positionalized, inserted into the observation cache, processed by the compressor, and the resulting layerwise query tokens are stored back into summary caches,
\[
\mathcal{C}_{q,n} \leftarrow \mathcal{M}(\mathcal{C}_{q,n}\cup q_{n,\tau}).
\]
Because working-memory length \(L\), episodic-memory size \(M\), and cache size \(S\) are fixed, the paper characterizes memory and compute per step as **nearly constant** [2603.04910].

An important implementation detail is that cached \(f_\tau\) and \(q_{n,\tau}\) are **detached from the computational graph**. This ensures that historical information is propagated only via summary tokens, reduces memory consumption, and is described as mitigating overfitting [2603.04910].

## 4. Diffusion-policy instantiation and learning objective

VPWEM is instantiated on diffusion policies, specifically **DP** and **MaIL**, so that action generation is conditioned on both short-term working memory and long-term episodic memory. Conceptually, the policy is written as
\[
\pi_\theta(a_t \mid w_t, e_\tau),
\]
or, for chunked actions,
\[
\pi_\theta(a_{t:t+H} \mid w_t, e_\tau).
\]
The diffusion backbone uses a Transformer decoder architecture and applies a cross-attention mask for conditioned action-chunk generation [2603.04910].

Given a clean action chunk \(\mathbf{a}_0 = a_{t:t+H}\), the forward diffusion process is
\[
\mathbf{a}_k = \sqrt{\bar{\alpha}_k}\,\mathbf{a}_0 + \sqrt{1-\bar{\alpha}_k}\,\epsilon,
\]
where \(\epsilon\sim\mathcal{N}(\mathbf{0},\mathbf{I})\). The denoiser predicts injected noise conditioned on noisy action, working memory, episodic memory, and diffusion timestep,
\[
\epsilon_\theta(\mathbf{a}_k, w_t, e_\tau, k).
\]
Training minimizes the DDPM-style denoising loss
\[
\mathcal{L} = \mathbb{E}_{(\cdot,\mathbf{a}_0),\epsilon} \left\| \epsilon - \epsilon_\theta\!\left( \sqrt{\bar\alpha_k}\mathbf{a}_0 + \sqrt{1-\bar\alpha_k}\epsilon,\, w_t,\, e_\tau,\, k \right) \right\|_2^2.
\]
The compressor is trained **end-to-end jointly** with this objective; it is not pretrained separately [2603.04910].

At inference time, VPWEM encodes the newest observation, updates the working-memory queue, inserts any evicted token into the observation cache, updates the summary caches by running the compressor, obtains episodic memory \(e_\tau\), and then performs reverse diffusion conditioned on \(w_t\) and \(e_\tau\). The reverse step is given in the paper as the standard DDPM update,
\[
\mathbf{a}_{k-1} = \frac{1}{\sqrt{\alpha_k} \left( \mathbf{a}_k - \frac{1-\alpha_k}{\sqrt{1-\bar{\alpha}_k} \epsilon_\theta(\mathbf{a}_k,w_t,e_\tau,k) \right) + \sigma_k \mathbf{z},
\]
with \(\mathbf{z}\sim\mathcal{N}(\mathbf{0},\mathbf{I})\) [2603.04910].

For **DP-VPWEM**, the reported settings are: DDPM with 50 sampling steps, action chunk length \(H=8\), temperature \(1.0\), 8 layers, embedding dimension \(D=256\), compressor depth \(N=2\), short-term tokens \(L=2\), long-term tokens \(M=2\), maximum cache size \(S=8\), dropout ratio of memory tokens \(0.3\), batch size \(64\), learning rate \(1\mathrm{e}{-4}\), EMA rate \(0.999\), and AdamW with \(\beta_1=0.9\), \(\beta_2=0.999\) [2603.04910].

## 5. Empirical evaluation and reported performance

VPWEM is evaluated on three benchmark families: **MIKASA**, **MoMaRT**, and **Robomimic**. MIKASA includes the memory-intensive tabletop tasks **ShellGameTouch-v0** and **RememberColor3-v0**. MoMaRT includes five long-horizon mobile-manipulation tasks: **Table Setup from Dishwasher**, **Table Setup from Dresser**, **Table Cleanup to Dishwasher**, **Table Cleanup to Sink**, and **Unload Dishwasher to Dresser**. Robomimic includes mostly near-Markovian tasks such as **Square** and **Transport**, together with **Long-Horizon Square**, where the robot must remember the initial block configuration [2603.04910].

The baseline set includes **RNN policy**, **Diffusion Policy (DP)**, **DP-PTP**, and **MaIL**. On MIKASA, the paper also reports comparisons against VLA baselines including **Octo-small**, **OpenVLA**, **\(\pi_0\)**, **SpatialVLA**, **CronusVLA**, and **MemoryVLA** [2603.04910].

The paper states that VPWEM “outperforms state-of-the-art baselines including diffusion policies and vision-language-action (VLA) models by more than 20% on the memory-intensive manipulation tasks in MIKASA” and achieves “an average 5% improvement on the mobile manipulation benchmark MoMaRT” [2603.04910]. On Robomimic, VPWEM performs **on par with baselines**, which the paper interprets as indicating no significant degradation in almost-Markovian settings [2603.04910].

A compute–performance comparison is reported on **MoMaRT unload-suboptimal**, contrasting long-context **DP-PTP** variants with **DP-VPWEM (L2S2)**:

| Method | Context | Model Size (M) | GPU Mem (MB) | Train Time (s/step) | Inference Time (s) | Success Rate (%) |
|---|---:|---:|---:|---:|---:|---:|
| DP-PTP | 4 | 50.67 | 656 | 0.02 | 0.17 | 46.1 |
| DP-PTP | 8 | 50.67 | 684 | 0.02 | 0.18 | 45.0 |
| DP-PTP | 16 | 50.68 | 794 | 0.08 | 0.27 | 49.1 |
| DP-PTP | 32 | 50.69 | 942 | 0.08 | 0.35 | 49.6 |
| DP-PTP | 64 | 50.70 | 1258 | 0.17 | 0.44 | 49.6 |
| DP-PTP | 128 | 50.73 | 1792 | 0.18 | 0.72 | 40.4 |
| **DP-VPWEM** | **L2S2** | **52.98** | **734** | **0.09** | **0.22** | **58.3** |

This result is used in the paper to support the claim that simply increasing context length causes substantial growth in memory and inference cost, whereas VPWEM attains the best reported success rate with bounded memory structures [2603.04910].

## 6. Ablations, limitations, and significance

The ablation studies are performed on **Long-Horizon Square**. They show that even **L1S1** achieves **65% success rate**, already outperforming the listed baselines. Increasing either short-term or long-term token count generally helps at first, but too many tokens can hurt, so the main experiments use **L2S2**. Increasing compressor depth improves performance, and increasing cache size helps until performance saturates beyond about **8**. For cache management, **FIFO**, **K-Means**, and **AdjSim** perform similarly, whereas **Random** performs worse; the main model therefore uses **FIFO** [2603.04910].

The paper frames several limitations and future directions. It suggests extending VPWEM to more base policies, adding auxiliary objectives such as reconstruction, and deploying on real robots [2603.04910]. A plausible implication is that the current formulation is benchmark-centered and primarily validated in simulation-oriented or standard offline imitation-learning settings rather than in broad real-world deployment.

In methodological terms, VPWEM’s significance lies in treating long-horizon robotic memory not as unbounded context accumulation but as a structured combination of explicit short-term history and learned compressed long-term state. The framework’s contribution is therefore architectural as much as empirical: it proposes a concrete mechanism by which diffusion-based policies can access temporally distant information without scaling attention over the full trajectory [2603.04910].

Source: https://www.emergentmind.com/topics/vpwem