VAMPO: Visual Dynamics Optimization

• VAMPO is a post-training framework for diffusion-based video models that enhances task-relevant visual dynamics for improved robot control.
• It reframes multi-step denoising as a sequential decision process optimized via reinforcement learning to achieve precise manipulation signals.
• Experimental results show that VAMPO improves downstream action generation, robustness, and generalization across simulated and real-world manipulation tasks.

VAMPO, introduced in "VAMPO: Policy Optimization for Improving Visual Dynamics in Video Action Models" (Ge et al., 19 Mar 2026), is a post-training framework for diffusion-based video prediction models that directly optimizes the precision of visual dynamics needed for robot control. Its central premise is that likelihood-surrogate objectives encourage globally plausible predictions but do not explicitly optimize manipulation-critical signals such as object pose, spatial relations, and contact timing. VAMPO addresses this objective mismatch by reframing multi-step denoising as a sequential decision process and optimizing the denoising policy with reinforcement learning, using verifiable rewards defined over expert visual dynamics in latent space. The reported effect is improved task-relevant visual dynamics, better downstream action generation, and stronger generalization across simulated and real-world manipulation tasks (Ge et al., 19 Mar 2026).

1. Problem setting and scope

VAMPO targets diffusion-based video prediction models used as a foundation for Vision–Language–Action systems. In this setting, video prediction models can learn visual dynamics from large-scale video data and transfer this knowledge to downstream robot control, but the training objective is misaligned with control requirements. Standard pretraining relies on likelihood surrogates such as ELBO variants or denoiser MSE on noise or clean targets. These objectives optimize global plausibility under the data distribution, yet they do not explicitly optimize precision-sensitive dynamics relevant for manipulation (Ge et al., 19 Mar 2026).

The framework is motivated by the observation that errors in pose, spatial relations, and contact timing may remain small but still be consequential. The paper describes this as a form of decision-boundary amplification: when subtle representation errors are passed to a downstream action-generation module, they can induce incorrect actions such as closing a gripper too early or misplacing relative to a receptacle. VAMPO therefore aligns the denoising objective with expert visual dynamics via reward-based post-training in latent space, which is described as more geometry- and relation-centric than pixel space and as directly impacting control (Ge et al., 19 Mar 2026).

Within this formulation, VAMPO is not a replacement for generative pretraining. It augments standard generative training with reward-based post-training aligned with task-relevant visual dynamics. The method presupposes a suitable pretrained video prediction model and a VAE encoder whose latent geometry is reliable enough to support reward computation and downstream policy improvement.

2. Sequential decision formulation of denoising

The mathematical core of VAMPO is a reformulation of multi-step denoising as a Markov decision process. The video prediction model is instantiated with a latent video diffusion model. If $N$ denotes the expert future video and $E$ the VAE encoder, the clean latent is

$x_0 = E(N).$

With EDM-style forward noising at continuous noise level $\sigma$,

$$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$

A conditional denoiser $D_\theta(x_\sigma,\sigma,c_v)$, under condition $c_v=(o,l)$, predicts the clean latent from the noisy latent. The pretraining objective is the denoiser MSE

$$\mathcal{L}_{Video} = \mathbb{E}_{x_0,\epsilon,\sigma} \left\| D_\theta(x_\sigma,\sigma,c_v) - x_0 \right\|^2.$$

At inference, the probability-flow ODE can be discretized with a first-order Euler step

$$x_{\sigma_{i-1}} = x_{\sigma_i} + (\sigma_{i-1} - \sigma_i)\, g_\theta(x_{\sigma_i}, \sigma_i, c_v),$$

with

$$g_\theta(x_{\sigma_i}, \sigma_i, c_v) = \frac{x_{\sigma_i} - D_\theta(x_{\sigma_i}, \sigma_i, c_v)}{\sigma_i}.$$

For Euler-Ancestral sampling, the update becomes

$E$0

where

$E$1

$E$2

This yields an explicit Gaussian transition density:

$E$3

The MDP is written as $E$4. The state is $E$5. The action can be represented either as the denoising direction $E$6 or directly as the next latent $E$7, depending on the sampler. The transition $E$8 follows the sampler dynamics. The initial state distribution samples $E$9 from data and $x_0 = E(N).$0. The reward is terminal only,

$x_0 = E(N).$1

so the optimization objective is

$x_0 = E(N).$2

This formulation makes the denoising policy $x_0 = E(N).$3 the object of reinforcement learning rather than treating denoising purely as a fixed sampling procedure (Ge et al., 19 Mar 2026).

3. Euler Hybrid sampler, latent-space reward, and GRPO

The principal algorithmic device in VAMPO is the Euler Hybrid sampler. It injects stochasticity only at the first denoising step and keeps all subsequent steps deterministic. Concretely, the first denoising step uses SDE Euler-Ancestral sampling, and the remaining steps use deterministic Euler ODE updates. The trajectory distribution is therefore concentrated on a single stochastic decision followed by a deterministic denoising path (Ge et al., 19 Mar 2026).

This design is introduced as a variance-reduction mechanism. Because only the first step is stochastic, the policy gradient reduces to a single REINFORCE term plus deterministic pathwise dependence carried through the sampler:

$x_0 = E(N).$4

where $x_0 = E(N).$5 is a baseline such as a group mean reward. The paper argues that this preserves denoising coherence from pretraining, concentrates exploration into the earliest denoising features that condition the downstream action-generation module, and enables tractable gradient computation with minimal variance (Ge et al., 19 Mar 2026).

The reward is a verifiable non-adversarial reward defined over latent-space alignment between the final predicted latent $x_0 = E(N).$6 and the expert latent $x_0 = E(N).$7:

$x_0 = E(N).$8

It is described as verifiable because it is computed deterministically from a known encoder $x_0 = E(N).$9 and known functions, uses no learned discriminator or implicit preferences, measures well-defined geometric alignment in latent space, and is reproducible across implementations. The stated implication is that the only way to increase the reward is to bring $\sigma$0 closer to $\sigma$1 in geometry and direction (Ge et al., 19 Mar 2026).

Optimization uses GRPO, adapted for diffusion post-training. For a group of $\sigma$2 rollouts under the same condition, rewards $\sigma$3 are converted into normalized advantages

$\sigma$4

and the PPO-style clipped objective is

$\sigma$5

with

$\sigma$6

In the paper’s interpretation, clipping acts as a practical trust region, while group-normalized advantages stabilize learning and avoid reward hacking by conditioning updates on relative preferences within the sampled group (Ge et al., 19 Mar 2026).

4. Training procedure and implementation details

The training pipeline starts from expert demonstrations containing current observation $\sigma$7, language instruction $\sigma$8, ground-truth action sequence $\sigma$9, and target future video $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$0. The expert future is encoded with a pretrained VAE encoder $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$1 to obtain $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$2. For each rollout, the algorithm samples initial noise $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$3, performs one stochastic Euler-Ancestral step, then executes deterministic Euler ODE updates until the final latent $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$4 is produced. Rewards are then computed in latent space and used for the GRPO update (Ge et al., 19 Mar 2026).

The reported pseudocode uses group rollouts with shared initial $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$5 within a group, explicitly to isolate the effect of first-step stochasticity on candidate diversity. The policy-update loop periodically sets $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$6. Typical hyperparameters are GRPO clip $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$7, group size $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$8, batch size $$x_\sigma = x_0 + \sigma \epsilon, \qquad \epsilon \sim \mathcal{N}(0, I).$$9, reward weights $D_\theta(x_\sigma,\sigma,c_v)$0 and $D_\theta(x_\sigma,\sigma,c_v)$1, post-training learning rate $D_\theta(x_\sigma,\sigma,c_v)$2, and fine-tuning learning rate $D_\theta(x_\sigma,\sigma,c_v)$3 (Ge et al., 19 Mar 2026).

The implementation notes specify that the video prediction model is instantiated with SVD-style latent diffusion, the denoiser is trained with an EDM-like target on $D_\theta(x_\sigma,\sigma,c_v)$4, and latent features for the downstream action-generation module are taken from early denoising states to keep inference fast. Post-training is reported at approximately $D_\theta(x_\sigma,\sigma,c_v)$5k steps on $D_\theta(x_\sigma,\sigma,c_v)$6 GPUs of H20 class for the video prediction model, while the action-generation module is trained for approximately $D_\theta(x_\sigma,\sigma,c_v)$7 epochs on $D_\theta(x_\sigma,\sigma,c_v)$8 GPUs. The project homepage is listed as https://vampo-robot.github.io/VAMPO/ (Ge et al., 19 Mar 2026).

5. Experimental results, ablations, and diagnostic measures

The experiments cover simulation and real-world manipulation. In simulation, the benchmarks are CALVIN ABC→D for language-conditioned manipulation and L-CALVIN for long-horizon sequences of $D_\theta(x_\sigma,\sigma,c_v)$9 steps, using Video Prediction Policy as the base model. Metrics include task completion in a row, average trajectory length, pose/spatial/contact alignment in latent space, downstream action success rates, and the effective rank of the Jacobian $c_v=(o,l)$0 as a measure of vision–action coupling (Ge et al., 19 Mar 2026).

Setting	Base policy	VAMPO
CALVIN ABC→D, tasks completed in a row (1–5)	96.0 / 91.3 / 86.4 / 80.4 / 74.7	98.0 / 94.8 / 91.3 / 88.3 / 83.1
CALVIN ABC→D, Avg. Len	4.28	4.56
L-CALVIN, Avg. Len	5.53	6.73
ER analysis, Avg. ER / Avg. ERR	29.28 / 0.0603	43.88 / 0.0814

On L-CALVIN, the reported VAMPO multi-step success rates across $c_v=(o,l)$1–$c_v=(o,l)$2 are $c_v=(o,l)$3, with growing gains at longer horizons relative to the base model. The ER analysis is used to support the claim that post-VAMPO policies exploit more independent latent-dynamics directions.

The real-world system is an Agibot Genie 01 dual-arm platform equipped with multi-view RGB-D cameras, specifically a head D455 and wrist D405 units. The tasks include grasp-in-clutter, precise single-arm placement, and bimanual coordinated grasp-and-place. The reported result is that VAMPO consistently improves video generation quality and downstream execution across varied scenes and embodiments (Ge et al., 19 Mar 2026).

Ablations isolate the components responsible for these gains. Injecting SDE noise only at the first denoising step yields the best performance and mitigates reward hacking, whereas multi-step SDE introduces credit-assignment issues and instability. Latent-space reward outperforms pixel-space reward, which is characterized as chasing appearance details not tightly coupled to control. GRPO, relative to DDPO, yields more stable updates and higher success rates. Improvements appear even at approximately $c_v=(o,l)$4 steps, best performance is reported at approximately $c_v=(o,l)$5 steps, and training remains stable relative to the base policy while still exhibiting reinforcement learning’s characteristic fluctuations. Robustness under visual distribution shift and longer horizons is presented as evidence that dynamics alignment is a generalizable principle (Ge et al., 19 Mar 2026).

6. Limitations, applicability, and terminological ambiguity

The method’s limitations are stated explicitly. Reward misspecification remains a central concern: if the latent encoder or reward weights are misaligned with task priorities, optimization may focus on suboptimal dynamics. Computational cost is nontrivial because post-training requires grouped rollouts and multiple denoising passes, although the design mitigates this through early feature extraction, small group size, and one-step stochasticity. Safety is not guaranteed; improved dynamics reduce downstream errors but do not eliminate edge cases, so integration with standard safety monitors and constraints in the controller remains necessary. Generality also depends on a suitable pretrained video prediction model and a VAE encoder whose latent geometry remains valid under deployment conditions; domain shifts that break encoder semantics may require re-encoding or adaptation (Ge et al., 19 Mar 2026).

The scope of VAMPO is therefore specific rather than universal. It is a post-training framework for improving control-relevant visual dynamics in diffusion-based video action models, not a general theorem about all generative video models or all robot-learning pipelines. A plausible implication is that its empirical success depends on the degree to which latent geometry is aligned with the downstream control objective.

A separate source of confusion is nomenclature. The acronym “VAMPO” is also sometimes used in searches for “visual-assisted mathematical problem solving,” but there is no official “VAMPO” benchmark alias in the paper introducing VAMPS; the correct benchmark name there is VAMPS: Visual-Assisted Mathematical Problem Solving Benchmark (Dabiriaghdam et al., 2 Jun 2026). In current usage, VAMPO refers to policy optimization for improving visual dynamics in video action models, whereas VAMPS refers to a graph-assisted mathematics benchmark.