---
title: 'PVE-DP: Enhanced Visuomotor Diffusion Policy'
url: https://www.emergentmind.com/topics/proprioceptive-visual-enhanced-diffusion-policy-pve-dp
type: topic
---

# PVE-DP: Enhanced Visuomotor Diffusion Policy

Proprioceptive-Visual Enhanced Diffusion Policy (PVE-DP) is a diffusion-based visuomotor imitation policy introduced within RoboMatch as the main fine-manipulation policy for learning from demonstrations in both simulation and real-world robot tasks. Its defining premise is that precise manipulation degrades when action denoising is conditioned only on ordinary visual features and joint angles, particularly under fine rotational alignment, small pose tolerances, and long chains of subtle corrections. PVE-DP therefore augments the conditioning pathway of a standard diffusion policy in two directions: it inserts a Discrete Wavelet Transform (DWT)-based visual module for multi-scale feature extraction, and it enriches proprioceptive input with high-precision end-effector IMU quaternions, so that the denoising process is informed by both time-frequency visual structure and explicit wrist orientation [2509.08522].

## 1. Motivation and problem formulation

PVE-DP is motivated by a specific failure mode of ordinary diffusion policies for robot manipulation. The RoboMatch paper states that “cumulative errors during inference, coupled with insufficient visual perception and proprioception, significantly impede precise decision-making and efficient robot control” [2509.08522]. In the paper’s framing, this is especially problematic for insertion, wiping, sorting, and grasping under small pose tolerances, where success depends less on coarse reaching and more on repeated micro-corrections in both translation and orientation.

The method targets two limitations simultaneously. First, standard diffusion policies do not explicitly model frequency-domain structure in the image stream, so they may miss scale-sensitive and edge-sensitive cues relevant to hand-object interaction. Second, joint angles alone do not fully specify end-effector orientation during contact-rich manipulation, particularly when small wrist roll or pitch errors dominate failure. PVE-DP addresses these issues by enriching both the visual stream and the robot self-state stream before action denoising.

Within RoboMatch, this policy sits inside a broader mobile-manipulation teleoperation and long-horizon execution system. The abstract reports that the overall approach improves data collection efficiency by over 20%, increases task success rates by 20–30% with PVE-DP, and enhances long-horizon inference performance by approximately 40% with the Auto-Matching Network (AMN) [2509.08522]. The article’s focus, however, is the fine-manipulation policy itself rather than the task-decomposition machinery of AMN.

## 2. Architectural composition

Architecturally, PVE-DP follows the standard diffusion-policy pattern but augments the conditioning pathway. The inputs are RGB visual observations from the robot’s cameras and proprioceptive state from the robot body, and the outputs are continuous action sequences for the manipulator. The paper describes the system as comprising three parts: an **FE-EMA visual module**, **rich proprioception**, and a **DP U-Net action denoiser** [2509.08522].

The proprioceptive branch concatenates joint-position input with IMU-derived quaternion input:
\[
P_{\text{fused}} = \text{Concat}(X_{\text{qpos}}, X_{\text{quat}}).
\]
The quaternion is represented as
\[
\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},
\]
with unit constraint
\[
|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.
\]

The visual branch produces a fused embedding that combines temporal attention with frequency-domain features:
\[
V_{\text{fused}} = \alpha \cdot X_{\text{time}} + (1-\alpha)\cdot X_{\text{fre}},
\]
where \(X_{\text{time}}\) is the temporal-domain feature produced by attention and group normalization, \(X_{\text{fre}}\) is the frequency-domain feature derived from wavelet processing, and \(\alpha\) balances the two.

This organization is important because PVE-DP is not described as a wholly new denoiser. Its novelty lies in how the observation condition is constructed before denoising. A common misconception is that its gains arise primarily from scaling model size; the paper’s ablation summary argues the opposite, namely that the policy performs better because it supplies the diffusion backbone with more task-relevant visual and proprioceptive signals rather than merely enlarging the network [2509.08522].

## 3. Frequency-enhanced visual representation

The visual enhancement in PVE-DP is implemented through a Frequency Enhanced EMA module that inserts DWT into the diffusion-policy backbone. The visual stream first undergoes wavelet-based decomposition, the high- and low-frequency components are recombined, and a cross-spatial attention mechanism is then applied [2509.08522].

For a 2D feature map, the paper gives the standard DWT decomposition
\[
\text{DWT}(X) = (cA, cH, cV, cD),
\]
where \(cA\) is the low-frequency approximation, \(cH\) horizontal details, \(cV\) vertical details, and \(cD\) diagonal details. A Haar wavelet filter is used. The high-frequency components are fused as
\[
F_H = cH + cV + cD,
\]
and then combined with the low-frequency branch to form
\[
X_{\text{fre}} = \text{ResBlock}(cA + F_H).
\]

In the paper’s interpretation, the DWT serves a narrowly defined role: it decomposes visual input into coarse structure and fine edge/detail information, which is useful when the robot must localize objects, edges, insertable parts, or target boundaries precisely. The “frequency enhanced” designation therefore refers to explicit multi-scale decomposition rather than to a generic spectral regularizer.

This has two immediate consequences. First, PVE-DP can condition on both global layout and local boundaries, rather than relying only on ordinary spatial features. Second, the fused visual representation is explicitly time-frequency in character, because the final embedding combines temporal-domain and frequency-domain features through \(V_{\text{fused}}\). This suggests that the method is designed for scenes in which contact geometry is visually subtle but operationally decisive.

## 4. Proprioceptive enrichment and diffusion objective

The proprioceptive enhancement comes from high-precision IMUs mounted on both end-effectors. These sensors provide real-time quaternion orientation data, which the paper describes as particularly valuable because many manipulation failures arise from small rotational errors at the gripper rather than from gross arm placement [2509.08522]. In preprocessing terms, the raw IMU orientation is represented as a normalized quaternion and concatenated with joint positions to form a richer self-state than joint angles alone.

The stated rationale is that joint positions alone can underspecify the actual hand pose due to kinematic coupling and accumulated control error. By adding quaternion orientation, the model can distinguish arm configurations that are similar in joint space but materially different in wrist pose. The paper ties this directly to insertion, grasp orientation, and alignment during contact-rich tasks, and states that this richer state reduces pose drift.

The diffusion policy is trained to predict noise added to action trajectories conditioned on fused visual and proprioceptive features:
\[
\mathcal{L}(\theta) = \mathbb{E}_{t, A_0^k, \epsilon}\left[\|\epsilon - \epsilon_\theta(P_{\text{fused}}, V_{\text{fused}}, A_t^k, t)\|^2\right].
\]
This is the standard conditional diffusion objective: the network predicts the noise on a noised action sequence rather than directly regressing the action sequence itself. At inference time, the model iteratively refines a sampled action sequence through denoising steps, each time conditioning on the current noisy action together with \(P_{\text{fused}}\) and \(V_{\text{fused}}\). The paper notes that during denoising, the policy “continuously refines the predicted action sequence by incorporating additional wrist perception data from robotic arms IMUs, along with time-frequency features extracted by the FE-EMA network” [2509.08522].

## 5. Evaluation, ablations, and computational profile

PVE-DP is evaluated on two simulation tasks from the ALOHA benchmark—**cube transfer** and **bimanual insertion**—and two real-world tasks—**wiping a table** and **sorting a workpiece**. The training data consist of 100 demonstrations for each simulated task and 50 demonstrations for each real task. The paper compares against ACT, DP U-Net, NL-ACT, InterACT, DiT-Block, DP with EMA, and ablated variants with quaternion-only or FE-EMA-only enhancements [2509.08522].

| Method | Average success across four tasks |
|---|---:|
| ACT | 52.0% |
| DP U-Net | 43.8% |
| NL-ACT | 53.8% |
| InterACT | 58.8% |
| DiT-Block | 56.8% |
| DP w. EMA | 54.0% |
| ACT w. quat. | 63.0% |
| DP w. quat. | 61.0% |
| DP w. FE-EMA | 59.0% |
| Full PVE-DP | 75.3% |

Task-wise, PVE-DP achieves **100% on Task 1**, **79% on Task 2**, **63% on Task 3**, and **59% on Task 4**. The paper interprets these results as evidence that both the DWT-based visual enhancement and the IMU quaternion branch contribute meaningfully, and that their combination is stronger than either component alone.

The computational overhead reported for these additions is small relative to baseline DP. Baseline DP has **101.7214M parameters** and **33.9266B FLOPs**; **DP + FE-EMA** has **101.7269M parameters** and **33.9662B FLOPs**; adding quaternion inputs yields **101.8416M parameters** and **33.9664B FLOPs**. The paper explicitly uses this to argue that the improvement is not explained by a large parameter increase.

The paper also reports long-horizon comparisons in which PVE-DP serves as the policy inside AMN. For the long tasks **Clean Trash** and **Deliver Tool**, PVE-DP alone outperforms DP and ACT on many subtasks, although the full AMN architecture achieves better end-to-end completion because it decomposes tasks into smaller segments. In **Clean Trash**, PVE-DP achieves **66% on Grasp** versus **30%** for DP; in **Deliver Tool**, it reaches **54% on grasping the screw** versus **26%** for DP, with subtask-level success rates measured over 50 trials [2509.08522]. These figures are consistent with the paper’s claim that PVE-DP is most valuable when success hinges on accurate hand positioning at critical contact moments.

## 6. Related formulations, distinctions, and limitations

A separate 2025 paper, **“Factorizing Diffusion Policies for Observation Modality Prioritization”**, presents what it explicitly calls a **PVE-DP interpretation** of Factorized Diffusion Policies (FDP) rather than the RoboMatch architecture itself [2509.16830]. In that formulation, a multimodal diffusion policy is factorized into a **base policy** trained on prioritized modalities \(y^{1:k}\) and a **residual policy** trained on all modalities \(y^{1:M}\), so that modality hierarchy is enforced in score space:
\[
\nabla_{x_t}\log p(x_t \mid y^{1:M}) =
\nabla_{x_t}\log p(x_t \mid y^{1:k}) +
\nabla_{x_t}\log p(y^{k+1:M} \mid x_t, y^{1:k}).
\]
The paper uses this to realize settings such as **proprioception \(>\) vision** and **vision \(>\) proprioception**, and reports that such prioritization improves low-data performance and robustness under distractors, clutter, and camera occlusions. In that usage, “PVE-DP” functions as a conceptual framing for hierarchical multimodal conditioning rather than as the specific DWT-plus-IMU architecture of RoboMatch.

Another related contemporaneous direction is **VGGT-DP**, a visuomotor imitation-learning framework that keeps the Diffusion Policy control formulation but replaces the usual lightweight vision backbone with the **Visual Geometry Grounded Transformer (VGGT)**, adds a **proprioception-guided visual learning strategy**, and introduces **Frame-Wise Token Reuse** and **random token pruning** for efficiency and robustness [2509.18778]. On 10 MetaWorld tasks, VGGT-DP achieves an **average success rate of 36.6%**, compared with **28.6% for DP3** and **19.1% for DP**, with especially strong gains on difficult spatial reasoning tasks. At the same time, the paper reports strong sensitivity to unseen viewpoints: in a Stick Pull viewpoint-perturbation test, performance drops from **39% at \(0^\circ\)** to **5% at \(5^\circ\)** and **0\%** at **\(10^\circ\)** and **\(15^\circ\)**.

These comparisons help delimit the meaning of PVE-DP. In the strict sense, PVE-DP refers to the RoboMatch fine-manipulation policy built from FE-EMA visual fusion, DWT-based multi-scale decomposition, and end-effector IMU quaternions [2509.08522]. In a broader interpretive sense, later work uses the label to discuss how proprioception and vision can be differentially prioritized within diffusion-based control [2509.16830]. A plausible implication is that the 2025 literature converges on a common thesis: diffusion-policy performance in manipulation depends critically on how multimodal observation structure is encoded, whether through wavelet-enhanced visual fusion, score-factorized modality hierarchy, or geometry-aware visual priors [2509.18778].

Source: https://www.emergentmind.com/topics/proprioceptive-visual-enhanced-diffusion-policy-pve-dp