PVE-DP: Enhanced Visuomotor Diffusion Policy

• The paper demonstrates PVE-DP's main contribution by enriching standard diffusion policies with wavelet-enhanced visual features and precise IMU-based proprioception to reduce manipulation errors.
• Methodologically, it fuses global and local visual cues using DWT and cross-attention while integrating high-precision wrist IMU quaternions to enhance fine rotational alignment.
• Empirical evaluations report a 20–30% improvement in task success rates and significant gains in long-horizon inference, underscoring its robust performance in complex manipulation tasks.

Proprioceptive-Visual Enhanced Diffusion Policy (PVE-DP) is a diffusion-based visuomotor imitation policy introduced within RoboMatch as the main fine-manipulation policy for learning from demonstrations in both simulation and real-world robot tasks. Its defining premise is that precise manipulation degrades when action denoising is conditioned only on ordinary visual features and joint angles, particularly under fine rotational alignment, small pose tolerances, and long chains of subtle corrections. PVE-DP therefore augments the conditioning pathway of a standard diffusion policy in two directions: it inserts a Discrete Wavelet Transform (DWT)-based visual module for multi-scale feature extraction, and it enriches proprioceptive input with high-precision end-effector IMU quaternions, so that the denoising process is informed by both time-frequency visual structure and explicit wrist orientation (Liu et al., 10 Sep 2025).

1. Motivation and problem formulation

PVE-DP is motivated by a specific failure mode of ordinary diffusion policies for robot manipulation. The RoboMatch paper states that “cumulative errors during inference, coupled with insufficient visual perception and proprioception, significantly impede precise decision-making and efficient robot control” (Liu et al., 10 Sep 2025). In the paper’s framing, this is especially problematic for insertion, wiping, sorting, and grasping under small pose tolerances, where success depends less on coarse reaching and more on repeated micro-corrections in both translation and orientation.

The method targets two limitations simultaneously. First, standard diffusion policies do not explicitly model frequency-domain structure in the image stream, so they may miss scale-sensitive and edge-sensitive cues relevant to hand-object interaction. Second, joint angles alone do not fully specify end-effector orientation during contact-rich manipulation, particularly when small wrist roll or pitch errors dominate failure. PVE-DP addresses these issues by enriching both the visual stream and the robot self-state stream before action denoising.

Within RoboMatch, this policy sits inside a broader mobile-manipulation teleoperation and long-horizon execution system. The abstract reports that the overall approach improves data collection efficiency by over 20%, increases task success rates by 20–30% with PVE-DP, and enhances long-horizon inference performance by approximately 40% with the Auto-Matching Network (AMN) (Liu et al., 10 Sep 2025). The article’s focus, however, is the fine-manipulation policy itself rather than the task-decomposition machinery of AMN.

2. Architectural composition

Architecturally, PVE-DP follows the standard diffusion-policy pattern but augments the conditioning pathway. The inputs are RGB visual observations from the robot’s cameras and proprioceptive state from the robot body, and the outputs are continuous action sequences for the manipulator. The paper describes the system as comprising three parts: an FE-EMA visual module, rich proprioception, and a DP U-Net action denoiser (Liu et al., 10 Sep 2025).

The proprioceptive branch concatenates joint-position input with IMU-derived quaternion input: $$P_{\text{fused}} = \text{Concat}(X_{\text{qpos}}, X_{\text{quat}}).$$ The quaternion is represented as

$$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$

with unit constraint

$|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$

The visual branch produces a fused embedding that combines temporal attention with frequency-domain features: $$V_{\text{fused}} = \alpha \cdot X_{\text{time}} + (1-\alpha)\cdot X_{\text{fre}},$$ where $X_{\text{time}}$ is the temporal-domain feature produced by attention and group normalization, $X_{\text{fre}}$ is the frequency-domain feature derived from wavelet processing, and $\alpha$ balances the two.

This organization is important because PVE-DP is not described as a wholly new denoiser. Its novelty lies in how the observation condition is constructed before denoising. A common misconception is that its gains arise primarily from scaling model size; the paper’s ablation summary argues the opposite, namely that the policy performs better because it supplies the diffusion backbone with more task-relevant visual and proprioceptive signals rather than merely enlarging the network (Liu et al., 10 Sep 2025).

3. Frequency-enhanced visual representation

The visual enhancement in PVE-DP is implemented through a Frequency Enhanced EMA module that inserts DWT into the diffusion-policy backbone. The visual stream first undergoes wavelet-based decomposition, the high- and low-frequency components are recombined, and a cross-spatial attention mechanism is then applied (Liu et al., 10 Sep 2025).

For a 2D feature map, the paper gives the standard DWT decomposition

$\text{DWT}(X) = (cA, cH, cV, cD),$

where $cA$ is the low-frequency approximation, $cH$ horizontal details, $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$0 vertical details, and $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$1 diagonal details. A Haar wavelet filter is used. The high-frequency components are fused as

$$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$2

and then combined with the low-frequency branch to form

$$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$3

In the paper’s interpretation, the DWT serves a narrowly defined role: it decomposes visual input into coarse structure and fine edge/detail information, which is useful when the robot must localize objects, edges, insertable parts, or target boundaries precisely. The “frequency enhanced” designation therefore refers to explicit multi-scale decomposition rather than to a generic spectral regularizer.

This has two immediate consequences. First, PVE-DP can condition on both global layout and local boundaries, rather than relying only on ordinary spatial features. Second, the fused visual representation is explicitly time-frequency in character, because the final embedding combines temporal-domain and frequency-domain features through $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$4. This suggests that the method is designed for scenes in which contact geometry is visually subtle but operationally decisive.

4. Proprioceptive enrichment and diffusion objective

The proprioceptive enhancement comes from high-precision IMUs mounted on both end-effectors. These sensors provide real-time quaternion orientation data, which the paper describes as particularly valuable because many manipulation failures arise from small rotational errors at the gripper rather than from gross arm placement (Liu et al., 10 Sep 2025). In preprocessing terms, the raw IMU orientation is represented as a normalized quaternion and concatenated with joint positions to form a richer self-state than joint angles alone.

The stated rationale is that joint positions alone can underspecify the actual hand pose due to kinematic coupling and accumulated control error. By adding quaternion orientation, the model can distinguish arm configurations that are similar in joint space but materially different in wrist pose. The paper ties this directly to insertion, grasp orientation, and alignment during contact-rich tasks, and states that this richer state reduces pose drift.

The diffusion policy is trained to predict noise added to action trajectories conditioned on fused visual and proprioceptive features: $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$5 This is the standard conditional diffusion objective: the network predicts the noise on a noised action sequence rather than directly regressing the action sequence itself. At inference time, the model iteratively refines a sampled action sequence through denoising steps, each time conditioning on the current noisy action together with $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$6 and $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$7. The paper notes that during denoising, the policy “continuously refines the predicted action sequence by incorporating additional wrist perception data from robotic arms IMUs, along with time-frequency features extracted by the FE-EMA network” (Liu et al., 10 Sep 2025).

5. Evaluation, ablations, and computational profile

PVE-DP is evaluated on two simulation tasks from the ALOHA benchmark—cube transfer and bimanual insertion—and two real-world tasks—wiping a table and sorting a workpiece. The training data consist of 100 demonstrations for each simulated task and 50 demonstrations for each real task. The paper compares against ACT, DP U-Net, NL-ACT, InterACT, DiT-Block, DP with EMA, and ablated variants with quaternion-only or FE-EMA-only enhancements (Liu et al., 10 Sep 2025).

Method	Average success across four tasks
ACT	52.0%
DP U-Net	43.8%
NL-ACT	53.8%
InterACT	58.8%
DiT-Block	56.8%
DP w. EMA	54.0%
ACT w. quat.	63.0%
DP w. quat.	61.0%
DP w. FE-EMA	59.0%
Full PVE-DP	75.3%

Task-wise, PVE-DP achieves 100% on Task 1, 79% on Task 2, 63% on Task 3, and 59% on Task 4. The paper interprets these results as evidence that both the DWT-based visual enhancement and the IMU quaternion branch contribute meaningfully, and that their combination is stronger than either component alone.

The computational overhead reported for these additions is small relative to baseline DP. Baseline DP has 101.7214M parameters and 33.9266B FLOPs; DP + FE-EMA has 101.7269M parameters and 33.9662B FLOPs; adding quaternion inputs yields 101.8416M parameters and 33.9664B FLOPs. The paper explicitly uses this to argue that the improvement is not explained by a large parameter increase.

The paper also reports long-horizon comparisons in which PVE-DP serves as the policy inside AMN. For the long tasks Clean Trash and Deliver Tool, PVE-DP alone outperforms DP and ACT on many subtasks, although the full AMN architecture achieves better end-to-end completion because it decomposes tasks into smaller segments. In Clean Trash, PVE-DP achieves 66% on Grasp versus 30% for DP; in Deliver Tool, it reaches 54% on grasping the screw versus 26% for DP, with subtask-level success rates measured over 50 trials (Liu et al., 10 Sep 2025). These figures are consistent with the paper’s claim that PVE-DP is most valuable when success hinges on accurate hand positioning at critical contact moments.

6. Related formulations, distinctions, and limitations

A separate 2025 paper, “Factorizing Diffusion Policies for Observation Modality Prioritization”, presents what it explicitly calls a PVE-DP interpretation of Factorized Diffusion Policies (FDP) rather than the RoboMatch architecture itself (Patil et al., 20 Sep 2025). In that formulation, a multimodal diffusion policy is factorized into a base policy trained on prioritized modalities $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$8 and a residual policy trained on all modalities $$\mathbf{q} = w + x \vec{i} + y \vec{j} + z \vec{k},$$9, so that modality hierarchy is enforced in score space: $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$0 The paper uses this to realize settings such as proprioception $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$1 vision and vision $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$2 proprioception, and reports that such prioritization improves low-data performance and robustness under distractors, clutter, and camera occlusions. In that usage, “PVE-DP” functions as a conceptual framing for hierarchical multimodal conditioning rather than as the specific DWT-plus-IMU architecture of RoboMatch.

Another related contemporaneous direction is VGGT-DP, a visuomotor imitation-learning framework that keeps the Diffusion Policy control formulation but replaces the usual lightweight vision backbone with the Visual Geometry Grounded Transformer (VGGT), adds a proprioception-guided visual learning strategy, and introduces Frame-Wise Token Reuse and random token pruning for efficiency and robustness (Ge et al., 23 Sep 2025). On 10 MetaWorld tasks, VGGT-DP achieves an average success rate of 36.6%, compared with 28.6% for DP3 and 19.1% for DP, with especially strong gains on difficult spatial reasoning tasks. At the same time, the paper reports strong sensitivity to unseen viewpoints: in a Stick Pull viewpoint-perturbation test, performance drops from 39% at $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$3 to 5% at $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$4 and 0\% at $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$5 and $|\mathbf{q}|^2 = w^2 + x^2 + y^2 + z^2 = 1.$6.

These comparisons help delimit the meaning of PVE-DP. In the strict sense, PVE-DP refers to the RoboMatch fine-manipulation policy built from FE-EMA visual fusion, DWT-based multi-scale decomposition, and end-effector IMU quaternions (Liu et al., 10 Sep 2025). In a broader interpretive sense, later work uses the label to discuss how proprioception and vision can be differentially prioritized within diffusion-based control (Patil et al., 20 Sep 2025). A plausible implication is that the 2025 literature converges on a common thesis: diffusion-policy performance in manipulation depends critically on how multimodal observation structure is encoded, whether through wavelet-enhanced visual fusion, score-factorized modality hierarchy, or geometry-aware visual priors (Ge et al., 23 Sep 2025).