Rectified Dynamic Mesh (R-DMesh)

• R-DMesh is a unified framework for video-guided 4D mesh generation that decouples discontinuous pose rectification from smooth motion dynamics.
• It decomposes the target sequence into base geometry, a rectification jump offset, and relative motion trajectories using a VAE architecture.
• The method employs geometry-aware Triflow Attention and a Rectified Flow diffusion transformer to achieve precise, video-aligned mesh animations.

Searching arXiv for the primary paper and closely related context papers to ground the article. arXiv search query: (Wu et al., 13 May 2026) R-DMesh Rectified Dynamic Mesh Flow Rectified Dynamic Mesh (R-DMesh) is a unified framework for video-guided 3D animation that generates a 4D mesh sequence from a static 3D mesh in an arbitrary pose and a monocular reference video showing a possibly different object performing some motion (Wu et al., 13 May 2026). Its defining operation is “rectification”: before continuous animation, the framework learns a one-time rectification jump offset that transforms the arbitrary input pose into the pose of the video’s first frame, after which it models smooth relative motion. The resulting output is a dynamic mesh $D=(V_{1:T},F)$ with fixed topology $F$ and time-varying vertices $V_t\in\mathbb{R}^{N\times 3}$, designed to respect the geometry of the input mesh while aligning pose and motion to video context.

1. Problem setting and the pose misalignment dilemma

R-DMesh addresses video-guided 3D animation, or equivalently 4D mesh generation, under a setting in which the conditional input mesh $M_{\text{cond}}=(V_{\text{cond}},F)$ is rarely aligned with the first frame of the driving video. The target is a mesh sequence with the same topology as the input but with vertices evolving over time so that the animation follows the video while preserving the identity and geometry of the original asset (Wu et al., 13 May 2026).

The central failure mode is the pose misalignment dilemma. If a model directly forces an arbitrarily posed mesh to follow a motion trajectory inferred from video, then the first update becomes a large, discontinuous jump,

$\Delta J = V_1 - V_{\text{cond}}.$

The data explicitly associates this discontinuity with two opposing failure modes: vertices may move very far in one step, leading to severe distortions such as stretched limbs or collapsing surfaces; or, if motion smoothness is heavily regularized, the model may ignore the jump and fail to reach the desired pose. R-DMesh isolates this discontinuity instead of treating it as ordinary motion, so that pose correction and subsequent animation are modeled separately (Wu et al., 13 May 2026).

The motivating use cases are content creation, pose and motion retargeting, and holistic video-to-4D generation. The framework is described as feed-forward and general-category, does not assume a specific skeleton such as SMPL, and directly predicts vertex trajectories. This suggests that its intended scope is broader than human-body motion transfer and includes stylized or nonstandard object categories.

2. Dynamic mesh representation and latent disentanglement

The representational core of R-DMesh is a decomposition of the target sequence into base geometry, a rectification jump, and relative motion trajectories. For training pairs with shared topology, the static conditional mesh is

$$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$

and the target dynamic sequence is

$D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$

Rather than reconstruct $V_{1:T}$ directly, R-DMesh decomposes the sequence into:

1. base geometry $V_{\text{cond}}$,
2. rectification jump offset

$\Delta J = V_1 - V_{\text{cond}},$

1. relative motion trajectories

$F$0

This yields the reconstruction relations

$F$1

The separation is operational rather than merely descriptive: $F$2 is intended to absorb the discontinuous pose correction, while $F$3 encodes smooth motion after rectification (Wu et al., 13 May 2026).

Before decomposition, the method applies canonicalization termed Dual-Norm in the ablations. The conditional mesh is centered at its centroid, the dynamic sequence is centered with respect to the centroid of $F$4, and both are normalized by the maximum absolute coordinate of the centered $F$5. The stated reason is that, in the video-conditioned setting, global translation is unidentifiable. The effect is to make $F$6 focus on local pose alignment and $F$7 focus on pure motion.

The representation is learned by an R-DMesh VAE. The encoder receives $F$8, $F$9, and $V_t\in\mathbb{R}^{N\times 3}$0, and produces a deterministic geometry latent $V_t\in\mathbb{R}^{N\times 3}$1 together with stochastic dynamic latents $V_t\in\mathbb{R}^{N\times 3}$2 and $V_t\in\mathbb{R}^{N\times 3}$3. The VAE treats only the dynamic components probabilistically:

$V_t\in\mathbb{R}^{N\times 3}$4

with KL penalties

$V_t\in\mathbb{R}^{N\times 3}$5

The base geometry latent is not stochastic because the conditional mesh is known at inference time (Wu et al., 13 May 2026).

The decoder reconstructs $V_t\in\mathbb{R}^{N\times 3}$6 and $V_t\in\mathbb{R}^{N\times 3}$7, and the VAE is trained with reconstruction MSE on both terms plus KL regularization:

$V_t\in\mathbb{R}^{N\times 3}$8

$V_t\in\mathbb{R}^{N\times 3}$9

The separate supervision on jumps and trajectories is termed Decoup-Loss and is reported as important for stable learning.

3. Triflow Attention and geometry-aware rectification

R-DMesh processes three coupled but disentangled streams: geometry flow, jump flow, and trajectory flow. The encoder first lifts $M_{\text{cond}}=(V_{\text{cond}},F)$0, $M_{\text{cond}}=(V_{\text{cond}},F)$1, and $M_{\text{cond}}=(V_{\text{cond}},F)$2 into high-dimensional features with positional encodings tailored to vertices and trajectories. Local mesh structure is incorporated by adjacency-masked self-attention on geometry features, after which the model compresses the representation with Farthest Point Sampling (FPS) and gathers corresponding features from all three modalities (Wu et al., 13 May 2026).

The architectural backbone is Triflow Attention. A shared attention map is computed only from geometry:

$M_{\text{cond}}=(V_{\text{cond}},F)$3

and then applied to geometry, jump, and trajectory features simultaneously:

$M_{\text{cond}}=(V_{\text{cond}},F)$4

This mechanism is used in both encoder and decoder stacks.

The stated motivation is physical consistency and local rigidity. Because the attention weights are geometry-defined, neighboring vertices receive similar aggregation patterns, and jump and motion features are propagated along geometric neighborhoods rather than arbitrary latent correlations. The paper explicitly associates this with avoiding unnatural shearing or tearing and with distributing pose changes in a way consistent with local rigidity (Wu et al., 13 May 2026).

Ablation results identify Jump-Decomp as the most critical design choice, but also attribute a significant role to Tri-Attn. Removing Jump-Decomp causes the initial frame to fail to move to the video pose and dramatically increases EucD; removing Tri-Attn increases reconstruction error and causes entanglement between motion and jump features. Dual-Norm and Decoup-Loss also improve performance. This suggests that the method’s robustness is not due to the generative model alone, but to the specific factorization of geometry, discontinuous alignment, and smooth dynamics.

4. Rectified Flow–based diffusion transformer and video conditioning

After the VAE is trained, R-DMesh learns a generative model over the dynamic latent

$M_{\text{cond}}=(V_{\text{cond}},F)$5

conditioned on the clean geometry latent $M_{\text{cond}}=(V_{\text{cond}},F)$6 and video features $M_{\text{cond}}=(V_{\text{cond}},F)$7 extracted from a pre-trained Video Diffusion Model. The generative component uses Rectified Flow, which constructs a straight interpolation between Gaussian noise $M_{\text{cond}}=(V_{\text{cond}},F)$8 and the target latent:

$M_{\text{cond}}=(V_{\text{cond}},F)$9

with target velocity

$\Delta J = V_1 - V_{\text{cond}}.$0

The training objective is the flow matching loss

$\Delta J = V_1 - V_{\text{cond}}.$1

At inference, the model samples $\Delta J = V_1 - V_{\text{cond}}.$2, integrates the learned ODE from $\Delta J = V_1 - V_{\text{cond}}.$3 to $\Delta J = V_1 - V_{\text{cond}}.$4, obtains $\Delta J = V_1 - V_{\text{cond}}.$5, and decodes it into a generated jump offset and relative trajectories (Wu et al., 13 May 2026).

Video conditioning is based on Wan 2.2 TI2V-5B. The model uses silhouette renderings of the reference video at resolution $\Delta J = V_1 - V_{\text{cond}}.$6, and extracts the 10-th DiT block feature as $\Delta J = V_1 - V_{\text{cond}}.$7. The data specifies that Wan operates in latent space with 30 DiT blocks and that the 10-th block output is a token tensor of shape $\Delta J = V_1 - V_{\text{cond}}.$8. Cross-attention injects these video latents into each Rectified Flow DiT block, with dynamic mesh latents as queries and video latents as keys and values. An ablation over layers $\Delta J = V_1 - V_{\text{cond}}.$9, $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$0, $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$1, $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$2, and multi-layer concatenations reports that layer 10 yields the lowest error and that multi-layer combinations underperform layer 10 alone (Wu et al., 13 May 2026).

Time conditioning uses AdaLN-Zero style modulation, and video conditioning uses classifier-free guidance enabled by randomly dropping video latents with probability $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$3 during training. The RF architecture is specified as 12 Transformer blocks with latent dimension 512. This design indicates that the video model is not retrained; instead, R-DMesh transfers spatio-temporal priors from a frozen VDM into the 3D latent space.

5. Dataset, training protocol, evaluation, and applications

To support the task, the authors construct Video-RDMesh, a dataset derived from dynamic assets in Objaverse and Objaverse-XL. The source pool contains 252,823 unique dynamic assets, mostly rig-based animations. After sequence extraction, slicing into 64-frame clips, static removal, and quality checks, the dataset contains 513,690 high-quality 64-frame vertex-trajectory clips. Each clip is paired with a Blender-rendered ground-truth video. Pose misalignment is simulated during training by choosing the conditional mesh as a random frame from the sequence rather than necessarily frame 1, forcing the model to learn arbitrary rectification jumps (Wu et al., 13 May 2026).

The training representation uses fixed topology per sample, clips of 64 frames, a maximum of 8192 vertices, a maximum of 20480 faces, and a face-to-vertex ratio below 2.5. The VAE uses 8 Triflow Attention layers in the encoder and 8 in the decoder, downsamples vertices by $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$4 with FPS, embeds inputs into 256-dimensional features, and uses latent dimensions $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$5, $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$6, and $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$7 for $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$8, $$M_{\text{cond}}=(V_{\text{cond}}\in\mathbb{R}^{N\times 3}, F\in\mathbb{Z}^{M\times 3}),$$9, and $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$0, respectively. It is trained with Adam for 200k iterations with a cosine learning-rate schedule from $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$1 to $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$2 on 32 NVIDIA H20 GPUs for about 54 hours. The RF model is trained for 300k iterations with constant learning rate $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$3 on the same cluster for about 120 hours.

Evaluation uses two 64-example test subsets from Video-RDMesh. One subset provides ground-truth dynamic meshes and frontal-view rendered videos, measured by Euclidean Distance (EucD) between generated and ground-truth vertices per frame. The second uses generated reference videos from WAN2.2-I2V-14B and evaluates PSNR, Subject Consistency (SC), and Motion Smoothness (SM) from VBench. The reported quantitative comparison is:

• SC4D: PSNR $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$4, SC $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$5, SM $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$6, runtime $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$7 m.
• L4GM: PSNR $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$8, SC $D=(V_{1:T}\in\mathbb{R}^{T\times N\times 3}, F).$9, SM $V_{1:T}$0, runtime $V_{1:T}$1 s.
• AAM: PSNR $V_{1:T}$2, SC $V_{1:T}$3, SM $V_{1:T}$4, runtime $V_{1:T}$5 s.
• PUPT: PSNR $V_{1:T}$6, SC $V_{1:T}$7, SM $V_{1:T}$8, EucD $V_{1:T}$9, runtime $V_{\text{cond}}$0 m.
• R-DMesh: PSNR $V_{\text{cond}}$1, SC $V_{\text{cond}}$2, SM $V_{\text{cond}}$3, EucD $V_{\text{cond}}$4, runtime $V_{\text{cond}}$5 s (Wu et al., 13 May 2026).

The qualitative comparisons in the data attribute characteristic failure modes to each baseline: SC4D and L4GM exhibit shape or color drift in novel views and sometimes severe spatial drift; AAM is text-driven and cannot control detailed video motions; PUPT uses optical flow and fails in radial or depth motion and cannot resolve pose mismatch. R-DMesh is described as producing 4D meshes that precisely follow video motion, maintain geometry in reference and novel views, and correctly rectify pose before moving.

The reported downstream applications are pose retargeting, motion retargeting, and holistic video-to-4D generation. For the latter, the pipeline uses Hunyuan3D 2.5 to reconstruct a static mesh from the first frame, SAM3 to segment the object across frames into silhouettes, and R-DMesh to animate the mesh. The method is stated to generalize to unseen object categories and topologies, to in-the-wild videos, and to cross-identity motion transfer, provided topology remains fixed per sample.

6. Relation to prior work, terminology, and reported limitations

Within its own research area, R-DMesh is positioned at the intersection of holistic 4D generation and skeleton-agnostic mesh animation. The comparison set includes SC4D, L4GM, AnimateAnyMesh, and Puppeteer, while the broader prior-work discussion contrasts the method with SMPL-based and skeleton-based pipelines such as MotionDiffuse and HumanMDM, and with video-guided mesh methods such as DriveAnyMesh and Mesh4D. The explicit claim is that existing video-guided methods do not explicitly disentangle pose rectification from motion, whereas R-DMesh introduces an explicit rectification jump $V_{\text{cond}}$6 to robustly handle misalignment (Wu et al., 13 May 2026).

A separate source of ambiguity is terminological. In numerical PDE literature, $V_{\text{cond}}$7-adaptivity or moving-mesh methods denote node repositioning under fixed connectivity, often driven by equidistribution, pseudo-elastic systems, or variational principles (Ameur et al., 2018, Mittal et al., 25 Jan 2026, Tyranowski et al., 2013). That body of work concerns mesh quality, validity, and solution-adaptive discretization for PDEs, not video-guided animation. This suggests that the “R” in R-DMesh should not be conflated with classical $V_{\text{cond}}$8-adaptive mesh motion, even though both settings share the broad idea of topology-preserving vertex relocation.

The reported limitations of R-DMesh are twofold. First, some generated sequences exhibit mesh self-interpenetration, attributed partly to noisy training data with self-intersections that are difficult to remove completely. Second, quality drops on rare or complex meshes, such as armored animals, because of data sparsity, leading to unnatural pose or motion. The future directions listed in the data are better data filtering for intersection-free ground truth, post-hoc optimization or constraints to reduce interpenetration, data augmentation or targeted collection for rare categories, and improved physical priors or explicit collision handling (Wu et al., 13 May 2026).

Taken together, these limitations clarify the present scope of the method. R-DMesh is a feed-forward framework for category-general, video-driven 4D mesh generation with explicit pose rectification, not a collision-aware simulation system and not a classical $V_{\text{cond}}$9-adaptive PDE mesh method. Its novelty lies in isolating discontinuous alignment from continuous motion, coupling these components through geometry-aware Triflow Attention, and generating dynamic latents with a Rectified Flow–based diffusion transformer conditioned on pre-trained video priors.