Motion-centric Dual-alignment DPO Trainer

• The paper introduces McDPO as a second-stage component that employs dual alignment to enforce both motion fidelity and text semantic consistency.
• It leverages automatically constructed preference pairs by sampling multiple motion candidates and selecting the highest and lowest scoring outputs for DPO training.
• McDPO mitigates error propagation from suboptimal video retrievals, yielding significant improvements in FID and R-Precision for 3D motion generation.

Searching arXiv for the cited papers to ground the article in current literature. {"query":"VimoRAG Video-based Retrieval-augmented 3D Motion Generation for Motion LLMs arXiv", "max_results": 5} {"query":"DenseDPO Fine-Grained Temporal Preference Optimization for Video Diffusion Models arXiv", "max_results": 5} {"query":"MoDiPO text-to-motion alignment via AI-feedback-driven Direct Preference Optimization arXiv", "max_results": 5} {"query":"(Xu et al., 16 Aug 2025)", "max_results": 5} The Motion-centric Dual-alignment DPO Trainer (McDPO) is the second-stage preference-optimization component of VimoRAG, a framework for video-based retrieval-augmented 3D motion generation for motion LLMs. Its purpose is to make a retrieval-conditioned motion LLM robust to imperfect retrieved videos by optimizing preferences over sampled motion sequences under multimodal conditioning. In VimoRAG, the generator receives a text prompt and a retrieved human-centric video, produces discrete motion tokens, and decodes them into a 3D motion sequence; McDPO then aligns this generator along two axes simultaneously—motion-modality alignment and text-motion semantic alignment—using a standard DPO objective with automatically constructed preference pairs rather than human annotations (Xu et al., 16 Aug 2025).

1. System placement and problem setting

McDPO is defined within a specific retrieval-augmented motion-generation pipeline rather than as a standalone generic DPO variant. VimoRAG addresses video-based retrieval-augmented 3D motion generation for motion LLMs, where the task is to generate a plausible, semantically aligned 3D human motion sequence from a free-form text description of a human action (Xu et al., 16 Aug 2025). The motivating data bottleneck is that HumanML3D has only about 14k motion clips, which makes motion LLMs vulnerable to out-of-domain (OOD) and out-of-vocabulary (OOV) text descriptions (Xu et al., 16 Aug 2025).

VimoRAG attempts to mitigate this limitation by retrieving relevant priors from a much larger in-the-wild human-centric video database. The retriever is the Gemini Motion Video Retriever (Gemini-MVR), and the generator is a motion LLM built on Phi-3-mini / Phi3-3.8B with LoRA finetuning (Xu et al., 16 Aug 2025). The retrieved result used in the paper is the rank-1 video. That video is embedded into segment-wise visual embeddings and fused with the textual prompt through an instruction-tuning template; the model then autoregressively generates discrete motion tokens that are decoded by VQ-VAE into the final 3D motion sequence (Xu et al., 16 Aug 2025).

McDPO sits only in the generation stage. The pipeline first performs visual demonstration-enhanced instruction tuning, producing a retrieval-conditioned motion LLM that becomes the reference model $\pi_{\text{ref}}$. McDPO then performs a second-stage preference optimization over the same text-plus-video conditioning context, training the policy model $\pi_\theta$ to produce better motion outputs under potentially noisy retrieved priors (Xu et al., 16 Aug 2025). The retriever is therefore upstream context, not the optimization target itself.

A central motivation is error propagation caused by suboptimal retrieval results. If the retrieved video is highly relevant, it can provide useful pose or action prior; if it is poor or only partially aligned with the text, conditioning on it can mislead the generator. McDPO is introduced because naive retrieval-conditioned supervised finetuning does not explicitly teach the model when to trust the retrieved video, when to ignore it, or how much to rely on it (Xu et al., 16 Aug 2025).

2. Dual alignment: motion-modality and text-motion semantics

The phrase “dual-alignment” has a precise meaning in VimoRAG. McDPO uses a preference reward that encourages generated motions to align along two axes simultaneously (Xu et al., 16 Aug 2025):

1. Motion-modality alignment: the generated motion should be close to the ground-truth motion in motion feature space.
2. Text-motion semantic alignment: the generated motion should be semantically consistent with the input text.

The aligned objects are the generated motion candidate $\hat{y}_i$, the ground-truth motion $y$, and the input text $x$. The retrieved video $v$ is part of the conditioning context, but the reward is not described as directly measuring video-motion similarity (Xu et al., 16 Aug 2025). This is a defining property of McDPO: the video acts as a potentially useful but potentially misleading prior, while preference optimization is anchored to motion fidelity and text consistency.

The first alignment compares $\hat{y}_i$ and $y$ using a motion-space distance $\ell(\hat{y}_i, y)$. The second alignment compares $\hat{y}_i$ and $\pi_\theta$0 using a semantic distance $\pi_\theta$1 (Xu et al., 16 Aug 2025). The resulting reward is

$\pi_\theta$2

The reward is negative because lower distances are better. The appendix gives the actual hyperparameters

$\pi_\theta$3

This establishes that the reward is predominantly motion-centric, with a lighter semantic correction from text (Xu et al., 16 Aug 2025).

The paper also states what remains underspecified. It says $\pi_\theta$4 is a “distribution distance” and $\pi_\theta$5 is a Euclidean distance “computed based on the features of the two inputs,” but it does not clearly specify the feature extractor used for motion features, the feature extractor mapping text and motion into the common semantic space for $\pi_\theta$6, or the exact form of the “distribution distance” (Xu et al., 16 Aug 2025). This means the dual-alignment concept is explicit, while some implementation details of the reward geometry remain implicit.

A plausible implication is that McDPO is best understood as a robustness-oriented preference design rather than a direct retrieval-consistency objective. Because the reward does not directly optimize video-motion agreement, the method can in principle learn to suppress misleading retrieved priors whenever better-scoring candidates are those that stay closer to the target motion and the text description.

3. Training procedure and mathematical formulation

McDPO is a two-stage process, not a simultaneous multi-loss mixture. Stage 1 optimizes supervised retrieval-conditioned motion generation; Stage 2 optimizes DPO on automatically constructed preference pairs (Xu et al., 16 Aug 2025).

The stage-1 input consists of text $\pi_\theta$7, a system prompt $\pi_\theta$8, an instruction template $\pi_\theta$9, and a retrieved video $\hat{y}_i$0. The video is embedded into $\hat{y}_i$1 segment-wise embeddings,

$\hat{y}_i$2

and fused with text as

$\hat{y}_i$3

The target 3D motion is encoded by VQ-VAE into discrete motion tokens

$\hat{y}_i$4

The supervised loss is

$\hat{y}_i$5

This stage produces the base model later frozen as the reference policy $\hat{y}_i$6 (Xu et al., 16 Aug 2025).

After stage 1, the reference model samples multiple candidate motions. The paper states that the model samples $\hat{y}_i$7 times to generate a candidate set, and the appendix specifies $\hat{y}_i$8 in experiments (Xu et al., 16 Aug 2025). The highest-reward candidate becomes the preferred motion $\hat{y}_i$9, and the lowest-reward candidate becomes the rejected motion $y$0, yielding the DPO dataset

$y$1

The final DPO loss is

$y$2

The appendix gives

$y$3

The optimization form is therefore standard DPO; the novelty lies in the multimodal conditioning $y$4, the motion-sequence output space, and the dual-alignment reward used for pair construction (Xu et al., 16 Aug 2025).

The training stages and roles can be summarized as follows.

Stage	Input/output role	Objective
Stage 1	Text $y$5 + retrieved video $y$6 $y$7 motion tokens $y$8	$y$9
Stage 2	Text $x$0 + retrieved video $x$1 + sampled candidates $x$2	$x$3

The paper explicitly states that McDPO is not a fundamentally new DPO equation; it is a motion-centric instantiation of DPO for retrieval-augmented text-to-motion generation (Xu et al., 16 Aug 2025). This distinguishes it from formulations that generalize DPO itself, such as Multi-Preference Lambda-weighted Listwise DPO, which extends DPO to simplex-weighted mixtures of multiple preference dimensions and listwise supervision but is not motion-specific (Sun et al., 24 Jun 2025).

4. Preference construction, representations, and retrieval-noise robustness

McDPO constructs supervision automatically rather than from human preference annotation. The process is: train the retrieval-conditioned motion LLM via SFT; freeze it as $x$4; sample multiple outputs $x$5 from $x$6; score each candidate with the dual-alignment reward; select the best as $x$7 and the worst as $x$8; and train the policy with DPO on these preference pairs (Xu et al., 16 Aug 2025).

This design makes the method explicitly self-referential in the sense used by the paper: the preference labels come from the reference model’s own sampled outputs plus a heuristic dual-alignment reward, not from manual annotation or a separately trained reward model (Xu et al., 16 Aug 2025). That structure resembles the AI-feedback logic used in MoDiPO, which also avoids human preference collection by ranking generated motions with pretrained text-motion retrieval models and then applying DPO to the resulting winner-loser pairs (Pappa et al., 2024). The difference is that MoDiPO is centered on text-to-motion diffusion alignment, whereas McDPO is centered on retrieval-conditioned motion language modeling (Pappa et al., 2024, Xu et al., 16 Aug 2025).

During McDPO training, the model sees the text prompt $x$9, the retrieved video $v$0, the ground-truth motion $v$1 for reward computation and stage-1 SFT, and the sampled candidate motions $v$2 from $v$3 (Xu et al., 16 Aug 2025). The generation model uses InternVideo2 as the video encoder, CLIP-Large as the image encoder, and a two-layer MLP visual projector; the maximum number of video frames during generation is 16 (Xu et al., 16 Aug 2025). Ground-truth 3D motion is discretized into motion tokens using VQ-VAE, and the appendix says the motion feature dimension is 263, following HumanML3D conventions, with 22 joints (Xu et al., 16 Aug 2025).

The instruction template used by the generator is also semantically significant:

Generate a sequence of motion tokens matching the following human motion description. You can use the video as a reference. Video information: {Retrieved Video $v$4} Motion description: {Input Text $v$5}

This wording frames the retrieved video as optional evidence rather than an authoritative target. McDPO sharpens that behavior through preference optimization (Xu et al., 16 Aug 2025).

The central practical claim is that McDPO mitigates retrieval-noise error propagation in three ways. First, it trains on the model’s own failure modes, because the candidate outputs are sampled from $v$6 under the same text-plus-video conditioning (Xu et al., 16 Aug 2025). Second, preference is anchored to text and target motion rather than retrieval agreement, so the optimized model is not rewarded for blindly matching the retrieved video (Xu et al., 16 Aug 2025). Third, DPO provides a relative training signal under noisy context: among several plausible outputs under the same retrieved-video context, the model is trained to prefer the more text-faithful and motion-faithful one (Xu et al., 16 Aug 2025).

This suggests a methodological affinity with DenseDPO, which argues that preference optimization for motion-rich video requires better alignment between compared samples and better localization of preference supervision over time (Wu et al., 4 Jun 2025). DenseDPO achieves that through structurally aligned pair construction and segment-level preferences (Wu et al., 4 Jun 2025). McDPO does not introduce temporal segment labels, but it addresses a parallel credit-assignment problem in retrieval-augmented motion generation: the generator must learn when retrieved evidence is informative and when it is misleading.

5. Implementation details and empirical evidence

The implementation details reported for McDPO are specific. In Stage 1, the backbone LLM is phi-3-mini / Phi3-3.8B; tuning uses LoRA; LoRA parameters and the visual adapter are trained while other modules are frozen (Xu et al., 16 Aug 2025). The main stage-1 settings are: 2 epochs on the HumanML3D training set, LoRA learning rate $v$7, visual adapter learning rate $v$8, LoRA rank $v$9, alpha $\hat{y}_i$0, batch size $\hat{y}_i$1, maximum context length $\hat{y}_i$2, precision bf16, maximum video frames during generation 16, and LLM temperature 0.9 (Xu et al., 16 Aug 2025).

In Stage 2, only the LLM is updated; all modules except for the LLM are kept frozen (Xu et al., 16 Aug 2025). The reported settings are 1 epoch, learning rate $\hat{y}_i$3, batch size $\hat{y}_i$4, weight decay $\hat{y}_i$5, and DPO coefficient $\hat{y}_i$6 (Xu et al., 16 Aug 2025). For DPO dataset construction, the appendix states that sampling is performed on a random 25% subset of the training set; for each input, the model samples $\hat{y}_i$7 outputs from $\hat{y}_i$8, computes the dual-alignment reward for each, and chooses the highest-reward output as positive $\hat{y}_i$9 and the lowest-reward output as negative $y$0 (Xu et al., 16 Aug 2025).

The largest reproducibility caveat remains the exact feature extraction used for $y$1 and $y$2, which the provided paper text does not fully operationalize (Xu et al., 16 Aug 2025). This is an important limitation because the reward model determines the preference pairs.

The empirical evidence specific to McDPO is strongest in the HumanML3D validation ablation. From Table 3 / Table $y$3, the paper reports (Xu et al., 16 Aug 2025):

• Gem+Mc: FID 0.148, Top-1 0.429, Top-2 0.625, Top-3 0.756
• Gem: FID 0.260, Top-1 0.403, Top-2 0.582, Top-3 0.682

Thus, adding McDPO to the same Gemini-MVR retrieval setup improves FID from 0.260 to 0.148 and Top-3 R-Precision from 0.682 to 0.756 (Xu et al., 16 Aug 2025). The paper presents this as the cleanest attribution of performance gain to McDPO itself.

The crossover robustness analysis is even more diagnostic. Figure 1 / Figure $y$4 compares four conditions (Xu et al., 16 Aug 2025):

• NMc-R1: no McDPO, rank-1 video at inference
• Mc-R1: McDPO, rank-1 video
• Mc-R$y$5: McDPO, random video
• NMc-R$y$6: no McDPO, random video

The reported values are (Xu et al., 16 Aug 2025):

• NMc-R1: FID = 0.260, Top-3 R-Precision = 0.682
• Mc-R1: FID = 0.148, Top-3 R-Precision = 0.756
• Mc-R$y$7: FID = 0.186, Top-3 R-Precision = 0.732
• NMc-R$y$8: FID = 0.346, Top-3 R-Precision = 0.670

The crucial result is that Mc-R$y$9 is better than NMc-R$\ell(\hat{y}_i, y)$0 even when inference uses a random video prior (Xu et al., 16 Aug 2025). The authors interpret this as evidence that McDPO enables the model to disregard non-informative video priors. Within the evidence presented, this is the clearest demonstration that McDPO functions as a retrieval-noise robustness mechanism rather than merely as a generic finetuning stage.

On the HumanML3D test set, the full VimoRAG system reports FID 0.131, Top-1 0.452, Top-2 0.655, Top-3 0.764, MM Dist 3.146, compared with MotionGPT (Phi3-3.8B) at FID 0.501, Top-1 0.396, Top-2 0.575, Top-3 0.673, MM Dist 3.724 (Xu et al., 16 Aug 2025). On IDEA400 zero-shot, the paper reports MotionGPT: FID 5.544 and VimoRAG: FID 2.388 (Xu et al., 16 Aug 2025). These gains are attributed to the entire VimoRAG framework rather than to McDPO alone, but the ablations indicate that McDPO is a major contributor.

6. Relation to adjacent DPO trainers, limitations, and interpretation

McDPO is best situated among a family of post-training preference-optimization methods that adapt DPO beyond standard language-response preference learning. MoDiPO demonstrates that DPO can be applied to text-to-motion diffusion by constructing motion-preference pairs from AI feedback and substantially improving FID while retaining RPrecision and Multi-Modality (Pappa et al., 2024). DenseDPO shows that for text-to-video diffusion, independent-noise DPO can induce a low-motion bias, and that better motion generation requires aligned pair construction and temporally dense preference supervision (Wu et al., 4 Jun 2025). McDPO differs from both in that it is targeted at retrieval-augmented 3D motion generation with a multimodal conditioning context $\ell(\hat{y}_i, y)$1 and a dual-alignment reward over complete motion candidates (Xu et al., 16 Aug 2025).

By contrast, Multi-Preference Lambda-weighted Listwise DPO extends DPO to multi-objective, simplex-weighted preference mixtures and listwise supervision, enabling dynamic interpolation across dimensions such as helpfulness and harmlessness (Sun et al., 24 Jun 2025). That work is directly relevant to multi-objective preference-optimization trainer design but only indirectly relevant to McDPO, because McDPO’s “dual alignment” does not use a simplex-weighted interpolation mechanism and is not formulated as dynamic deployment-time control over two explicit preference channels (Sun et al., 24 Jun 2025). A plausible implication is that McDPO could be viewed as a fixed-weight two-axis preference design, whereas lambda-weighted listwise DPO is a more general framework for controllable preference mixtures.

A different but conceptually related line is divergence-point preference learning for multi-turn tool-calling agents, where DPO supervision is localized at matched branch points and trained under the same control context used at inference (Tang, 22 Jun 2026). The connection is methodological rather than domain-specific: both approaches attempt to align preference learning with the actual failure modes encountered under deployment-like conditioning. McDPO does this by sampling motion candidates under retrieved-video conditioning and constructing preferences from those failures; divergence-point preference learning does it by aligning trajectories at comparable states or prefixes before applying DPO (Tang, 22 Jun 2026).

Several limitations are explicit or strongly implied in the McDPO description. Preference pairs depend on the quality and diversity of candidates sampled from $\ell(\hat{y}_i, y)$2; if those samples are all poor, the preference signal may be weak (Xu et al., 16 Aug 2025). The reward is heuristic and not based on human judgments (Xu et al., 16 Aug 2025). The exact feature encoders behind $\ell(\hat{y}_i, y)$3 and $\ell(\hat{y}_i, y)$4 are underspecified (Xu et al., 16 Aug 2025). The dual alignment is between generated motion and reference motion, and between generated motion and text; it is not explicitly a text-video and motion-video dual alignment (Xu et al., 16 Aug 2025). The paper’s explicit limitation section also emphasizes latency, since an LLM-based generator is slower than specialist motion models (Xu et al., 16 Aug 2025).

The most concise interpretation is that McDPO is the mechanism that makes retrieval augmentation in VimoRAG selective rather than obedient. It does not assume that the retrieved video is always helpful. Instead, it uses DPO over sampled motion sequences to teach a retrieval-conditioned motion LLM to exploit the video when it is informative and suppress it when it is misleading (Xu et al., 16 Aug 2025). In that sense, the “motion-centric” qualifier refers both to the dominant reward weight $\ell(\hat{y}_i, y)$5 and to the fact that the compared candidates are complete motion sequences rather than language outputs (Xu et al., 16 Aug 2025).