Motion Condition Injection in Diffusion Models

• Motion Condition Injection is a design pattern that uses explicit motion cues as conditioning variables for diffusion-based motion generation and control.
• It employs various methods such as concatenation, residual branches, and part-aware attention to integrate motion structures into generative models.
• Applications span robot imitation learning, multi-condition human motion synthesis, and unified motion editing with measurable performance gains.

Searching arXiv for papers and disambiguations of “Motion Condition Injection” and closely related usages. arXiv search query: "Motion Condition Injection" OR motion-conditioned injection diffusion motion generation manipulation Motion Condition Injection (MCI), as used in recent diffusion-based manipulation and motion-generation work, denotes the use of motion representations as explicit conditioning variables inside a downstream generator or denoiser, rather than as auxiliary outputs. In this formulation, predicted or provided motion—such as an object pose trajectory, localized body-part style, trajectory constraints, or source motion—is injected into action generation or motion synthesis so that the model reasons over motion structure while producing control or kinematic outputs. Recent implementations span robot imitation learning, multi-condition human motion synthesis, part-aware speech-driven 3D motion generation, and unified motion generation and editing (Su et al., 2024, Ling et al., 2023, Liu et al., 15 Oct 2025, Guo et al., 4 Feb 2025).

1. Conceptual formulations

A canonical factorization appears in MBA (Motion Before Action), which writes

$$p(\mathbf{M},\mathbf{A}|\mathbf{O}) = p(\mathbf{M}|\mathbf{O})\, p(\mathbf{A}|\mathbf{M},\mathbf{O}),$$

where $\mathbf{O}$ is the observation, $\mathbf{M}$ is the object motion or pose sequence, and $\mathbf{A}$ is the robot action sequence. In this factorization, $p(\mathbf{M}|\mathbf{O})$ is the motion predictor and $p(\mathbf{A}|\mathbf{M},\mathbf{O})$ is the action generator conditioned on motion. The defining operation is that the predicted object motion is not merely decoded; it is reused as a conditioning variable for the action diffusion head (Su et al., 2024).

A broader motion-centric formulation appears in MotionLab, which organizes tasks around source motion $M_S$, condition $C$, and target motion $M_T$, and learns the mapping

$(M_S, C) \rightarrow M_T.$

In this scheme, generation tasks are the special case in which source motion is empty, whereas editing tasks are those in which source motion is present and the model produces a modified target motion based on the specified conditions. This makes motion itself part of the conditioning interface, rather than only the output domain (Guo et al., 4 Feb 2025).

These formulations indicate that MCI is not tied to a single architectural motif. A plausible implication is that the term is best understood as a design pattern in which motion variables mediate downstream generation, with the exact mechanism varying across concatenation-based conditioning, residual control branches, part-aware attention, and source-to-target flow modeling.

2. Motion-conditioned action generation in robotic manipulation

MBA implements MCI as a two-stage cascade for robot imitation learning. The pipeline is: encode the current observation into observation features; predict a future object motion sequence from those features using a diffusion model; convert the predicted motion sequence into a motion feature vector; inject that motion feature as an additional condition into the policy’s diffusion action head; and generate robot action sequences under both observation and object-motion guidance. The motion sequence is

$\mathbf{O}$0

and each object pose $\mathbf{O}$1 is a $\mathbf{O}$2-dimensional vector consisting of $\mathbf{O}$3D translation and a $\mathbf{O}$4D rotation representation from $\mathbf{O}$5, so $\mathbf{O}$6 and $\mathbf{O}$7. The paper explicitly enforces $\mathbf{O}$8 so that the predicted motion horizon is at least as long as the action horizon (Su et al., 2024).

The two diffusion stages use parallel reverse-update forms. For motion diffusion,

$\mathbf{O}$9

and for motion-guided action diffusion,

$\mathbf{M}$0

After motion diffusion produces a clean sequence $\mathbf{M}$1, the sequence is unfolded along the $\mathbf{M}$2 axis into $\mathbf{M}$3, then passed through MLP layers $\mathbf{M}$4, producing the motion feature vector $\mathbf{M}$5. The action noise predictor receives $\mathbf{M}$6, $\mathbf{M}$7, $\mathbf{M}$8, and diffusion step $\mathbf{M}$9, and the paper states that the features are concatenated to predict noise jointly. It also states that MBA does not describe a special cross-attention block for MBA itself; instead, it augments the existing diffusion action head with the motion feature. MoCap is required for training supervision in real-world experiments, but not used at deployment (Su et al., 2024).

MBA is presented as a plug-and-play module for policies with diffusion action heads, including DP, DP3, and RISE. On 57 tasks across Adroit, DexArt, and MetaWorld Easy/Medium/Hard/VeryHard, the reported average success rates are 71.3% for DP3, 77.5% for DP3 + MBA, 53.6% for DP, and 67.8% for DP + MBA, corresponding to gains of +6.2% over DP3 and +14.2% over DP. The paper also reports that MBA reduces variance. In real-world tasks, MBA improves RISE on Cut Clay, Put Bread into Pot, Open Drawer, and Pour Balls; for example, Open Drawer success changes from 37.5% to 52.5%, and in Put Bread into Pot the success changes from 80% to 95% while redundant attempts change from 7 to 1 (Su et al., 2024).

3. Residual control injection in multi-condition human motion synthesis

MCM treats motion condition injection as a ControlNet-style add-on for DDPM-like motion generators. Its central design is a two-branch architecture consisting of a main branch and a control branch. The main branch is the base motion diffusion model; the control branch has the same architecture as the main branch and is initialized with the parameters of the main branch. During control-branch training, all parameters except the control branch and the bridge modules are frozen. The bridge modules are fully connected layers or $\mathbf{A}$0D convolutions, initialized to zero, and their outputs are added to the corresponding main-branch features. Because the bridge starts at zero, the control branch does not disrupt the main branch early in training and instead learns residual conditioning signals (Ling et al., 2023).

MCM is explicitly designed to work with any DDPM-like backbone, including MotionDiffuse, MDM, and the paper’s own MWNet. It supports text, music, speech, and other control signals. The implementation uses pretrained encoders: CLIP-base for text, Jukebox for music, and HuBERT-base for speech. The extracted features are projected to a common dimension and concatenated. The paper emphasizes that MCM can accept multiple control signals simultaneously and generalize to combinations not seen during training, such as text + music for dance generation (Ling et al., 2023).

The MWNet backbone extends this control framework with channel-dimension self-attention. In addition to standard time-wise self-attention,

$\mathbf{A}$1

MWNet introduces channel-wise self-attention,

$\mathbf{A}$2

to model spatial complexity and inter-joint correlations in motion sequences. The paper’s motion representation is 263-dimensional per frame, with 22 joints from SMPL, a frame rate of 20 FPS, 1000 diffusion steps, and a linear $\mathbf{A}$3 schedule from 0.0001 to 0.02 (Ling et al., 2023).

On HumanML3D, MWNet reports R-Precision Top-1 $\mathbf{A}$4, Top-2 $\mathbf{A}$5, Top-3 $\mathbf{A}$6, FID $\mathbf{A}$7, MultiModal Distance $\mathbf{A}$8, Diversity $\mathbf{A}$9, and MultiModality $p(\mathbf{M}|\mathbf{O})$0. On AIST++, MWNet + MCM reports FID $p(\mathbf{M}|\mathbf{O})$1, Div $p(\mathbf{M}|\mathbf{O})$2, and BAS $p(\mathbf{M}|\mathbf{O})$3, compared with EDGE at FID $p(\mathbf{M}|\mathbf{O})$4, Div $p(\mathbf{M}|\mathbf{O})$5, and BAS $p(\mathbf{M}|\mathbf{O})$6. The paper states that qualitative results show effective multi-condition modal control across text-to-motion, music-to-dance, and speech-to-gesture (Ling et al., 2023).

4. Part-aware injection of style, rhythm, and emotion

MimicParts defines MCI as a part-aware, multi-condition conditioning strategy inside a latent diffusion model. The framework has three main components: a Part-aware Style Encoder, a Part-aware Motion Latent Diffusion Model, and an RVQ-VAE decoder. The body is explicitly divided into three regions—upper body $p(\mathbf{M}|\mathbf{O})$7, lower body $p(\mathbf{M}|\mathbf{O})$8, and hands $p(\mathbf{M}|\mathbf{O})$9—and this decomposition is used both for localized style encoding and for part-aware denoising and attention. The paper argues that motion style is not uniform over the full body, and that global style extractors lose regional differences (Liu et al., 15 Oct 2025).

Instead of using one global style embedding, MimicParts trains separate style encoders per body part. The style encoder is trained with the NT-Xent loss,

$p(\mathbf{A}|\mathbf{M},\mathbf{O})$0

under the assumption that short, temporally adjacent segments from the same person tend to share the same style. The denoiser is conditioned on localized style $p(\mathbf{A}|\mathbf{M},\mathbf{O})$1, content $p(\mathbf{A}|\mathbf{M},\mathbf{O})$2, rhythm $p(\mathbf{A}|\mathbf{M},\mathbf{O})$3, and emotion $p(\mathbf{A}|\mathbf{M},\mathbf{O})$4,

$p(\mathbf{A}|\mathbf{M},\mathbf{O})$5

with training loss

$p(\mathbf{A}|\mathbf{M},\mathbf{O})$6

The paper’s part-aware attention block first splits features $p(\mathbf{A}|\mathbf{M},\mathbf{O})$7 and then applies region-specific attention,

$p(\mathbf{A}|\mathbf{M},\mathbf{O})$8

Content and localized style are combined by

$p(\mathbf{A}|\mathbf{M},\mathbf{O})$9

after which rhythm and emotion are progressively injected through cross-attention (Liu et al., 15 Oct 2025).

At inference time, MimicParts uses incremental classifier-free guidance: $M_S$0 where $M_S$1, $M_S$2, and $M_S$3 control content guidance, style guidance, and rhythm/emotion guidance, respectively, and training-time condition dropout uses probability $M_S$4. The speech-side features are extracted by Wav2Vec2 for content, BEATs for rhythm, and Emotion2Vec for emotion (Liu et al., 15 Oct 2025).

The reported single-speaker results are best BC $M_S$5, best Diversity $M_S$6, and second-best FGD $M_S$7. For multi-speaker results, the reported values are FGD $M_S$8, BC $M_S$9, Diversity $C$0, and SRA $C$1. Ablation results make the conditioning role explicit: removing the part-aware style encoder changes FGD to $C$2, BC to $C$3, Diversity to $C$4, and SRA to $C$5; removing rhythm and emotion changes FGD to $C$6, BC to $C$7, Diversity to $C$8, and SRA to $C$9 (Liu et al., 15 Oct 2025).

5. Motion-as-condition in unified generation and editing

MotionLab extends MCI from conditional generation to a unified motion transformation framework. Its Motion-Condition-Motion paradigm treats source motion, condition, and target motion as first-class variables, and uses rectified flows to learn the mapping between them. The generic rectified-flow objective is

$M_T$0

and for straight-line interpolation $M_T$1, the velocity becomes $M_T$2. In MotionLab terms, the model learns how to transform $M_T$3 into $M_T$4 based on $M_T$5 (Guo et al., 4 Feb 2025).

The architectural carrier of this conditioning is the MotionFlow Transformer (MFT), which combines Joint Attention, a Condition Path, and Aligned ROPE. Joint Attention concatenates tokens from different modalities into a single sequence so that source motion, target motion, condition, and instruction can interact directly. The Condition Path introduces adaptive LayerNorm (adaLN) and a modulation mechanism for each modality, allowing the same attention machinery to behave differently for source motion, target motion, text, trajectory, style, and instruction tokens. Task Specified Instruction Modulation adds a task-specific natural-language instruction

$M_T$6

extracted from CLIP, with examples such as “generate motion by given text.” and “edit source motion by given text.” (Guo et al., 4 Feb 2025).

MotionLab also uses Aligned ROPE, described as a 1D ROPE with appropriate temporal alignment across modalities, to preserve synchronization between source and target motion. The training procedure is Motion Curriculum Learning, split into self-supervised pre-training and supervised fine-tuning. The pre-training anchor task is masked source motion reconstruction, and the fine-tuning stage introduces tasks in an easy-to-hard sequence while mixing previously learned tasks to mitigate forgetting. The unified task set includes unconditional generation, masked reconstruction, reconstruction, text-based generation, trajectory-based generation, motion in-between, style-based generation, text-based editing, trajectory-based editing, and style transfer (Guo et al., 4 Feb 2025).

At inference time, MotionLab uses classifier-free guidance for both generation and editing. For generation,

$M_T$7

and for editing,

$M_T$8

The paper reports promising generalization capabilities and inference efficiency across multiple benchmarks for human motion, and its ablation study states that removing rectified flows, the MotionFlow Transformer, Aligned ROPE, task instruction modulation, or motion curriculum learning degrades performance across tasks (Guo et al., 4 Feb 2025).

6. Scope, distinctions, and acronym disambiguation

A recurring source of confusion is that “MCI” is not a stable acronym across arXiv. In the motion-generation and manipulation papers discussed above, it refers to motion-conditioned injection mechanisms. In other literatures, however, the same acronym denotes unrelated concepts. In “Generalized TASEP on open chains with a modified injection condition,” the relevant term is a modified injection condition at the left boundary of an open-chain generalized TASEP, with $M_T$9 in the analytically solvable version; this is not motion conditioning in the machine-learning sense (Povolotsky et al., 2022). In “GFE-Mamba: Mamba-based AD Multi-modal Progression Assessment via Generative Feature Extraction from MCI,” MCI means Mild Cognitive Impairment, specifically the MCI-to-AD progression prediction problem (Fang et al., 2024). In “A closed-form approximate incoherent GN-model supporting MCI contributions,” MCI means multi-channel interference, defined as all triplets that are neither SCI nor XCI in the GN-model decomposition (Zefreh et al., 2019).

Within motion-related research itself, MCI also does not imply one mandatory operator. MBA uses feature-level injection by concatenating a motion feature $(M_S, C) \rightarrow M_T.$0 with observation features in the action diffusion head rather than introducing a special cross-attention block (Su et al., 2024). MimicParts uses part-aware self-attention, cross-attention, and incremental classifier-free guidance to inject localized style, rhythm, and emotion (Liu et al., 15 Oct 2025). MotionLab moves further by treating source motion itself as an input modality and combining it with conditions through joint attention, modality-specific modulation, and task instructions (Guo et al., 4 Feb 2025). This suggests that the unifying feature of MCI is not a specific layer type, but the explicit operational role of motion as a conditioning variable in generation, control, or editing.

A second misconception is that motion conditioning is equivalent to post hoc correction. The papers do not describe it that way. MBA conditions the action denoiser directly on predicted object motion; MimicParts injects regional style and speech-derived cues during denoising; and MotionLab learns source-to-target motion transformation under conditions rather than generating first and editing afterward (Su et al., 2024, Liu et al., 15 Oct 2025, Guo et al., 4 Feb 2025). In this sense, MCI is best viewed as internal conditional computation over motion structure rather than external post-processing.