Part-aware Motion Modulation (PMM)

• Part-aware Motion Modulation (PMM) is a method that decomposes human motion into distinct body parts with frame-wise editability for precise and localized control.
• It employs semantic body decomposition and transformer-based feature encoding to generate part-specific modulation weights, ensuring dynamic and interpretable motion synthesis.
• Empirical results show PMM significantly improves text-driven motion editing and robust motion generation from noisy or incomplete real-world video data.

Part-aware Motion Modulation (PMM) is a methodological paradigm for spatiotemporal localized control in generative models of human motion. Rather than treating the human body as a globally unified entity, PMM frameworks decompose the body into discrete, semantically meaningful parts and predict part-specific, temporally resolved “editability” or modulation weights. This enables dynamic, interpretable, and fine-grained selective editing or synthesis of motion in direct response to conditioning signals such as text or multimodal inputs. Recent instantiations of PMM have facilitated significant advancements in both text-driven motion editing and robust motion generation from partially observed, noisy real-world video data (Yang et al., 30 Dec 2025, Li et al., 14 Dec 2025).

1. Formalization and Overall Role of PMM

In canonical PMM frameworks, the human skeleton is decomposed into $P=5$ standard parts: torso, left arm, right arm, left leg, and right leg. For each of these components, PMM predicts a continuous, frame-wise modulation weight that gates the degree of motion feature modification in response to user intent or data reliability. This weight matrix $R \in [0,1]^{T \times 5}$ (with $T$ denoting sequence length) enables precise spatial isolation: only those parts whose $R_{i, t}$ are high are subject to strong editing or generative transformations, preserving coherence elsewhere. In text-driven editing architectures such as PartMotionEdit (Yang et al., 30 Dec 2025), PMM is realized as an intermediary between cross-modal encoding modules and the generative denoiser, taking as input the instruction-aware motion features and outputting modulated representations for diffusion-based synthesis.

2. Body Part Decomposition and Feature Encoding

A standardized body decomposition underpins the semantics and technical execution of PMM. Using a 22-joint kinematic skeleton, five sets of joints correspond to the torso (root, spine segments, neck, head), bilateral arms (shoulder, elbow, wrist), and bilateral legs (hip, knee, ankle). Each part possesses a learnable query vector (dimension $D$) serving as a semantic prototype. Temporal softmax attention aligns these queries with feature tensors across $T$ frames to yield part-specific attention maps $A_i \in \mathbb{R}^T$. Aggregated via weighted sums, these produce part embeddings $z_i \in \mathbb{R}^D$, concatenated across parts and further processed via a compact transformer (2 layers, $d=256$, 4 heads) to model inter-part correlations. This architecture enables both spatial (part-level) and temporal (frame-level) localization of modulation signals (Yang et al., 30 Dec 2025).

3. Modulation Weight Prediction and Application

The central output of PMM is the matrix $R \in [0,1]^{T\times 5}$, computed by passing the part embeddings through a nonlinear mapping comprising GELU activations and sigmoid gating:

$$R = \sigma(W_2 \cdot \mathrm{GELU}(W_1 \cdot \hat{Z}))$$

where $W_1, W_2$ are learned projections. Each $R_{i,t}$ specifies the editability for part $i$ at time $t$. The modulation is realized via a gated residual pathway:

$$F^{\prime\prime}_m = F^{\prime}_m + R \odot \mathrm{MLP}(F^{\prime}_m)$$

with $\odot$ denoting broadcasted element-wise multiplication. This structure ensures that the diffusion backbone only substantially modifies features for parts and times where $R_{i, t}$ approaches 1, effecting fine-grained, user-controllable edits or robust generation (Yang et al., 30 Dec 2025).

4. Supervisory Signals and Loss Functions

PMM is supervised by ground-truth similarity curves that measure per-part, per-frame distances between source and target motions. For each part $g_i$ at frame $t$: $$D_{i,t}^{pos} = \frac{1}{|g_i|} \sum_{j \in g_i} \|X_{t,j}^{src} - X_{t,j}^{tgt}\|_2$$

$$D_{i,t}^{rot} = \frac{1}{|g_i|} \sum_{j \in g_i} \|R_{t,j}^{src} - R_{t,j}^{tgt}\|_2$$

Resulting similarity scores are normalized twice (across dataset and within motion) to produce $Y_{i, t} \in [0,1]$. The regression loss enforces $R \approx Y$: $$\mathcal{L}_{PSM} = \frac{1}{N T} \sum_{i=1}^{5} \sum_{t=1}^T \|R_{i, t} - Y_{i, t}\|_2$$ Temporal smoothness is further encouraged: $$\mathcal{L}_{smooth} = \frac{1}{5 (T-1)} \sum_{i=1}^{5} \sum_{t=1}^{T-1} |R_{i, t+1} - R_{i, t}|_1$$ so that PMM outputs temporally coherent gating. When combined ($$\mathcal{L}_{PMM} = \mathcal{L}_{PSM} + \lambda_s\mathcal{L}_{smooth}$$, $\lambda_s = 0.1$), and joined with the denoising loss ($\mathcal{L}_{DDPM}$), these signals yield high editability fidelity and robust alignment with semantic intent (Yang et al., 30 Dec 2025).

5. Extensions to Robust Motion Generation from Videos

PMM principles are also integral to learning from noisy, in-the-wild video data, where part observability is variable. In RoPAR (Li et al., 14 Dec 2025), a part-level variational autoencoder encodes only “credible” parts ($C_p > \tau$ confidence) into latent tokens, masking noisy/unreliable regions. A part-aware autoregressive transformer predicts masked credible tokens conditioned on text and a credible-part mask $M$, while all noisy parts are omitted from reconstruction and gradients. This selective modulation, coupled with a final diffusion denoising stage, enables robust full-body motion generation even with incomplete observations: $$p(Z|T,M) = \prod_{i=1}^N \prod_{p=1}^P p(z^p_i|Z \circ M, T)$$

$$\mathcal{L}_{AR} = -\sum_{(i, p) \in \mathcal{I}} \log p_\theta\left(z^p_i \mid Z_{\setminus(i,p)}, T\right)$$

Performance on benchmarks such as K700-M demonstrates that by isolating credible part signals and enforcing part-wise masking, RoPAR outperforms prior global and unstructured modeling approaches under noisy data conditions (Li et al., 14 Dec 2025).

6. Empirical Results and Ablations

Ablation studies show that PMM yields incremental benefits in editability even in the absence of part-similarity supervision, but that explicit similarity curve supervision provides a larger gain:

• No PMM, no similarity supervision: AvgR=2.31
• PMM only: AvgR=2.29
• Similarity curve supervision only: AvgR=2.19
• PMM with part-level similarity (full): AvgR=1.92 (best) (Yang et al., 30 Dec 2025)

Qualitative analysis confirms that in tasks such as “raise right arm only,” PMM generates high, temporally localized $R_{i, t}$ values solely for the right arm, avoiding unwanted crosstalk across parts and temporal leakage endemic to previous global models.

7. Context and Significance

Part-aware Motion Modulation represents a paradigm shift in 3D human motion modeling, enabling spatially selective and temporally precise control over generative processes. By organizing computation and supervision along part boundaries, PMM frameworks close the gap between global semantic conditioning and localized, interpretable motion edits. This approach yields superior text–motion alignment, robustness to noisy or incomplete part observations, and preservation of global motion coherence. The co-evolution of PMM with architectures such as PartMotionEdit and RoPAR underlines the efficacy of decoupling part-level representation learning from global motion synthesis (Yang et al., 30 Dec 2025, Li et al., 14 Dec 2025).

A plausible implication is that future research will increasingly leverage part-aware modulation not only for human body synthesis, but for other articulated agents and in heterogeneous sensory integration, as the partitioned attention–modulation pipeline inherently supports modular, selective processing regimes.