---
title: Motion Score Distillation (MSD)
url: https://www.emergentmind.com/topics/motion-score-distillation-msd
type: topic
---

# Motion Score Distillation (MSD)

Searching arXiv for recent papers on Motion Score Distillation and related formulations.
Searching arXiv for "motion score distillation".
Motion Score Distillation (MSD) denotes a family of score-distillation procedures that use pretrained diffusion priors to guide motion-preserving video editing, text-driven 3D animation, or direct motion-sequence refinement. In the cited literature, the term is not tied to a single universally fixed loss. Rather, it is used for closely related constructions that distill denoising scores from video diffusion models or motion diffusion models into an optimizable target, with the target taking the form of video frames, a deformable body model, a Gaussian-based 3D asset, or an explicit motion sequence. Across these variants, the central idea is to replace full reverse diffusion with gradient-based optimization driven by a pretrained prior, while introducing additional constraints to preserve motion, appearance, structure, or physical plausibility [2411.12831][2403.12002][2512.12534][2511.00248].

## 1. Terminological scope and historical placement

MSD is best understood as an outgrowth of Score Distillation Sampling (SDS). In the standard SDS view, a pretrained diffusion model supplies a score estimate that can be back-propagated through a downstream generator. "Towards motion from video diffusion models" extends this logic from images to videos through a temporal SDS variant, asking whether publicly available text-to-video diffusion models can guide realistic human body animation by deforming an SMPL-X representation [2411.12831].

"DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editing" uses the expression "Motion Score Distillation" for an extension of SDS or Delta Denoising Score to videos. In that setting, the optimizable object is not a newly synthesized motion source but an existing source video whose real motion should be preserved while new appearance is injected by a pretrained text-to-video prior [2403.12002].

"Animus3D: Text-driven 3D Animation via Motion Score Distillation" assigns MSD a more specific meaning. It introduces MSD as an SDS alternative that explicitly contrasts a static source distribution with a dynamic target distribution, using the difference between dynamic and static denoiser predictions as the motion-driving term. This formulation is designed for animating a static 3D asset while preserving appearance [2512.12534].

A related but distinct nomenclature appears in "Object-Aware 4D Human Motion Generation," which proposes Motion Diffusion Score Distillation Sampling (MSDS). There the distilled prior is a pretrained motion diffusion model operating directly in motion space rather than a video diffusion model operating on rendered frames or latents [2511.00248].

This usage pattern suggests that MSD is less a single algorithm than a research direction: score distillation specialized to motion-centric optimization problems.

## 2. Core mathematical formulations

The common starting point is the SDS identity used with a latent diffusion denoiser $\hat\epsilon_\phi$. In "Towards motion from video diffusion models," the generator parameters $\theta$ are updated by
$$
\nabla_\theta L_{\mathrm{SDS}}(\phi,z)
=
\mathbb{E}_{t,\epsilon}
\left[
w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta}
\right],
$$
with $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$. The paper interprets $(\hat\epsilon-\epsilon)$ as an unbiased estimator of the diffusion model's score and extends this to videos through temporal SDS on a latent tensor $Z$, yielding
$$
\nabla_\alpha L_{\mathrm{SDS-T}}
=
\lambda_{\mathrm{SDS}}
\mathbb{E}_{\sigma,\epsilon}
\left[
w(\sigma)\cdot (\hat\epsilon(Z_{\alpha,\sigma,\epsilon}\mid y,\sigma)-\epsilon)\cdot \frac{\partial Z_\alpha}{\partial \alpha}
\right],
$$
where $\alpha$ denotes the PoseField parameters and $\lambda_{\mathrm{SDS}}$ is set to $1\mathrm{e}{-3}$ [2411.12831].

DreamMotion formulates video-level motion-preserving distillation through Video-DDS:
$$
L_{\mathrm{V-DDS}}(\theta;y)
=
\mathbb{E}_{t,\epsilon}
\left\|
\epsilon_\phi^w(x_t^{1:N}(\theta),t,y)
-
\epsilon_\phi^w(\hat x_t^{1:N},t,\hat y)
\right\|_2^2.
$$
Here the target video and source video are noised by the same $(t,\epsilon)$. This is coupled with spatial and temporal self-similarity matching losses extracted from U-Net self-attention features, producing a total objective
$$
L_{\mathrm{total}}(\theta)
=
L_{\mathrm{V-DDS}}(\theta;y)
+\lambda_s L_{\mathrm{S-SSM}}(\theta)
+\lambda_t L_{\mathrm{T-SSM}}(\theta),
$$
with $\lambda_s,\lambda_t$ typically chosen near $1$ [2403.12002].

Animus3D defines MSD through an explicit differential motion step,
$$
\epsilon_{\mathrm{motion}}^{*}
=
\epsilon_{\mathrm{dynamic}}-\epsilon_{\mathrm{static}},
$$
and uses the gradient
$$
\nabla_\phi \mathcal{L}_{\mathrm{MSD}}
=
\mathbb{E}_t
\left[
w(t)\,(\epsilon_{\mathrm{dynamic}}-\epsilon_{\mathrm{static}})\,\frac{\partial x_t}{\partial \phi}
\right].
$$
The dynamic branch uses a motion prompt, while the static branch uses a LoRA-adapted denoiser conditioned on a static reference prompt. In this formulation, MSD is intended to isolate motion cues from appearance cues by subtracting a static-video prediction from a dynamic-video prediction [2512.12534].

MSDS in the motion-diffusion setting instead optimizes a motion sequence $X$ directly:
$$
\nabla_X\mathcal{L}_{\mathrm{MSDS}}
=
\mathbb{E}_{t,\varepsilon}
\left[
w(t)\,(X-\mathrm{MDM}_\phi(X_t,t,c))
\right],
$$
where $X_t=\sqrt{\alpha_t}X+\sqrt{1-\alpha_t}\varepsilon$. This pulls the optimized sequence toward what the pretrained motion diffusion model considers in-distribution at the current noise level [2511.00248].

| Formulation | Optimized target | Core score signal |
|---|---|---|
| Temporal SDS | SMPL-X PoseField / rendered video latent | $\hat\epsilon-\epsilon$ |
| DreamMotion MSD | Source video frames | Conditional target-source denoiser difference |
| Animus3D MSD | Dynamic 3D asset parameters | $\epsilon_{\mathrm{dynamic}}-\epsilon_{\mathrm{static}}$ |
| MSDS | Motion sequence $X$ | $X-\mathrm{MDM}_\phi(X_t,t,c)$ |

## 3. Optimization targets and representational choices

The representation being optimized is a primary axis along which MSD variants differ. In "Towards motion from video diffusion models," the body model is SMPL-X,
$$
M(\beta,\theta,\psi)=\mathrm{LBS}(T_p(\beta,\theta,\psi),J(\beta),\theta,W),
$$
with $\theta=(\theta_b,\theta_f,\theta_h)$. The method fixes $\beta,\theta_f,\theta_h,\psi$ and optimizes only the major body joints $\theta_b\in\mathbb{R}^{3\cdot 21}$. Rather than optimizing each frame independently, it introduces a 2-layer MLP PoseField $P_\alpha:\tau\mapsto\theta_b$ with positional encoding. Each frame is rendered by placing a camera on a circle around the SMPL-X character and rasterizing with nvdiffrast under fixed texture and lighting. The rendered frames are concatenated into a video $V_\alpha$ [2411.12831].

DreamMotion operates directly on the target video frames. The optimization is initialized so that the target video equals the source video, and each iteration perturbs the target video under a pretrained text-to-video U-Net without any weight fine-tuning. The method is model-agnostic and applies to both cascaded and non-cascaded video diffusion frameworks. In the cascaded Show-1 case, optimization is confined to the Keyframe Generation U-Net, while Temporal Interpolation and Super-Resolution remain untouched. A binary editing mask $m^{1:N}$ filters Video-DDS gradients to avoid over-editing unintended regions [2403.12002].

Animus3D uses 3D Gaussian-based representations rendered into static and dynamic videos. It introduces a dual-branch construction: static Gaussians $\mathcal{G}_c$ produce a static video, while dynamic Gaussians $\mathcal{G}_{0:T-1}$ produce a motion-conditioned video. A LoRA-enhanced video diffusion model is trained on static-video data to provide a faithful static denoiser, and DDIM inversion is used to produce deterministic, appearance-preserving noise estimates [2512.12534].

MSDS in object-aware 4D human motion generation is yet another target choice. It optimizes a continuous motion sequence whose frames encode SMPL pose parameters, global translation, and orientation. Human and object geometry are represented as 3D Gaussian splats; human Gaussians are skinned via SMPL-X barycentric maps, while object Gaussians remain fixed. A large language model provides a coarse trajectory prior, and MSDS refines the motion so that it remains realistic under the motion diffusion prior while respecting trajectory and collision constraints [2511.00248].

## 4. Mechanisms for preserving motion, structure, and appearance

Motion-centric score distillation is typically under-constrained if only the diffusion prior is used. The corresponding literature therefore introduces task-specific regularizers.

DreamMotion adds two self-similarity terms derived from U-Net self-attention key features. Spatial Self-Similarity Matching computes, for each frame, a Gram-like cosine-similarity matrix over spatial positions and penalizes deviations between source and target. Temporal Self-Similarity Matching collapses spatial dimensions by marginal mean, computes frame-to-frame cosine similarities, and penalizes deviations in temporal affinity. According to the reported ablations, omitting S-SSM causes drift in object shape, while omitting T-SSM causes flicker; only the full combination of V-DDS, S-SSM, and T-SSM consistently preserves structure and motion [2403.12002].

Animus3D addresses a different failure mode: entanglement between motion and appearance in the denoising residual. Its static-video LoRA branch exists because a vanilla video diffusion model tends to hallucinate small motions even under static prompts. DDIM inversion is used to replace stochastic noise sampling with deterministic inversion so that the noise predictions reflect motion cues rather than random appearance shifts. The method further adds temporal smoothness through a 3D total variation penalty and spatial rigidity through an As-Rigid-As-Possible regularizer. This combination is intended to suppress temporal jitter and spatial distortion in the learned motion field [2512.12534].

The SMPL-X deformation pipeline of "Towards motion from video diffusion models" uses a simpler regularization scheme. Its total loss combines temporal SDS, image SDS on each frame, and a motion-smoothness regularizer
$$
L_{\mathrm{reg}}
=
\lambda_{\mathrm{reg}}
\sum_{i=1}^{F-1}
\|\theta_b(i+1)-\theta_b(i)\|^2,
$$
with $\lambda_{\mathrm{reg}}=1\mathrm{e}{-3}$. The PoseField is initialized at the mean pose of the target action and clamped within $\pm 3$ standard deviations of that mean. The paper also reports occasional mode collapse in which the mesh oscillates or drifts out of anatomical range if $\lambda_{\mathrm{reg}}$ is too small [2411.12831].

A common misconception is that motion-specific score distillation is equivalent to simply increasing classifier-free guidance in vanilla SDS. The available ablations do not support that equivalence. In Animus3D, vanilla SDS with low guidance $(\mathrm{CFG}=7.5)$ produces almost no visible motion, while high guidance $(\mathrm{CFG}=100)$ produces erratic, appearance-destroying motion; full MSD is reported to yield stable, large, semantically correct motions that align with text [2512.12534].

## 5. Empirical behavior across tasks

In the SMPL-X synthesis setting, "Towards motion from video diffusion models" evaluates prompts including "walking," "running," "punching," and "doing a cartwheel." All three tested video diffusion models—ModelScope, ZeroScope, and VideoCrafter2—produce plausible motion for very common actions such as walking and running. "Punching" yields only slight upper-body motion, and "cartwheel" fails entirely, with the mesh remaining near a static T-pose. Among the tested priors, VideoCrafter2 outperforms ModelScope and ZeroScope in frame-to-frame variation and natural trajectories. To isolate whether the failure lies in the mesh parameterization, the paper freezes rendering and directly optimizes video latents under the same temporal SDS loss; "running" remains coherent, whereas "punching" still shows minimal variation, supporting the conclusion that the limitation is inherent to the video diffusion priors rather than to the renderer or PoseField [2411.12831].

DreamMotion reports both automatic and human evaluation over 26 DAVIS/WebVid pairs. The automatic metrics are Text-Align, defined as average CLIP cosine between prompt and frame, and Frame-Con, defined as average CLIP cosine between all pairs of frames. On ZeroScope, DreamMotion achieves Text-Align $0.8209$ and Frame-Con $0.9726$, both marked as best, with human scores of $4.14$ for Edit Accuracy, $4.21$ for Frame Consistency, and $4.33$ for Structure and Motion Preservation. On the cascaded Show-1 pipeline, it reports Text-Align $0.7747$, Frame-Con $0.9755$, human Edit Accuracy $3.97$, Frame Consistency $3.74$, and Structure and Motion Preservation $4.30$ [2403.12002].

Animus3D emphasizes ablation-based qualitative evidence rather than the benchmark structure summarized for DreamMotion. Its reported observations are that dual distributions without faithful noise produce motion but corrupt appearance, whereas full MSD with faithful inversion yields stable, large, semantically correct motions aligned with text. The motion refinement module further upsamples the animation from $T$ to $T'=2T-1$ frames and refines it with a large pretrained rectified-flow video model to obtain smoother, more detailed motion beyond the original temporal resolution [2512.12534].

MSDS is evaluated in an object-aware 4D human motion scenario using the prompt "the human jumps onto the table." The reported case study states that without MSDS, the human floats and misses the table, whereas the full MSDS-plus-constraints system makes the feet contact the table and keeps collision loss near zero. The quantitative comparison against 4D-fy, averaged over 4 views and 5 runs, reports Pose Plausibility $(\times 10^{-1})$ of $0.62$ for 4D-fy versus $0.48$ for MSDI, Pose Variation of $0.035$ versus $0.089$, Trajectory Length of $0.87$ m versus $1.94$ m, and Optical Flow Score of $0.45$ versus $1.12$ [2511.00248].

## 6. Limitations, misconceptions, and research directions

A recurrent empirical limitation is the dependence of score distillation on the coverage of the underlying prior. The SMPL-X study states that common human activities such as walking and running emerge robustly purely from text guidance, but rare or complex motions such as cartwheels and karate punches are not well represented in the diffusion models' training data, so SDS cannot recover them [2411.12831]. This suggests that MSD quality is bounded not only by the optimization procedure but also by the motion statistics internalized by the frozen prior.

Another limitation concerns structural scope. DreamMotion is explicitly designed to preserve existing structure and motion; it is therefore not intended for large structural reconfigurations such as changing object topology. Its optimization is also iterative and can take several minutes per video [2403.12002]. By contrast, Animus3D and MSDS target motion generation rather than conservative editing, but they introduce their own dependencies on auxiliary mechanisms such as LoRA adaptation, deterministic inversion, motion refinement, LLM trajectory proposals, and collision handling [2512.12534][2511.00248].

The literature also points toward several extensions. "Towards motion from video diffusion models" lists stronger or larger video-diffusion models, motion-capture fine-tuned diffusion priors such as MDM or PriorMDM, explicit kinematic priors, velocity penalties, multi-view consistency losses, silhouette-based constraints, and joint optimization of latent and PoseField parameters for hybrid latent-mesh fitting [2411.12831]. DreamMotion proposes adaptive weighting schedules $\lambda_s(t)$ and $\lambda_t(t)$, learned masks or semantic segmentation for automatic region-of-interest selection, and application to 3D-aware NeRF or mesh editing by treating novel 3D renderings as video frames [2403.12002]. Animus3D adds a plausible implication: once static and dynamic distributions are explicitly separated, motion-specific score distillation can be interpreted as a route toward appearance-preserving animation of static assets rather than direct content synthesis from noise [2512.12534].

Taken together, these works position MSD as a technically diverse but conceptually coherent class of methods: diffusion scores are distilled into a target representation, and task-specific constraints determine whether the result behaves as motion-preserving video editing, motion extraction from video priors, text-driven 3D animation, or object-aware 4D human motion generation.

Source: https://www.emergentmind.com/topics/motion-score-distillation-msd