Motion Score Distillation (MSD)

• Motion Score Distillation (MSD) is a family of methods that leverages pretrained diffusion priors to optimize motion in videos, 3D animations, and motion sequences.
• It replaces full reverse diffusion with gradient-based optimization while integrating constraints to preserve motion, appearance, and structure.
• Different variants like Temporal SDS, DreamMotion, and Animus3D tailor MSD with specific targets and regularizers to address challenges in realistic motion generation.

Searching arXiv for papers on Motion Score Distillation and related formulations. Searching arXiv for "motion score distillation". Motion Score Distillation (MSD) denotes a family of score-distillation procedures that use pretrained diffusion priors to guide motion-preserving video editing, text-driven 3D animation, or direct motion-sequence refinement. In the cited literature, the term is not tied to a single universally fixed loss. Rather, it is used for closely related constructions that distill denoising scores from video diffusion models or motion diffusion models into an optimizable target, with the target taking the form of video frames, a deformable body model, a Gaussian-based 3D asset, or an explicit motion sequence. Across these variants, the central idea is to replace full reverse diffusion with gradient-based optimization driven by a pretrained prior, while introducing additional constraints to preserve motion, appearance, structure, or physical plausibility (Janson et al., 2024, Jeong et al., 2024, Sun et al., 14 Dec 2025, Gui et al., 31 Oct 2025).

1. Terminological scope and historical placement

MSD is best understood as an outgrowth of Score Distillation Sampling (SDS). In the standard SDS view, a pretrained diffusion model supplies a score estimate that can be back-propagated through a downstream generator. "Towards motion from video diffusion models" extends this logic from images to videos through a temporal SDS variant, asking whether publicly available text-to-video diffusion models can guide realistic human body animation by deforming an SMPL-X representation (Janson et al., 2024).

"DreamMotion: Space-Time Self-Similar Score Distillation for Zero-Shot Video Editing" uses the expression "Motion Score Distillation" for an extension of SDS or Delta Denoising Score to videos. In that setting, the optimizable object is not a newly synthesized motion source but an existing source video whose real motion should be preserved while new appearance is injected by a pretrained text-to-video prior (Jeong et al., 2024).

"Animus3D: Text-driven 3D Animation via Motion Score Distillation" assigns MSD a more specific meaning. It introduces MSD as an SDS alternative that explicitly contrasts a static source distribution with a dynamic target distribution, using the difference between dynamic and static denoiser predictions as the motion-driving term. This formulation is designed for animating a static 3D asset while preserving appearance (Sun et al., 14 Dec 2025).

A related but distinct nomenclature appears in "Object-Aware 4D Human Motion Generation," which proposes Motion Diffusion Score Distillation Sampling (MSDS). There the distilled prior is a pretrained motion diffusion model operating directly in motion space rather than a video diffusion model operating on rendered frames or latents (Gui et al., 31 Oct 2025).

This usage pattern suggests that MSD is less a single algorithm than a research direction: score distillation specialized to motion-centric optimization problems.

2. Core mathematical formulations

The common starting point is the SDS identity used with a latent diffusion denoiser $\hat\epsilon_\phi$. In "Towards motion from video diffusion models," the generator parameters $\theta$ are updated by

$$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$

with $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$. The paper interprets $(\hat\epsilon-\epsilon)$ as an unbiased estimator of the diffusion model's score and extends this to videos through temporal SDS on a latent tensor $Z$, yielding

$$\nabla_\alpha L_{\mathrm{SDS-T}} = \lambda_{\mathrm{SDS}} \mathbb{E}_{\sigma,\epsilon} \left[ w(\sigma)\cdot (\hat\epsilon(Z_{\alpha,\sigma,\epsilon}\mid y,\sigma)-\epsilon)\cdot \frac{\partial Z_\alpha}{\partial \alpha} \right],$$

where $\alpha$ denotes the PoseField parameters and $\lambda_{\mathrm{SDS}}$ is set to $1\mathrm{e}{-3}$ (Janson et al., 2024).

DreamMotion formulates video-level motion-preserving distillation through Video-DDS:

$\theta$0

Here the target video and source video are noised by the same $\theta$1. This is coupled with spatial and temporal self-similarity matching losses extracted from U-Net self-attention features, producing a total objective

$\theta$2

with $\theta$3 typically chosen near $\theta$4 (Jeong et al., 2024).

Animus3D defines MSD through an explicit differential motion step,

$\theta$5

and uses the gradient

$\theta$6

The dynamic branch uses a motion prompt, while the static branch uses a LoRA-adapted denoiser conditioned on a static reference prompt. In this formulation, MSD is intended to isolate motion cues from appearance cues by subtracting a static-video prediction from a dynamic-video prediction (Sun et al., 14 Dec 2025).

MSDS in the motion-diffusion setting instead optimizes a motion sequence $\theta$7 directly:

$\theta$8

where $\theta$9. This pulls the optimized sequence toward what the pretrained motion diffusion model considers in-distribution at the current noise level (Gui et al., 31 Oct 2025).

Formulation	Optimized target	Core score signal
Temporal SDS	SMPL-X PoseField / rendered video latent	$$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$0
DreamMotion MSD	Source video frames	Conditional target-source denoiser difference
Animus3D MSD	Dynamic 3D asset parameters	$$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$1
MSDS	Motion sequence $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$2	$$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$3

3. Optimization targets and representational choices

The representation being optimized is a primary axis along which MSD variants differ. In "Towards motion from video diffusion models," the body model is SMPL-X,

$$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$4

with $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$5. The method fixes $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$6 and optimizes only the major body joints $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$7. Rather than optimizing each frame independently, it introduces a 2-layer MLP PoseField $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$8 with positional encoding. Each frame is rendered by placing a camera on a circle around the SMPL-X character and rasterizing with nvdiffrast under fixed texture and lighting. The rendered frames are concatenated into a video $$\nabla_\theta L_{\mathrm{SDS}}(\phi,z) = \mathbb{E}_{t,\epsilon} \left[ w(t)\cdot (\hat\epsilon_\phi(z_t;y,t)-\epsilon)\cdot \frac{\partial z}{\partial \theta} \right],$$9 (Janson et al., 2024).

DreamMotion operates directly on the target video frames. The optimization is initialized so that the target video equals the source video, and each iteration perturbs the target video under a pretrained text-to-video U-Net without any weight fine-tuning. The method is model-agnostic and applies to both cascaded and non-cascaded video diffusion frameworks. In the cascaded Show-1 case, optimization is confined to the Keyframe Generation U-Net, while Temporal Interpolation and Super-Resolution remain untouched. A binary editing mask $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$0 filters Video-DDS gradients to avoid over-editing unintended regions (Jeong et al., 2024).

Animus3D uses 3D Gaussian-based representations rendered into static and dynamic videos. It introduces a dual-branch construction: static Gaussians $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$1 produce a static video, while dynamic Gaussians $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$2 produce a motion-conditioned video. A LoRA-enhanced video diffusion model is trained on static-video data to provide a faithful static denoiser, and DDIM inversion is used to produce deterministic, appearance-preserving noise estimates (Sun et al., 14 Dec 2025).

MSDS in object-aware 4D human motion generation is yet another target choice. It optimizes a continuous motion sequence whose frames encode SMPL pose parameters, global translation, and orientation. Human and object geometry are represented as 3D Gaussian splats; human Gaussians are skinned via SMPL-X barycentric maps, while object Gaussians remain fixed. A LLM provides a coarse trajectory prior, and MSDS refines the motion so that it remains realistic under the motion diffusion prior while respecting trajectory and collision constraints (Gui et al., 31 Oct 2025).

4. Mechanisms for preserving motion, structure, and appearance

Motion-centric score distillation is typically under-constrained if only the diffusion prior is used. The corresponding literature therefore introduces task-specific regularizers.

DreamMotion adds two self-similarity terms derived from U-Net self-attention key features. Spatial Self-Similarity Matching computes, for each frame, a Gram-like cosine-similarity matrix over spatial positions and penalizes deviations between source and target. Temporal Self-Similarity Matching collapses spatial dimensions by marginal mean, computes frame-to-frame cosine similarities, and penalizes deviations in temporal affinity. According to the reported ablations, omitting S-SSM causes drift in object shape, while omitting T-SSM causes flicker; only the full combination of V-DDS, S-SSM, and T-SSM consistently preserves structure and motion (Jeong et al., 2024).

Animus3D addresses a different failure mode: entanglement between motion and appearance in the denoising residual. Its static-video LoRA branch exists because a vanilla video diffusion model tends to hallucinate small motions even under static prompts. DDIM inversion is used to replace stochastic noise sampling with deterministic inversion so that the noise predictions reflect motion cues rather than random appearance shifts. The method further adds temporal smoothness through a 3D total variation penalty and spatial rigidity through an As-Rigid-As-Possible regularizer. This combination is intended to suppress temporal jitter and spatial distortion in the learned motion field (Sun et al., 14 Dec 2025).

The SMPL-X deformation pipeline of "Towards motion from video diffusion models" uses a simpler regularization scheme. Its total loss combines temporal SDS, image SDS on each frame, and a motion-smoothness regularizer

$z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$3

with $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$4. The PoseField is initialized at the mean pose of the target action and clamped within $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$5 standard deviations of that mean. The paper also reports occasional mode collapse in which the mesh oscillates or drifts out of anatomical range if $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$6 is too small (Janson et al., 2024).

A common misconception is that motion-specific score distillation is equivalent to simply increasing classifier-free guidance in vanilla SDS. The available ablations do not support that equivalence. In Animus3D, vanilla SDS with low guidance $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$7 produces almost no visible motion, while high guidance $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$8 produces erratic, appearance-destroying motion; full MSD is reported to yield stable, large, semantically correct motions that align with text (Sun et al., 14 Dec 2025).

5. Empirical behavior across tasks

In the SMPL-X synthesis setting, "Towards motion from video diffusion models" evaluates prompts including "walking," "running," "punching," and "doing a cartwheel." All three tested video diffusion models—ModelScope, ZeroScope, and VideoCrafter2—produce plausible motion for very common actions such as walking and running. "Punching" yields only slight upper-body motion, and "cartwheel" fails entirely, with the mesh remaining near a static T-pose. Among the tested priors, VideoCrafter2 outperforms ModelScope and ZeroScope in frame-to-frame variation and natural trajectories. To isolate whether the failure lies in the mesh parameterization, the paper freezes rendering and directly optimizes video latents under the same temporal SDS loss; "running" remains coherent, whereas "punching" still shows minimal variation, supporting the conclusion that the limitation is inherent to the video diffusion priors rather than to the renderer or PoseField (Janson et al., 2024).

DreamMotion reports both automatic and human evaluation over 26 DAVIS/WebVid pairs. The automatic metrics are Text-Align, defined as average CLIP cosine between prompt and frame, and Frame-Con, defined as average CLIP cosine between all pairs of frames. On ZeroScope, DreamMotion achieves Text-Align $z_t=\sqrt{\alpha_t}z+\sqrt{1-\alpha_t}\epsilon$9 and Frame-Con $(\hat\epsilon-\epsilon)$0, both marked as best, with human scores of $(\hat\epsilon-\epsilon)$1 for Edit Accuracy, $(\hat\epsilon-\epsilon)$2 for Frame Consistency, and $(\hat\epsilon-\epsilon)$3 for Structure and Motion Preservation. On the cascaded Show-1 pipeline, it reports Text-Align $(\hat\epsilon-\epsilon)$4, Frame-Con $(\hat\epsilon-\epsilon)$5, human Edit Accuracy $(\hat\epsilon-\epsilon)$6, Frame Consistency $(\hat\epsilon-\epsilon)$7, and Structure and Motion Preservation $(\hat\epsilon-\epsilon)$8 (Jeong et al., 2024).

Animus3D emphasizes ablation-based qualitative evidence rather than the benchmark structure summarized for DreamMotion. Its reported observations are that dual distributions without faithful noise produce motion but corrupt appearance, whereas full MSD with faithful inversion yields stable, large, semantically correct motions aligned with text. The motion refinement module further upsamples the animation from $(\hat\epsilon-\epsilon)$9 to $Z$0 frames and refines it with a large pretrained rectified-flow video model to obtain smoother, more detailed motion beyond the original temporal resolution (Sun et al., 14 Dec 2025).

MSDS is evaluated in an object-aware 4D human motion scenario using the prompt "the human jumps onto the table." The reported case study states that without MSDS, the human floats and misses the table, whereas the full MSDS-plus-constraints system makes the feet contact the table and keeps collision loss near zero. The quantitative comparison against 4D-fy, averaged over 4 views and 5 runs, reports Pose Plausibility $Z$1 of $Z$2 for 4D-fy versus $Z$3 for MSDI, Pose Variation of $Z$4 versus $Z$5, Trajectory Length of $Z$6 m versus $Z$7 m, and Optical Flow Score of $Z$8 versus $Z$9 (Gui et al., 31 Oct 2025).

6. Limitations, misconceptions, and research directions

A recurrent empirical limitation is the dependence of score distillation on the coverage of the underlying prior. The SMPL-X study states that common human activities such as walking and running emerge robustly purely from text guidance, but rare or complex motions such as cartwheels and karate punches are not well represented in the diffusion models' training data, so SDS cannot recover them (Janson et al., 2024). This suggests that MSD quality is bounded not only by the optimization procedure but also by the motion statistics internalized by the frozen prior.

Another limitation concerns structural scope. DreamMotion is explicitly designed to preserve existing structure and motion; it is therefore not intended for large structural reconfigurations such as changing object topology. Its optimization is also iterative and can take several minutes per video (Jeong et al., 2024). By contrast, Animus3D and MSDS target motion generation rather than conservative editing, but they introduce their own dependencies on auxiliary mechanisms such as LoRA adaptation, deterministic inversion, motion refinement, LLM trajectory proposals, and collision handling (Sun et al., 14 Dec 2025, Gui et al., 31 Oct 2025).

The literature also points toward several extensions. "Towards motion from video diffusion models" lists stronger or larger video-diffusion models, motion-capture fine-tuned diffusion priors such as MDM or PriorMDM, explicit kinematic priors, velocity penalties, multi-view consistency losses, silhouette-based constraints, and joint optimization of latent and PoseField parameters for hybrid latent-mesh fitting (Janson et al., 2024). DreamMotion proposes adaptive weighting schedules $$\nabla_\alpha L_{\mathrm{SDS-T}} = \lambda_{\mathrm{SDS}} \mathbb{E}_{\sigma,\epsilon} \left[ w(\sigma)\cdot (\hat\epsilon(Z_{\alpha,\sigma,\epsilon}\mid y,\sigma)-\epsilon)\cdot \frac{\partial Z_\alpha}{\partial \alpha} \right],$$0 and $$\nabla_\alpha L_{\mathrm{SDS-T}} = \lambda_{\mathrm{SDS}} \mathbb{E}_{\sigma,\epsilon} \left[ w(\sigma)\cdot (\hat\epsilon(Z_{\alpha,\sigma,\epsilon}\mid y,\sigma)-\epsilon)\cdot \frac{\partial Z_\alpha}{\partial \alpha} \right],$$1, learned masks or semantic segmentation for automatic region-of-interest selection, and application to 3D-aware NeRF or mesh editing by treating novel 3D renderings as video frames (Jeong et al., 2024). Animus3D adds a plausible implication: once static and dynamic distributions are explicitly separated, motion-specific score distillation can be interpreted as a route toward appearance-preserving animation of static assets rather than direct content synthesis from noise (Sun et al., 14 Dec 2025).

Taken together, these works position MSD as a technically diverse but conceptually coherent class of methods: diffusion scores are distilled into a target representation, and task-specific constraints determine whether the result behaves as motion-preserving video editing, motion extraction from video priors, text-driven 3D animation, or object-aware 4D human motion generation.