Score-Matching Motion Priors (SMP)

• Score-Matching Motion Priors (SMP) are data-driven models that use diffusion-based score matching to capture high-fidelity motion distributions across diverse domains.
• They employ rigorous feature normalization and weight balancing techniques to maintain stable training and adapt architectures to task-specific needs.
• SMP models enable unconditional generative modeling, motion estimation, and reward shaping, achieving state-of-the-art performance in benchmarks.

Score-Matching Motion Priors (SMP) represent a class of data-driven models that encode high-fidelity motion distributions using diffusion-based score-matching objectives. These priors can be leveraged for unconditional generative modeling, motion estimation, reward shaping in character control, and artifact correction across diverse domains including human animation, image rectification, and medical imaging. The unifying concept is the direct training of models to approximate the score (gradient of log-density) of motion distributions under increasing noise, enabling both sample generation and downstream guidance. Recent SMP advances emphasize rigorous normalization, weight balancing for stable training, architectural adaptation, modularity for reuse, and empirical validation against state-of-the-art benchmarks.

1. Theoretical Foundations of SMP

The core of SMP utilizes score-based diffusion models to approximate the data distribution over motion vectors, images, or volumetric measurements. For a motion (or motion-related) vector $x \in \mathbb{R}^{nL}$ or similar, the forward generative process is specified by a Stochastic Differential Equation (SDE):

$$dx(t) = d(x(0)) + d(t\,\varepsilon) \implies x(t) = x(0) + t\,\varepsilon, \quad \varepsilon \sim \mathcal{N}(0, I)$$

Training is performed by sampling noise level $t$ and residual noise $\varepsilon$, injecting both into the features prior to denoising. The fundamental objective is L2 score-matching:

$$\mathcal{L}_{\rm score} = \mathbb{E}_{t, x(0), \varepsilon} \left[ w(t)\, \|s_\theta(x(t), t) - \nabla_x \log p_t(x(t))\|_2^2 \right]$$

where the model learns to discriminate high-density regions in the data space by denoising samples corrupted at variable noise levels. The score, under standard preconditioning,

$$s_\theta(x(t), t) = \frac{D_\theta(x(t), t) - x(t)}{t}$$

can be indirectly targeted by expressing the objective in terms of the denoiser's output:

$$\mathcal{L}(\theta) = \mathbb{E}_{t, x(0), \varepsilon} \left[ w(t)\, \|D_\theta(x(t), t) - x(0)\|_2^2 \right]$$

Weightings $w(t)$ are analytically derived from the expected gradient magnitudes, ensuring balanced backpropagation across timesteps and feature groups (Björkstrand et al., 14 Oct 2025, Mu et al., 2 Dec 2025).

2. Feature-Space Normalization and Weight Balancing

SMP implementations emphasize the normalization of input features according to their geometric or physical properties. For human motion, one SMPL frame is vectorized as $x_i = [J_i, \Phi_i, \tau_i, \beta]$ of dimension 139: joint rotations $J$ and orientations $\Phi$ (6D), global translation $\tau$, and shape $\beta$. Rotations are scaled to preserve manifold structure, translation is z-scored in 3D, and shape parameters are individually standardized. This ensures that each group contributes comparably to loss gradients, avoiding feature group domination.

Weight balancing is achieved via time-dependent and group-dependent scalars, e.g., via a learned uncertainty field $u_\psi(t)$, whose exponential acts as a normalization factor:

$$w(t) = \frac{\sqrt{\lambda(t)}}{\sqrt{e^{u_\psi(t)}}}$$

This prevents the accumulation of gradient-magnitude imbalances and maintains stable training dynamics across both feature and temporal axes. Per-group weighting (e.g. $w^k$ for feature group $k$) further equalizes contributions (Björkstrand et al., 14 Oct 2025).

3. Network Architectures and Algorithmic Implementations

Diffusion-based SMP architectures typically employ U-Net variants tailored to the task domain:

• Motion Generation (Björkstrand et al., 14 Oct 2025): EDM2 U-Net in 1D for SMPL motion, channels=192, [1,2,3,4] multipliers, attention at lower resolutions, dropout 0.1.
• Image-to-Motion Estimation (Wang et al., 10 May 2025): Latent Diffusion SD backbone, fine-tuned VAE encoder/decoder for both image and flow fields, U-Net extended to process concatenated condition/flow latents.
• Physics-based Character Control (Mu et al., 2 Dec 2025): Transformer encoder with time/style-adaptive normalization, two-layer structure with $\sim$3M parameters.
• Medical Imaging (Zhang et al., 4 Nov 2025): 2D U-Nets in the wavelet domain, residual blocks with wavelet convolutions (WTConv) expanding receptive field efficiently.

Training and sampling procedures are task-specific. For motion generation, SGD is performed over denoiser and uncertainty networks, with preconditioning and ablated group normalization. In image-to-motion settings, only the flow latent is noised at each step, and architecture is adapted for multi-channel input. In medical imaging, alternating pseudo-3D inference is performed using 2D priors on orthogonal slices, accelerating via wavelet transformations (Björkstrand et al., 14 Oct 2025, Wang et al., 10 May 2025, Zhang et al., 4 Nov 2025, Mu et al., 2 Dec 2025).

4. Task-Specific Adaptations and Applications

SMP models are applied across a range of domains:

• Unconditional Human Motion and Shape Generation: SMP achieves state-of-the-art results (FID=1.81, diversity=8.73, limb $\sigma=0.02$mm) for direct SMPL motion+shape synthesis, matching previous bests with lower computational cost (31 noise function evaluations) (Björkstrand et al., 14 Oct 2025).
• Reusable Priors for Physics-Based Character Control: SMP is trained once on a large dataset, then frozen and reused as a scalar reward in reinforcement learning, enabling modular and style-conditioned control without dependence on reference motion data. It outperforms adversarial methods (AMP, AMP-Frozen) and matches tracking performance of DeepMimic, with style conditioning enabling synthesis and composition of new styles (Mu et al., 2 Dec 2025).
• Image-to-Motion Estimation: SMP serves as a motion estimator in tasks such as Stitched Image Rectangling (SIR) and Rolling Shutter Correction (RSC), leveraging adaptive ensemble strategies for output consistency and enabling one-step DDIM inference, which is empirically shown to outperform multi-step inference in accuracy and speed (Wang et al., 10 May 2025).
• 3D MRI Motion Artifact Correction: SMP fuses two orthogonal 2D score priors in a pseudo-3D restoration pipeline, guided by a mean-reverting SDE, and employing wavelet-domain U-Nets for accelerated and coherent artifact removal in 3D medical volumes (Zhang et al., 4 Nov 2025).

5. Empirical Evaluation and Ablations

Extensive empirical evaluation across motion generation, character control, image rectification, and MRI motion correction consistently indicates that SMP achieves, and in some cases surpasses, state-of-the-art performance according to canonical metrics:

Task	SMP (Best FID/PSNR/Return)	Previous SOTA
Human motion generation (Björkstrand et al., 14 Oct 2025)	FID=1.81	VAE (FID$\sim$1.17), MDM (FID=3.58)
Physics-based control (Mu et al., 2 Dec 2025)	Return=0.914 (steering)	AMP=0.634, AMP-Frozen=0.243
Image rectification (Wang et al., 10 May 2025)	1-step inference, state-of-art accuracy	Prior methods, slower inference
3D MRI correction (Zhang et al., 4 Nov 2025)	PSNR$\sim$34.2 dB (mild), 29-30 dB (severe)	PFAD, SDE-MRI (lower PSNR, hours per volume)

Ablation studies demonstrate the necessity of feature-space normalization, per-group balancing, and weight analytics. For example, baseline models (no normalization, unbalanced gradients) result in degraded FID and lower diversity; successive introduction of SMP components recovers empirical quality (Björkstrand et al., 14 Oct 2025).

6. Limitations and Open Research Directions

Current SMP models exhibit certain limitations:

• Motion Generation: Residual foot-skating and mesh self-intersection; limited vertical stability (STAIR-ASCENT drift); potential for further improvement with conditional priors or alternate pose models (Björkstrand et al., 14 Oct 2025).
• Character Control: Style conditioning accuracy saturates when reference classifier is poor; limited compositional granularity when blending priors; generative initialization can exhibit out-of-distribution behaviors unless denoising is well constrained (Mu et al., 2 Dec 2025).
• Image/MRI Applications: Performance depends on pretrained priors, which may underrepresent pathological or rare artifacts; pseudo-3D strategies mitigate but may not eliminate all slice-discontinuities; wavelet transforms increase domain-specific complexity (Zhang et al., 4 Nov 2025).

Future directions include conditional SMPs (text, constraints), integration with hybrid pose models (SMPL, hand, face), adaptation to human-scene joint modeling, fine-grained score composition for style transfer, and more robust handling of domain-specific feature imbalances (Björkstrand et al., 14 Oct 2025, Mu et al., 2 Dec 2025, Zhang et al., 4 Nov 2025).

7. Connections to Related Approaches

SMPs generalize the score-matching philosophy beyond adversarial, autoencoding, or GAN-based priors by leveraging diffusion objectives for both sample generation and reward shaping. They offer modularity, reusability, and compositionality—qualities that adversarial methods lack due to requirement for retraining per task and reference-motion retention. SMPs also facilitate tractable likelihood estimation and efficient sampling via ODE-based methods and closed-form simulations, and they highlight the value of analytic normalization and task-driven adaptation as central to stable, high-quality motion modeling (Björkstrand et al., 14 Oct 2025, Wang et al., 10 May 2025, Mu et al., 2 Dec 2025, Zhang et al., 4 Nov 2025).