Differentiable Motion Trajectory (DMT)

• Differentiable Motion Trajectory (DMT) is a representation that models motion with smooth polynomial functions to ensure continuous, flicker-free temporal evolution.
• It embeds control-point motions in a differentiable computational graph, allowing gradient backpropagation across frames to optimize global trajectory parameters.
• DMT is applied in sketch animation, robotics, and autonomous driving, demonstrating improved temporal consistency and efficient trajectory optimization.

Searching arXiv for papers on “Differentiable Motion Trajectory” and closely related formulations.

Searching arXiv for "Differentiable Motion Trajectory"

Differentiable Motion Trajectory (DMT) denotes a motion representation in which temporal evolution is encoded by differentiable functions, so that supervision applied at individual times can update shared trajectory parameters across an entire sequence. In its explicit named form, DMT was introduced for vector sketch animation by modeling each Bézier control point as a smooth polynomial trajectory over normalized time, thereby replacing frame-wise sketch generation with a single global spatiotemporal object (Zhu et al., 30 Sep 2025). This suggests a broader research usage in which closely related formulations appear in robotics, visuomotor imitation, markerless motion capture, kinodynamic planning, and integrated prediction-planning, even when the term “DMT” itself is not used (Xu et al., 3 Jul 2025, Xu et al., 2024, Cotton, 2024, Lee, 2024).

1. Definition and conceptual scope

In the sketch-animation formulation, DMT is the representation of frame-wise stroke motion by differentiable polynomial trajectories assigned to stroke control points. Its stated purpose is to address temporal coherence: frame-wise methods exhibit flickering, temporal popping/jitter, weak semantic consistency, and poor long-video behavior, whereas DMT enforces smooth control-point motion and allows loss from one frame to update global trajectory parameters shared across frames (Zhu et al., 30 Sep 2025).

A broader, technically coherent interpretation emerges across adjacent literature. A differential-drive mobile manipulator can be parameterized by base arc length $s(t)$, base yaw $\theta(t)$, and manipulator joints $q_k(t)$, with $x(t),y(t)$ reconstructed by integration rather than optimized directly (Xu et al., 3 Jul 2025). A visuomotor policy can output a finite-horizon action trajectory as the solution of a differentiable constrained QP layer (Xu et al., 2024). Markerless motion capture can optimize an implicit neural function of time whose outputs are pose parameters passed through differentiable biomechanics and camera models (Cotton, 2024). Kinodynamic planning can search a low-dimensional manifold of continuous-time differentiable motions $q(t)=\hat q_\beta(z,t)$ rather than the full trajectory space (Lee, 2024). Autonomous-driving planners can place a differentiable nonlinear optimizer inside a learned prediction stack so that planning errors supervise both prediction and cost weights (Huang et al., 2022). These formulations share a common structural thesis: the trajectory is not merely predicted; it is embedded in a differentiable computational graph.

Domain	Trajectory representation	Differentiability role
Vector sketch animation (Zhu et al., 30 Sep 2025)	Polynomial control-point motion over time	Global semantic gradient propagation across frames
Differential-drive mobile manipulation (Xu et al., 3 Jul 2025)	Quintic piecewise polynomials in $(s,\theta,q)$	Gradient-based smooth whole-body trajectory optimization
Visuomotor imitation (Xu et al., 2024)	Horizon action sequence from a constrained QP	End-to-end learning with position/velocity/acceleration constraints
Markerless motion capture (Cotton, 2024)	Implicit MLP $f_{\phi_n}(t)$ for pose parameters	End-to-end fitting through biomechanics and reprojection
Kinodynamic planning (Lee, 2024)	Differentiable motion manifold $q(t)=\hat q_\beta(z,t)$	Constraint-aware manifold shaping and fast online sampling

A common misconception is that DMT necessarily implies a fully end-to-end differentiable planner-controller stack. Several relevant systems are only partially differentiable: topology search, RRT-based initialization, primitive selection, or post-hoc trajectory optimization may remain external modules (Xu et al., 3 Jul 2025, Xu et al., 5 Apr 2026).

2. Canonical mathematical formulation

The most explicit mathematical definition of DMT appears in vector sketch animation. Each frame contains $N_s$ strokes, and each stroke is a Bézier curve

$$C(u) = \sum_{i=0}^{m} B_{m,i}(u) P_i, \qquad B_{m,i}(u) = \binom{m}{i} u^{i} (1-u)^{m-i},$$

where $\theta(t)$0 are control points (Zhu et al., 30 Sep 2025).

DMT models the motion of each control point over normalized time $\theta(t)$1 as a polynomial trajectory

$\theta(t)$2

Because polynomial functions are infinitely differentiable, the representation is differentiable in the ordinary analytic sense. The central implication is compositional: rasterized frame appearance depends on Bézier geometry, Bézier geometry depends on frame control points, and frame control points depend on global trajectory coefficients $\theta(t)$3. A loss at one frame therefore backpropagates into parameters that affect many or all frames (Zhu et al., 30 Sep 2025).

The formal justification is given through two appendix theorems. First, if a Bézier curve changes continuously over time, then its control points must also vary continuously. For sampled $\theta(t)$4,

$\theta(t)$5

Second, a continuously changing Bézier curve can be arbitrarily approximated by a Bézier curve whose control points are polynomial functions of time. If each control-point trajectory $\theta(t)$6 is continuous, then there exists a polynomial $\theta(t)$7 such that $\theta(t)$8, and the approximate curve

$\theta(t)$9

satisfies

$q_k(t)$0

Within that formulation, DMT is not a heuristic smoothing layer; it is the primary temporal parameterization of the animated object (Zhu et al., 30 Sep 2025).

The same pattern recurs in robotics with different state choices. For a differential-drive mobile manipulator,

$q_k(t)$1

is reconstructed from arc length and yaw through

$q_k(t)$2

This absorbs the nonholonomic constraint into the representation rather than imposing it afterward (Xu et al., 3 Jul 2025).

3. Basis choice, optimization structure, and differentiable losses

Although the control-point motion is first written in the power basis, the sketch-animation method does not optimize DMT in that form. It uses a Bernstein basis

$q_k(t)$3

because the paper identifies a sensitivity imbalance in the power basis. For

$q_k(t)$4

the $q_k(t)$5 sensitivity norm is

$q_k(t)$6

which varies from $q_k(t)$7 at $q_k(t)$8 to $q_k(t)$9 at $x(t),y(t)$0. The paper interprets this as causing gradient vanishing for early frames and gradient explosion for late frames. Under the Bernstein basis,

$x(t),y(t)$1

so parameter sensitivity is uniform over time (Zhu et al., 30 Sep 2025).

The optimization loop evaluates DMT trajectories at each frame, constructs Bézier strokes, renders them with a differentiable rasterizer, and minimizes

$x(t),y(t)$2

The semantic term is

$x(t),y(t)$3

the geometric term is

$x(t),y(t)$4

and the temporal consistency term is defined over sampled points on each stroke: $x(t),y(t)$5 This combination matters: continuity comes from the representation, whereas track-based correspondence loss anchors that continuity to observed motion (Zhu et al., 30 Sep 2025).

The paper also details numerical stabilization and systems engineering needed for high-degree temporal polynomials. For large $x(t),y(t)$6, direct evaluation of $x(t),y(t)$7 is numerically unstable in 32-bit arithmetic, so Bernstein terms are computed in log space. For long videos, gradients of CLIP losses are accumulated with alternating computation-and-release, reducing memory complexity from $x(t),y(t)$8 to $x(t),y(t)$9 with respect to frame count. Reported GPU memory falls from 17.2 GB for SketchVideo at 50 frames to 2.47 GB for the DMT method at 50 frames, while the method uses 3.37 GB at 400 frames (Zhu et al., 30 Sep 2025).

A related optimization pattern appears in mobile-manipulator planning. There, the trajectory is represented by piecewise quintic polynomials that are “quintic piecewise polynomials with four times continuously differentiable at the segmented points,” and $q(t)=\hat q_\beta(z,t)$0 are reconstructed by Simpson’s rule. Inequality constraints are sampled in time and converted to smooth penalties, while terminal pose equalities are handled by Powell-Hestenes-Rockafellar augmented Lagrangian with L-BFGS as the inner solver (Xu et al., 3 Jul 2025).

4. Trajectory representations beyond the explicit DMT name

Outside sketch animation, the literature instantiates DMT-like ideas through several technically distinct representations.

In constrained visuomotor imitation, a differentiable trajectory optimization layer serves as the policy head. The optimization variable is a stacked action sequence $q(t)=\hat q_\beta(z,t)$1; velocities are extracted by a selection matrix, accelerations are computed by finite differencing, and positions by discrete integration. The forward pass solves

$q(t)=\hat q_\beta(z,t)$2

subject to linear position, velocity, and acceleration constraints (Xu et al., 2024). Here the trajectory is discrete-time rather than spline-based, but it remains differentiable because the optimizer is embedded as a QP layer.

In markerless motion capture, each sequence has its own implicit neural trajectory $q(t)=\hat q_\beta(z,t)$3, implemented as an MLP with sinusoidal positional encoding and time scaled from $q(t)=\hat q_\beta(z,t)$4 to $q(t)=\hat q_\beta(z,t)$5. The output is pose parameters $q(t)=\hat q_\beta(z,t)$6, and marker positions are obtained by differentiable biomechanics: $q(t)=\hat q_\beta(z,t)$7 Those 3D markers are then reprojected into cameras and optimized against 2D detections, with gradients flowing through the trajectory network, the biomechanical model, and the camera model (Cotton, 2024).

In kinodynamic planning, a continuous-time manifold decoder

$q(t)=\hat q_\beta(z,t)$8

defines a differentiable motion manifold. Time derivatives

$q(t)=\hat q_\beta(z,t)$9

are available directly, allowing penalties on joint velocity, acceleration, jerk, torque, end-effector velocity, and self-collision during manifold shaping (Lee, 2024).

In autonomous driving, integrated prediction-planning methods use differentiable nonlinear optimization rather than an explicit closed-form trajectory basis. A kinematic bicycle rollout

$(s,\theta,q)$0

is embedded in a least-squares planner whose cost weights are partly learned, making the optimized ego trajectory differentiable with respect to upstream prediction and learned cost parameters (Huang et al., 2022).

A plausible implication is that DMT is better understood as a family resemblance than a single canonical architecture: polynomial control-point motion, discrete-time QP trajectories, implicit neural functions of time, differentiable planners, and motion manifolds all qualify when the trajectory itself is a differentiable object through which task losses, geometric penalties, or planning objectives are propagated.

5. Applications and empirical evidence

In vector sketch animation, DMT is used for short-video and long-video sketch synthesis, 3D animation conversion, and text-to-video conversion. On DAVIS-scale short sequences, the paper reports that CLIPasso has poor temporal consistency and visible flicker, SketchVideo improves coherence but still jitters, Fang et al. suppresses more flicker but remains unstable in details, and DMT-based results have no visible flickering with better semantic consistency. On longer sequences, the method is reported on a 400-frame dance video, a 300-frame surfing video from LVOS, and 500-frame 3D animation; because trajectories are continuous in time, it also supports frame interpolation / high-framerate output, including a case with 6 FPS input and 24 FPS output (Zhu et al., 30 Sep 2025).

The Bernstein-basis ablation isolates a core DMT design choice. The power basis converges more slowly, requires more than 1000 iterations to get reasonable output, and produces distortions in later frames as $(s,\theta,q)$1 increases; the Bernstein basis reaches comparable quality within about 500 iterations and avoids distortion artifacts across all frames. The method also preserves recognizable structure under very small stroke budgets; with very few strokes such as 4, prior methods lose semantics more severely, whereas DMT is reported to preserve recognizable structure through global semantic modeling across frames (Zhu et al., 30 Sep 2025).

In mobile-manipulator planning, the hierarchical arc length-yaw formulation is evaluated against REMANI on three start-goal distance ranges with 500 random collision-free problem instances per range. For Small (3m–8m), success rises from 62.4% to 92.0%, planning time falls from 318.9 ms to 107.7 ms, and trajectory duration changes from 13.8 s to 13.2 s. For Medium (8m–15m), success rises from 73.8% to 91.0%, and planning time falls from 810.8 ms to 227.5 ms. For Large (15m–30m), success rises from 51.4% to 79.8%, and planning time falls from 2799.8 ms to 522.2 ms (Xu et al., 3 Jul 2025).

In constrained visuomotor imitation, differentiable trajectory optimization improves smoothness and physical reliability without large success degradation. On the real-world “Move-the-stack” task, LeTO achieves 13/20 success with acc-peak 0.250, compared with 4/20 for both LSTM-GMM and DiffusionPolicy. On “Arrange-chopsticks,” DiffusionPolicy achieves 18/20 success with avg-contact-max 13.9 N and contact-peak 24.8 N, while LeTO achieves 17/20 success with avg-contact-max 11.7 N and contact-peak 16.1 N (Xu et al., 2024).

In markerless motion capture, end-to-end differentiable biomechanics improves reprojection quality and gait metrics. Reported $(s,\theta,q)$2 rises from 0.70 to 0.72 for LocoMujoco with bundle adjustment and from 0.69 to 0.72 for MyoSkeleton with bundle adjustment, and bundle adjustment yields a significant 1.9% increase in $(s,\theta,q)$3. Step-width, stride-length, and step-length errors are also reported against GaitRite, with control-population errors around 8–10 mm for several measures (Cotton, 2024).

In integrated prediction-planning for autonomous driving, differentiable planning produces better open-loop and closed-loop outcomes than separated prediction and planning. Open-loop planning error at $(s,\theta,q)$4 s improves from 3.466 m to 3.335 m, collision rate improves from 1.813% to 1.802%, and prediction ADE/FDE improves from 0.766 / 1.908 to 0.740 / 1.814. In closed-loop evaluation, collision falls from 7% to 5%, and $(s,\theta,q)$5 s position error improves from 10.17 m to 9.365 m (Huang et al., 2022).

These results do not establish a single cross-domain benchmark for DMT, but they do show recurring empirical advantages: improved temporal coherence in graphics, smoother and safer motion in imitation learning, better trajectory fit in inverse problems, and higher success with lower planning time in robot planning.

6. Limitations, misconceptions, and open directions

DMT does not by itself guarantee physical validity, global optimality, or full end-to-end differentiability. In sketch animation, the method depends on tracking quality; the consistency loss may degrade under rapid motion or severe occlusion, optimization remains iterative and expensive, and the system focuses on foreground objects rather than full-scene animation (Zhu et al., 30 Sep 2025). In differential-drive mobile-manipulator planning, inequality constraints are enforced only at discrete time samples, differentiability is used for solving but the method does not expose an end-to-end differentiable layer or implicit gradients through the optimizer, and topology search plus RRT-based initialization remain non-differentiable external components (Xu et al., 3 Jul 2025).

A second misconception is that any differentiable trajectory representation is equally expressive or stable. The power-basis versus Bernstein-basis ablation shows that basis choice can materially affect gradient conditioning (Zhu et al., 30 Sep 2025). In graph-based trajectory prediction with differential motion models, the numerical solver can have a substantial impact on performance, and the study explicitly advises against first-order Euler forward in favor of second-order Heun’s method (Westny et al., 2023). In differentiable imitation, convex position, velocity, and acceleration constraints are tractable, but nonlinear collision avoidance, contact-rich feasibility, and full robot dynamics are خارج scope; the method is “best viewed as task-space constrained trajectory shaping, not general motion planning” (Xu et al., 2024).

A third misconception is that differentiable trajectory methods are necessarily black-box learning systems. Several of the strongest examples are explicitly gray-box or hybrid. LeTO combines an LSTM encoder with a constrained QP layer (Xu et al., 2024). DIPP combines a Transformer-based predictor with a differentiable nonlinear planner and learned cost weights (Huang et al., 2022). DMMP/DMMFP learns a differentiable manifold offline, then samples or rejects candidates online rather than directly optimizing in the original trajectory space (Lee, 2024). Primitive-based truncated diffusion for differential-drive mobile manipulators uses differentiable task conditioning and denoising, but still relies on post-hoc trajectory optimization to ensure dynamic feasibility and task-specific optimality (Xu et al., 5 Apr 2026).

Open directions stated or implied across the literature include handling topology change more natively, integrating collision and dynamics constraints without coarse temporal sampling, extending differentiable models to dynamic environments and richer contact, improving uncertainty handling, and making the full proposal-to-execution stack differentiable rather than only the representation-learning front end. A plausible implication is that future DMT systems will be increasingly hybrid: differentiable representations for temporal structure, learned priors for multimodality, and optimization modules for hard geometric or dynamic feasibility.