Follow-Your-Shape: Geometry in Motion & Control

• Follow-Your-Shape is a design principle that explicitly models geometry to condition outputs in motion synthesis, image editing, and robotic control.
• It employs techniques like FSQ-VAE to couple discrete motion tokens with continuous SMPL shape parameters, preserving kinematic style and enhancing text-driven synthesis.
• The approach extends to formation control, continuum robotics, and glyph recognition, providing robust, shape-aware outputs across various technical settings.

Searching arXiv for papers using the term "Follow-Your-Shape" and the primary motion-synthesis paper. “Follow-Your-Shape” is a label that has been used in multiple technical settings to denote systems whose outputs are constrained by an explicit target morphology, contour, or trajectory. In text-driven human motion synthesis, it denotes the capability to generate human motions that not only follow a text description of an action, but also adapt the style and kinematics of that action to a specified body shape (Liao et al., 4 Apr 2025). The same label has also been used for shape-aware image editing, arbitrary-shape multi-agent formation, real-time glyph recognition, and follow-the-leader continuum-robot planning (Long et al., 11 Aug 2025, Fujioka et al., 2024, Lee, 2011, Shentu et al., 12 May 2026). This suggests a recurring research pattern: geometry is not treated as a secondary by-product, but as a first-class conditioning signal or control objective.

1. Human-motion meaning of the term

In the motion-synthesis literature, “Follow-Your-Shape” addresses a limitation of existing text-to-motion generation methods: they often learn a homogenized, canonical body shape because it is easier to model, but this homogenization can distort the natural correlations between different body shapes and their motion dynamics (Liao et al., 4 Apr 2025). The central problem is therefore not only semantic alignment between text and action, but also morphological alignment between action realization and body shape.

The formulation in “Shape My Moves” separates content and style in a specific way. Motion content is represented by discrete tokens learned from shape-normalized motion, while body shape is represented continuously through SMPL shape parameters. A single sequence model is then trained to predict both the continuous shape and the motion tokens from text, after which the motion is decoded back into a continuous, shape-aware trajectory. The stated result is a continuous $3$D joint trajectory $\hat X$ whose overall form, including stride length, joint sweep, and center-of-mass height, naturally “follows” the predicted shape $\hat\beta$.

A common misconception in this setting is that shape normalization merely removes nuisance variation. The motion paper argues the opposite: normalizing away shape too aggressively may erase correlations that are physically and perceptually meaningful. Under this view, body shape is not only an appearance variable; it contributes to kinematic style.

2. Shape-aware FSQ-VAE and joint text conditioning

The motion architecture begins with a shape-normalized motion sequence

$X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$

which is encoded as

$$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$

The encoder output is quantized by a finite scalar quantizer. In the paper’s notation, each latent is first bounded with a per-dimension range $L$, transformed with $\tanh$ and scaling, and then rounded with straight-through gradients: $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$ An equivalent vector-form description is

$z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$

Shape is reintroduced in the decoder rather than in the discrete codebook. Let $\beta$ be the SMPL shape parameter. It is projected through an MLP projector $\hat X$0 to obtain a time-invariant shape feature

$\hat X$1

This feature is concatenated with each quantized code and decoded as

$\hat X$2

The training objective reconstructs the shape-aware ground-truth motion $\hat X$3, adds a rotation reconstruction term, and imposes physics losses $\hat X$4, $\hat X$5, and $\hat X$6, which penalize ground-penetration, foot skating, and bone-length variation. Classic VQ-VAE commitment and codebook regularizers are also included.

The same work explicitly avoids discretizing shape. Instead, the continuous SMPL shape vector $\hat X$7, described as six measurement attributes plus the canonical ten SMPL parameters, is regressed from a learned transformer embedding: $\hat X$8 The sequence model is a pretrained T5 transformer whose vocabulary is extended by $\hat X$9 motion codes $\hat\beta$0, a shape token $\hat\beta$1, and start/end tokens for motion. Given a text prompt, it autoregressively predicts

$\hat\beta$2

with joint conditional probability

$\hat\beta$3

The training losses combine token prediction and shape regression, with

$\hat\beta$4

To expose the VAE to diverse shapes, the training procedure replaces a random $\hat\beta$5 of ground-truth $\hat\beta$6 values with synthetics from the Shapy/A2S model; for these synthetic shapes, only the physics losses are applied (Liao et al., 4 Apr 2025).

3. End-to-end synthesis pipeline, training regime, and significance

At inference time, the motion pipeline is explicitly staged. A free-form text prompt is tokenized and passed through the transformer, which predicts the special shape token $\hat\beta$7 and a sequence of motion codes. The shape embedding yields

$\hat\beta$8

the motion codes are de-quantized to $\hat\beta$9, the shape style feature is computed as $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$0, and the final motion is decoded by

$X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$1

The paper further notes that $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$2 can optionally be retargeted to arbitrary skeletons via standard IK (Liao et al., 4 Apr 2025).

The reported training setup is specific. The FSQ-VAE uses AdamW with learning rate $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$3 for $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$4K iterations and $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$5 for $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$6K iterations, batch size $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$7, on A100. Its loss weights are $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$8, $X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D},$9, and $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$0. The transformer uses AdamW with learning rate $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$1 and $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$2, with two-stage fine-tuning: $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$3K iterations on text↔motion and text→motion tasks, followed by $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$4K iterations on text→motion only; batch size is $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$5 across $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$6 A100s.

The evaluation is described as quantitative, qualitative, and supplemented by a comprehensive perceptual study. No metric values are given in the provided material, but the paper’s stated conclusion is that disentangling motion content and shape style in a quantized VAE, predicting both jointly from text through a single transformer, and reassembling them in the decoder yields motions that match the action description while also following the specific body shape predicted from language.

4. Shape-aware image editing under the same label

In image editing, “Follow-Your-Shape” names a different method class: a training-free and mask-free framework for prompt-driven object-shape transformations while strictly preserving non-target content (Long et al., 11 Aug 2025). The motivation is that recent flow-based image editing models often struggle with large-scale shape transformations. The paper identifies three common failure modes in prior editors: reliance on external segmentation masks, reliance on noisy cross-attention maps, and unconditional KV-injection that preserves background but “over-anchors” and suppresses edits.

The key construct is the Trajectory Divergence Map (TDM). The method performs DDIM/RF inversion of the source image and an editing denoising pass under the target prompt, then compares token-wise velocity differences between inversion and denoising trajectories. For token $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$7 at timestep $$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$8,

$$Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T.$$9

Each $L$0 is min-max normalized over spatial tokens, aggregated over a denoising window $L$1 by softmax fusion, smoothed with a Gaussian kernel, and thresholded at $L$2, with default $L$3, to produce a binary mask $L$4. The mask localizes where the shape edit should occur.

This mask drives Scheduled KV Injection over three denoising phases. In Phase 1, $L$5 everywhere, producing unconditional KV injection of inversion features for initial trajectory stabilization. In Phase 2, keys and values are blended token-wise: $L$6 In Phase 3, $L$7, so only target-prompt features are injected. Because the mask is derived from the model’s own velocity differences, no external mask is required.

The benchmark introduced for this task, ReShapeBench, comprises $L$8 new images and enriched prompt pairs specifically curated for shape-aware editing. Its composition is given as $L$9 single-object images, $\tanh$0 multi-object images, and a $\tanh$1-image evaluation split mixing these with selected PIE-Bench examples; all images are $\tanh$2, and prompts are generated by Qwen-2.5-VL and manually validated. Metrics are LAION Aesthetic Score, PSNR, LPIPS$\tanh$3, and CLIP Similarity. The reported numbers are: Follow-Your-Shape achieves Aesthetic $\tanh$4, PSNR $\tanh$5, LPIPS $\tanh$6, and CLIP $\tanh$7; the next best methods are reported as Aesthetic $\tanh$8–$\tanh$9, PSNR $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$0–$$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$1, LPIPS $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$2–$$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$3, and CLIP $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$4–$$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$5. The same paper also states limitations: prompt and hyperparameter sensitivity, and temporal instability when extending TDM to video in Wan $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$6, where instability in $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$7 across frames can cause flicker or inconsistent edits.

5. Shape following in distributed control and continuum robotics

In multi-agent formation control, a related “Follow-Your-Shape” description is built on cyclic pursuit for arbitrary desired closed curves (Fujioka et al., 2024). Agents move in $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$8, with positions $$z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}.$$9, headings $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$0, and predecessor index

$z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$1

The desired shape is a closed planar curve $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$2 represented as a truncated Fourier series,

$z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$3

with $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$4 for all $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$5.

The controller maintains an internal phase variable $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$6 and uses

$z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$7

so that agents remain evenly spaced in phase. Position is updated by

$z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$8

where $z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2.$9 is the shape-following velocity and

$\beta$0

pulls the agent toward the predecessor’s trajectory. The paper distinguishes two scenarios: one in which each agent knows its own absolute heading and the full Fourier series of $\beta$1, and a local-information scenario in which agents know only their initial heading and one-step predecessor displacement, embed shape information via

$\beta$2

and sporadically realign orientation with probability $\beta$3. The numerically most robust law is the “Achievement–Decrease” rule

$\beta$4

Simulation details include initial conditions on a circle of radius $\beta$5, parameters $\beta$6, $\beta$7, $\beta$8 with later tests varying $\beta$9 from $\hat X$00 to $\hat X$01, and convergence measured by the average discrete Fréchet distance to the best-fit scaled, rotated, translated $\hat X$02. The method is reported to form multiple shapes, including those represented as Fourier series, while collision avoidance is explicitly not handled.

In continuum robotics, the cognate notion is follow-the-leader motion for manipulator-mounted continuum robots (Shentu et al., 12 May 2026). The problem is defined over waypoint sequence

$\hat X$03

configuration space

$\hat X$04

and discretized backbone

$\hat X$05

with objectives of exact tip tracking and minimal shape deviation. The errors are

$\hat X$06

The planner builds an offline library

$\hat X$07

then performs online search over active shapes, closed-form base-pose alignment, and Chamfer-distance selection. Base alignment is decomposed into translation, rotation about the tip via Rodrigues’ formula, and axial rotation; the combined base pose is

$\hat X$08

Theoretical guarantees include resolution complete shape search, exact tip tracking at each waypoint and each interpolation step, and an asymptotic tip-error bound

$\hat X$09

On $\hat X$10 simulated paths over $\hat X$11 test classes, the paper reports $\hat X$12 tip error and $\hat X$13 mean shape deviation at $\hat X$14 success rate, with clustered search reducing planning time from about $\hat X$15 s to about $\hat X$16 s in the PCC benchmark. Hardware experiments on a $\hat X$17-DOF tendon-driven continuum robot mounted on a serial manipulator report nonzero execution errors, attributed to unmodeled tendon stretch, friction, and hysteresis rather than planner failure.

6. Gesture interfaces, recognition, and broader interpretation

An older but conceptually related usage appears in the Squiggle glyph recognizer, which describes how a “Follow-Your-Shape” interface can be built around affine template matching and real-time shadow rendering (Lee, 2011). Raw pen samples are regularized into fixed-length segments, given as $\hat X$18 px in the prototype, then interpolated to exactly $\hat X$19 milestones with $\hat X$20. The total path length

$\hat X$21

is recorded for triangle-area normalization.

Affine alignment is constructed from corresponding triangles. For each glyph $\hat X$22, the triangle matrix is

$\hat X$23

with determinant normalized as

$\hat X$24

The recognizer selects the largest-area triangles from the input glyph, typically the top $\hat X$25 with $\hat X$26–$\hat X$27, and for each template forms the affine map

$\hat X$28

provided the template triangle is nondegenerate. Match quality is then measured by the sum of squared aligned point distances,

$\hat X$29

Incremental matching updates only the tail of the path, rebuilds the determinant structure only every few input points, and renders “shadows” of the top-$\hat X$30 matches, with $\hat X$31 given as an example. The prototype parameters include $\hat X$32, degenerate-triangle threshold $\hat X$33, line-detection threshold $\hat X$34–$\hat X$35, orientation-similarity cutoff $\hat X$36, shadow opacity $\hat X$37–$\hat X$38, and runtime of $\hat X$39–$\hat X$40 Hz on modern tablets.

Across these literatures, a plausible implication is that “Follow-Your-Shape” is less a single formalism than a recurring design principle. In motion synthesis, shape is a conditioning variable that modifies kinematic realization; in image editing, it is a region-localized structural transformation inferred from trajectory divergence; in formation control and continuum robotics, it is a target curve or path to be tracked; in glyph recognition, it is the geometric object against which incremental input is affinely aligned. What remains constant is the insistence that geometry be explicitly modeled rather than absorbed into a residual latent.