--- title: 'Follow-Your-Shape: Geometry in Motion & Control' url: https://www.emergentmind.com/topics/follow-your-shape type: topic --- # Follow-Your-Shape: Geometry in Motion & Control Searching arXiv for recent papers using the term "Follow-Your-Shape" and the primary motion-synthesis paper. “Follow-Your-Shape” is a label that has been used in multiple technical settings to denote systems whose outputs are constrained by an explicit target morphology, contour, or trajectory. In text-driven human motion synthesis, it denotes the capability to generate human motions that not only follow a text description of an action, but also adapt the style and kinematics of that action to a specified body shape [2504.03639]. The same label has also been used for shape-aware image editing, arbitrary-shape multi-agent formation, real-time glyph recognition, and follow-the-leader continuum-robot planning [2508.08134], [2403.17417], [1109.5323], [2605.11618]. This suggests a recurring research pattern: geometry is not treated as a secondary by-product, but as a first-class conditioning signal or control objective. ## 1. Human-motion meaning of the term In the motion-synthesis literature, “Follow-Your-Shape” addresses a limitation of existing text-to-motion generation methods: they often learn a homogenized, canonical body shape because it is easier to model, but this homogenization can distort the natural correlations between different body shapes and their motion dynamics [2504.03639]. The central problem is therefore not only semantic alignment between text and action, but also morphological alignment between action realization and body shape. The formulation in “Shape My Moves” separates *content* and *style* in a specific way. Motion content is represented by discrete tokens learned from shape-normalized motion, while body shape is represented continuously through SMPL shape parameters. A single sequence model is then trained to predict both the continuous shape and the motion tokens from text, after which the motion is decoded back into a continuous, shape-aware trajectory. The stated result is a continuous \(3\)D joint trajectory \(\hat X\) whose overall form, including stride length, joint sweep, and center-of-mass height, naturally “follows” the predicted shape \(\hat\beta\). A common misconception in this setting is that shape normalization merely removes nuisance variation. The motion paper argues the opposite: normalizing away shape too aggressively may erase correlations that are physically and perceptually meaningful. Under this view, body shape is not only an appearance variable; it contributes to kinematic style. ## 2. Shape-aware FSQ-VAE and joint text conditioning The motion architecture begins with a shape-normalized motion sequence \[ X^N = [x^N_1,\dots,x^N_T]\in\mathbb R^{T\times D}, \] which is encoded as \[ Z=[z_1,\dots,z_\tau]=\mathcal E(X^N), \qquad z_i\in\mathbb R^d, \qquad \tau\ll T. \] The encoder output is quantized by a finite scalar quantizer. In the paper’s notation, each latent is first bounded with a per-dimension range \(L\), transformed with \(\tanh\) and scaling, and then rounded with straight-through gradients: \[ z^{(\mathrm e)} \mapsto z^{(\mathrm b)} = f(L)\odot \tanh(z^{(\mathrm e)}), \qquad \hat z = \bigl\lfloor z^{(\mathrm b)} \bigr\rceil_{\mathrm{ste}}. \] An equivalent vector-form description is \[ z_q=\arg\min_{e_j\in\mathcal E}\|z_e-e_j\|^2. \] Shape is reintroduced in the decoder rather than in the discrete codebook. Let \(\beta\) be the SMPL shape parameter. It is projected through an MLP projector \(P_{\theta_s}\) to obtain a time-invariant shape feature \[ \tilde\beta=P_{\theta_s}(\beta)\in\mathbb R^d. \] This feature is concatenated with each quantized code and decoded as \[ \hat X=\mathcal D\bigl[(\hat z_1\oplus \tilde\beta),\dots,(\hat z_\tau\oplus \tilde\beta)\bigr]. \] The training objective reconstructs the shape-aware ground-truth motion \(X^R\), adds a rotation reconstruction term, and imposes physics losses \(L_{\mathrm{float}}\), \(L_{\mathrm{slide}}\), and \(L_{\mathrm{bone}}\), which penalize ground-penetration, foot skating, and bone-length variation. Classic VQ-VAE commitment and codebook regularizers are also included. The same work explicitly avoids discretizing shape. Instead, the continuous SMPL shape vector \(\beta\), described as six measurement attributes plus the canonical ten SMPL parameters, is regressed from a learned transformer embedding: \[ \hat\beta = P_{\theta_e}(M_{\mathrm{[BETA]}}). \] The sequence model is a pretrained T5 transformer whose vocabulary is extended by \(k\) motion codes \(\{c_1,\dots,c_k\}\), a shape token \(\mathrm{[BETA]}\), and start/end tokens for motion. Given a text prompt, it autoregressively predicts \[ y = [\mathrm{[BETA]},\, c_{i_1},\, c_{i_2},\, \dots,\, c_{i_\tau}], \] with joint conditional probability \[ p(\beta,t\mid \text{text}) = p(\mathrm{[BETA]}\mid \text{txt}) \,p(c_{i_1}\mid \text{txt}) \cdots p(c_{i_\tau}\mid \text{txt},c_{i_{1:\tau-1}}). \] The training losses combine token prediction and shape regression, with \[ L_{\mathrm{shape}}=\lambda_\beta \|\beta-\hat\beta\|_1. \] To expose the VAE to diverse shapes, the training procedure replaces a random \(q\%\) of ground-truth \(\beta\) values with synthetics from the Shapy/A2S model; for these synthetic shapes, only the physics losses are applied [2504.03639]. ## 3. End-to-end synthesis pipeline, training regime, and significance At inference time, the motion pipeline is explicitly staged. A free-form text prompt is tokenized and passed through the transformer, which predicts the special shape token \(\mathrm{[BETA]}\) and a sequence of motion codes. The shape embedding yields \[ \hat\beta=P_{\theta_e}(M_{\mathrm{[BETA]}}), \] the motion codes are de-quantized to \(\hat Z=[\hat z_1,\dots,\hat z_\tau]\), the shape style feature is computed as \(\tilde\beta=P_{\theta_s}(\hat\beta)\), and the final motion is decoded by \[ \hat X=\mathcal D\bigl[(\hat z_1\oplus\tilde\beta),\dots,(\hat z_\tau\oplus\tilde\beta)\bigr]. \] The paper further notes that \(\hat X\) can optionally be retargeted to arbitrary skeletons via standard IK [2504.03639]. The reported training setup is specific. The FSQ-VAE uses AdamW with learning rate \(2\times 10^{-4}\) for \(200\)K iterations and \(10^{-5}\) for \(100\)K iterations, batch size \(256\), on A100. Its loss weights are \(\lambda_f=\lambda_s=\lambda_b=10\), \(\lambda_{\mathrm{rot}}=0.5\), and \(\beta_{\mathrm{vq}}=\gamma_{\mathrm{vq}}=1\). The transformer uses AdamW with learning rate \(8\times 10^{-4}\) and \(\lambda_\beta=0.5\), with two-stage fine-tuning: \(120\)K iterations on text↔motion and text→motion tasks, followed by \(30\)K iterations on text→motion only; batch size is \(64\) across \(8\) A100s. The evaluation is described as quantitative, qualitative, and supplemented by a comprehensive perceptual study. No metric values are given in the provided material, but the paper’s stated conclusion is that disentangling motion content and shape style in a quantized VAE, predicting both jointly from text through a single transformer, and reassembling them in the decoder yields motions that match the action description while also following the specific body shape predicted from language. ## 4. Shape-aware image editing under the same label In image editing, “Follow-Your-Shape” names a different method class: a training-free and mask-free framework for prompt-driven object-shape transformations while strictly preserving non-target content [2508.08134]. The motivation is that recent flow-based image editing models often struggle with large-scale shape transformations. The paper identifies three common failure modes in prior editors: reliance on external segmentation masks, reliance on noisy cross-attention maps, and unconditional KV-injection that preserves background but “over-anchors” and suppresses edits. The key construct is the Trajectory Divergence Map (TDM). The method performs DDIM/RF inversion of the source image and an editing denoising pass under the target prompt, then compares token-wise velocity differences between inversion and denoising trajectories. For token \(i\) at timestep \(t\), \[ \Delta v_t^{(i)} = v_{\mathrm{tgt}}(z_t^{(i)}, t)-v_{\mathrm{src}}(x_t^{(i)}, t), \qquad \delta_t^{(i)}=\|\Delta v_t^{(i)}\|_2. \] Each \(\delta_t\) is min-max normalized over spatial tokens, aggregated over a denoising window \(T_2\) by softmax fusion, smoothed with a Gaussian kernel, and thresholded at \(\tau\), with default \(0.35\), to produce a binary mask \(M_S\in\{0,1\}^{H\times W}\). The mask localizes where the shape edit should occur. This mask drives Scheduled KV Injection over three denoising phases. In Phase 1, \(M_S=0\) everywhere, producing unconditional KV injection of inversion features for initial trajectory stabilization. In Phase 2, keys and values are blended token-wise: \[ K^* = M_S \odot K_{\mathrm{tgt}} + (1-M_S)\odot K_{\mathrm{inv}}, \qquad V^* = M_S \odot V_{\mathrm{tgt}} + (1-M_S)\odot V_{\mathrm{inv}}. \] In Phase 3, \(M_S=1\), so only target-prompt features are injected. Because the mask is derived from the model’s own velocity differences, no external mask is required. The benchmark introduced for this task, ReShapeBench, comprises \(120\) new images and enriched prompt pairs specifically curated for shape-aware editing. Its composition is given as \(70\) single-object images, \(50\) multi-object images, and a \(50\)-image evaluation split mixing these with selected PIE-Bench examples; all images are \(512\times 512\), and prompts are generated by Qwen-2.5-VL and manually validated. Metrics are LAION Aesthetic Score, PSNR, LPIPS\(\times 10^3\), and CLIP Similarity. The reported numbers are: Follow-Your-Shape achieves Aesthetic \(6.57\), PSNR \(35.79\), LPIPS \(8.23\), and CLIP \(33.71\); the next best methods are reported as Aesthetic \(6.53\)–\(6.52\), PSNR \(32.9\)–\(34.7\), LPIPS \(16\)–\(18\), and CLIP \(26\)–\(30\). The same paper also states limitations: prompt and hyperparameter sensitivity, and temporal instability when extending TDM to video in Wan \(2.1\), where instability in \(\delta_t\) across frames can cause flicker or inconsistent edits. ## 5. Shape following in distributed control and continuum robotics In multi-agent formation control, a related “Follow-Your-Shape” description is built on cyclic pursuit for arbitrary desired closed curves [2403.17417]. Agents move in \(\mathbb R^2\), with positions \(x_i(k)\), headings \(\phi_i(k)\), and predecessor index \[ n(i)= \begin{cases} i+1,& i0, \] pulls the agent toward the predecessor’s trajectory. The paper distinguishes two scenarios: one in which each agent knows its own absolute heading and the full Fourier series of \(\gamma\), and a local-information scenario in which agents know only their initial heading and one-step predecessor displacement, embed shape information via \[ v'_{i1}(k)=R(\theta_i(k))\,\dot\gamma(\tau_i(k)), \] and sporadically realign orientation with probability \(\beta\). The numerically most robust law is the “Achievement–Decrease” rule \[ \beta=c_d\,A_i(k). \] Simulation details include initial conditions on a circle of radius \(15\), parameters \(\eta=0.01\), \(\alpha=0.01\), \(N=3\) with later tests varying \(N\) from \(5\) to \(30\), and convergence measured by the average discrete Fréchet distance to the best-fit scaled, rotated, translated \(\gamma\). The method is reported to form multiple shapes, including those represented as Fourier series, while collision avoidance is explicitly not handled. In continuum robotics, the cognate notion is follow-the-leader motion for manipulator-mounted continuum robots [2605.11618]. The problem is defined over waypoint sequence \[ \mathcal W=\{w_1,\dots,w_n\}, \qquad w_i\in\mathbb R^3, \] configuration space \[ \mathcal Q\subset\mathbb R^d, \qquad \mathcal K = SE(3)\times \mathcal Q, \] and discretized backbone \[ f(q)=P=[p_0,\dots,p_D]^\top\in\mathbb R^{(D+1)\times 3}, \] with objectives of exact tip tracking and minimal shape deviation. The errors are \[ e_{\mathrm{tip}}(i)=\|p_D-w_i\|, \qquad e_{\mathrm{shape}}(i)=\mathrm{Chamfer}(P_{\mathrm{act}},\mathcal W_i). \] The planner builds an offline library \[ \mathcal L=\{(q^{(j)},P^{(j)})\}_{j=1}^{N_{\mathrm{lib}}}, \qquad q^{(j)}\sim \mathrm{Uniform}(\mathcal Q), \] then performs online search over active shapes, closed-form base-pose alignment, and Chamfer-distance selection. Base alignment is decomposed into translation, rotation about the tip via Rodrigues’ formula, and axial rotation; the combined base pose is \[ T_b = T_{\mathrm{axial}(\phi)}\cdot T_{\mathrm{bend}(\theta)}\cdot T_{\mathrm{trans}(t)} \in SE(3). \] Theoretical guarantees include resolution complete shape search, exact tip tracking at each waypoint and each interpolation step, and an asymptotic tip-error bound \[ \max_\beta \|\mathbf p_{\mathrm{tip}}(\beta)-\mathbf p_{\mathrm{ftl}}(\beta)\| \le \frac{C}{h^2}\,\Delta_{\max}^2. \] On \(120\) simulated paths over \(3\) test classes, the paper reports \(0\%\) tip error and \(1.9\%\) mean shape deviation at \(100\%\) success rate, with clustered search reducing planning time from about \(46\) s to about \(2.3\) s in the PCC benchmark. Hardware experiments on a \(6\)-DOF tendon-driven continuum robot mounted on a serial manipulator report nonzero execution errors, attributed to unmodeled tendon stretch, friction, and hysteresis rather than planner failure. ## 6. Gesture interfaces, recognition, and broader interpretation An older but conceptually related usage appears in the Squiggle glyph recognizer, which describes how a “Follow-Your-Shape” interface can be built around affine template matching and real-time shadow rendering [1109.5323]. Raw pen samples are regularized into fixed-length segments, given as \(3\) px in the prototype, then interpolated to exactly \(n\) milestones with \(n\approx 16\). The total path length \[ \lambda(g)=\sum_i \|g_{i+1}-g_i\| \] is recorded for triangle-area normalization. Affine alignment is constructed from corresponding triangles. For each glyph \(p\), the triangle matrix is \[ M(p)_{abc} = [\,p_b-p_a,\; p_c-p_a\,], \] with determinant normalized as \[ D(p)_{abc}= \frac{\det(M(p)_{abc})}{(\lambda(p)/2)^2}. \] The recognizer selects the largest-area triangles from the input glyph, typically the top \(m\) with \(m\approx 8\)–\(16\), and for each template forms the affine map \[ T_{abc}=\hat H_{abc}^{-1}\hat G_{abc}, \] provided the template triangle is nondegenerate. Match quality is then measured by the sum of squared aligned point distances, \[ \mathrm{Metric}(g,r)=\sum_{i=0}^{n-1}\bigl[(g_i.x-r_i.x)^2+(g_i.y-r_i.y)^2\bigr]. \] Incremental matching updates only the tail of the path, rebuilds the determinant structure only every few input points, and renders “shadows” of the top-\(k\) matches, with \(k=3\) given as an example. The prototype parameters include \(m=10\), degenerate-triangle threshold \(10^{-8}\), line-detection threshold \(0.002\)–\(0.004\), orientation-similarity cutoff \(2.12\), shadow opacity \(15\%\)–\(25\%\), and runtime of \(30\)–\(60\) Hz on modern tablets. Across these literatures, a plausible implication is that “Follow-Your-Shape” is less a single formalism than a recurring design principle. In motion synthesis, shape is a conditioning variable that modifies kinematic realization; in image editing, it is a region-localized structural transformation inferred from trajectory divergence; in formation control and continuum robotics, it is a target curve or path to be tracked; in glyph recognition, it is the geometric object against which incremental input is affinely aligned. What remains constant is the insistence that geometry be explicitly modeled rather than absorbed into a residual latent. Source: https://www.emergentmind.com/topics/follow-your-shape