Shape-Aware Unit: Conditioning Motion on Geometry

• Shape-Aware Unit is a computational module that explicitly integrates shape parameters, like the SMPL beta vector, into the motion synthesis process.
• The architecture jointly processes discrete motion tokens and continuous shape embeddings to generate motion that adapts dynamically to varying body geometries.
• Empirical results show that incorporating shape awareness reduces artifacts such as bone-length errors and foot sliding, thereby improving overall motion realism.

Shape-Aware Unit denotes a class of mechanisms that make shape, geometry, or morphology an explicit computational variable rather than a post-hoc correction. In text-driven human motion synthesis, the clearest formulation appears in “Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions,” where the functional unit is the decoder of a Shape-Aware FSQ-VAE that receives de-quantized motion latents together with a continuous SMPL shape vector $\boldsymbol{\beta}$, while a paired text model predicts both motion tokens and $\boldsymbol{\beta}$ from language (Liao et al., 4 Apr 2025). A plausible broader interpretation is that a shape-aware unit is any module whose internal representation or decoding is explicitly anchored to shape-dependent structure, whether that structure is a body model, a latent 3D grid, a floorplan partition, a deformation field, or a contour descriptor.

1. Conceptual definition and problem setting

In the motion-synthesis setting, “shape-aware” means that human motion is not treated as independent of body shape. A tall, long-legged runner and a short, stocky runner performing “the same motion” differ in stride length, foot clearance, timing, joint excursions, and balance adjustments. Existing text-to-motion systems typically normalize motions to a canonical skeleton or body, learn text $\to$ canonical motion, and only afterward retarget that motion to different shapes. That homogenization simplifies learning, but it destroys the natural correlation between shape and dynamics and can induce foot sliding, penetration, self-intersections, and unnatural joint limits after retargeting (Liao et al., 4 Apr 2025).

Within that formulation, a Shape-Aware Unit is not merely a retargeting stage. It is a mechanism that jointly generates or reconstructs a body shape and a motion consistent with that shape, with shape entering the generative pathway as an explicit continuous signal. In the primary instantiation, shape is represented by the SMPL parameter vector $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$, and the motion representation is a sequence

$$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$

including root-relative rotation and velocity, root height, joint 3D locations and velocities, joint rotations, and foot contact labels (Liao et al., 4 Apr 2025).

A common misconception is to equate shape awareness with later retargeting to a new skeleton. In the shape-aware formulation, shape is not an afterthought. It modulates decoding itself, so that the reconstructed trajectory is expected to respect the limb lengths and other shape-dependent structure implied by $\boldsymbol{\beta}$ (Liao et al., 4 Apr 2025).

2. Architectural realization in text-driven motion synthesis

The canonical realization consists of two stages. First, a Shape-Aware FSQ-VAE learns a discrete motion token space for shape-normalized motions $X^N$ but reconstructs shape-aware motions $\hat{X}^R$ when given both motion tokens and $\boldsymbol{\beta}$. Second, a T5-based model, called ShapeMove, takes text and predicts both a motion-token sequence and a dedicated shape token $[\text{BETA}]$, whose embedding is projected into continuous $\boldsymbol{\beta}$0 (Liao et al., 4 Apr 2025).

The encoder maps the normalized motion to a latent sequence $\boldsymbol{\beta}$1, which is quantized by finite scalar quantization. The latent is first projected into FSQ space, bounded with a $\boldsymbol{\beta}$2-based transform, and quantized dimension-wise with a straight-through estimator. The resulting integer-valued representation $\boldsymbol{\beta}$3 indexes an implicit codebook; with levels $\boldsymbol{\beta}$4, the number of resulting indices $\boldsymbol{\beta}$5 is around 1000. The temporal encoder is inherited from the T2M-GPT setting and downsamples time, for example $\boldsymbol{\beta}$6 (Liao et al., 4 Apr 2025).

The Shape-Aware Unit proper is the decoder-side conditioning mechanism: $\boldsymbol{\beta}$7 Here $\boldsymbol{\beta}$8 is an MLP that projects shape to a decoder feature, the projected shape is broadcast in time, and each quantized motion feature is concatenated with that shape embedding before temporal decoding. In the paper’s interpretation, the tokens encode canonical motion content, while the shape embedding encodes style in the sense of skeleton, limb lengths, and mass-distribution cues (Liao et al., 4 Apr 2025).

ShapeMove extends T5’s vocabulary with $\boldsymbol{\beta}$9 motion tokens, $\to$0, $\to$1, and $\to$2. For each sample, the output sequence is

$\to$3

The embedding at the $\to$4 position, $\to$5, is mapped to continuous shape by

$\to$6

This division of labor is central: shape is not hidden implicitly inside motion tokens, but supervised explicitly through a dedicated head (Liao et al., 4 Apr 2025).

3. Objectives that enforce shape awareness

The Shape-Aware FSQ-VAE is trained with a reconstruction-plus-physics objective. The reconstruction term is Smooth L1 on motion and rotations,

$\to$7

Three geometric or physical losses are then added: a float loss $\to$8 penalizing feet or other lowest joints that remain too far above the ground, a foot sliding loss $\to$9 penalizing horizontal foot velocity during contact, and a bone length loss $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$0 penalizing differences between ground-truth and reconstructed bone lengths. The full objective is

$\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$1

Bone-length consistency directly couples the decoder to the skeleton implied by $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$2, while float and sliding penalties couple contact realism to shape-dependent limb geometry (Liao et al., 4 Apr 2025).

ShapeMove itself uses a standard autoregressive cross-entropy token loss and an explicit shape-prediction loss,

$\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$3

This forces the $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$4 embedding to encode a text-conditioned shape vector, while the token sequence remains aligned with motion semantics (Liao et al., 4 Apr 2025).

The paper also augments shape diversity by replacing $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$5 of ground-truth $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$6 with synthetic $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$7 from Shapy’s Attributes-to-Shape model. Since no paired motion exists for those synthetic shapes, training on them uses only geometric losses $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$8 and $\boldsymbol{\beta} \in \mathbb{R}^{d_\beta}$9. This suggests a broader design principle: a shape-aware unit can often be regularized on unpaired geometric states even when full trajectory supervision is absent (Liao et al., 4 Apr 2025).

4. Empirical evidence and ablation findings

The quantitative evidence is strongest in four places. First, in quantizer reconstruction, the Shape-Aware FSQ-VAE achieves FID $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$0, Bone Length Difference $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$1 mm, and Jitter Difference $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$2, outperforming VQ-based baselines such as T2M-GPT, MotionGPT, and TM2T. The reduced bone-length error is presented as direct evidence that reconstruction conforms to subject-specific geometry (Liao et al., 4 Apr 2025).

Second, in text-to-motion generation, the full system achieves Penetrate $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$3 cm, Skate $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$4, Bone Length Variances $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$5, and FID $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$6, together with best or near-best RPrecision across Top-1, Top-2, and Top-3. The interpretation given is that the model maintains consistent bone lengths over time and adapts contacts better than baselines that do not condition on $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$7 (Liao et al., 4 Apr 2025).

Third, the explicit shape head is accurate in anthropometric space. Differences between predicted and intended measurements are Height $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$8 cm, Arm length $$X^R = \{x^R_1,\dots,x^R_T\}, \qquad x^R_t \in \mathbb{R}^{263},$$9 cm, Leg length $\boldsymbol{\beta}$0 cm, Chest circumference $\boldsymbol{\beta}$1 cm, Waist circumference $\boldsymbol{\beta}$2 cm, and Hip circumference $\boldsymbol{\beta}$3 cm. These are reported as small errors indicating that $\boldsymbol{\beta}$4 is a reliable conditioning channel (Liao et al., 4 Apr 2025).

Fourth, the ablation study shows that the conditioning pathway itself is essential. Without shape conditioning (“No sc”), the quantizer has FID $\boldsymbol{\beta}$5, Bone length diff $\boldsymbol{\beta}$6 mm, Float $\boldsymbol{\beta}$7 cm, and Skate $\boldsymbol{\beta}$8. Adding shape conditioning (“sc”) improves FID to $\boldsymbol{\beta}$9 and Bone length diff to $X^N$0 mm. Adding $X^N$1 yields Bone length diff $X^N$2 mm, and the full model (“sc + $X^N$3”) reaches FID $X^N$4, Bone length diff $X^N$5 mm, Float $X^N$6, and Skate $X^N$7. The paper interprets this as direct evidence that the shape-aware conditioning mechanism, rather than only extra losses, is the decisive ingredient (Liao et al., 4 Apr 2025).

A human perceptual study further measures Shape-to-Text, Motion-to-Text, and Plausibility of Motion with Shape. The method is preferred nearly as often as ground truth and is chosen 12–38% more often than baselines depending on the metric, with the largest effect on plausibility of motion with shape (Liao et al., 4 Apr 2025).

5. Cross-domain realizations of the same pattern

A plausible cross-domain reading is that “Shape-Aware Unit” names a reusable architectural motif rather than a domain-specific block. Across the supplied literature, the same motif reappears as a spatially grounded token, region, deformation controller, or loss component that encodes local or global shape explicitly.

Domain	Shape-aware unit	Primary function
Human motion	Motion tokens + continuous $X^N$8 decoder	Shape-consistent motion synthesis
Industrial CAD	Latent grid token with local geometric statistics	Geometry representation and attribution
Floorplans	Boundary-adaptive unit region with latent code	Geometry-aware planning, retrieval, generation
Video editing	Atlas-space deformation and UV-map warping module	Shape-consistent temporal editing
Skeleton-free animation	Handle predictor + mesh-conditioned diffusion	Mesh-specific motion adaptation
Medical segmentation	Fourier-descriptor loss term	Shape-preserving supervision

In industrial CAD analysis, “Shape” defines a latent grid

$X^N$9

where each token is tied to a grid cell and summarizes local geometric statistics of nearby points—positions, normals, and curvature—through a multiscale geometry-aware tokenizer. The token is explicitly described as a field of shape-aware units, and masked-token reconstruction plus contrastive consistency make each token reconstructible from context while remaining spatially localized for attribution (Mounmo et al., 19 Apr 2026).

In floorplan representation, Unit Region Encoding replaces pixels or room nodes with boundary-adaptive unit regions derived from a geometry-aware density map

$\hat{X}^R$0

and each region receives a latent code via dense feature extraction and region pooling. On 3D-Front and RPLAN, this yields on the order of $\hat{X}^R$1 regions instead of $\hat{X}^R$2 pixels while preserving room-label IoU above 99% on 3D-Front for suitable slicing strategies (Zhang et al., 19 Jan 2025).

In layered video editing, the shape-aware mechanism is a deformation module on top of Neural Layered Atlas. A keyframe deformation field $\hat{X}^R$3 is estimated by semantic correspondence, back-projected into atlas space, propagated to all frames, and used to deform UV maps and alpha masks. Diffusion-guided atlas optimization then refines both appearance and deformation, allowing shape change rather than only texture change (Lee et al., 2023).

In skeleton-free animation, TapMo’s shape-aware unit is the combination of a GCN-based mesh handle predictor and a diffusion model conditioned on a mesh deformation feature $\hat{X}^R$4. The predictor yields adaptive handles, skinning weights, and a shape descriptor; the diffusion module then predicts handle-level motion and mesh-specific local translation corrections, with geometry-aware spring, adversarial, and ARAP-style objectives preserving deformation plausibility (Zhang et al., 2023).

In medical segmentation, FourierLoss introduces a shape-aware loss based on Fourier descriptors of contour radial-distance functions. The shape discrepancy

$\hat{X}^R$5

multiplies pixel-wise cross-entropy as

$\hat{X}^R$6

Because the weights $\hat{X}^R$7 are trainable, the loss can shift emphasis between coarse outline and contour detail during training (Erden et al., 2023).

These examples differ operationally, but they share a common structure: a shape variable is first encoded in a representation that is stable and interpretable in its own domain, and that representation is then used either to route information, to condition generation, or to regularize prediction.

6. Limitations, misconceptions, and future directions

The motion-synthesis formulation has explicit limitations. It relies on a preprocessing step that converts SMPL $\hat{X}^R$8 into six hand-crafted attributes and then into text in a specific template, which limits the variety of shape language and the coverage of subtle or unconventional body types. Shape diversity still derives from AMASS and HumanML3D, even though Shapy A2S augments $\hat{X}^R$9, so extreme shapes and non-human topologies are not explored. The paper suggests richer body models such as SMPL-X, dynamic soft tissue, dynamic shape, stronger physics priors, and more powerful LLMs that can support free-form shape descriptions (Liao et al., 4 Apr 2025).

A second misconception is that shape-aware units must always be explicit geometric decoders. Other works show weaker but still meaningful forms. In monocular 3D detection, a center-based instance-segmentation head is used only during training to make shared features aware of visible shape; the branch is dropped at inference, yet it still improves occluded-object performance and Average Depth Similarity (Chen et al., 2022). In image editing, trajectory-based region control can infer an edit mask from divergence between inversion and editing flows without any explicit mesh or skeletal model; the resulting mechanism is still shape-aware because it localizes structural edits while preserving non-target regions (Long et al., 11 Aug 2025).

The broader implication is that a Shape-Aware Unit is best understood as a design pattern for injecting geometry into learning systems at the point where representation, decoding, or supervision would otherwise become shape-agnostic. In the strongest version, exemplified by the motion model, the unit receives a discrete or compressed content representation and a continuous shape variable and produces an output whose kinematics or geometry are conditioned on both. In weaker but still operative versions, it appears as a structured token field, a boundary-adaptive region set, a deformation controller, or a shape-sensitive loss. Across these forms, the recurring objective is the same: to preserve the correlation between form and behavior that canonicalized or purely appearance-driven models tend to erase.