MoMask++: Multi-Scale Text-to-Motion Generation

• The paper introduces MoMask++, a framework that redefines motion tokenization using multi-scale residual quantization with a shared codebook.
• It employs a single masked transformer for unified token generation, enabling precise control over execution details from long, expressive text prompts.
• Empirical results on benchmarks like SnapMoGen demonstrate enhanced text-motion alignment and improved generation quality under complex, detailed descriptions.

Searching arXiv for the specified paper and closely related work to ground the article in current literature. MoMask++ is a text-to-motion generation framework introduced in "SnapMoGen: Human Motion Generation from Expressive Texts" for high-fidelity, text-aligned 3D human motion synthesis from expressive prompts, with an explicit emphasis on fine-grained control over execution details, style, direction, speed, timing, and body parts (Guo et al., 12 Jul 2025). Within the SnapMoGen formulation, MoMask++ reworks residual vector quantization and masked token generation by transforming motion into multi-scale token sequences, using a shared codebook across quantization layers, and learning to generate all tokens with a single generative masked transformer rather than a hierarchical stack. The method is presented in direct response to the constraints of short and semantically sparse text-motion datasets, which the paper identifies as a major obstacle to nuanced controllability and generalization to casual user prompts.

1. Conceptual setting and motivation

MoMask++ is situated in the text-to-motion literature as a generative masked modeling approach designed for expressive, long-form textual descriptions rather than short, generic prompts (Guo et al., 12 Jul 2025). The motivating claim is that existing datasets, exemplified by HumanML3D with average descriptions of 12 words, do not provide sufficient semantic detail for precise motion control. SnapMoGen addresses this by supplying text of substantially greater density, with an average of 48 words per description, thereby exposing models to richer text-motion correspondences.

The framework is motivated by three limitations attributed to prior methods, including MoMask. First, residual VQ in MoMask operates at a single full temporal scale, so tokens in deeper residual layers carry progressively less information. The reported consequence is overproduction of tokens with uneven information content, which reduces efficiency in text-to-token learning. Second, layer-specific codebooks create heterogeneous vocabularies across residual layers, which in turn necessitates hierarchical token generation through a primary transformer for first-layer tokens and a secondary transformer for remaining tokens. The paper characterizes this as reducing flexibility and limiting the representational capacity of non-first-layer tokens. Third, diffusion and autoregressive baselines are reported to struggle with complex, long prompts and with maintaining multimodal coherence under the SnapMoGen evaluation setting.

These design pressures explain why MoMask++ is framed not merely as an incremental architecture change but as a reformulation of how motion should be tokenized and generated under long, expressive text supervision. A plausible implication is that the method treats token economy and token semantics as central issues in text-to-motion alignment, rather than viewing motion tokenization as a preprocessing step detached from generation quality.

2. Motion representation and multi-scale residual quantization

MoMask++ adopts pose features that largely follow HumanML3D but are adapted to SnapMoGen’s 24-joint skeleton and are directly convertible to mocap formats such as BVH (Guo et al., 12 Jul 2025). The motion feature vector is 296-dimensional and includes root angular velocity on $Y$, root linear velocity on $XZ$, root height on $Y$, together with 6D local joint rotations, local joint positions, and local joint velocities. This representation is the input to a VQ-VAE whose encoder and decoder each contain three dilated res-blocks with temporal downscale or upscale factor 4, with self-attention layers inserted after res-blocks to improve reconstruction fidelity.

The core technical change is multi-scale residual VQ. In standard VQ, the latent sequence $f \in \mathbb{R}^{n \times d}$ is quantized as

$$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$

$$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$

Conventional residual quantization, as summarized in the paper, refines the approximation layer by layer:

$$q^v = Q^v(r^v), \qquad \hat{f}^v = \mathrm{lookup}(C^v, q^v), \qquad r^{v+1} = r^v - \hat{f}^v,$$

with final approximation

$\hat{f} = \sum_{v=0}^{V} \hat{f}^v.$

MoMask++ replaces this with a multi-scale residual formulation in which a common codebook $C$ is shared across all layers, and each layer operates at a temporal resolution $h^v$ satisfying $XZ$0:

$XZ$1

$XZ$2

$XZ$3

where $XZ$4 denotes bilinear interpolation to length $XZ$5.

The training objective preserves the residual alignment formulation rather than reverting to standard VQ codebook and commitment losses:

$XZ$6

and augments it with an essential rotational feature term:

$XZ$7

The temporal scales progress as $XZ$8, moving from coarse to fine. HumanML3D uses 4 quantization layers with shared codebook size $XZ$9, whereas SnapMoGen uses 2 quantization layers with shared codebook size $Y$0. A concrete example given for a 320-frame sequence uses 10 scales with token counts $Y$1, totaling 266 tokens; the paper reports that this outperforms a 6-layer 480-token full-scale RVQ reconstruction while using 45% fewer tokens. This is presented as evidence that multi-scale tokenization continues to allocate meaningful semantics to deeper layers rather than relegating them to low-information residual corrections.

3. Single-transformer masked generation and text conditioning

The generative stage of MoMask++ uses a bidirectional transformer that predicts motion tokens conditioned on text, with tokens from all temporal scales concatenated into a single sequence of length

$Y$2

(Guo et al., 12 Jul 2025). This single-sequence design is a direct consequence of the shared codebook and is intended to eliminate the hierarchical conditioning structure required by earlier MoMask-style token generation.

Text conditioning is implemented in two variants. In the in-context setting, text token embeddings and motion token embeddings are concatenated into one transformer input sequence. In the cross-attention setting, dedicated multi-head cross-attention layers allow motion features to query text features. T5-base provides the word-level text features in both cases. The paper reports a dataset-dependent behavior: in-context conditioning generally performs better on HumanML3D, whereas cross-attention is less prone to overfitting and performs better on SnapMoGen’s longer descriptions.

Training uses variable-rate uniform masking with masking fraction

$Y$3

and a masked-token cross-entropy objective

$Y$4

where $Y$5 is the partially masked sequence and $Y$6 is the text condition. The method also uses a replacing-and-remasking strategy in the style of BERT and MoMask, together with classifier-free guidance implemented by dropping text conditioning with probability 10%.

Inference follows a MaskGIT-like parallel decoding regime. The token sequence is initialized as $Y$7, and the model runs for a fixed number of iterations $Y$8. At each iteration, it predicts distributions for all masked positions, samples token identities, computes confidence values, and then re-masks

$Y$9

lowest-confidence tokens. CFG is applied with guidance scale $f \in \mathbb{R}^{n \times d}$0, with $f \in \mathbb{R}^{n \times d}$1 on SnapMoGen and $f \in \mathbb{R}^{n \times d}$2 on HumanML3D. After the final iteration, the token sequence is decoded through the VQ-VAE decoder into motion.

This architecture concentrates the entire generation problem into masked parallel refinement over a unified token vocabulary. The paper’s interpretation is that shared-vocabulary multi-scale tokenization allows the transformer to learn all scales jointly, better exploiting token capacity and avoiding the representational asymmetry of first-layer versus deeper-layer tokens.

4. SnapMoGen dataset and its role in MoMask++

MoMask++ is introduced together with SnapMoGen, and the architecture is tightly coupled to the dataset’s design assumptions (Guo et al., 12 Jul 2025). SnapMoGen contains 20,450 motion clips covering 43.7 hours and 4.7M frames at 30 fps. Each clip spans 4–12 seconds with average length 7.8 seconds. The clips are segmented from long sequences while preserving temporal continuity, a property used to motivate later discussion of long-term generation and blending. The dataset includes 122,565 text descriptions, comprising 2 human and 4 LLM-augmented descriptions per clip, with average length 48 words. It involves 10 performers and includes diverse activities and styles, including examples such as elderly and zombie.

Segmentation is described as velocity-trough-based probabilistic segmentation at motionless moments with hard minimum and maximum durations. Motion capture uses Xsens and Rokoko, and mirrored augmentation is applied throughout experiments. These dataset details are not incidental: they define the empirical environment in which MoMask++ is expected to learn temporally coherent and semantically dense motion-text mappings.

The paper also integrates LLM rewriting into both dataset construction and inference-time use. During augmentation, an LLM corrects typos and grammar and rewrites human annotations for diversity while preserving semantics. At inference time, an LLM expands casual inputs into 60–100-word expressive, motion-focused prompts specifying body posture, timing, stylistic elements, and duration within 4–12 seconds. The stated purpose is to align user inputs with the expressive narration style present in SnapMoGen. This suggests that MoMask++ is designed not only as a generative model but as part of a broader prompting pipeline in which textual style normalization materially affects controllability.

The framework is also positioned for long-term motion generation and blending because the segmentation preserves continuity from long sequences. The paper presents qualitative long-sequence generations, and it argues that the coarse-to-fine temporal structure of multi-scale tokens is conducive to maintaining longer-horizon coherence. It does not, however, define a dedicated long-horizon evaluation protocol.

5. Empirical performance and benchmark behavior

The reported evaluations cover both HumanML3D and SnapMoGen, with MoMask++ assessed under in-context and cross-attention conditioning variants (Guo et al., 12 Jul 2025). On HumanML3D, the in-context variant obtains R-Precision Top-1/Top-2/Top-3 of $f \in \mathbb{R}^{n \times d}$3, FID $f \in \mathbb{R}^{n \times d}$4, MM Dist $f \in \mathbb{R}^{n \times d}$5, and Multimodality $f \in \mathbb{R}^{n \times d}$6. The cross-attention variant obtains R-Precision $f \in \mathbb{R}^{n \times d}$7, FID $f \in \mathbb{R}^{n \times d}$8, MM Dist $f \in \mathbb{R}^{n \times d}$9, and Multimodality $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$0. The paper compares these with MoMask at $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$1 R-Precision, FID $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$2, MM Dist $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$3, and Multimodality $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$4. The stated interpretation is that MoMask++ generally improves R-Precision on HumanML3D, while MoMask retains a lower FID on that dataset.

On SnapMoGen, the in-context variant obtains R-Precision Top-1/Top-2/Top-3 of $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$5, FID $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$6, CLIP score $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$7, and Multimodality $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$8. The cross-attention variant obtains $$f = E(m), \qquad q_i = \arg\min_{k \in [K]} \| \mathrm{lookup}(C, k) - f_i \|_2,$$9, FID $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$0, CLIP score $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$1, and Multimodality $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$2. MoMask is reported at $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$3, FID $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$4, CLIP score $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$5, and Multimodality $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$6. The paper characterizes MoMask++ as state of the art on SnapMoGen in both motion-text alignment and motion quality, while diffusion-based baselines are described as yielding higher FID and lower alignment scores under long prompts.

A compact comparison is as follows.

Benchmark	Variant	Main reported outcome
HumanML3D	MoMask++ in-context	Best reported R-Precision among MoMask++ variants
HumanML3D	MoMask	Lower FID than MoMask++
SnapMoGen	MoMask++ cross-attention	Best reported FID and CLIP score among MoMask++ variants
SnapMoGen	MoMask++ in-context	Best reported Top-1 R-Precision among MoMask++ variants
SnapMoGen	MoMask	Lower alignment and worse FID than MoMask++

These results matter because they do not present a uniform dominance claim across all metrics and datasets. Instead, they indicate a trade-off structure: MoMask++ appears especially effective when the supervision regime includes long, expressive texts, whereas on HumanML3D its main advantage is stronger alignment rather than consistently superior distributional quality under FID.

6. Ablations, limitations, and practical interpretation

The ablation study isolates several factors that shape MoMask++ behavior (Guo et al., 12 Jul 2025). Compact 148-dimensional essentials improve VQ reconstruction but slightly reduce generation quality relative to the full 296-dimensional pose feature set. Larger codebooks improve reconstruction and CLIP alignment but do not monotonically improve FID, and an intermediate size can be optimal. Increasing the number of quantization layers improves reconstruction but can complicate generation, with 2 layers performing best on SnapMoGen. Full-scale layers can yield better raw VQ metrics but worse generation because of inefficient token utilization. Adding self-attention inside the VQ model improves both reconstruction and generation. CLIP is described as inadequate for long, complex texts, so T5-base is preferred. Larger transformers are reported to overfit on long prompts, and base-size cross-attention works best.

These observations clarify a recurring point in the paper: better token reconstruction does not necessarily imply better text-conditioned generation. The authors explicitly note that multi-scale residual quantization can outperform full-scale quantization in downstream generation even when the latter attains lower VQ reconstruction error. This suggests that the relevant bottleneck is not only latent approximation fidelity but also how token semantics are distributed across the sequence seen by the generator.

The limitations are equally explicit. VQ quantization error can degrade motion quality. Rare or uncommon motion patterns and prompts remain challenging. Physical plausibility, including foot contacts, is not guaranteed. The dataset itself contains inertial mocap noise, including global position imprecision and jitter, and lacks very skilled or outdoor actions. Failure cases are also noted for unusual or underspecified prompts when using the LLM rewriting pipeline.

From a deployment perspective, the paper’s practical guidance is consistent with these findings: prefer T5-base text features, multi-scale RVQ with a shared codebook, and a single masked transformer over all tokens; use classifier-free guidance and the MaskGIT-like remasking schedule; and tune codebook size and the number of quantization layers to balance reconstruction against generation quality. The reported implementation uses an 8-layer transformer with feedforward size 1024, latent dimension 384, 6 attention heads, and dropout 0.2; VQ-VAE and transformer batch sizes are 256 and 64 respectively; loss weights are $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$7 and $$\hat{f} = \mathrm{lookup}(C, q), \qquad \hat{m} = D(\hat{f}).$$8; and training is reported on a single Tesla V100 GPU for MoMask++.

MoMask++ is therefore best understood as a text-to-motion framework in which dataset design, tokenization strategy, and masked parallel decoding are tightly co-optimized. Its principal technical claim is that a shared-codebook, multi-scale tokenization scheme allows a single masked transformer to exploit token capacity more effectively under expressive text supervision, particularly on the SnapMoGen benchmark where long, detailed prompts are central to the task formulation.