MotionMillion: Text-Annotated Motion Corpus

• MotionMillion is a million-scale, text-annotated human motion dataset and benchmarking suite that standardizes evaluation for generative and zero-shot text-to-motion models.
• It employs a six-stage automated curation pipeline—integrating shot segmentation, human detection, SMPL estimation, and GPT-4o text annotation—to ensure high data diversity and annotation quality.
• Empirical results demonstrate state-of-the-art performance with significant improvements in FID and R-Precision, driving scalable and robust text-to-motion synthesis.

MotionMillion is a million-scale, text-annotated human motion corpus and benchmarking suite that has emerged as a primary resource for large-scale generative modeling, zero-shot generalization, and benchmarking of text-to-motion models. As the field shifts towards generalist motion models inspired by advances in LLMs, MotionMillion defines a new standard for data diversity, annotation quality, and evaluation rigor in human motion generation research (Fan et al., 9 Jul 2025).

1. Dataset Construction and Characteristics

MotionMillion aggregates curated data from eleven major preexisting motion datasets—including MotionX, InterHuman, Inter-X, BABEL, Fitness, PhantomDance, GDance, FineDance, HI4D, TRUMANS, HumanSC3D—and further expands coverage by mining web videos at unprecedented scale (Fan et al., 9 Jul 2025). The curation pipeline consists of six automated stages:

• Shot Segmentation: Raw videos are split into ≤200-frame shots using PySceneDetect, with sharpness filtering via Laplacian operators.
• Human Detection & Tracking: Grounding DINO (conf > 0.85) followed by SAM2 propagates masks; per-frame IoU checks and confidence filtering handle occlusions.
• Bounding Box & Transition Filtering: Sudden bounding-box jumps are detected by center distance thresholds; cuts are introduced at scene changes.
• SMPL Motion Estimation: GVHMR recovers SMPL parameters $\{t, R, \theta, \beta\}$ in the gravity-view frame.
• Motion Filtering: Outliers are removed by computing orientation differences $\Delta\theta=\mathrm{Transform}(R_i R_{i-1}^{-1})$ and “jerk” ($\dddot{J}_i$), followed by Isolation Forest filtering.
• Text Annotation: GPT-4o generates detailed, multi-aspect English captions per segment; each caption is paraphrased 20 times by LLaMA 3.1-8B to maximize text diversity.

The resulting corpus surpasses 2 million sequences, each at least 1 s in duration (30 fps; >2,000 hours), all with rich semantic coverage spanning daily life, sports, dance, combat, multi-person scenarios, and complex compositional instructions. All clips are aligned to full-body, SMPL-format skeletons (without hands/facial detail), with pose records as

$$x^i = \{ \dot{r}^x, \dot{r}^z, \dot{r}^a, p^i, v^i, r^i \}$$

where $\dot{r}^{x,z}$ are root velocities, $\dot{r}^a$ is 6D angular velocity, $p^i$/$v^i$ are per-joint positions/velocities, and $r^i$ are 6D joint rotations relative to root, losslessly convertible to SMPL/BVH (Fan et al., 9 Jul 2025).

Quality assurance includes frame-level outlier elimination, visual sharpness gating, and human-in-the-loop bounding box verification.

2. Data Structure, Annotation, and Diversity

MotionMillion emphasizes hierarchical, high-fidelity annotation. Each sequence is paired with:

• Semantic-rich captions: Generated by GPT-4o, then paraphrased twenty ways for linguistic diversity.
• Coverage: Activities span walking, sitting, complex dances, inhuman or compositional actions (“zombie walk,” “run, then jump,” etc.).
• Pose diversity and smoothness: Principal component t-SNE visualizations reveal dense semantic clusters not present in smaller datasets; motion smoothness (low jerk) approaches the best prior corpora.

All annotations are currently English-only; captions contain semantics (action, style, emotion, subject traits, environment).

Compared to previous datasets (e.g., Motion-X: 81,000 clips; HumanML3D: 29,000), MotionMillion is at least 15× larger (Wang et al., 2024). Its multimodal, large-scale structure supports a much broader class of generative modeling and retrieval tasks.

3. Evaluation Protocol: MotionMillion-Eval

MotionMillion-Eval is an evaluation suite tailored to measure zero-shot text-to-motion generation and compositional generalization (Fan et al., 9 Jul 2025). Key features:

• Test-only prompts: 126 expert-written prompts spanning seven categories (Daily Life, Work, Arts/Dance, Communication, Combat, Sports, Non-Human Behaviors); all held out for testing.
• Automatic and human metrics:
  • FID: Fréchet Inception Distance between generated and ground-truth motion embeddings.
  • R-Precision (R@k): Fraction of cases where ground truth text is among top-k nearest retrievals for generated motion/text embeddings.
  • Human evaluation: Text alignment (TA), motion smoothness (MS), and physical plausibility (PP), scored 1–4 by expert annotators.

Evaluation uses a dedicated motion-text retrieval model (trained on MotionMillion) and a blinded A/B testing protocol for human judgment. The suite stresses generalization, compositionality, and coverage beyond mere in-domain actions.

4. Model Architectures and Training on MotionMillion

MotionMillion has fostered the development of scalable text-to-motion architectures. Notable methodologies include:

• Autoregressive models (as in (Fan et al., 9 Jul 2025)):
  • Motion tokenization: Finite Scalar Quantization (FSQ) discretizes continuous motion latents, with Haar wavelet preprocessing to suppress high-frequency jitter.
  • Backbone: Text encoder (e.g., T5-XL), LLAMA-style Hybrid Attention Blocks (HAB), causal motion sequence modeling.
  • Losses: FSQ reconstruction; autoregressive cross-entropy token prediction.
  • Scaling: Model sizes from 1B to 7B; emergent zero-shot capabilities at 7B.
  • Optimization: AdamW, large batch (512), mixed precision, cosine learning rate schedule, gradient checkpointing.
• Being-M0 and Motionbook (Wang et al., 2024):
  • Decoder-only causal Transformer backbone.
  • High-capacity, 2D lookup-free quantizer (2D-LFQ) for lossless, part-wise motion tokenization (token index by sign bits; codebook capacity up to $2^{16}$), avoiding lookup inefficiency and codebook collapse.
  • Hierarchical feature representation for whole-body and per-part motion granularity.
  • Empirically, 2D-LFQ exhibits monotonic codebook utilization and superior FID scaling.

A further class of methods replaces Euclidean representations with geometry-aware approaches:

• Riemannian Motion Generation (RMG) (Miao et al., 16 Mar 2026):
  • Represents motion as a product manifold $\Delta\theta=\mathrm{Transform}(R_i R_{i-1}^{-1})$0 (translations and joint/universal quaternions).
  • Uses Riemannian flow matching, tangent-space supervision, and manifold-preserving ODE integrators to learn geodesic dynamics.
  • Output post-processed into canonical joint formats for comparison.
  • Demonstrates state-of-the-art FID and R@1 on MotionMillion-Eval.

5. Empirical Performance and Scaling Insights

A summary of key empirical results is provided in the table below (all metrics from (Fan et al., 9 Jul 2025, Wang et al., 2024, Miao et al., 16 Mar 2026)):

Model/Size	FID (↓)	R@1 (↑)	Notes
ScaMo-3B	89.0	0.67	Baseline autoregressive model
MotionMillion-3B	10.8	0.79	Improved by scale/data/model
MotionMillion-7B	10.3	0.79	Emergent zero-shot generalization at this scale
Being-M0 (13B)	78.7	0.131	GPT2/LLaMA-2 baseline with Motionbook tokenizer
RMG (1.7B, guid=3)	7.8	0.86	Geometry-aware (T,R) manifold, classifier-free guidance

Performance scales with both data size and model size. For example, increasing from 0.02 M to 1 M samples improves R@1 by ~3× on MotionBase (Wang et al., 2024). 2D-LFQ quantization maintains strong scaling and avoids expressivity collapse seen with standard VQ/RVQ.

Zero-shot and OOD generalization is directly measured in the UNSEEN-90K and 126-prompt MotionMillion-Eval scenarios, where scaled models and geometry-aware architectures retain fidelity and retrieval performance well above prior approaches (Fan et al., 9 Jul 2025, Miao et al., 16 Mar 2026).

6. Limitations and Future Directions

Several dataset and methodological limitations remain:

• Absence of detailed hand/facial keypoints; current annotations and models encode only coarse SMPL body motions.
• All text annotations are English-only.
• Coverage bias toward certain activities and demographics due to web-video sourcing.
• Autoregressive decoding limits real-time streaming; alternative architectures (diffusion, mixture-of-experts) may alleviate this.
• Current codebooks in tokenizers (e.g., FSQ, 2D-LFQ) are uniform and unadaptive.

Future work is projected to address: integration of SMPL-X/FLAME/Mano for hand/face; multilingual and cross-lingual annotation and generation; physically grounded generative priors; richer human-object and multi-person interaction models; and expansion of the evaluation benchmark to more varied and fine-grained compositional actions (Fan et al., 9 Jul 2025).

7. Significance and Impact

MotionMillion constitutes the most extensive and systematically curated resource for human motion modeling, supporting the development of robust, generalist, and zero-shot-capable text-to-motion models. It demonstrates, through systematic scaling studies, that both dataset size and model capacity are crucial for high-fidelity, generalizable motion generation. By hosting open benchmarks and public code, it standardizes evaluation and accelerates research in large-scale human motion synthesis, retrieval, and downstream embodied AI fields (Fan et al., 9 Jul 2025, Wang et al., 2024, Miao et al., 16 Mar 2026).