---
title: 'MotionMillion: Text-Annotated Motion Corpus'
url: https://www.emergentmind.com/topics/motionmillion
type: topic
---

# MotionMillion: Text-Annotated Motion Corpus

MotionMillion is a million-scale, text-annotated human motion corpus and benchmarking suite that has emerged as a primary resource for large-scale generative modeling, zero-shot generalization, and benchmarking of text-to-motion models. As the field shifts towards generalist motion models inspired by advances in large language models, MotionMillion defines a new standard for data diversity, annotation quality, and evaluation rigor in human motion generation research [2507.07095].

## 1. Dataset Construction and Characteristics

MotionMillion aggregates curated data from eleven major preexisting motion datasets—including MotionX, InterHuman, Inter-X, BABEL, Fitness, PhantomDance, GDance, FineDance, HI4D, TRUMANS, HumanSC3D—and further expands coverage by mining web videos at unprecedented scale [2507.07095]. The curation pipeline consists of six automated stages:

- **Shot Segmentation:** Raw videos are split into ≤200-frame shots using PySceneDetect, with sharpness filtering via Laplacian operators.
- **Human Detection & Tracking:** Grounding DINO (conf > 0.85) followed by SAM2 propagates masks; per-frame IoU checks and confidence filtering handle occlusions.
- **Bounding Box & Transition Filtering:** Sudden bounding-box jumps are detected by center distance thresholds; cuts are introduced at scene changes.
- **SMPL Motion Estimation:** GVHMR recovers SMPL parameters \(\{t, R, \theta, \beta\}\) in the gravity-view frame.
- **Motion Filtering:** Outliers are removed by computing orientation differences \(\Delta\theta=\mathrm{Transform}(R_i R_{i-1}^{-1})\) and “jerk” (\(\dddot{J}_i\)), followed by Isolation Forest filtering.
- **Text Annotation:** GPT-4o generates detailed, multi-aspect English captions per segment; each caption is paraphrased 20 times by LLaMA 3.1-8B to maximize text diversity.

The resulting corpus surpasses 2 million sequences, each at least 1 s in duration (30 fps; >2,000 hours), all with rich semantic coverage spanning daily life, sports, dance, combat, multi-person scenarios, and complex compositional instructions. All clips are aligned to full-body, SMPL-format skeletons (without hands/facial detail), with pose records as
\[
x^i = \{ \dot{r}^x, \dot{r}^z, \dot{r}^a, p^i, v^i, r^i \}
\]
where \(\dot{r}^{x,z}\) are root velocities, \(\dot{r}^a\) is 6D angular velocity, \(p^i\)/\(v^i\) are per-joint positions/velocities, and \(r^i\) are 6D joint rotations relative to root, losslessly convertible to SMPL/BVH [2507.07095].

Quality assurance includes frame-level outlier elimination, visual sharpness gating, and human-in-the-loop bounding box verification.

## 2. Data Structure, Annotation, and Diversity

MotionMillion emphasizes hierarchical, high-fidelity annotation. Each sequence is paired with:

- **Semantic-rich captions**: Generated by GPT-4o, then paraphrased twenty ways for linguistic diversity.
- **Coverage**: Activities span walking, sitting, complex dances, inhuman or compositional actions (“zombie walk,” “run, then jump,” etc.).
- **Pose diversity and smoothness**: Principal component t-SNE visualizations reveal dense semantic clusters not present in smaller datasets; motion smoothness (low jerk) approaches the best prior corpora.

All annotations are currently English-only; captions contain semantics (action, style, emotion, subject traits, environment).

Compared to previous datasets (e.g., Motion-X: 81,000 clips; HumanML3D: 29,000), MotionMillion is at least 15× larger [2410.03311]. Its multimodal, large-scale structure supports a much broader class of generative modeling and retrieval tasks.

## 3. Evaluation Protocol: MotionMillion-Eval

MotionMillion-Eval is an evaluation suite tailored to measure zero-shot text-to-motion generation and compositional generalization [2507.07095]. Key features:

- **Test-only prompts**: 126 expert-written prompts spanning seven categories (Daily Life, Work, Arts/Dance, Communication, Combat, Sports, Non-Human Behaviors); all held out for testing.
- **Automatic and human metrics**:
  - *FID*: Fréchet Inception Distance between generated and ground-truth motion embeddings.
  - *R-Precision (R@k)*: Fraction of cases where ground truth text is among top-k nearest retrievals for generated motion/text embeddings.
  - *Human evaluation*: Text alignment (TA), motion smoothness (MS), and physical plausibility (PP), scored 1–4 by expert annotators.

Evaluation uses a dedicated motion-text retrieval model (trained on MotionMillion) and a blinded A/B testing protocol for human judgment. The suite stresses generalization, compositionality, and coverage beyond mere in-domain actions.

## 4. Model Architectures and Training on MotionMillion

MotionMillion has fostered the development of scalable text-to-motion architectures. Notable methodologies include:

- **Autoregressive models** (as in [2507.07095]):
    - *Motion tokenization*: Finite Scalar Quantization (FSQ) discretizes continuous motion latents, with Haar wavelet preprocessing to suppress high-frequency jitter.
    - *Backbone*: Text encoder (e.g., T5-XL), LLAMA-style Hybrid Attention Blocks (HAB), causal motion sequence modeling.
    - *Losses*: FSQ reconstruction; autoregressive cross-entropy token prediction.
    - *Scaling*: Model sizes from 1B to 7B; emergent zero-shot capabilities at 7B.
    - *Optimization*: AdamW, large batch (512), mixed precision, cosine learning rate schedule, gradient checkpointing.
- **Being-M0 and Motionbook** [2410.03311]:
    - Decoder-only causal Transformer backbone.
    - High-capacity, 2D lookup-free quantizer (2D-LFQ) for lossless, part-wise motion tokenization (token index by sign bits; codebook capacity up to \(2^{16}\)), avoiding lookup inefficiency and codebook collapse.
    - Hierarchical feature representation for whole-body and per-part motion granularity.
    - Empirically, 2D-LFQ exhibits monotonic codebook utilization and superior FID scaling.

A further class of methods replaces Euclidean representations with geometry-aware approaches:

- **Riemannian Motion Generation (RMG)** [2603.15016]:
    - Represents motion as a product manifold \(M = \mathbb R^3 \times (\mathbb S^3)^{J+1}\) (translations and joint/universal quaternions).
    - Uses Riemannian flow matching, tangent-space supervision, and manifold-preserving ODE integrators to learn geodesic dynamics.
    - Output post-processed into canonical joint formats for comparison.
    - Demonstrates state-of-the-art FID and R@1 on MotionMillion-Eval.

## 5. Empirical Performance and Scaling Insights

A summary of key empirical results is provided in the table below (all metrics from [2507.07095], [2410.03311], [2603.15016]):

| Model/Size         | FID (↓) | R@1 (↑) | Notes                                                      |
|--------------------|---------|---------|------------------------------------------------------------|
| ScaMo-3B           | 89.0    | 0.67    | Baseline autoregressive model                              |
| MotionMillion-3B   | 10.8    | 0.79    | Improved by scale/data/model                               |
| MotionMillion-7B   | 10.3    | 0.79    | Emergent zero-shot generalization at this scale            |
| Being-M0 (13B)     | 78.7    | 0.131   | GPT2/LLaMA-2 baseline with Motionbook tokenizer            |
| RMG (1.7B, guid=3) | 7.8     | 0.86    | Geometry-aware (T,R) manifold, classifier-free guidance    |

Performance scales with both data size and model size. For example, increasing from 0.02 M to 1 M samples improves R@1 by ~3× on MotionBase [2410.03311]. 2D-LFQ quantization maintains strong scaling and avoids expressivity collapse seen with standard VQ/RVQ.

Zero-shot and OOD generalization is directly measured in the UNSEEN-90K and 126-prompt MotionMillion-Eval scenarios, where scaled models and geometry-aware architectures retain fidelity and retrieval performance well above prior approaches [2507.07095][2603.15016].

## 6. Limitations and Future Directions

Several dataset and methodological limitations remain:

- Absence of detailed hand/facial keypoints; current annotations and models encode only coarse SMPL body motions.
- All text annotations are English-only.
- Coverage bias toward certain activities and demographics due to web-video sourcing.
- Autoregressive decoding limits real-time streaming; alternative architectures (diffusion, mixture-of-experts) may alleviate this.
- Current codebooks in tokenizers (e.g., FSQ, 2D-LFQ) are uniform and unadaptive.

Future work is projected to address: integration of SMPL-X/FLAME/Mano for hand/face; multilingual and cross-lingual annotation and generation; physically grounded generative priors; richer human-object and multi-person interaction models; and expansion of the evaluation benchmark to more varied and fine-grained compositional actions [2507.07095].

## 7. Significance and Impact

MotionMillion constitutes the most extensive and systematically curated resource for human motion modeling, supporting the development of robust, generalist, and zero-shot-capable text-to-motion models. It demonstrates, through systematic scaling studies, that both dataset size and model capacity are crucial for high-fidelity, generalizable motion generation. By hosting open benchmarks and public code, it standardizes evaluation and accelerates research in large-scale human motion synthesis, retrieval, and downstream embodied AI fields [2507.07095][2410.03311][2603.15016].

Source: https://www.emergentmind.com/topics/motionmillion