HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation

Abstract: We present HY-Motion 1.0, a series of state-of-the-art, large-scale, motion generation models capable of generating 3D human motions from textual descriptions. HY-Motion 1.0 represents the first successful attempt to scale up Diffusion Transformer (DiT)-based flow matching models to the billion-parameter scale within the motion generation domain, delivering instruction-following capabilities that significantly outperform current open-source benchmarks. Uniquely, we introduce a comprehensive, full-stage training paradigm -- including large-scale pretraining on over 3,000 hours of motion data, high-quality fine-tuning on 400 hours of curated data, and reinforcement learning from both human feedback and reward models -- to ensure precise alignment with the text instruction and high motion quality. This framework is supported by our meticulous data processing pipeline, which performs rigorous motion cleaning and captioning. Consequently, our model achieves the most extensive coverage, spanning over 200 motion categories across 6 major classes. We release HY-Motion 1.0 to the open-source community to foster future research and accelerate the transition of 3D human motion generation models towards commercial maturity.

• The paper introduces a scalable DiT-based flow matching model that leverages billion-parameter architecture to enhance semantic alignment and physical realism in text-to-motion generation.
• It implements a three-phase training paradigm—including large-scale pretraining, high-quality fine-tuning, and reinforcement learning—to refine motion fidelity and eliminate artifacts.
• The integrated methodology combines joint multimodal attention and an extensive curated dataset, setting a new open-source benchmark for commercial-grade 3D motion synthesis.

HY-Motion 1.0: Scalable Flow Matching for High-Fidelity Text-to-3D Human Motion Generation

Introduction and Motivation

HY-Motion 1.0 introduces a scalable and instruction-aligned framework for text-to-motion generation, employing billion-parameter Diffusion Transformer (DiT) models adapted for flow matching. Motivated by the limitations of existing approaches—either small-scale diffusion models with weak semantics or LLM-tokenizer hybrids with compromised motion quality due to quantization—HY-Motion 1.0 leverages the scaling law of diffusion architectures and a rigorous multi-stage training paradigm. The model is the first in its domain to scale DiT-based flow matching to the billion-parameter regime, delivering substantial improvements in both semantic alignment and physical realism over prior works.

Figure 1: Comparison of HY-Motion 1.0 against prior text-to-motion models and visual samples of generated motions demonstrating retargeting capabilities.

Data Curation and Annotation Pipeline

The success of HY-Motion 1.0 is predicated on an extensive, multi-source data corpus. The pipeline integrates 12 million in-the-wild video clips, motion capture records, and high-quality 3D animation assets. All motions are converted to a unified SMPL-H skeleton, followed by aggressive filtering and canonicalization (e.g., abnormal joint velocities, artifacts, and retargeted skeletal mapping). The result is a 3,000-hour motion dataset, with 400 hours receiving manual, high-fidelity curation.

Motion captioning combines automated VLM annotation with manual corrections, followed by LLM-driven diversification to ensure semantic breadth and accurate text-motion alignment. The processed dataset spans six coarse motion classes and over 200 fine-grained subclasses, enabling unmatched coverage.

Figure 2: Overview of the data processing pipeline integrating cleaning, captioning, and manual refinement for motion-text pair construction.

Figure 3: Hierarchical taxonomy of motion categories across the full dataset.

HY-Motion 1.0 Architecture

The core model is a hybrid DiT, integrating both dual-stream and single-stream Transformer blocks for joint motion-text distribution modeling. Motion and text tokens interact via specialized joint attention, merging later for deep multimodal fusion. Text conditioning utilizes both Qwen3-8B (token-wise semantic guidance with bidirectional refinement for non-autoregressive generation) and CLIP-L (global semantic embedding via AdaLN modulation).

Attention is managed via asymmetric multimodal masking (motion tokens freely attend to text, text tokens are masked from motion latents) and a localized window (121-frame) temporal mask to enforce kinematic locality. RoPE encodes global positional relations across concatenated motion and text embeddings. For generative training, a flow matching objective interpolates between Gaussian noise and motion data, with inference performed via continuous ODE integration.

Figure 4: Pipeline overview highlighting prompt parsing, semantic enrichment, duration prediction, and 3D motion synthesis.

Figure 5: Model architecture diagram for HY-Motion DiT, featuring multimodal attention mechanisms and hierarchical text guidance.

A parallel LLM module predicts motion duration and rewrites prompts, fine-tuned both in supervised and RL setups (Group Relative Policy Optimization, GRPO), enhancing semantic robustness and temporal plausibility.

Multi-Stage Training Paradigm

HY-Motion 1.0 adopts a three-phase curriculum:

• Large-Scale Pretraining: Training on the entire 3,000-hour corpus achieves robust semantic priors and kinematic diversity, though with residual artifacts due to noisy data.
• High-Quality Fine-Tuning: A targeted 400-hour curation sharpens fidelity, eliminates artifacts (e.g., foot sliding), and enhances anatomical and semantic specificity.
• Reinforcement Learning Alignment: Final RL stages, employing human preference-driven DPO and Flow-GRPO, optimize semantic alignment and constraint satisfaction beyond supervised likelihood—a necessary step to close the perceptual gap between model outputs and human expectations.

Evaluation and Results

HY-Motion 1.0 surpasses four leading benchmarks (DART, LoM, GoToZero, MoMask) in both human and automatic evaluations. On a diverse, 2000-prompt test set, HY-Motion achieves superior scores in instruction-following ($\text{Avg}=3.24$ vs. $2.17\text{--}2.31$) and motion quality ($\text{Avg}=3.43$ vs. $2.79\text{--}3.11$), with an SSAE alignment of $78.6\%$ (vs. $42.7\text{--}58.0\%$ for baselines).

Scaling experiments confirm monotonic gains in instruction alignment up to a billion parameters, with motion quality saturating above $\sim0.5$B parameters, supporting the need for large networks but also rigorous data quality management.

Figure 6: Qualitative comparisons illustrating visual improvements by HY-Motion 1.0 over prior models for representative prompts.

Implications and Future Directions

Practically, HY-Motion 1.0 sets a new open-source baseline suitable for commercial-grade 3D motion synthesis. The separation of concerns between broad semantic learning (data scale) and precise fidelity (data quality) shapes future scaling strategies for motion generation. The model's RL-based alignment strategy may inform broader multimodal generative modeling beyond the motion domain.

Key outstanding challenges include handling highly complex or nuanced instructions (limited by current captioning accuracy and semantic annotation) and advancing human-object interaction (HOI), as existing datasets focus on body kinematics without explicit object context. Future research should target dataset expansion with richer interaction modalities and improved semantic captioning pipelines.

Conclusion

HY-Motion 1.0 substantiates the scaling law for text-to-motion generation, demonstrating that billion-parameter DiT-based flow matching models yield significant advances in instruction adherence and motion realism. By coupling extensive data curation and staged optimization (pretraining, fine-tuning, RL), the work delivers an open-source solution with strong empirical performance and a blueprint for future large-scale, instruction-following 3D Motion AI research.

---