Bidirectional Momentum Text-Motion Model

• Bidirectional momentum text-motion models are generative frameworks that unify text-to-motion and motion-to-text mapping using momentum-inspired propagation for robust cross-modal synthesis.
• They integrate autoregressive, diffusion, and masked modeling techniques to achieve flexible motion generation, editing, and completion across diverse applications.
• Empirical evaluations on benchmarks like HumanML3D demonstrate improved metrics, including lower FID and higher R-Precision, thanks to cascaded decoding and joint inference mechanisms.

A Bidirectional Momentum Text-Motion Model is a class of generative models designed to unify text-to-motion and motion-to-text mapping, with a particular focus on jointly modeling both modalities under a single or tightly coupled framework. These models are characterized by bidirectional generation capabilities, momentum-like information propagation, and support for powerful inference and editability paradigms. Modern approaches employ either autoregressive masked modeling, discrete or continuous diffusion, or a hybrid thereof. Notable representatives include BAMM, MoTe, DiMo, and PackDiT, each capturing nuanced text-motion relationships and enabling flexibility in generation, completion, and editing tasks (Pinyoanuntapong et al., 2024, Wu et al., 2024, Jiang et al., 27 Jan 2025, Zhang et al., 4 Feb 2026).

1. Bidirectional Text-Motion Modeling: Motivation and Scope

The dominant approach in text-to-motion generation historically relied on unidirectional denoising diffusion models or autoregressive language-modeling-inspired frameworks. Most of these systems are incapable of reciprocal mapping (i.e., motion-to-text), are limited to one generation direction, or require fixed-length motion sequences a priori. Bidirectional text-motion models address these limitations by:

• Learning both $p(\text{motion}|\text{text})$ and $p(\text{text}|\text{motion})$ (plus joint distributions) within one unified parameterization (Wu et al., 2024, Zhang et al., 4 Feb 2026, Jiang et al., 27 Jan 2025).
• Supporting application domains including text-conditioned motion synthesis, automatic motion captioning, text-controlled editing/inpainting, unconditional joint sampling, and motion completion.
• Allowing for "momentum-like" propagation—either algorithmically (e.g., temporally smoothed updates in PackDiT and momentum-biased ODE integrators) or architecturally (multi-pass cascades in BAMM, iterative refinement in DiMo).

A key driver is pragmatic flexibility: bidirectional models enable robust toolkits for animation, semantic understanding, and content generation by accommodating arbitrary input/output directionality and sequence length (Pinyoanuntapong et al., 2024, Jiang et al., 27 Jan 2025).

2. Architectural Building Blocks and Tokenization

State-of-the-art approaches share a common pipeline consisting of:

• Motion Tokenization: Transforming high-dimensional motion sequences into discrete tokens through vector quantization—using VQ-VAE (Pinyoanuntapong et al., 2024), multi-stage Residual Vector Quantization (RVQ) (Zhang et al., 4 Feb 2026), or transformer-based tokenizers (Wu et al., 2024, Jiang et al., 27 Jan 2025). Quantization fundamentally reduces reconstruction error, allows discrete sequence modeling, and forms the core of transformer input.
• Text Embedding: Leveraging frozen or finetuned large-scale text encoders (CLIP, BERT, GPT-2) to map textual prompts or captions into dense, transformation-compatible embeddings. These embeddings can be processed in parallel or integrated via cross-modal blocks in the model backbone (Wu et al., 2024, Jiang et al., 27 Jan 2025).

Table: Representative Model Tokenization Strategies

Model	Motion Tokenizer	Text Encoder
BAMM	VQ-VAE	CLIP text embedding
DiMo	RVQ (6 layers, 1024)	BERT
MoTe	Transformer-VAE	CLIP + projection, GPT-2
PackDiT	Linear + BERT	BERT + projection, GPT-2

Motion tokenization is fundamental, enabling discrete modeling and architectural compatibility between motion and language streams.

3. Core Modeling Paradigms: Autoregressive, Diffusion, and Masked Approaches

Bidirectional momentum text-motion models bifurcate along autoregressive and diffusion-masked modeling methodologies, with hybridizations observed in recent advances:

• Masked Autoregressive (BAMM): Utilizes a hybrid self-attention Transformer that alternates between unidirectional (causal for AR sampling) and bidirectional (random masking for denoising) attention masks during training (Pinyoanuntapong et al., 2024). The model unrolls autoregressively during inference (predicting until an [END] token), followed by a bidirectional refinement stage for quality and editability enhancement.
• Continuous/Discrete Diffusion (MoTe, DiMo, PackDiT): These frameworks apply iterative forward noising (diffusion) and learned denoising processes. MoTe realizes this on continuous latent embeddings, while DiMo employs discrete masked token refinement, modeling both text and motion as sequences subject to progressive denoising (Wu et al., 2024, Zhang et al., 4 Feb 2026). PackDiT utilizes twin diffusion transformers with "mutual prompting" (cross-attention) blocks for tight cross-modal integration (Jiang et al., 27 Jan 2025).
• Momentum-Inspired Extensions (PackDiT, BAMM): Explicitly proposed in PackDiT, where exponential moving averages of hidden representations or score functions are incorporated within the diffusion process to facilitate smoother, information-preserving updates across modalities and timesteps (Jiang et al., 27 Jan 2025). BAMM cascades decoding across AR and bidirectional passes, functionally mimicking a temporal update with momentum.

These modeling choices implicitly encode dependencies forward and backward in time and across modalities, unifying denoising (for editability) with AR (for adaptive endpoint prediction and causal coherence).

4. Hybrid Decoding, Inference Workflow, and Editing Capabilities

Advanced bidirectional momentum models support powerful and flexible inference routines:

• Cascaded Decoding (BAMM): Inference consists of an initial AR pass—sampling one token at a time until [END]—followed by a bidirectional refinement wherein low-confidence or random tokens are resampled considering full bidirectional context. This duality enables both high-quality sampling and versatile editing, including inpainting, outpainting, and prefix/suffix completion (Pinyoanuntapong et al., 2024).
• Parallel Progressive Denoising (DiMo): Iterative masked-token updates, where each round refines a user-defined fraction of masked tokens, allow explicit trading of inference speed for output quality. This enables applications including text-free motion completion and motion caption correction (Zhang et al., 4 Feb 2026).
• Mutual Prompting and Joint Sampling (PackDiT, MoTe): At inference, deterministic or stochastic denoising allows conditioning on arbitrary cross-modal inputs (text, motion, or both). Zero-shot editing is achieved by masking desired temporal spans and leveraging the model's denoising capability (Jiang et al., 27 Jan 2025, Wu et al., 2024).

Momentum-style architectural mechanisms—such as exponential moving averages or second-order ODE updates in the diffusion solver—are proposed as extensions and plausibly improve smoothness and alignment when jointly synthesizing or editing both modalities (Jiang et al., 27 Jan 2025).

5. Training Objectives, Optimization, and Guidance Mechanisms

• Hybrid Masking Loss (BAMM): BAMM samples between causal (AR) and bidirectional (random masking) attention masks during training, and computes a weighted negative log-likelihood across both (Pinyoanuntapong et al., 2024).
• Joint Denoising Loss (MoTe, DiMo, PackDiT): Losses are typically denoising mean squared error objectives, defined over both conditional and joint settings, covering all mode combinations (text→motion, motion→text, joint, unconditional) (Wu et al., 2024, Jiang et al., 27 Jan 2025, Zhang et al., 4 Feb 2026).
• Group Relative Policy Optimization (GRPO) for Alignment (DiMo): After maximum likelihood training, semantic alignment is further improved by groupwise relative policy optimization, leveraging task-specific CLIP and captioning rewards (Zhang et al., 4 Feb 2026).
• Classifier-Free Guidance (All): Employed during both training and inference to amplify conditional fidelity. Model predictions are blended between text/motion-conditioned and unconditional logits using a tunable scale parameter, controlling adherence to the conditioning signal (Pinyoanuntapong et al., 2024, Wu et al., 2024, Zhang et al., 4 Feb 2026).

Optimization with AdamW, scheduled learning rates, and large batch sizes are standard; architectural variants include transformer depth, fusion block style, and codebook size.

6. Quantitative Results, Trade-Offs, and Model Comparisons

Models are benchmarked on HumanML3D and KIT-ML with the following key metrics: R-Precision (retrieval), FID (Fréchet Inception Distance), MMDist, BLEU, CIDEr, ROUGE, and BERTScore (Pinyoanuntapong et al., 2024, Wu et al., 2024, Jiang et al., 27 Jan 2025, Zhang et al., 4 Feb 2026). Representative quantitative results include:

Model	HumanML3D R@1	FID	BLEU@4	CIDEr
BAMM	0.525	0.055	—	—
PackDiT-Small	0.510	0.106	11.82	25.5
MoTe	0.548	0.075	11.76	32.3
DiMo (with GRPO)	0.528	0.047	22.7	58.1

Key empirical observations:

• DiMo achieves the lowest FID among unified models (0.047 on HumanML3D), with strong BLEU and CIDEr in motion captioning after GRPO (Zhang et al., 4 Feb 2026).
• PackDiT sets the HumanML3D text-to-motion FID record (0.106), exceeding all prior diffusion and autoregressive baselines (Jiang et al., 27 Jan 2025).
• BAMM demonstrates superior text alignment, editability, and length prediction, benefiting from its cascaded decoding (Pinyoanuntapong et al., 2024).
• MoTe and DiMo support reciprocal inference within the same network, with MoTe yielding best-in-class R-Precision and MMDist (Wu et al., 2024, Zhang et al., 4 Feb 2026).

Inference steps can be tuned to trade off latency with generation fidelity, as shown in DiMo.

7. Applications, Extensions, and Limitations

Bidirectional momentum text-motion models enable applications including:

• High-fidelity text-to-motion synthesis with variable or implicit motion length
• Automatic motion captioning (motion-to-text)
• Inpainting, outpainting, and temporal editing (zero-shot)
• Motion prediction and in-betweening
• Joint generation of novel text-motion pairs

Extensions proposed in the literature include momentum propagation in the solver (e.g., second-order ODE steps) and dynamic noise schedules responsive to cross-modal similarity (Jiang et al., 27 Jan 2025). Failure cases include degraded control in highly fine-grained motion synthesis, out-of-domain generalization challenges, and occasional token repetition in language generation (Wu et al., 2024).

These unified architectures close the gap between one-way and reciprocal modeling of complex spatiotemporal semantics, providing a foundation for future multimodal motion-language generation systems.