Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM

Abstract: Text-to-motion (T2M) generation has broad applications in character animation, virtual avatars, and human-robot interaction. Existing methods typically generate pose trajectories or motion tokens directly from language, forcing a single model to handle semantic interpretation, long-horizon structure, and low-level physical realization. This coupling makes them costly and often unreliable for long, compositional, or semantically dense prompts. We propose Text2BFM, the first framework that aligns natural language with pretrained Behavioral Foundation Models (BFMs) for T2M generation without relying on heavy end-to-end motion generators. Text2BFM operates in the latent policy space of a frozen BFM, using it as an executable motion prior. A text-aligned variational behavioral bottleneck compresses BFM policy-latent sequences into compact motion representations that are compatible with language and preserve long-horizon behavioral structure. Generation is performed in this compact behavioral manifold with a lightweight conditional generator, and the resulting latent encoded behaviors are decoded into policy latents that drive the pretrained frozen BFM. By decoupling semantic planning from motion execution, Text2BFM achieves efficient, robust T2M generation and strong performance on long, compositional textual descriptions.

• The paper introduces Text2BFM, a framework that decouples semantic planning from motion execution using a compressed behavioral bottleneck.
• It employs transformer-based flow matching and contrastive text alignment to enhance text-motion retrieval and compositional accuracy.
• Experimental results on HumanML3D and KIT-ML benchmarks show superior R-Precision and smoother phase transitions compared to prior methods.

Text2BFM: Decoupling Semantic Planning and Physically Grounded Motion Execution

Introduction and Motivation

Text-to-motion (T2M) generation serves as a critical capability in the domains of character animation, virtual avatars, and embodied agent control. Conventional approaches synthesize motion by directly mapping text to pose trajectories or discrete motion tokens using a monolithic model, with the generator tasked simultaneously with semantic parsing, high-level motion planning, and low-level physical realization. This entanglement frequently results in motions that are visually plausible yet suffer from artifacts such as foot sliding, poor contact handling, or limited temporal compositionality, especially when exposed to long-horizon or richly compositional instructions.

The paper "Plan, Don't Pose: Long Composite Motion Generation with Text-Aligned BFM" (2605.29906) addresses these limitations by reframing T2M as the problem of generating compact, semantically meaningful behavioral programs that are decoded to latent policy controls executable by a frozen Behavioral Foundation Model (BFM). This methodological disentanglement separates high-level semantic planning from low-level motor execution, yielding improved compositional fidelity and efficiency in generating long, hierarchical behaviors.

Figure 1: Overview of the Text2BFM method showing training and generation pipelines, emphasizing bottleneck compression, semantic alignment, and rollout via a frozen BFM.

Methodology

Text2BFM delineates motion synthesis into a multi-stage process, leveraging the latent policy space of a pretrained BFM as the interface for executable behaviors. The pipeline comprises:

1. Latent Extraction and Compression: For each reference motion trajectory, the Forward-Backward (FB) representations of future-conditioned states are averaged and projected as local policy latents $z_{1:T}$. Rather than generating these long-horizon sequences directly, a variational behavioral bottleneck (VBB) compresses $z_{1:T}$ into a more compact behavioral program $m_{1:T_m}$, where $T_m \ll T$.
2. Textual Alignment: The compact program space is regularized not only for faithful latent reconstruction but also for semantic alignment with the textual prompt. A contrastive loss is applied over frame-program-token and text-token embeddings, enforcing local temporal-textual correspondences.
3. Conditional Generation via Flow Matching: A lightweight conditional generator operates in the compact behavioral manifold, generating $m_{1:T_m}$ from text via transformer-based flow matching. This approach is more efficient and achieves superior generative quality compared with standard diffusion in the latent space.
4. Decoding and Execution: The synthesized behavioral program is decoded to policy latents, which are then rolled out through the frozen BFM in a physics-based simulator, enabling motion trajectories that are physically consistent and contact-rich.

Experimental Results

Standard Text-to-Motion Benchmarks

Text2BFM is evaluated on HumanML3D and KIT-ML, following standard T2M evaluation protocols (R-Precision, FID, MM-Distance, and MultiModality). The approach achieves:

• Best R-Precision Top-3: $0.876$ (HumanML3D), $0.901$ (KIT-ML)
• Lowest MultiModal Distance: $2.498$ (HumanML3D), $2.658$ (KIT-ML)

Quantitatively, Text2BFM establishes new SOTA in retrieval-based semantic fidelity and text-motion alignment, surpassing both diffusion- and token-based competitors. Notably, FID scores are not strictly optimal; this is attributed to the inductive biases in the BFM latent priors and training domain mismatch.

Figure 2: Text2BFM rollouts accurately capturing compositional, multi-stage instructions from the prompt.

Compositional Motion and Long-Horizon Consistency

The architecture demonstrates clear advantages in scenarios involving multi-phase, compositional prompts. Through hierarchical motion evaluation on $N \geq 3$ stage instructions, Text2BFM—especially in its “Compose” variant where subclauses are mapped to modular behavioral programs—exhibits marked improvements:

• Order Accuracy (N=3 stages): 0.671 (Text2BFM-Compose, versus 0.395 for naive generation)
• Transition Score: Substantially reduced, indicating motion with smoother phase transitions and fewer boundary artifacts

Figure 3: Method comparison (Kimodo vs. Text2BFM) for a 7-clause compositional prompt; Text2BFM-Compose precisely tracks each stage with clean transitions.

Ablation and Analysis

Ablation studies elucidate several insights:

• Compression in Latent Space: Aggressive temporal compression (factors of 4–8) retains latent fidelity with only minor degradation in Action KL, whereas higher ratios (16x) exhibit ballooning error, highlighting an optimal compression regime.
• Semantic Alignment: Enabling the contrastive motion-text alignment penalty significantly improves retrieval metrics and MM-Dist while preserving BFM latent reconstruction.
• Generator Backbone: Conditional flow matching outperforms latent diffusion in both sample quality and wall-clock generation, attaining similar or higher text-motion alignment in fewer steps.

Figure 4: Additional qualitative samples showcase the accuracy of instruction following for diverse, multi-stage physical behaviors.

Theoretical Implications and Limitations

The paper offers a rigorous theoretical exposition connecting future-conditioned BFM latent smoothness and the compressibility of latent trajectories, supported by total-variation bounds and closed-loop system stability. This justifies the use of temporally compact, semantically aligned control programs as an information bottleneck for policy-conditioned motion generation.

However, Text2BFM’s capacity is fundamentally circumscribed by the expressivity of the frozen BFM. Behaviors not captured within the BFM’s latent space or those requiring significant out-of-distribution generalization remain unattainable. Furthermore, rare or object-centric actions challenge current representations.

Impact and Future Directions

Practically, decoupling semantic planning from low-level control extends the applicability of T2M paradigms to complex, long-horizon, hierarchical tasks, with ramifications for animation, interactive avatars, simulation-based robotics, and language-driven embodied AI research. The approach provides a framework for safe behavior prototyping in simulation and scalable motion synthesis from free-form language descriptions.

Several promising directions remain. These include joint optimization of BFM and text generator, explicit modeling of object interaction, and multi-agent extensions. Ensuring robust OOD generalization and targeted diversity will further expand the limits of controllable human-centric generative models.

Conclusion

Text2BFM introduces a principled, modular framework for text-to-motion generation rooted in behavioral modeling and semantic alignment. By compressing policy-latent trajectories into text-discriminative behavioral programs, it achieves strong compositional generalization, smooth physical execution, and substantial improvements in retrieval-based and semantic fidelity metrics on established benchmarks. The explicit separation between semantic and physical levels presents a meaningful step towards more robust, extensible, and controllable motion generation systems.