MoGIC: Motion Generation with Intent & Visual Context

• MoGIC is a multimodal framework for 3D human motion synthesis that enhances text-driven approaches by incorporating explicit intention modeling and visual grounding.
• It employs modality-specific encoders and a shared Conditional Masked Transformer to align language, vision, and motion data for robust causal reasoning.
• The adaptive mixture-of-attention mechanism and dual output heads achieve significant performance gains in motion fidelity and intention prediction across multiple benchmarks.

Searching arXiv for the exact MoGIC paper and nearby naming collisions to ground the article. arXiv search query: (Shi et al., 3 Oct 2025) MoGIC motion generation intention visual context MoGIC, short for Motion Generation via Intention understanding and visual Context, is a multimodal human motion generation and understanding framework that integrates intention modeling and visual priors into motion synthesis. It is designed to address two limitations identified in prior text-driven motion generation: the failure to capture the causal logic and human intentions that drive behavior, and the lack of visual grounding needed for fine-grained spatiotemporal specification. The framework jointly optimizes multimodal-conditioned motion generation and intention prediction, introduces a mixture-of-attention mechanism with adaptive scope for local alignment between conditional tokens and motion subsequences, and is paired with Mo440H, a 440-hour benchmark assembled from 21 high-quality motion datasets (Shi et al., 3 Oct 2025).

1. Problem formulation and scope

MoGIC is positioned against text-driven motion generation methods that treat synthesis primarily as a bidirectional mapping between language and motion. In that formulation, motion is modeled as a sequence aligned with text, but the underlying why of motion remains weakly represented. MoGIC instead treats intention as a first-class target and incorporates visual context so that motion generation is not conditioned on language alone (Shi et al., 3 Oct 2025).

The framework accepts language, vision, and motion as inputs, in any subset, and produces 3D human motion sequences together with natural-language intention descriptions. The task repertoire explicitly includes text-to-motion (L2M), vision+text-to-motion (VL2M), vision-only motion generation (V2M), motion completion/in-betweening (M2M), intention prediction, and motion captioning (M2T) (Shi et al., 3 Oct 2025).

A central claim of the method is that language alone is often underdetermined for motion control. Prompts such as a person walking, turning, or picking up an object may specify high-level semantics while omitting turning angle, object placement, or trajectory details. MoGIC addresses this by using visual priors and by forcing the shared latent representation to support both motion generation and intention prediction, thereby encoding higher-level causal structure rather than only surface correspondence between text and kinematics (Shi et al., 3 Oct 2025).

2. Architectural organization

MoGIC consists of modality-specific encoders, a shared Conditional Masked Transformer (CMT) backbone, and two disentangled output heads. The modality encoders are a temporal convolutional motion encoder-decoder, a frozen CLIP text encoder, and a vision encoder that operates on low-frame-rate image sequences and uses attention-based global pooling (Shi et al., 3 Oct 2025).

The CMT receives masked motion latent tokens together with multimodal condition tokens. It combines three mechanisms. First, it applies self-attention over motion tokens. Second, it performs semantic-level modulation through adaptive LayerNorm conditioned on global text and vision context. Third, it applies a fine-grained cross-attention mechanism over token-level text and vision representations (Shi et al., 3 Oct 2025).

The global context is defined as

$z^g = z_t^g + z_v^g,$

where $z_t^g$ and $z_v^g$ are the global text and vision embeddings. These are projected to adaptive LayerNorm parameters,

$$(\alpha_c, \beta_c, \gamma_c) = W_{\text{ada}}(z^g),$$

which modulate normalized motion tokens: $$z_m \gets z_m + \gamma_c \odot h(\alpha_c \odot \bar{z}_m + \beta_c).$$ This provides semantic conditioning at the block level rather than only through token concatenation (Shi et al., 3 Oct 2025).

The two output heads are deliberately separated. The Motion Generation Head (MGH) is a continuous-time interpolant model in latent space. The Intention Prediction Head (IPH) is a T5-style autoregressive decoder that generates intention text from the shared CMT representation. Missing modalities are handled by replacing absent text or vision with a learnable placeholder embedding, allowing a single backbone to cover L2M, VL2M, V2M, M2M, intention prediction, and captioning without changing the architecture (Shi et al., 3 Oct 2025).

3. Intention modeling and visual priors

In MoGIC, intention is represented as a textual description of the underlying goal or semantic summary of motion, rather than as a full caption of visible kinematics. During intention prediction, the framework hides the latter 50% of motion tokens and predicts a text description of the latent goal behind the incomplete sequence. The intention objective is the autoregressive cross-entropy

$$\mathcal{L}_{\text{intent}} = \mathbb{E}_{(x, y)} \Bigl[ - \sum_{i=1}^T \log P(y_i \mid y_{<i}, z) \Bigr].$$

Because both IPH and MGH are conditioned on the same shared representation $z$, the joint objective forces that representation to encode goal structure useful for both explanation and generation (Shi et al., 3 Oct 2025).

Visual priors are supplied through low-frame-rate image sequences (~1 fps). These may be true RGB frames or rendered SMPL meshes when RGB is absent. The design choice is notable because the method does not require dense pose tracking from video. Instead, the visual input provides weak but informative cues about scene layout, human-object relations, and coarse trajectory constraints (Shi et al., 3 Oct 2025).

Frame embeddings $z_v^p = f_v(v^p)$ are aggregated by attention into a global vision vector,

$z_v^g = \text{Attn}(q_v, z_v, z_v),$

while frame-level tokens are retained for token-level fusion. This creates a two-scale conditioning path: global semantic modulation through $z_v^g$, and local alignment through frame tokens $z_t^g$0. The paper argues that this resolves ambiguities that remain under text-only conditioning, especially in scenarios where object location or body orientation cannot be inferred reliably from language alone (Shi et al., 3 Oct 2025).

4. Mixture-of-attention with adaptive scope

The most distinctive conditioning mechanism in MoGIC is a mixture-of-attention module with adaptive scope. It operates over motion queries $z_t^g$1 and multimodal condition tokens

$z_t^g$2

For expert $z_t^g$3, the projected queries, keys, and values are

$z_t^g$4

Attention weights are computed in the usual scaled-dot-product form, then sparsified by an adaptive top-$z_t^g$5 rule (Shi et al., 3 Oct 2025).

For each motion query, MoGIC sorts the attention weights and chooses the smallest $z_t^g$6 whose cumulative mass exceeds a threshold $z_t^g$7, subject to expert-specific lower and upper bounds: $z_t^g$8 Weights outside the selected support are zeroed and renormalized, producing expert output

$z_t^g$9

This mechanism allows one expert to retain broad, global attention and another to focus on small local supports such as short text phrases or a few visual frames (Shi et al., 3 Oct 2025).

The default configuration uses two experts: one with $z_v^g$0 and $z_v^g$1, and one with $z_v^g$2 and $z_v^g$3. According to the reported ablations, removing cross-attention harms performance, increasing the number of experts improves FID, and two experts provide the best trade-off for R-precision and motion quality (Shi et al., 3 Oct 2025).

5. Training regime and the Mo440H benchmark

MoGIC is trained in two stages. The first stage is cross-modal generative pretraining over five tasks: L2M, VL2M, V2M, M2M, and intention prediction. The second stage is optional task-specific finetuning, particularly for L2M and M2T (Shi et al., 3 Oct 2025).

The motion generation loss is formulated in latent space using an interpolant-style objective. With clean latent $z_v^g$4, noisy latent

$z_v^g$5

and predicted velocity field $z_v^g$6, the motion loss is

$z_v^g$7

The overall objective is

$z_v^g$8

This explicit multi-task formulation is central to the claim that intention supervision improves the generative latent representation rather than acting as an auxiliary output alone (Shi et al., 3 Oct 2025).

The accompanying dataset, Mo440H, contains 440 hours of motion, approximately 50M frames, and is assembled from 21 high-quality motion datasets. The representation is standardized to a 22-joint skeleton, 30 fps, and a maximum sequence length of 10 s. The corpus includes about 210k text descriptions and about 140k visual sequences at 1 fps. Three subsets are defined: Mo440H-All for autoencoder and cross-modal generative training, Mo440H-ML for motion-language tasks, and Mo440H-MLV for motion-language-vision tasks (Shi et al., 3 Oct 2025).

Training details reported in the paper include Adam with learning rate $z_v^g$9, 2000 warm-up steps, 500 epochs on HumanML3D, and 10M iterations on Mo440H. The CMT uses 384 channels, with 1 layer on HumanML3D and 2 layers on Mo440H. The IPH is a 3-layer T5-style decoder, while the MGH is a 10-layer MLP with 1280 channels (Shi et al., 3 Oct 2025).

6. Empirical performance, interpretation, and naming context

MoGIC reports substantial gains on standard motion-generation benchmarks. On HumanML3D, the paper reports FID 0.070, compared with approximately 0.114–0.116 for strong prior baselines, and states that this corresponds to a 38.6% reduction relative to the best prior result. On the same benchmark, R-Precision improves to 0.545 / 0.741 / 0.835 for Top-1/2/3. On Mo440H-ML, the finetuned model reports FID 0.123, described as a 34.6% reduction versus strong baselines, with Top-3 R-Precision 0.926 (Shi et al., 3 Oct 2025).

The reported gains extend beyond pure L2M. In motion in-betweening on Mo440H, MoGIC improves the prefix w/ language setting from 0.194 to 0.128 FID relative to MARDM. For captioning on HumanML3D, the framework reaches BLEU-1 53.13, ROUGE 40.6, and BERTScore 40.7, while on Mo440H it outperforms T2MT and MG-MotionLLM across BLEU, ROUGE, and BERTScore. For vision-augmented generation, LV2M improves FID from 0.330 to 0.266 relative to L2M without finetuning, and qualitative examples indicate that visual conditioning corrects unrealistic text-only motions such as inappropriate overhead lifting in a weightlifting scenario (Shi et al., 3 Oct 2025).

The ablation evidence attributes much of the performance gain to explicit intention modeling. Removing intention loss is reported to degrade FID by approximately 35% and to worsen retrieval metrics. This suggests that intention supervision is not merely auxiliary captioning; it acts as a structural regularizer on the shared motion-text-vision latent space (Shi et al., 3 Oct 2025).

The framework also has clear limitations. The visual input is sparse (1 fps), so a plausible implication is that fine-grained temporal alignment remains incomplete. Intention is represented as short text, which suggests a relatively shallow model of goals and subgoals. The method focuses primarily on single-character motion, although the dataset includes interaction scenarios. These limitations coexist with a broad set of supported tasks and with a public code release (Shi et al., 3 Oct 2025).

The acronym should also be distinguished from several unrelated arXiv works with similar names. MoG denotes a graph-based retrieval-augmented generation method (Yuan et al., 29 May 2026); MoGE is a mixture-of-group-experts regularization method for invariant representations (Kang et al., 12 Apr 2025); MGIC refers to multigrid-in-channels CNN architectures (Eliasof et al., 2020); GAMIC is a graph-aligned molecular in-context learning method grounded on Morgan fingerprints (Al-Lawati et al., 8 Feb 2025); MagiC is a benchmark for grounded multimodal cognition in vision-LLMs (Wu et al., 9 Jul 2025); and MOGC is a multi-order graph clustering model with adaptive node-level weight learning (Liu et al., 2024). In contemporary arXiv usage, MoGIC specifically names the intention-aware, vision-grounded motion synthesis framework introduced in 2025 (Shi et al., 3 Oct 2025).