Interact2Ar: Autoregressive Human Motion

• The paper introduces Interact2Ar, the first end-to-end text-conditioned autoregressive diffusion model for full-body interactions with fine-grained hand kinematics.
• It employs specialized transformer heads and a Mixed Memory mechanism to efficiently manage large temporal contexts and reactive motion generation.
• Evaluation metrics such as R-Precision, FID, and temporal smoothness demonstrate that Interact2Ar outperforms previous models in generating coherent human motion.

Interact2Ar is a text-conditioned autoregressive diffusion model for generating full-body human-human interactions. It was introduced as the first end-to-end text-conditioned autoregressive diffusion model for full-body human-human interactions with detailed hand kinematics, and it targets a specific difficulty in motion synthesis: producing individually plausible body and hand motion while preserving coherent coordination between interactants over time. Its central design combines cooperative denoisers, specialized trajectory/body/hand prediction heads, and an autoregressive pipeline with a Mixed Memory mechanism that supports efficient large temporal context windows and reactive generation (Ruiz-Ponce et al., 22 Dec 2025).

1. Problem definition and scope

Interact2Ar addresses the generation of realistic, text-conditioned, full-body human-human interaction motion. In this formulation, an interaction $x$ is a dyad comprising two people whose motions must be individually plausible and jointly coherent over time, including fine-grained hand articulation and inter-person spatial relations such as contacts and proximity. The model is motivated by the observation that hands are critical to non-verbal communication and physical interaction, including handshakes, hugs, and pushing, but that the hand degrees of freedom alone exceed those of the body, making joint learning difficult (Ruiz-Ponce et al., 22 Dec 2025).

The method is positioned against three limitations of prior work. First, many diffusion- and transformer-based human motion generators either omit hands entirely or model them separately with insufficient body context, which yields inefficiency or incoherent hand-body coupling. Second, most diffusion pipelines denoise an entire motion at once, which limits their ability to react to a partner’s evolving motion and can produce weak adaptivity and repetitive artifacts on long horizons. Third, non-autoregressive generators cannot adapt mid-generation to state perturbations, goal changes, or partner motion variations. Interact2Ar responds to these limitations by coupling text conditioning with autoregressive chunk-wise synthesis and explicit modeling of detailed hand kinematics (Ruiz-Ponce et al., 22 Dec 2025).

A common point of interpretive importance is the suffix “Ar.” The underlying model is a motion-generation architecture rather than an augmented-reality interface. This suggests that “Ar” refers to the autoregressive character of the model rather than to AR in the usual augmented-reality sense.

2. Motion representation and cooperative architecture

Interact2Ar uses a per-person SMPL-X representation that remains non-redundant. For each individual $i \in \{a,b\}$, a frame is represented by root translation $r \in \mathbb{R}^3$, root rotation $\phi \in \mathbb{R}^6$, body joint rotations $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$, and hand joint rotations $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$. A dyadic sequence is written as $x=\{x^a, x^b\}$. Body shapes are normalized to a neutral template because of limited shape diversity in the Inter-X dataset. The rationale is to keep the output space compact and physically meaningful while relying on forward kinematics during training for kinematic losses (Ruiz-Ponce et al., 22 Dec 2025).

The model architecture is built around cooperative denoisers with specialized heads. Two parallel transformer-encoder streams, one per person, share weights. Cross-attention between streams exchanges hidden states so that each person’s prediction is conditioned on the other’s motion context. This parameter-sharing strategy is intended to preserve interpersonal dependencies while reducing model size. A shared motion encoder consumes the noised motion chunk, the condition $c$, and the diffusion timestep $t$, and produces a latent representation used by three specialized decoders operating in parallel: a trajectory head, a body head, and a hands head (Ruiz-Ponce et al., 22 Dec 2025).

The specialization of these heads is explicit. The trajectory head predicts global root translation and orientation trajectory and uses 4 transformer blocks, 4 heads, hidden dimension 256, and FFN dimension 512. The body head predicts body joint rotations and uses 8 transformer blocks, 8 heads, hidden dimension 512, and FFN dimension 1024. The hands head uses the same depth and dimensions as the body head. Because all three heads are conditioned on the same encoded latent, the architecture aims to preserve body-hand-trajectory coherence while still handling the distinct kinematic scale and dimensionality of each component (Ruiz-Ponce et al., 22 Dec 2025).

3. Autoregressive diffusion and Mixed Memory

Interact2Ar factorizes a sequence of length $N$ into non-overlapping sub-motions of length $i \in \{a,b\}$0:

$i \in \{a,b\}$1

Instead of denoising the entire motion in one pass, the model generates chunk $i \in \{a,b\}$2 conditioned on a memory of previously generated frames, then advances to chunk $i \in \{a,b\}$3. This chunk-wise formulation is the mechanism by which the method realizes reactive, state-aware synthesis (Ruiz-Ponce et al., 22 Dec 2025).

The memory design is the main architectural distinction. At step $i \in \{a,b\}$4, the short-term memory consists of the most recent $i \in \{a,b\}$5 generated frames,

$i \in \{a,b\}$6

while the long-term memory adds a temporally downsampled history over a larger window $i \in \{a,b\}$7 with stride $i \in \{a,b\}$8,

$i \in \{a,b\}$9

The complete memory is the concatenation $r \in \mathbb{R}^3$0. The denoiser then predicts the clean sub-motion as

$r \in \mathbb{R}^3$1

After chunk synthesis, the memory is updated by appending the newly generated clean frames and discarding older ones according to $r \in \mathbb{R}^3$2, $r \in \mathbb{R}^3$3, and $r \in \mathbb{R}^3$4 (Ruiz-Ponce et al., 22 Dec 2025).

The stated intuition is that short-term memory enforces seamless transitions, whereas long-term downsampled history preserves broader narrative intent without prohibitive compute. The reported example is a 60-frame context using only 24 stored frames when $r \in \mathbb{R}^3$5, $r \in \mathbb{R}^3$6, and $r \in \mathbb{R}^3$7, corresponding to up to a $r \in \mathbb{R}^3$8 memory reduction at equal context coverage. In the reported ablations, simply enlarging short-term context can degrade FID and gives diminishing returns because of increased learning complexity, whereas Mixed Memory sustains or improves quality as context grows with a much smaller memory footprint (Ruiz-Ponce et al., 22 Dec 2025).

4. Diffusion objective, kinematic supervision, and training setup

Interact2Ar follows a standard DDPM/DDIM-style Gaussian noising process and directly predicts $r \in \mathbb{R}^3$9 at each denoising step rather than $\phi \in \mathbb{R}^6$0 or $\phi \in \mathbb{R}^6$1. This parameterization is used so that supervision can be applied both in representation space and in forward-kinematics-derived kinematic spaces. The total loss is

$\phi \in \mathbb{R}^6$2

where $\phi \in \mathbb{R}^6$3 is an $\phi \in \mathbb{R}^6$4 loss on SMPL-X parameters, $\phi \in \mathbb{R}^6$5 penalizes root orientation errors for both people, $\phi \in \mathbb{R}^6$6 and $\phi \in \mathbb{R}^6$7 are $\phi \in \mathbb{R}^6$8 losses on global joint positions and finite-difference velocities, $\phi \in \mathbb{R}^6$9 penalizes foot velocities during contact, and $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$0 enforces inter-person proximity through pairwise distance maps between joints of person $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$1 and person $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$2, masked to near-contact pairs (Ruiz-Ponce et al., 22 Dec 2025).

The distance term is important because it couples the two interactants in global space rather than only at the level of individual pose quality. In the model definition,

$\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$3

and the loss penalizes differences between ground-truth and predicted inter-person distance maps under a mask $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$4 that retains joint pairs close in the ground truth. This makes the training objective sensitive to contact-like or near-contact structure rather than to pose accuracy alone (Ruiz-Ponce et al., 22 Dec 2025).

Training is conducted on Inter-X, described as 11K full-body dyadic interactions across 40 actions with rich text annotations and full SMPL-X including hands. Optimization uses AdamW with learning rate $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$5, weight decay $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$6, batch size 128, 5000 epochs, and EMA. The supplementary loss weights are reported as $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$7, $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$8, $\theta_{\text{body}} \in \mathbb{R}^{21\times 6}$9, $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$0, $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$1, and $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$2. For sampling, the non-autoregressive variant uses DDIM-50, whereas the autoregressive model achieves its best empirical results with only 10 denoising steps per chunk (Ruiz-Ponce et al., 22 Dec 2025).

5. Evaluation protocol and empirical results

The evaluation protocol combines standard text-to-motion metrics with specialized evaluators retrained for interaction realism. Baselines are T2M, InterGen, and InterMask. Standard metrics include R-Precision at Top-1/2/3, FID, Multimodal Distance, Diversity, and MultiModality. The distinctive methodological contribution is a set of robust evaluators trained on global joint positions rather than rotations and partitioned into full-body, body-only, and hand-only heads. These evaluators are intended to reduce bias and better capture global placement between interactants; they are trained with contrastive motion-text encoders following T2M for 300 epochs with learning rate $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$3 and feature dimension 512 (Ruiz-Ponce et al., 22 Dec 2025).

On the main quantitative results for Inter-X, the autoregressive Interact2Ar variant outperforms InterMask across the reported evaluators. Under the full-body evaluator, R-Precision Top-1 improves from 0.415 to 0.453, FID decreases from 0.671 to 0.277, MM Dist decreases from 3.487 to 3.095, Diversity is reported as comparable, and MultiModality changes from 1.686 to 1.427, with lower being better in this setting. Under the body-only evaluator, R-Precision Top-1 improves from 0.386 to 0.469, FID from 6.720 to 0.352, and MM Dist from 4.616 to 3.173. Under the hands-only evaluator, R-Precision Top-1 improves from 0.360 to 0.422, FID from 1.960 to 0.257, and MM Dist from 3.794 to 3.111. The reported summary is that autoregression consistently outperforms the non-autoregressive variant across full, body, and hands (Ruiz-Ponce et al., 22 Dec 2025).

The model is also evaluated for adaptivity through long-horizon transition smoothness. Peak Jerk and Area Under the Jerk are computed on long sequences formed by concatenating 8 motions; for non-autoregressive baselines, concatenation is done by inpainting. Interact2Ar reduces Peak Jerk from 2.328 for InterMask to 0.136 and AUJ from 61.74 to 8.84, which is presented as evidence of markedly smoother temporal composition and disturbance adaptation. A 35-participant user study further reports preference for Interact2Ar over InterMask and InterGen for both overall quality/text alignment and hand realism, with performance approaching ground-truth quality (Ruiz-Ponce et al., 22 Dec 2025).

A further technical point concerns evaluator robustness. The original Inter-X evaluator is described as insensitive to severe trajectory degradations, including 10% noise on trajectory or swapping trajectories between people. By contrast, the retrained global-position evaluators sharply penalize these degradations. One reported example is a trajectory swap causing R-Precision to fall from 0.740 for ground truth to 0.558 and FID to rise to 8.65. This matters because the claimed gains of Interact2Ar are tied not only to better generation but also to an evaluation protocol designed to be more sensitive to inter-person spatial coherence (Ruiz-Ponce et al., 22 Dec 2025).

6. Downstream capabilities, limitations, and relation to adjacent interaction literatures

The autoregressive plus Mixed Memory design is used to support three downstream capabilities. The first is temporal motion composition, where prompts can be switched on the fly and the model generates smooth transitions without offline stitching or inpainting artifacts. The second is real-time adaptation to disturbances, where external perturbations such as sudden root displacement or unexpected contact can be injected between chunks and the next chunk adapts naturally. The third is sequential multi-person interaction, where one person can interact with different partners in succession; after one interaction ends, a new partner and prompt can be introduced and memory-conditioned generation continues smoothly. Parallel multi-human modeling is described as feasible but not trained end-to-end in the reported system (Ruiz-Ponce et al., 22 Dec 2025).

The main limitation identified for the method is dataset-dependent. Because Inter-X normalizes body shape to a neutral template, accurate modeling of shape-dependent contacts is constrained, especially precise hand-hand and hand-body contacts. A plausible implication is that contact fidelity is bounded not only by model architecture but also by available geometric diversity in the training set. Future extensions are described in terms of training with more diverse shapes and scaling beyond sequential to fully parallel multi-human joint generation (Ruiz-Ponce et al., 22 Dec 2025).

Interact2Ar also sits beside a separate body of research in which the supplied comparative notes describe several augmented-reality systems as relevant to an “Interact2Ar-style system.” In that adjacent literature, “Sketched Reality” operationalizes AR-mediated bi-directional interaction between virtual sketches and actuated tangible interfaces (Kaimoto et al., 2022); “AR-based interaction for safe human-robot collaborative manufacturing” formalizes a depth-sensor workspace model with binary masks $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$4, $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$5, and $\theta_{\text{hands}} \in \mathbb{R}^{30\times 6}$6 and guarded AR confirmations for safe collaboration (Hietanen et al., 2019); “Avatar-centred AR Collaborative Mobile Interaction” presents a marker-centered, multi-user mobile AR architecture with Photon-synchronized avatar behavior and a reported SUS score of 85.87 (Marques et al., 2023); “Accessible Gesture-Driven Augmented Reality Interaction System” proposes multimodal gesture recognition with ViT, TCN, GAT, FedAvg, and RL-based interface adaptation for motor-impaired users (Wang, 18 Jun 2025); and “Semantic Reality” introduces a scene-anchored semantic graph over eight relation types for interactive visualization of inter-object connectivity in AR (Liu et al., 6 Apr 2026). These systems belong to AR interaction design rather than dyadic motion synthesis, but the supplied material presents them as conceptually relevant to broader interaction-centered system design.