Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animation (2510.14976v1)
Abstract: Close-proximity human-human interactive poses convey rich contextual information about interaction dynamics. Given such poses, humans can intuitively infer the context and anticipate possible past and future dynamics, drawing on strong priors of human behavior. Inspired by this observation, we propose Ponimator, a simple framework anchored on proximal interactive poses for versatile interaction animation. Our training data consists of close-contact two-person poses and their surrounding temporal context from motion-capture interaction datasets. Leveraging interactive pose priors, Ponimator employs two conditional diffusion models: (1) a pose animator that uses the temporal prior to generate dynamic motion sequences from interactive poses, and (2) a pose generator that applies the spatial prior to synthesize interactive poses from a single pose, text, or both when interactive poses are unavailable. Collectively, Ponimator supports diverse tasks, including image-based interaction animation, reaction animation, and text-to-interaction synthesis, facilitating the transfer of interaction knowledge from high-quality mocap data to open-world scenarios. Empirical experiments across diverse datasets and applications demonstrate the universality of the pose prior and the effectiveness and robustness of our framework.
Summary
- The paper introduces a unified diffusion framework that leverages interactive poses as strong spatial and temporal priors to generate dynamic human interaction animations.
- The architecture combines a pose generator and animator trained on motion capture datasets, achieving superior performance metrics such as an FID of 22.6 and a 68.1% contact ratio.
- The framework supports diverse inputs including text and images, enabling realistic two-person and multi-person interactions across both in-domain and out-of-domain datasets.
Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animation
Introduction and Motivation
Ponimator introduces a unified framework for modeling and generating human-human interaction dynamics by anchoring on interactive poses—defined as two-person poses in close proximity and contact. The central hypothesis is that such interactive poses encode strong spatial and temporal priors, enabling intuitive inference of past and future interaction dynamics. This approach leverages high-quality motion capture datasets (Inter-X, Dual-Human) to learn these priors and applies them to diverse tasks, including image-based interaction animation, reaction animation, and text-to-interaction synthesis.
Framework Overview
Ponimator consists of two main components: a pose generator and a pose animator, bridged by interactive poses. The generator synthesizes interactive poses from a single pose, text, or both, while the animator generates dynamic motion sequences from these poses.
Figure 1: Framework overview. Ponimator consists of a pose generator and animator, bridged by interactive poses. The generator takes a single pose, text, or both as input to produce interactive poses, while the animator unleashes interaction dynamics from static poses.
The pose generator and animator are both implemented as conditional diffusion models, with the generator focusing on spatial priors and the animator on temporal priors. This modular design enables flexible conditioning and supports a wide range of input modalities.
Interactive Pose and Motion Modeling
Interactive poses are parameterized using the SMPL-X body model, capturing joint rotations, global orientation, translation, and shape. The interaction motion is modeled as a short sequence of poses centered around an interaction moment, with the interactive pose serving as the anchor. The joint distribution is decomposed as:
where is the temporal prior (animator) and is the spatial prior (generator). Both are learned via diffusion models trained on mocap data, with interactive poses extracted by proximity-based heuristics.
Pose Animator: Temporal Prior Modeling
The pose animator generates motion sequences conditioned on an interactive pose and shape. The denoising target is defined as motion residuals with respect to the interactive pose, enforcing contextual dynamics shaped by the anchor. A one-hot vector encodes the interaction time index, and an imputation strategy ensures the interactive pose is preserved during diffusion. Conditions are encoded via SMPL-X joint positions and injected using AdaIN.
The architecture is based on DiT (Diffusion Transformer), with alternating spatial and temporal attention blocks. Training employs a composite loss: diffusion loss, SMPL loss, interaction loss (contact and orientation), and velocity loss for motion coherence. Data augmentation with random noise improves robustness to real-world pose estimation errors.
Pose Generator: Spatial Prior Modeling
The pose generator synthesizes interactive poses from text, single-person pose, or both. Conditioning is unified via binary masks for text and pose presence, enabling flexible input combinations. The diffusion target includes joint positions to capture shape and gender, with inverse kinematics used for recovery post-generation.
The architecture mirrors the animator but omits temporal attention. Text is encoded via CLIP and injected using AdaLN. Training uses diffusion, SMPL, and bone length losses.
Applications
Ponimator supports:
- Two-person image animation: Extracts interactive poses from images and animates them using the animator.
- Single-person interaction generation: Generates a partner pose from a single-person input (optionally with text), then animates the interaction.
- Text-to-interaction synthesis: Directly generates interactive poses from text and animates the resulting interaction.
(Figure 2)
Figure 2: Applications. Ponimator enables two-person image animation, single-person interaction generation, and text-to-interaction synthesis.
Experimental Results
Ponimator demonstrates strong performance across multiple datasets and tasks. Key metrics include FID, precision, recall, diversity, contact ratio, and penetration. Notably, Ponimator achieves:
- Superior motion realism and contact: FID of 22.6 and contact ratio of 68.1 on Inter-X, outperforming baselines (MDM*, InterGen, RIG, ComMDM).
- Generalization: Robust results on in-domain (Inter-X, Dual-Human) and out-of-domain datasets (Duolando, Hi4D, Interhuman), including multi-person interactions without retraining.
- Flexible input handling: Effective synthesis from single pose, text, or image, with diverse and realistic interaction dynamics.
(Figure 3)
Figure 3: Interactive pose animation generalizes across datasets and supports multi-person interactions.
(Figure 4)
Figure 4: Single-person image interaction generation and text-driven synthesis yield plausible and diverse interaction dynamics.
Implementation Details
- Interactive pose extraction: Proximity-based contact detection using SMPL-X mesh vertices.
- Model architecture: DiT-based, 8-layer Transformer, 1024 latent dim, spatial/temporal attention, AdaIN/AdaLN for conditioning.
- Training: AdamW optimizer, cosine scheduler, DDIM sampling, batch sizes 256/512, 4000 epochs, 4×A100 GPUs.
- Inference speed: 0.21s for pose generation, 0.24s for 3s motion animation at 10fps.
Limitations
- Short-term modeling: Focused on short interaction segments; chaining enables longer sequences but with diminishing prior benefits.
- Scene context: Ignores environment, leading to possible physical implausibility.
- Contact accuracy: Dependent on pose estimation/generation quality; errors propagate to motion.
- Inter-person penetration: No explicit modeling, resulting in occasional unrealistic contact.
(Figure 5)
Figure 5: Failure modes include inter-person penetration, lack of scene awareness, and inaccurate contact.
Implications and Future Directions
Ponimator's anchoring on interactive poses provides a robust and interpretable prior for human-human interaction modeling, enabling generalization to open-world scenarios and diverse input modalities. The modular diffusion-based design facilitates integration with downstream video synthesis pipelines and supports flexible conditioning.
Future work should address long-term interaction modeling, explicit physical constraints (penetration avoidance), and integration of scene context for enhanced realism. Incorporating text conditioning into the animator and leveraging multi-modal inputs (e.g., images, audio) could further expand the framework's applicability.
Conclusion
Ponimator presents a principled approach to human-human interaction animation by leveraging interactive pose priors via conditional diffusion models. The framework demonstrates strong generalization, realism, and versatility across datasets and tasks, establishing interactive poses as a universal anchor for modeling and generating social interaction dynamics.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Overview
This paper introduces Ponimator, a new tool that turns static pictures of people into short, realistic animation clips showing how they interact. It focuses on “interactive poses” — moments when two people are close and touching (like a hug, a handshake, or a push). The main idea is simple: if you know how two people are positioned at one key moment, you can guess what happened just before and what will happen next.
Key Objectives
The paper aims to:
- Learn how people move during close interactions by studying one “freeze-frame” pose of two people in contact.
- Animate pairs of people in photos so the image becomes a believable short video clip.
- Create a second person’s pose for a single-person image (and then animate them together).
- Generate two-person interactions from text descriptions (like “two people hug” or “one person lifts another”).
How It Works (Explained Simply)
Think of Ponimator as two smart helpers that work together:
- The Pose Animator:
- Input: a pair of people’s poses at a key moment (the “interactive pose”).
- Output: a short animation showing what happened before and after that moment.
- Analogy: Imagine a single snapshot of a handshake. From that snapshot, the animator smoothly fills in the motion: how they reached out, shook hands, and then let go.
- Trick it uses: a “diffusion model,” which is like starting with a noisy, blurry version of the movement and gradually cleaning it up until it looks realistic. It uses known patterns from many recorded human motions to make the animation look natural.
- The Pose Generator:
- Input: a single-person pose, or a text prompt, or both.
- Output: a matching partner’s pose so the two people are in contact (an interactive pose).
- Analogy: If you have a picture of one person reaching out, the generator can create a second person who is being hugged, pushed, or high-fived, depending on your text prompt.
- It can understand short text (via a language-vision model called CLIP) and combine it with body posture to create a suitable partner pose.
What is an “interactive pose”?
- It’s a moment where two people are close and physically connected (e.g., hands touching in a handshake).
- These poses carry “prior knowledge”: from them, you can guess likely past and future moves, and the spatial relationship between the two bodies.
Where does Ponimator learn from?
- It trains on high-quality motion-capture datasets of real two-person interactions (like Inter-X and Dual-Human). These datasets include many examples of people interacting in close contact.
Main Findings and Why They’re Important
Here are the main results:
- Better realism: Ponimator produces more natural, believable motions than several existing methods. In tests, it scored better on motion-quality measures and made more physically plausible contact between people.
- Works on real-world photos: It can take pictures from the internet, estimate the poses, and animate them into short clips.
- Versatile:
- Two-person image animation: Turn a static two-person photo into an interaction clip.
- Single-person image completion: Add a second person with a fitting pose, then animate both.
- Text-to-interaction: Create short two-person interaction clips from a simple text description.
- Generalizes well: It handles different datasets, unseen subjects, and even more than two people in some cases without special retraining.
Why this matters:
- Many tools focus only on static poses or require long video input. Ponimator uses a simple anchor — one interactive pose — to unlock rich, short-term interaction dynamics. This simplicity helps it work more reliably across many situations.
Implications and Potential Impact
Ponimator could help:
- Creators, animators, and game developers quickly turn images or ideas into short, realistic interaction clips.
- Social media and AR apps bring photos to life with believable motion.
- Education and research on body language and social interactions, by visualizing how interactions unfold.
Big picture:
- Anchoring on interactive poses is a practical, robust strategy. It avoids complex physics engines and heavy hand-crafted rules, yet still produces physically plausible contact and smooth motion.
- With careful use, this approach can make digital content more engaging. However, developers should consider privacy and fairness when applying it to images of real people and ensure it’s used responsibly.
Knowledge Gaps
Knowledge Gaps, Limitations, and Open Questions
Below is a concise, actionable list of what remains missing, uncertain, or unexplored in the paper, intended to guide future research.
- Limited temporal scope: the system is trained and evaluated on 3-second clips at 10 fps; it does not address longer interactions, multi-stage activities, or smooth chaining/transitioning of segments into long-form sequences.
- Dependency on pose estimation quality in the wild: image-based applications rely on off-the-shelf 3D pose estimators (BUDDI, SMPLer-X) without a quantitative robustness analysis to occlusions, truncations, multi-person scenes, extreme camera viewpoints, depth/scale ambiguities, or noisy SMPL-X fits.
- No quantitative evaluation on in-the-wild images: the paper presents qualitative image animations but lacks metrics (e.g., MPJPE/PA-MPJPE, contact consistency, image-plane alignment) to measure accuracy and realism when transferring mocap priors to real photos.
- Physical plausibility is not enforced: there is no physics simulation or constraint handling for balance, foot-ground contact, center-of-mass stability, friction, or self-collision; penetration remains non-zero and foot skidding or ground contact errors are not measured.
- Ambiguous shape/gender recovery: the pose generator outputs rest-pose joint positions and recovers SMPL-X shapes via inverse kinematics, but the method is underconstrained; there is no evaluation of shape/gender accuracy, stability, or failure modes of the IK step.
- Heuristic contact detection: interactive frames are selected via a proximity threshold over SMPL-X vertices; sensitivity to threshold choice, false positives/negatives, and mislabeling of interaction moments is not analyzed.
- Unvalidated probabilistic factorization: the decomposition p(𝒳,β)=p(𝒳; x_I,β)·p(x_I,β) is assumed but not theoretically justified or empirically validated (e.g., via conditional independence checks or likelihood estimates).
- Multi-person (>2) interactions: while a qualitative example suggests emergent multi-person capability, there is no formal method or evaluation for scaling to more than two agents, handling multiple simultaneous contacts, or resolving conflicts among overlapping priors.
- Limited fine-grained contact modeling: SMPL-X is used with relatively coarse joint settings (e.g., 21×3), and hand/finger articulation, facial expressions, and micro-contact semantics (e.g., grasps, finger placement in handshakes) are not explicitly modeled or evaluated.
- Scene and environment obliviousness: interactions are generated without scene constraints (e.g., ground plane, obstacles, furniture), camera intrinsics/extrinsics, or background alignment; there is no rendering/alignment into the original image domain or evaluation of 2D plausibility.
- Text conditioning scope and control: the CLIP-based conditioning is demonstrated with short phrases but lacks evaluation on complex, compositional prompts, role assignment (who initiates), intensity/speed, style/emotion, and temporal control beyond the single interaction timestep I.
- Reaction animation to full partner motion: although “reaction animation” is mentioned, the method does not condition on a full observed motion sequence of one person to generate the other’s responsive dynamics; only single-pose and text conditions are supported.
- Forecasting accuracy not measured: when generating past/future dynamics around x_I, the paper does not report forecasting metrics (e.g., MPJPE/ACC for past/future relative to GT) or analyze uncertainty in motion predictions.
- Dataset bias and coverage: training focuses on close-contact interactions from Inter-X and Dual-Human; the diversity across demographics, body types (children/elderly), cultural interaction styles, and rare or complex activities is not characterized.
- Diversity vs. plausibility trade-offs: while some diversity metrics (e.g., MModality) are reported, there is no systematic analysis of how sampling temperature, mask probabilities (p_text, p_pose), or model stochasticity affects the balance between variety and contact/physical realism.
- Robustness to extreme or out-of-distribution inputs: failure cases (e.g., adversarial text, atypical single poses, extreme motions like lifts/falls) are not cataloged, and model reliability under such conditions remains unclear.
- Fairness of comparisons: baselines (e.g., MDM adapted for two-person) may be disadvantaged; the paper does not standardize text prompt sets, training schedules, or hyperparameters across methods to ensure fair, reproducible cross-benchmark comparisons.
- Ethical and societal implications: risks of bias, misrepresentation of social interactions, and potential misuse in sensitive contexts (e.g., surveillance, deepfakes) are not discussed; no guidelines for responsible deployment are provided.
- Computational footprint and deployment: training/inference efficiency is measured on A100 GPUs; scalability to resource-constrained settings, latency for interactive applications, and memory/compute optimization are not addressed.
- Extension to multi-modal inputs: integrating audio, video context, or scene semantics to refine pose generation and animation is not explored; only pose and text are considered.
- Handling non-contact interactions: the framework is anchored on close-contact poses; extension to non-contact yet socially meaningful interactions (e.g., gestural exchanges at distance) and modeling their dynamics is not covered.
- Camera- and world-frame consistency: global orientation/translation in SMPL-X are used, but alignment to camera coordinates for image-based tasks is not modeled; the effect of camera motion or viewpoint changes is not analyzed.
- Calibration and uncertainty quantification: the diffusion outputs are stochastic, but there is no calibration of confidence, uncertainty visualization, or mechanisms to let users select among multiple plausible interaction trajectories.
Practical Applications
Practical Applications Derived from Ponimator
Ponimator introduces an interactive-pose–anchored framework with two conditional diffusion models: a pose animator (temporal prior) and a pose generator (spatial prior). It transfers motion knowledge from high-quality mocap datasets (Inter-X, Dual-Human) to open-world images and text, enabling near-term animation pipelines and informing longer-term, safety-critical systems. Below are actionable use cases across industry, academia, policy, and daily life.
Immediate Applications
- Image-to-interaction animation for media production
- Sector: media/entertainment, advertising, social media
- Use case: Animate two-person images (e.g., handshake, hug, dance) into short, realistic interaction clips (≈3s, 10 fps) for storyboards, social posts, ad creatives, and previsualization.
- Tools/workflows:
- A Blender/Unreal/Unity plugin “Interactive Pose Animator” that ingests a 2-person image, runs BUDDI for pose extraction, and applies Ponimator’s pose animator to produce animation.
- A web service/API that takes an image, infers interactive poses, and returns a short clip or GLB/FBX motion.
- Assumptions/dependencies: Requires accurate 3D pose recovery (SMPL-X) from images; best for close-contact interactions and short sequences; quality depends on occlusion and camera viewpoint.
- Single-person reaction animation for content creation
- Sector: media/entertainment, education, social apps
- Use case: Generate a plausible partner pose and short interaction (e.g., “high-five,” “lift the other”) from a single-person image or pose, optionally conditioned by text.
- Tools/workflows:
- A “Reaction Generator” extension in video editors that estimates the visible person’s pose (e.g., SMPLer-X) and uses Ponimator’s pose generator then animator to produce two-person motion.
- Assumptions/dependencies: Text understanding via CLIP must match intended interaction; single-person pose must be recoverable; motion realism bounded by training data coverage.
- Text-to-interaction synthesis for rapid prototyping
- Sector: game development, XR, virtual production
- Use case: Author short two-person interactions directly from text prompts (e.g., “two people hug,” “push”), exporting as mocap-style motion clips for NPC behaviors or previs.
- Tools/workflows:
- A Unity editor tool where designers type prompts and obtain short interaction clips to assign to NPC pairs; batch generation for variation.
- Assumptions/dependencies: Short-term interactions (≈3s) favored; semantic grounding relies on CLIP; domain mismatch may occur for rare/complex actions.
- AR filters and lenses featuring interactive avatars
- Sector: AR/VR, social media (e.g., camera apps)
- Use case: Generate short interactive motions between user avatar and a virtual partner (greeting, dance step) triggered by text or gesture.
- Tools/workflows:
- A server-side “Interaction Lens” that uploads the user’s pose, generates a partner pose and motion via Ponimator, and renders an AR overlay.
- Assumptions/dependencies: Real-time on-device performance may be limited; consistent tracking and latency management required; privacy/consent considerations for live images.
- Training data augmentation for interaction recognition
- Sector: computer vision, machine learning research
- Use case: Generate diverse two-person close-contact motion clips to expand datasets for training interaction recognition, social understanding, or contact modeling models.
- Tools/workflows:
- A dataset augmentation pipeline that samples diverse interactive poses and motion variants (via Ponimator’s generator+animator) and exports to standard formats with labels.
- Assumptions/dependencies: Synthetic-to-real domain gap must be measured; balancing class distributions and minimizing bias requires curation.
- Rapid prototyping of multi-person interaction beats
- Sector: animation, choreography, live performance visualization
- Use case: Compose short sequences where multiple characters interact in close proximity (paper shows generalization to >2 persons without retraining), useful for testing staging and beats.
- Tools/workflows:
- A layout tool that places characters, specifies interactive moments (time index I), and uses Ponimator to fill in short motions.
- Assumptions/dependencies: Beyond two-person interactions is emergent, not guaranteed; staging and collision handling may require manual cleanup.
- Educational visualizations of social cues and contact dynamics
- Sector: education, social skills training, sports coaching
- Use case: Visualize likely past/future motions around a contact moment to teach social signals (greeting etiquette) or coaching cues (grappling, dance holds).
- Tools/workflows:
- A classroom app that animates textbook images or tutor-uploaded photos to show interaction context.
- Assumptions/dependencies: Motions are plausible but not biomechanics-grade; professional domains should treat outputs as illustrative, not prescriptive.
- Motion library generation for game/NPC interaction sets
- Sector: gaming
- Use case: Batch-generate short interaction clips (push, handshake, hug variants) to populate NPC behavior trees.
- Tools/workflows:
- A “Ponimator Batch” CLI that takes a list of text prompts and exports motion files for animation blending systems.
- Assumptions/dependencies: Clips are short and close-contact; multi-step or long-horizon behaviors need chaining/authoring.
Long-Term Applications
- Real-time, on-device interactive motion synthesis
- Sector: AR/VR, mobile apps
- Use case: Instant generation of partner poses and short interactions on smartphones/AR glasses during live capture.
- Tools/products: On-device optimized models (distillation/quantization), incremental sampling, hardware acceleration.
- Assumptions/dependencies: Significant model compression and latency reduction; robust pose recovery under occlusion; energy/battery constraints.
- Human–robot close-contact intent prediction and safe planning
- Sector: robotics, HRI, industrial safety
- Use case: Predict likely near-term human motion in close proximity, enabling robots to plan safer responses (handover, avoid accidental contact).
- Tools/products: A planning module that consumes Ponimator-like interactive-pose priors to anticipate short-horizon dynamics.
- Assumptions/dependencies: Requires rigorous validation and domain adaptation (human-human → human-robot); physical simulation and safety certification needed.
- Clinical and caregiving training simulators for assisted mobility
- Sector: healthcare
- Use case: Simulate caregiver–patient interactions (support, lift assist, transfers) for training modules and scenario rehearsal.
- Tools/products: A simulator integrating interaction priors with physics engines and patient-specific biomechanical models.
- Assumptions/dependencies: Must incorporate medically accurate constraints; reduce penetration and ensure realistic contact forces; ethical oversight and clinical validation essential.
- Surveillance and public safety interaction forecasting
- Sector: public safety, security
- Use case: Forecast short-term motion around contact events to assist operators in situational awareness (e.g., crowd management).
- Tools/products: Video analytics module that flags close-contact interactions and predicts likely motions for proactive response.
- Assumptions/dependencies: Robustness to noisy, crowded scenes; strong governance, privacy, consent, and fairness frameworks; minimize false positives and harmful profiling.
- Long-horizon multi-person social simulation and crowd animation
- Sector: film/TV, urban planning, simulation
- Use case: Scale short interaction priors to multi-minute sequences with multiple agents and evolving social contexts.
- Tools/products: Hierarchical controller that stitches pose-anchored short clips with scene/intent planners and physics-based crowd systems.
- Assumptions/dependencies: Requires higher-level behavior modeling, scene semantics, collision avoidance, and continuity constraints.
- Physically grounded interaction generation with contact dynamics
- Sector: animation, robotics
- Use case: Coupling Ponimator with physics to reduce interpenetration and model forces/torques during contact.
- Tools/products: Hybrid differentiable physics + diffusion pipelines; contact-aware loss functions; material/force models.
- Assumptions/dependencies: Increased compute and complexity; careful tuning to balance realism and sample diversity.
- Domain adaptation for specialized interactions (sports, dance, emergency response)
- Sector: sports, performing arts, emergency services
- Use case: Tailor the interactive pose prior to specific domains (e.g., judo throws, tango holds, rescue carries).
- Tools/products: Fine-tuning pipelines with curated domain mocap; content-authoring interfaces for domain experts.
- Assumptions/dependencies: Access to high-quality, domain-specific mocap; licensing; expert evaluation for correctness.
- Policy frameworks for consent, provenance, and responsible synthesis
- Sector: policy/regulation, platform governance
- Use case: Establish guidelines for animating real people’s images (consent, watermarking), dataset governance (bias audits), and provenance (content signatures).
- Tools/products: Content provenance standards (e.g., C2PA integration), platform-level synthesis disclosures, bias/fairness documentation.
- Assumptions/dependencies: Multistakeholder collaboration (industry, academia, civil society); evolving regulatory landscape; enforcement mechanisms.
- Co-creative systems where AI avatars respond to users in collaborative XR
- Sector: XR/Metaverse, education/training
- Use case: Interactive training or role-play where virtual partners react plausibly to user movements and text/dialogue in near real time.
- Tools/products: XR middleware combining pose anchoring, speech/text conditioning, and motion blending.
- Assumptions/dependencies: Sensing fidelity for user motion; low-latency generation; social acceptability and comfort; safety for close-contact cues.
- Synthetic dataset generation for social behavior research at scale
- Sector: academia/research
- Use case: Large-scale generation of controlled two-person interactions (with annotations) to paper social signals, contact patterns, and generalization.
- Tools/products: A data factory that varies pose, shape, camera, context, and interaction text to produce labeled corpora.
- Assumptions/dependencies: Annotation reliability; openness and reproducibility; managing synthetic bias and ensuring ethical use.
Notes on feasibility across applications:
- The framework excels at short, close-contact interactions anchored on interactive poses; longer, multi-stage behaviors require composition or higher-level planning.
- Performance relies on accurate 3D pose estimation (SMPL-X) and robust text grounding (CLIP); domain shift (in-the-wild images, rare actions) can reduce fidelity.
- Physical plausibility is improved compared to baselines but not guaranteed; penetration metrics suggest additional physics coupling is beneficial for safety-critical domains.
- Compute constraints: reported inference uses 50 DDIM steps on an A100 for ≈0.24s per 3s clip; mobile or real-time scenarios need optimization.
Glossary
- AdaIN: Adaptive Instance Normalization used to inject conditioning signals into network layers. "and injected into the model layers via AdaIN~\cite{huang2017arbitrary}"
- AdaLN: Adaptive Layer Normalization variant for conditioning within transformer layers. "and injected by AdaLN~\cite{huang2017arbitrary}"
- AdamW: An optimizer that decouples weight decay from the gradient update, improving training stability. "trained using AdamW~\cite{loshchilov2017decoupled} (LR "
- Bone length loss: A loss enforcing realistic skeleton proportions by matching predicted bone lengths to ground truth. "and bone length loss $\mathcal{L}_{\text{bone}$ minimizes the MSE with ground-truth lengths in the SMPLX~\cite{SMPL-X:2019} kinematic tree."
- CLIP: A vision-LLM used to encode text conditions for generation. "The text condition is encoded via CLIP~\cite{radford2021learning}"
- CLIP-ViTL/14: A specific CLIP text encoder variant used (ViT-L, patch size 14). "a frozen CLIP-ViTL/14~\cite{radford2021learning} text encoder."
- Conditional diffusion model: A generative diffusion model that produces samples guided by input conditions. "Leveraging interactive pose priors, Ponimator employs two conditional diffusion models"
- Contact Frame Ratio (CR): Percentage of frames with physical contact between individuals, indicating interaction plausibility. "Contact Frame Ratio (CR., )—proportion of frames with two-person contact"
- DDIM: Denoising Diffusion Implicit Models; a fast deterministic sampler for diffusion models. "At inference, DDIM~\cite{song2020denoising} samples 50 steps"
- DiT: Diffusion Transformer architecture that uses transformer blocks for diffusion-based generation. "We adopt the DiT~\cite{peebles2023scalable} architecture as our diffusion model"
- Forward kinematics (FK): Computing joint positions from poses and shape using the kinematic chain. "SMPLX joint forward kinematics (FK) function "
- Frechet Inception Distance (FID): A metric comparing feature distributions of generated vs real data to assess quality. "Frechet Inception Distance (FID), the feature distribution against ground truth (GT)."
- Inverse kinematics (IK): Recovering model parameters (e.g., shape) from target joint positions. "we can recover $\boldsymbol{\beta}^{\{a,b\}$ from $j_{\text{rest}^{\{a,b\}$ using inverse kinematics (IK)."
- Interactive pose prior: Prior knowledge encoded by close-contact two-person poses that guides motion generation. "Leveraging interactive pose priors, Ponimator employs two conditional diffusion models"
- Kinematic tree: The hierarchical skeletal structure defining joint connectivity and lengths. "SMPLX~\cite{SMPL-X:2019} kinematic tree."
- MModality: Metric quantifying the diversity of motions generated from the same text prompt. "MModality—the ability to generate diverse interactions from the same text~\cite{liang2024intergen, tevet2023human}."
- Mocap: Motion capture data used to train priors and models for realistic human movement. "trained on high-quality mocap data"
- SMPL loss: A reconstruction loss on SMPL/SMPL-X parameters or outputs to match ground truth poses. "we apply the SMPL loss $\mathcal{L}_{\text{smpl}$"
- SMPLX parametric body model: A 3D human body model with expressive hands, face, and body parameters. "we use the SMPLX parametric body model~\cite{SMPL-X:2019}"
- Spatial attention: Attention mechanism focused on spatial relationships (e.g., contact) within a frame. "that alternate spatial attention for human contact and temporal attention for motion dynamics."
- Temporal attention: Attention mechanism modeling dependencies across time steps for motion dynamics. "that alternate spatial attention for human contact and temporal attention for motion dynamics."
- Velocity loss: A loss encouraging smooth and coherent motion by penalizing undesirable velocity patterns. "and a velocity loss ~\cite{tevet2023human}"
Collections
Sign up for free to add this paper to one or more collections.