---
title: 'HumanoidGen: Human Generation & Animation'
url: https://www.emergentmind.com/topics/humanoidgen
type: topic
---

# HumanoidGen: Human Generation & Animation

Searching arXiv for recent papers explicitly relevant to "HumanoidGen" and adjacent humanoid generation/animation/control work.
Searching for "humanoid generation 3D human generation animatable avatars Gaussian splatting diffusion".
arXiv search query: humanoid generation 3D human generation animatable avatars gaussian splatting diffusion
“HumanoidGen” is an *Editor’s term* for a research area concerned with generating, reconstructing, animating, and controlling humanoid entities across 2D images, 3D avatars, rigged assets, contact-rich multi-human interactions, and physically grounded robot behaviors. In the available literature, no single canonical method defines the term; notably, HumanGen explicitly states that its paper does not use the name “HumanoidGen,” which suggests that the label is better understood as an umbrella for related methods than as a standardized model family [2212.05321]. Under that umbrella, the field spans pose-conditioned human image synthesis, text-driven 3D human generation, expressive SMPL-X-controlled avatars, anthropometric human modeling, contact-guided interaction synthesis, 4D human-scene generation, locomotion controllers, humanoid-object interaction generation, and whole-body data generation for imitation learning [2103.06902], [2306.09329], [2311.13574], [2506.04351], [2508.09858], [2605.24592].

## 1. Scope and research taxonomy

The literature grouped here divides into several technically distinct but increasingly interconnected strands.

| Strand | Core problem | Representative papers |
|---|---|---|
| 2D human generation | Pose-conditioned image synthesis and part control | HumanGAN [2103.06902] |
| 3D human and avatar generation | Text-driven humans, radiance fields, Gaussian avatars | HumanGen [2212.05321], DreamHuman [2306.09329], HumanGaussian [2311.17061], HuGeDiff [2506.04351] |
| Expressive and rigged assets | Body, face, hand control and animate-ready rigs | XAGen [2311.13574], AniGen [2604.08746] |
| Human interaction and 4D synthesis | Contact-guided partners and synthetic human dynamics | ContactGen [2401.17212], HumanGenesis [2508.09858] |
| Physically grounded humanoids | Locomotion, humanoid-object interaction, loco-manipulation data generation | MuGen [2605.24592], SimGenHOI [2508.14120], HumanoidMimicGen [2605.27724] |

At the 2D end, HumanGAN formulates a conditional VAE-GAN for dressed human images with DensePose conditioning, pose-normalized UV appearance, and part-based latent codes; it supports global appearance sampling, pose transfer, parts transfer, garment transfer, and part sampling in one framework [2103.06902]. At the 3D end, HumanGen, DreamHuman, HumanGaussian, and HuGeDiff differ primarily in representation—tri-planes and SDF, pose-conditioned NeRF, 3D Gaussian Splatting optimized by SDS, and diffusion directly in 3DGS parameter space, respectively [2212.05321], [2306.09329], [2311.17061], [2506.04351].

A plausible implication is that “HumanoidGen” is less a single architecture than a convergence zone: image generation, 3D reconstruction, avatar animation, and robot control increasingly share representations, priors, and conditioning mechanisms.

## 2. Representational foundations

A defining characteristic of this literature is the choice of representation. HumanGAN uses a pose-normalized SMPL UV texture map, DensePose part assignments, and a part-structured latent space with \(M=24\) body parts and latent dimension \(N=16\) per part, yielding \(512 \times 512\) human images with localized semantic control [2103.06902]. HumanGen instead couples a pre-trained 2D human StyleGAN with an EG3D-style tri-plane generator, a PIFuHD-based reconstructor, an SDF geometry branch, and a two-stage texture blending mechanism, enabling \(360^\circ\) free-view rendering and direct \(512 \times 256\) rendering without a separate super-resolution head [2212.05321].

DreamHuman represents an avatar as a pose- and shape-conditioned NeRF over imGHUM semantic signed distance features, so that pose changes can be applied after optimization and the avatar remains animatable [2306.09329]. HumanGaussian and HuGeDiff both adopt 3D Gaussian Splatting, but in different roles. HumanGaussian initializes approximately \(100k\) Gaussians from a posed SMPL-X mesh and optimizes them with structure-aware SDS using RGB and depth branches [2311.17061]. HuGeDiff builds a weakly supervised pipeline in which approximately \(10k\) photorealistic humans are generated with FLUX, reconstructed into SMPL-X-anchored 3DGS, and then used to train a text-conditioned diffusion model directly over 3DGS parameters; its densified mesh reaches \(84{,}317\) vertices [2506.04351].

Rigged generation pushes representation further. XAGen uses multi-scale, multi-part tri-planes for body, face, and hands, inverse linear blend skinning driven by SMPL-X, and part-specific rendering and discriminators [2311.13574]. AniGen formalizes animate-ready generation as joint prediction of shape, skeleton, and skinning through unified \(S^3\) fields over a shared spatial domain, augmented by a confidence-decaying skeleton field and a dual skin feature field that allows a fixed-architecture network to support arbitrary rig complexity [2604.08746]. AnthroNet occupies a different point in the design space: it models humans by 36 anthropometric measurements plus a sex indicator and outputs a high-resolution mesh with approximately \(36{,}078\) vertices and \(68{,}414\) triangles, then applies a learned skinner for posing [2309.03812].

This suggests that representation choice now encodes a research commitment. Tri-planes and NeRFs emphasize view-consistent rendering; 3DGS emphasizes efficiency and explicit surface-aligned detail; anthropometric parameterizations emphasize measurable control; unified rig fields emphasize animation-readiness.

## 3. Conditioning and controllability

Control signals in HumanoidGen-style systems range from geometry-preserving part codes to text, pose, interaction labels, anthropometrics, and multimodal dialogue context. HumanGAN disentangles appearance from pose by encoding appearance in canonical UV space and warping it to a target DensePose; because appearance is factorized by body parts, the model can hold most of a person fixed while resampling only the head, garment regions, or other parts [2103.06902].

Text conditioning dominates recent 3D work. DreamHuman takes text-only input and optimizes a realistic animatable 3D human avatar without paired text–3D supervision; it samples poses from a motion prior during training and uses semantic zoom on six regions—head, upper body, lower body, midsection, left arm, and right arm—to improve local detail [2306.09329]. HumanGaussian conditions on a text prompt \(y\) and, optionally, a pose skeleton \(\mathbf{p}\); its diffusion source model jointly denoises RGB and depth conditioned on pose and text, and it uses annealed negative prompt guidance to mitigate over-saturation [2311.17061]. HuGeDiff preserves text conditioning through its entire loop: text prompts drive FLUX image synthesis, the same prompts condition the 3D diffusion model, and classifier-free guidance controls the strength of semantic alignment [2506.04351].

Other systems broaden the control vocabulary. XAGen uses SMPL-X parameters for body pose, facial expression, jaw pose, hand pose, and body shape, with separate body, face, and hand cameras and discriminators [2311.13574]. AnthroNet conditions on a \(37\)-dimensional vector comprising 36 anthropometric measurements plus sex, encoded with random Fourier features, and can then produce bodies in arbitrary pose [2309.03812]. ContactGen takes as input a 3D partner human and an interaction label, predicts potential contact regions between the two humans, and uses those estimated regions as guidance inside a guided diffusion model [2401.17212]. SimGenHOI conditions on text prompts, object geometry encoded as a Basis Point Set, sparse object waypoints, and the initial humanoid pose to generate key whole-body actions for humanoid-object interaction [2508.14120].

The broadest conditioning interface appears in “Body of Her,” which extends a pre-trained LLM into a real-time, duplex, interactive multimodal model that integrates audio and visual inputs and is further modulated by text prompts, trajectory signals, and identity embeddings; the reported training corpus comprises approximately \(200{,}000\) hours of audio, around \(130{,}000\) hours of video data, and about \(20{,}000\) alignment samples [2408.02879]. A reasonable interpretation is that the field is moving from single-modality conditioning toward compound control stacks in which language, pose, geometry, and interaction state coexist.

## 4. Animation, interaction, and 4D dynamics

A central transition in this literature is from static generation to animatable and interactive generation. DreamHuman explicitly targets animatable 3D avatars rather than fixed assets: the NeRF is conditioned on pose and shape, and the optimized avatar can be re-posed by changing \(\theta\) without retraining [2306.09329]. XAGen extends this paradigm to expressive full-body avatars, adding independent control over body, face, jaw, and hands, together with multi-part rendering and multi-part discriminators to improve small-scale regions such as eyes, mouth, and fingers [2311.13574].

ContactGen shifts attention from single avatars to multi-human interaction. Its task is defined by a partner human and an interaction label; the method introduces a contact prediction module that adaptively estimates potential contact regions between two humans according to the interaction label, then enforces those regions inside a guided diffusion framework. The reported evaluation on CHI3D emphasizes physically plausible and diverse poses relative to comparison methods [2401.17212]. This is notable because the conditioning variable is not simply “pose” but “contact structure.”

HumanGenesis moves to 4D human-scene synthesis. Its four-agent system comprises a Reconstructor, a Critique Agent, a Pose Guider, and a Video Harmonizer. The Reconstructor builds 3D-consistent human-scene representations from monocular video using 3D Gaussian Splatting and deformation decomposition; the Critique Agent uses multi-round MLLM-based reflection to localize and refine poor regions; the Pose Guider supplies expressive pose sequences; and the Video Harmonizer applies a hybrid rendering-plus-diffusion pipeline, coupled to a Back-to-4D feedback loop that refines the underlying geometry [2508.09858]. In “Body of Her,” the output space broadens again: the model generates speech, full-body movements for talking, responding, idling, and manipulation, and does so in real time under duplex conversational constraints [2408.02879].

AniGen addresses a related but distinct bottleneck: even high-quality 3D generators typically produce static assets, after which post-hoc auto-rigging is brittle. Its two-stage flow-matching pipeline first synthesizes a sparse structural scaffold and then dense geometry plus articulation in a structured latent space, directly producing animate-ready assets rather than requiring a later rigging pass [2604.08746].

Taken together, these systems indicate a shift from “generate a human shape” toward “generate a controllable temporal entity”: one that can be re-posed, re-targeted, brought into contact, inserted into scenes, or animated under explicit skeletal structure.

## 5. Physics-grounded humanoids and robotics

In robotics-oriented work, the term “generation” often refers not to image or mesh synthesis but to the synthesis of executable motion, physically valid morphology, or large demonstration corpora. MuGen is representative: it learns a multi-skill locomotion controller for humanoid robots using VQ-VAEs trained with model-based reinforcement learning, then distills the privileged-information teacher into a deployable student policy. The same latent space supports both tracking of unseen human motions and a generative mode in which a prior encoder selects motion codes from state alone [2605.24592].

Humanoid-Gym provides the complementary infrastructure perspective. It is an Isaac Gym-based reinforcement-learning framework for humanoid locomotion with zero-shot sim-to-real transfer, verified on RobotEra’s XBot-S and XBot-L, and includes a sim-to-sim bridge from Isaac Gym to MuJoCo to test robustness across physics engines [2404.05695]. “From Universal Humanoid Control to Automatic Physically Valid Character Creation” extends generation one step earlier in the pipeline: given pre-specified human motion sequences, it optimizes humanoid physical attributes so that the resulting body is physically valid and better suited to imitate those motions under a pretrained universal controller [2206.09286].

SimGenHOI fuses generative modeling and control more tightly. A Diffusion Transformer predicts key whole-body actions conditioned on text prompts, object geometry, sparse waypoints, and the initial humanoid pose; these key actions are interpolated into trajectories that a contact-aware RL policy tracks while correcting penetration and foot sliding. The model and policy are then mutually fine-tuned so that the generator shifts toward the controller’s feasible region and the controller shifts toward the generator’s motion distribution [2508.14120]. HumanoidMimicGen addresses data scarcity for loco-manipulation by adapting contact-rich whole-body skills from a handful of source demonstrations, interleaving them with whole-body locomotion and manipulation planning, and using the resulting synthetic corpus to train whole-body visuomotor policies that outperform policies trained only on real-world data by \(20\%\) on a new nine-task benchmark [2605.27724].

A plausible implication is that a robotics interpretation of HumanoidGen is now emerging in which generation, planning, and control are no longer separate stages. Instead, the generator proposes sparse structure—motion codes, key actions, demonstrations, or body designs—and a controller or planner enforces dynamic validity.

## 6. Evaluation, limitations, and likely trajectories

Reported evaluations indicate rapid progress, but they also reveal where the field remains structurally weak.

| Paper | Reported result | Context |
|---|---|---|
| HumanGen [2212.05321] | FID 20.97, depth 0.0201 | 3D human radiance fields |
| HumanGaussian [2311.17061] | ~1 hour per prompt on a single NVIDIA A100 40GB | Text-driven 3D humans |
| HuGeDiff [2506.04351] | 0.45 minutes per sample | Text-to-3D human generation |
| XAGen [2311.13574] | FID 8.55 on DeepFashion | Expressive avatar generation |
| HumanGenesis [2508.09858] | SSIM 0.891, PSNR 26.992, LPIPS 0.081 | NeuMan reconstruction |
| MuGen [2605.24592] | SR\(_\text{unseen}\) 0.9535 | Generative locomotion control |
| HumanoidMimicGen [2605.27724] | 20% improvement over real-only training | Loco-manipulation imitation |

The limitations are also strikingly consistent across subfields. HumanGaussian reports persistent weaknesses on hands and feet and attributes this partly to weaknesses inherited from current text-to-image models [2311.17061]. DreamHuman notes geometry–albedo ambiguity, limited resolution, and the lack of explicit temporal supervision for clothing dynamics [2306.09329]. HuGeDiff identifies limited clothing pattern detail, limited hair modeling due to SMPL-X constraints, domain gap on real datasets, and unnatural shading under pose changes [2506.04351]. XAGen still depends heavily on SMPL-X fitting quality and remains weak on loose clothing and inverse-skinning errors [2311.13574]. AniGen generalizes to humanoids but still reports sensitivity in hands, face, and image-only conditioning, especially where rig precision or temporal coherence would be required [2604.08746]. HumanGenesis highlights dynamic scenes, fine facial and hand detail, and extreme motions or viewpoints as failure cases [2508.09858]. In interactive agent settings, “Body of Her” explicitly reports physical realism failures such as objects appearing abruptly, cut paper reattaching, hand anatomy errors, and left-right confusion [2408.02879]. In robotics, MuGen identifies difficulty with standing still, limited world-model fidelity, and incomplete sim-to-real adaptation as open problems [2605.24592].

Across these strands, the likely research trajectory is clear. Geometry alone is insufficient without articulation; articulation alone is insufficient without contact and temporal coherence; temporal coherence alone is insufficient without physical executability. This suggests that the most durable future formulations of HumanoidGen will likely combine explicit structural priors such as SMPL-X, imGHUM, or skeleton fields; efficient 3D representations such as tri-planes or Gaussian splats; multimodal conditioning from text, pose, geometry, and interaction state; and some form of physics-grounded refinement or control loop [2212.05321], [2508.14120].

Source: https://www.emergentmind.com/topics/humanoidgen