HumanoidGen: Human Generation & Animation

• HumanoidGen is a research field that encompasses methods for generating, reconstructing, and animating humanoid figures across 2D images, 3D avatars, rigged assets, and robotic behaviors.
• The field leverages diverse representations such as SMPL-X, tri-planes, and 3D Gaussian Splatting to enable multi-modal conditioning and precise control over pose, shape, and texture.
• Recent research in HumanoidGen demonstrates rapid progress with state-of-the-art metrics and practical improvements in animation, interaction, and physics-grounded control.

Searching arXiv for papers explicitly relevant to "HumanoidGen" and adjacent humanoid generation/animation/control work. Searching for "humanoid generation 3D human generation animatable avatars Gaussian splatting diffusion". arXiv search query: humanoid generation 3D human generation animatable avatars gaussian splatting diffusion “HumanoidGen” is an Editor’s term for a research area concerned with generating, reconstructing, animating, and controlling humanoid entities across 2D images, 3D avatars, rigged assets, contact-rich multi-human interactions, and physically grounded robot behaviors. In the available literature, no single canonical method defines the term; notably, HumanGen explicitly states that its paper does not use the name “HumanoidGen,” which suggests that the label is better understood as an umbrella for related methods than as a standardized model family (Jiang et al., 2022). Under that umbrella, the field spans pose-conditioned human image synthesis, text-driven 3D human generation, expressive SMPL-X-controlled avatars, anthropometric human modeling, contact-guided interaction synthesis, 4D human-scene generation, locomotion controllers, humanoid-object interaction generation, and whole-body data generation for imitation learning (Sarkar et al., 2021, Kolotouros et al., 2023, Xu et al., 2023, Ivashechkin et al., 4 Jun 2025, Li et al., 13 Aug 2025, Feng et al., 23 May 2026).

1. Scope and research taxonomy

The literature grouped here divides into several technically distinct but increasingly interconnected strands.

Strand	Core problem	Representative papers
2D human generation	Pose-conditioned image synthesis and part control	HumanGAN (Sarkar et al., 2021)
3D human and avatar generation	Text-driven humans, radiance fields, Gaussian avatars	HumanGen (Jiang et al., 2022), DreamHuman (Kolotouros et al., 2023), HumanGaussian (Liu et al., 2023), HuGeDiff (Ivashechkin et al., 4 Jun 2025)
Expressive and rigged assets	Body, face, hand control and animate-ready rigs	XAGen (Xu et al., 2023), AniGen (Huang et al., 9 Apr 2026)
Human interaction and 4D synthesis	Contact-guided partners and synthetic human dynamics	ContactGen (Gu et al., 2024), HumanGenesis (Li et al., 13 Aug 2025)
Physically grounded humanoids	Locomotion, humanoid-object interaction, loco-manipulation data generation	MuGen (Feng et al., 23 May 2026), SimGenHOI (Lin et al., 18 Aug 2025), HumanoidMimicGen (Lin et al., 26 May 2026)

At the 2D end, HumanGAN formulates a conditional VAE-GAN for dressed human images with DensePose conditioning, pose-normalized UV appearance, and part-based latent codes; it supports global appearance sampling, pose transfer, parts transfer, garment transfer, and part sampling in one framework (Sarkar et al., 2021). At the 3D end, HumanGen, DreamHuman, HumanGaussian, and HuGeDiff differ primarily in representation—tri-planes and SDF, pose-conditioned NeRF, 3D Gaussian Splatting optimized by SDS, and diffusion directly in 3DGS parameter space, respectively (Jiang et al., 2022, Kolotouros et al., 2023, Liu et al., 2023, Ivashechkin et al., 4 Jun 2025).

A plausible implication is that “HumanoidGen” is less a single architecture than a convergence zone: image generation, 3D reconstruction, avatar animation, and robot control increasingly share representations, priors, and conditioning mechanisms.

2. Representational foundations

A defining characteristic of this literature is the choice of representation. HumanGAN uses a pose-normalized SMPL UV texture map, DensePose part assignments, and a part-structured latent space with $M=24$ body parts and latent dimension $N=16$ per part, yielding $512 \times 512$ human images with localized semantic control (Sarkar et al., 2021). HumanGen instead couples a pre-trained 2D human StyleGAN with an EG3D-style tri-plane generator, a PIFuHD-based reconstructor, an SDF geometry branch, and a two-stage texture blending mechanism, enabling $360^\circ$ free-view rendering and direct $512 \times 256$ rendering without a separate super-resolution head (Jiang et al., 2022).

DreamHuman represents an avatar as a pose- and shape-conditioned NeRF over imGHUM semantic signed distance features, so that pose changes can be applied after optimization and the avatar remains animatable (Kolotouros et al., 2023). HumanGaussian and HuGeDiff both adopt 3D Gaussian Splatting, but in different roles. HumanGaussian initializes approximately $100k$ Gaussians from a posed SMPL-X mesh and optimizes them with structure-aware SDS using RGB and depth branches (Liu et al., 2023). HuGeDiff builds a weakly supervised pipeline in which approximately $10k$ photorealistic humans are generated with FLUX, reconstructed into SMPL-X-anchored 3DGS, and then used to train a text-conditioned diffusion model directly over 3DGS parameters; its densified mesh reaches $84{,}317$ vertices (Ivashechkin et al., 4 Jun 2025).

Rigged generation pushes representation further. XAGen uses multi-scale, multi-part tri-planes for body, face, and hands, inverse linear blend skinning driven by SMPL-X, and part-specific rendering and discriminators (Xu et al., 2023). AniGen formalizes animate-ready generation as joint prediction of shape, skeleton, and skinning through unified $S^3$ fields over a shared spatial domain, augmented by a confidence-decaying skeleton field and a dual skin feature field that allows a fixed-architecture network to support arbitrary rig complexity (Huang et al., 9 Apr 2026). AnthroNet occupies a different point in the design space: it models humans by 36 anthropometric measurements plus a sex indicator and outputs a high-resolution mesh with approximately $36{,}078$ vertices and $N=16$0 triangles, then applies a learned skinner for posing (Picetti et al., 2023).

This suggests that representation choice now encodes a research commitment. Tri-planes and NeRFs emphasize view-consistent rendering; 3DGS emphasizes efficiency and explicit surface-aligned detail; anthropometric parameterizations emphasize measurable control; unified rig fields emphasize animation-readiness.

3. Conditioning and controllability

Control signals in HumanoidGen-style systems range from geometry-preserving part codes to text, pose, interaction labels, anthropometrics, and multimodal dialogue context. HumanGAN disentangles appearance from pose by encoding appearance in canonical UV space and warping it to a target DensePose; because appearance is factorized by body parts, the model can hold most of a person fixed while resampling only the head, garment regions, or other parts (Sarkar et al., 2021).

Text conditioning dominates recent 3D work. DreamHuman takes text-only input and optimizes a realistic animatable 3D human avatar without paired text–3D supervision; it samples poses from a motion prior during training and uses semantic zoom on six regions—head, upper body, lower body, midsection, left arm, and right arm—to improve local detail (Kolotouros et al., 2023). HumanGaussian conditions on a text prompt $N=16$1 and, optionally, a pose skeleton $N=16$2; its diffusion source model jointly denoises RGB and depth conditioned on pose and text, and it uses annealed negative prompt guidance to mitigate over-saturation (Liu et al., 2023). HuGeDiff preserves text conditioning through its entire loop: text prompts drive FLUX image synthesis, the same prompts condition the 3D diffusion model, and classifier-free guidance controls the strength of semantic alignment (Ivashechkin et al., 4 Jun 2025).

Other systems broaden the control vocabulary. XAGen uses SMPL-X parameters for body pose, facial expression, jaw pose, hand pose, and body shape, with separate body, face, and hand cameras and discriminators (Xu et al., 2023). AnthroNet conditions on a $N=16$3-dimensional vector comprising 36 anthropometric measurements plus sex, encoded with random Fourier features, and can then produce bodies in arbitrary pose (Picetti et al., 2023). ContactGen takes as input a 3D partner human and an interaction label, predicts potential contact regions between the two humans, and uses those estimated regions as guidance inside a guided diffusion model (Gu et al., 2024). SimGenHOI conditions on text prompts, object geometry encoded as a Basis Point Set, sparse object waypoints, and the initial humanoid pose to generate key whole-body actions for humanoid-object interaction (Lin et al., 18 Aug 2025).

The broadest conditioning interface appears in “Body of Her,” which extends a pre-trained LLM into a real-time, duplex, interactive multimodal model that integrates audio and visual inputs and is further modulated by text prompts, trajectory signals, and identity embeddings; the reported training corpus comprises approximately $N=16$4 hours of audio, around $N=16$5 hours of video data, and about $N=16$6 alignment samples (Ao, 2024). A reasonable interpretation is that the field is moving from single-modality conditioning toward compound control stacks in which language, pose, geometry, and interaction state coexist.

4. Animation, interaction, and 4D dynamics

A central transition in this literature is from static generation to animatable and interactive generation. DreamHuman explicitly targets animatable 3D avatars rather than fixed assets: the NeRF is conditioned on pose and shape, and the optimized avatar can be re-posed by changing $N=16$7 without retraining (Kolotouros et al., 2023). XAGen extends this paradigm to expressive full-body avatars, adding independent control over body, face, jaw, and hands, together with multi-part rendering and multi-part discriminators to improve small-scale regions such as eyes, mouth, and fingers (Xu et al., 2023).

ContactGen shifts attention from single avatars to multi-human interaction. Its task is defined by a partner human and an interaction label; the method introduces a contact prediction module that adaptively estimates potential contact regions between two humans according to the interaction label, then enforces those regions inside a guided diffusion framework. The reported evaluation on CHI3D emphasizes physically plausible and diverse poses relative to comparison methods (Gu et al., 2024). This is notable because the conditioning variable is not simply “pose” but “contact structure.”

HumanGenesis moves to 4D human-scene synthesis. Its four-agent system comprises a Reconstructor, a Critique Agent, a Pose Guider, and a Video Harmonizer. The Reconstructor builds 3D-consistent human-scene representations from monocular video using 3D Gaussian Splatting and deformation decomposition; the Critique Agent uses multi-round MLLM-based reflection to localize and refine poor regions; the Pose Guider supplies expressive pose sequences; and the Video Harmonizer applies a hybrid rendering-plus-diffusion pipeline, coupled to a Back-to-4D feedback loop that refines the underlying geometry (Li et al., 13 Aug 2025). In “Body of Her,” the output space broadens again: the model generates speech, full-body movements for talking, responding, idling, and manipulation, and does so in real time under duplex conversational constraints (Ao, 2024).

AniGen addresses a related but distinct bottleneck: even high-quality 3D generators typically produce static assets, after which post-hoc auto-rigging is brittle. Its two-stage flow-matching pipeline first synthesizes a sparse structural scaffold and then dense geometry plus articulation in a structured latent space, directly producing animate-ready assets rather than requiring a later rigging pass (Huang et al., 9 Apr 2026).

Taken together, these systems indicate a shift from “generate a human shape” toward “generate a controllable temporal entity”: one that can be re-posed, re-targeted, brought into contact, inserted into scenes, or animated under explicit skeletal structure.

5. Physics-grounded humanoids and robotics

In robotics-oriented work, the term “generation” often refers not to image or mesh synthesis but to the synthesis of executable motion, physically valid morphology, or large demonstration corpora. MuGen is representative: it learns a multi-skill locomotion controller for humanoid robots using VQ-VAEs trained with model-based reinforcement learning, then distills the privileged-information teacher into a deployable student policy. The same latent space supports both tracking of unseen human motions and a generative mode in which a prior encoder selects motion codes from state alone (Feng et al., 23 May 2026).

Humanoid-Gym provides the complementary infrastructure perspective. It is an Isaac Gym-based reinforcement-learning framework for humanoid locomotion with zero-shot sim-to-real transfer, verified on RobotEra’s XBot-S and XBot-L, and includes a sim-to-sim bridge from Isaac Gym to MuJoCo to test robustness across physics engines (Gu et al., 2024). “From Universal Humanoid Control to Automatic Physically Valid Character Creation” extends generation one step earlier in the pipeline: given pre-specified human motion sequences, it optimizes humanoid physical attributes so that the resulting body is physically valid and better suited to imitate those motions under a pretrained universal controller (Luo et al., 2022).

SimGenHOI fuses generative modeling and control more tightly. A Diffusion Transformer predicts key whole-body actions conditioned on text prompts, object geometry, sparse waypoints, and the initial humanoid pose; these key actions are interpolated into trajectories that a contact-aware RL policy tracks while correcting penetration and foot sliding. The model and policy are then mutually fine-tuned so that the generator shifts toward the controller’s feasible region and the controller shifts toward the generator’s motion distribution (Lin et al., 18 Aug 2025). HumanoidMimicGen addresses data scarcity for loco-manipulation by adapting contact-rich whole-body skills from a handful of source demonstrations, interleaving them with whole-body locomotion and manipulation planning, and using the resulting synthetic corpus to train whole-body visuomotor policies that outperform policies trained only on real-world data by $N=16$8 on a new nine-task benchmark (Lin et al., 26 May 2026).

A plausible implication is that a robotics interpretation of HumanoidGen is now emerging in which generation, planning, and control are no longer separate stages. Instead, the generator proposes sparse structure—motion codes, key actions, demonstrations, or body designs—and a controller or planner enforces dynamic validity.

6. Evaluation, limitations, and likely trajectories

Reported evaluations indicate rapid progress, but they also reveal where the field remains structurally weak.

Paper	Reported result	Context
HumanGen (Jiang et al., 2022)	FID 20.97, depth 0.0201	3D human radiance fields
HumanGaussian (Liu et al., 2023)	~1 hour per prompt on a single NVIDIA A100 40GB	Text-driven 3D humans
HuGeDiff (Ivashechkin et al., 4 Jun 2025)	0.45 minutes per sample	Text-to-3D human generation
XAGen (Xu et al., 2023)	FID 8.55 on DeepFashion	Expressive avatar generation
HumanGenesis (Li et al., 13 Aug 2025)	SSIM 0.891, PSNR 26.992, LPIPS 0.081	NeuMan reconstruction
MuGen (Feng et al., 23 May 2026)	SR$N=16$9 0.9535	Generative locomotion control
HumanoidMimicGen (Lin et al., 26 May 2026)	20% improvement over real-only training	Loco-manipulation imitation

The limitations are also strikingly consistent across subfields. HumanGaussian reports persistent weaknesses on hands and feet and attributes this partly to weaknesses inherited from current text-to-image models (Liu et al., 2023). DreamHuman notes geometry–albedo ambiguity, limited resolution, and the lack of explicit temporal supervision for clothing dynamics (Kolotouros et al., 2023). HuGeDiff identifies limited clothing pattern detail, limited hair modeling due to SMPL-X constraints, domain gap on real datasets, and unnatural shading under pose changes (Ivashechkin et al., 4 Jun 2025). XAGen still depends heavily on SMPL-X fitting quality and remains weak on loose clothing and inverse-skinning errors (Xu et al., 2023). AniGen generalizes to humanoids but still reports sensitivity in hands, face, and image-only conditioning, especially where rig precision or temporal coherence would be required (Huang et al., 9 Apr 2026). HumanGenesis highlights dynamic scenes, fine facial and hand detail, and extreme motions or viewpoints as failure cases (Li et al., 13 Aug 2025). In interactive agent settings, “Body of Her” explicitly reports physical realism failures such as objects appearing abruptly, cut paper reattaching, hand anatomy errors, and left-right confusion (Ao, 2024). In robotics, MuGen identifies difficulty with standing still, limited world-model fidelity, and incomplete sim-to-real adaptation as open problems (Feng et al., 23 May 2026).

Across these strands, the likely research trajectory is clear. Geometry alone is insufficient without articulation; articulation alone is insufficient without contact and temporal coherence; temporal coherence alone is insufficient without physical executability. This suggests that the most durable future formulations of HumanoidGen will likely combine explicit structural priors such as SMPL-X, imGHUM, or skeleton fields; efficient 3D representations such as tri-planes or Gaussian splats; multimodal conditioning from text, pose, geometry, and interaction state; and some form of physics-grounded refinement or control loop (Jiang et al., 2022, Lin et al., 18 Aug 2025).