HumanGaussian: 3D Human Models via Gaussian Splatting

• HumanGaussian is a method that represents humans with sets of explicit 3D Gaussians anchored by priors like SMPL or SMPL-X.
• It leverages techniques such as UV alignment, linear blend skinning, and feature splatting to enable efficient novel-view synthesis and pose estimation.
• Applications include text-driven 3D human generation, sparse-view reconstruction, and real-time avatar animation with high FPS and low error metrics.

Searching arXiv for papers on HumanGaussian and closely related human Gaussian splatting methods. HumanGaussian denotes a family of human-centered 3D Gaussian splatting formulations in which a person is represented by a set of 3D Gaussians and rendered by explicit splatting rather than volumetric ray marching. In the literature, the term appears both as the title of a specific text-driven generation method, "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" (Liu et al., 2023), and as a broader research direction encompassing generalizable sparse-view reconstruction, animatable avatars, and feature-aware human representations such as "Generalizable Human Gaussians for Sparse View Synthesis" (Kwon et al., 2024), "Generalizable Human Gaussians from Single-View Image" (Chen et al., 2024), and "HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features" (Dey et al., 2024). Across these works, the common premise is that humans can be modeled as explicit Gaussian primitives carrying geometry and appearance, often coupled with human priors such as SMPL or SMPL-X, and optimized or predicted for tasks including novel-view rendering, novel-pose animation, dense pose estimation, and text- or image-conditioned 3D generation.

1. Terminological scope and research lineage

The name "HumanGaussian" has two closely related uses in the literature. First, it names the text-to-3D method "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" (Liu et al., 2023), which generates 3D humans from text prompts by combining 3D Gaussian Splatting with Structure-Aware SDS, RGB-depth joint guidance, and Annealed Negative Prompt Guidance. Second, the term functions more broadly as a descriptor for human-focused Gaussian splatting systems, including generalizable reconstruction from sparse views (Kwon et al., 2024), single-image human Gaussian prediction (Chen et al., 2024), integrated biomechanical feature prediction (Dey et al., 2024), animatable Gaussian avatars (Hu et al., 2023), and monocular or sparse-view avatar priors (Peng et al., 7 Jun 2025).

Within this broader usage, the central shift is from scene-specific Gaussian optimization toward human-specific structure. Several papers explicitly anchor Gaussians to SMPL or SMPL-X, use UV-aligned parameterizations, or deform canonical Gaussians through linear blend skinning. For example, "Generalizable Human Gaussians for Sparse View Synthesis" reformulates the learning of 3D Gaussian parameters into a regression process defined on the 2D UV space of a human template and introduces a multi-scaffold to represent offset details (Kwon et al., 2024). "Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modeling" uses a UV-aligned latent identity map and a disentangled Multi-Head U-Net to predict Gaussian attributes conditioned on pose and view (Peng et al., 7 Jun 2025).

This suggests that HumanGaussian research is best understood as the human-specialized branch of 3D Gaussian splatting. A plausible implication is that the term increasingly refers less to a single algorithm and more to a design space: explicit Gaussian primitives, human priors, and task-specific conditioning for rendering, reconstruction, or animation.

2. Core representation and rendering model

Most HumanGaussian methods inherit the standard 3D Gaussian representation. "HFGaussian" defines a 3D Gaussian as

$$G(x) = \exp\left(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)\right),$$

with mean $\mu \in \mathbb{R}^3$, covariance $\Sigma \in \mathbb{R}^{3\times3}$, color $c$, and opacity $\alpha$, and factorizes covariance as

$\Sigma = R S S^T R^T,$

where $R \in SO(3)$ is rotation and $S$ is diagonal with scales along principal axes (Dey et al., 2024). Closely related formulations appear in GauHuman (Hu et al., 2023), Animatable 3D Gaussian (Liu et al., 2023), SIGMAN (Yang et al., 9 Apr 2025), and the sparse-view generalizable human Gaussian model (Kwon et al., 2024).

Projection and rasterization also follow standard 3DGS practice. HFGaussian gives the 2D covariance as

$\Sigma' = J W \Sigma W^T J^T,$

where $W$ is the view transform and $\mu \in \mathbb{R}^3$0 the Jacobian of the projective transform’s affine approximation, and uses ordered alpha compositing

$\mu \in \mathbb{R}^3$1

for pixels overlapped by depth-sorted Gaussians (Dey et al., 2024). Comparable compositing equations are stated in GauHuman (Hu et al., 2023), Animatable 3D Gaussian (Liu et al., 2023), AniGaussian (Li et al., 24 Feb 2025), and HumanGaussian (Liu et al., 2023).

What distinguishes HumanGaussian systems from generic 3DGS is not the renderer but the parameterization and conditioning of the Gaussian set. In "Generalizable Human Gaussians from Single-View Image," a human is represented by a set $\mu \in \mathbb{R}^3$2, where each $\mu \in \mathbb{R}^3$3, and the model predicts $\mu \in \mathbb{R}^3$4 Gaussians per view at $\mu \in \mathbb{R}^3$5 output resolution, or $\mu \in \mathbb{R}^3$6 Gaussians in the two-view refinement stage (Chen et al., 2024). In "Generalizable Human Gaussians for Sparse View Synthesis," Gaussians are placed on SMPL-X scaffolds and their parameters are regressed in UV space (Kwon et al., 2024). In SIGMAN, a Gaussian primitive is $\mu \in \mathbb{R}^3$7, but canonical parameters are defined as offsets from a SMPL-X template in T-pose (Yang et al., 9 Apr 2025).

A recurrent misconception is that Gaussian splatting alone provides human structure. The literature instead shows that explicit human structure usually enters through template anchoring, skinning, UV alignment, point-cloud regression, or feature supervision rather than the Gaussian primitive itself (Kwon et al., 2024, Dey et al., 2024, Peng et al., 7 Jun 2025).

3. Human priors, canonical spaces, and articulation

A major organizing principle in HumanGaussian research is the use of canonical human space. GauHuman represents the 3D human in canonical space and transforms 3D Gaussians from canonical space to posed space with linear blend skinning (Hu et al., 2023). For a canonical-space point $\mu \in \mathbb{R}^3$8, its posed version is given by SMPL-based LBS, and GauHuman extends this to Gaussian means and covariances: $\mu \in \mathbb{R}^3$9

$\Sigma \in \mathbb{R}^{3\times3}$0

(Hu et al., 2023).

Animatable 3D Gaussian adopts the same canonical-versus-posed decomposition. It defines a skinned Gaussian

$\Sigma \in \mathbb{R}^{3\times3}$1

with fixed skinning weights from a standard human model, a learned vertex displacement $\Sigma \in \mathbb{R}^{3\times3}$2, and pose-driven LBS of Gaussian centers and orientations (Liu et al., 2023). AniGaussian likewise decomposes motion into non-rigid local deformation and rigid SMPL-based deformation, with a pose guided deformation strategy that constrains the dynamic Gaussian avatar with SMPL pose guidance (Li et al., 24 Feb 2025).

Other works embed the prior in UV space rather than directly in skeletal transforms. "Generalizable Human Gaussians for Sparse View Synthesis" anchors Gaussians on the SMPL-X mesh surface and multiple dilated scaffolds, with one Gaussian per foreground UV pixel per scaffold level (Kwon et al., 2024). PGHM defines UV maps for Gaussian position offsets, scale, and color, and maps them to posed space via SMPL-X LBS (Peng et al., 7 Jun 2025). HGM uses an SMPL-guided branch that projects image features onto a densified SMPL mesh, propagates them through a sparse 3D volume, and fuses them with an image-based UNet branch to infer human Gaussians from a single image (Chen et al., 2024).

These designs separate the field into two broad classes. One class uses explicit deformation of canonical Gaussians via SMPL or SMPL-X for animatable avatars (Hu et al., 2023, Liu et al., 2023, Li et al., 24 Feb 2025, Peng et al., 7 Jun 2025). The other class uses SMPL or SMPL-X as a structured prior for generalizable feed-forward Gaussian prediction (Kwon et al., 2024, Chen et al., 2024, Dey et al., 2024). This suggests that HumanGaussian methods treat the body model less as the final representation than as an organizing coordinate system for Gaussian placement, motion, or supervision.

4. Methodological paradigms

HumanGaussian methods now occupy several distinct paradigms. Optimization-based avatar systems reconstruct a subject from monocular or multi-view video by optimizing Gaussians directly. GauHuman reports training speed of 1–2 minutes per subject and rendering up to 189 FPS at 1024p resolution, with about $\Sigma \in \mathbb{R}^{3\times3}$3k 3D Gaussians (Hu et al., 2023). Animatable 3D Gaussian reports fast reconstruction in seconds, real-time rendering around $\Sigma \in \mathbb{R}^{3\times3}$4 FPS at $\Sigma \in \mathbb{R}^{3\times3}$5, and scalability to multi-human scenes (Liu et al., 2023). AniGaussian targets animatable clothed human avatars from monocular RGB video, with approximately 30 minutes of training and about 45 FPS rendering on RTX 4090 (Li et al., 24 Feb 2025).

Generalizable feed-forward reconstruction replaces per-subject optimization with learned Gaussian prediction. "Generalizable Human Gaussians for Sparse View Synthesis" learns Gaussian parameters from sparse views in a feed-forward manner and reports strong within-dataset and cross-dataset generalization, outperforming NHP and NIA in LPIPS and FID while remaining far faster than NeRF-like rendering (Kwon et al., 2024). "Generalizable Human Gaussians from Single-View Image" introduces a generate-then-refine pipeline with SMPL and diffusion priors, renders a coarse back view, refines it with a ControlNet-based diffusion model, and reconstructs refined Human Gaussians from the front view and refined back view (Chen et al., 2024). On HuMMan, it reports PSNR $\Sigma \in \mathbb{R}^{3\times3}$6, SSIM $\Sigma \in \mathbb{R}^{3\times3}$7, LPIPS $\Sigma \in \mathbb{R}^{3\times3}$8, and 382 FPS, compared with SHERF at PSNR $\Sigma \in \mathbb{R}^{3\times3}$9, SSIM $c$0, LPIPS $c$1, and 1.33 FPS (Chen et al., 2024).

Feature-aware human representations extend Gaussian outputs beyond RGB. HFGaussian is a human-centric 3D Gaussian splatting framework that synthesizes novel-view RGB images, predicts 3D skeleton or keypoints, predicts 2D keypoints, and predicts dense pose from a few RGB views, all in real time at about 25 FPS and in a generalizable fashion (Dey et al., 2024). It uses feature splatting, in which each Gaussian carries a latent feature vector rendered via the same alpha compositing rule as colors, and a DGCNN+PointNet hybrid pose regressor operating on a point cloud derived from depth maps (Dey et al., 2024).

Text-driven HumanGaussian methods form another paradigm. "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" uses SMPL-X initialization, a structure-aware SDS with RGB and depth branches, Annealed Negative Prompt Guidance, and a size-based prune-only phase to generate 3D humans from text in about one hour on a single A100 (Liu et al., 2023). Its user study reports the highest mean opinion scores among compared methods for Texture Quality, Geometry Quality, and Text Alignment (Liu et al., 2023).

Large-scale latent generative paradigms have recently emerged. SIGMAN compresses multi-view images into Gaussian latents with a UV-structured VAE and uses a DiT-based conditional generator to produce 3D human Gaussians from an image or text condition. It is trained on HGS-1M, a dataset with $c$2 million 3D Gaussian assets, more than $c$3M rendered views, and about $c$4K identities (Yang et al., 9 Apr 2025). This suggests a shift from direct regression toward distribution learning in a structured HumanGaussian latent space.

5. Representative systems and empirical profile

The literature spans several use cases and empirical regimes.

System	Input/task	Key mechanism	Reported outcomes
HFGaussian (Dey et al., 2024)	Sparse RGB views	Generalizable Gaussian backbone, feature splatting, point-cloud pose regressor	24.37 FPS; THuman2.0: PSNR 32.43, Dense pose MSE 0.0017, MPJPE 0.0704, [email protected] 0.8707
GHG (Kwon et al., 2024)	Sparse views	UV-space regression on SMPL-X scaffolds, inpainting, multi-scaffold output	Better LPIPS/FID than NHP and NIA; 4 FPS with inpainting, 15 FPS without
HGM (Chen et al., 2024)	Single image	UNet + SMPL-guided branch + ControlNet back-view refinement	HuMMan: PSNR 23.86, SSIM 0.95, LPIPS 0.062, 382 FPS
GauHuman (Hu et al., 2023)	Monocular video	Canonical-space Gaussians, LBS refinement, KL-guided split/clone and merge	~13k Gaussians, 1–2 min training, up to 189 FPS
PGHM (Peng et al., 7 Jun 2025)	Monocular video	UV-aligned latent identity map, disentangled Multi-Head U-Net	~20 min personalization; NeuMan: PSNR 31.85, SSIM 0.987, LPIPS 0.0171
HumanGaussian (Liu et al., 2023)	Text prompt	Structure-Aware SDS, RGB+depth diffusion, annealed negative guidance	~1 hour on one A100; best user-study scores among listed baselines
SIGMAN (Yang et al., 9 Apr 2025)	Image/text-to-3D generation	UV-structured VAE + MM-DiT + HGS-1M	HGS-1M with 1M Gaussian assets; image-to-3D PSNR 25.45, LPIPS 0.044, SSIM 0.939

Several cross-paper patterns are clear. Optimization-based systems still achieve very strong per-subject quality and control (Hu et al., 2023, Li et al., 24 Feb 2025), but feed-forward systems now offer broad generalization and near-real-time or real-time inference (Kwon et al., 2024, Chen et al., 2024, Dey et al., 2024). Parametric priors reduce adaptation time dramatically: PGHM reports about 20 minutes per subject, which is much lower than methods requiring hours or days of optimization (Peng et al., 7 Jun 2025). Large-scale latent modeling pushes the field toward foundation-style HumanGaussian generation, with SIGMAN explicitly framing the task as latent distribution learning rather than direct 2D-to-3D regression (Yang et al., 9 Apr 2025).

It would be misleading to compare all metrics directly across papers because datasets, task formulations, and outputs differ. A plausible implication is that HumanGaussian research is no longer a single benchmark race; it is a collection of related representation strategies spanning rendering, pose estimation, generation, and animation.

6. Limitations, controversies, and open directions

Several limitations recur across the literature. Many methods remain single-person systems. HFGaussian is trained and evaluated on single-person scenes and identifies extension to multi-person scenes as an open direction (Dey et al., 2024). CrowdGaussian directly targets multi-person 3D Gaussian crowd reconstruction from a single image, addressing occlusion, low clarity, and many diverse appearances with an occlusion-adapted large human model and Self-Calibrated Learning for diffusion refinement (Song et al., 18 Mar 2026). This suggests that crowd-scale HumanGaussian modeling is emerging but not yet standard.

Dependence on human priors is both a strength and a constraint. PGHM, AniGaussian, GauHuman, and related avatar models depend on reasonably accurate SMPL or SMPL-X fitting and often struggle with very loose or highly non-rigid clothing (Peng et al., 7 Jun 2025, Li et al., 24 Feb 2025). HGM notes robustness to imperfect SMPL estimates because SMPL enters as a guiding feature volume rather than a hard runtime constraint, but still identifies the prior as important for plausible geometry (Chen et al., 2024). A common misconception is that Gaussian splatting eliminates the need for parametric body priors; in practice, most high-performing human systems still exploit such priors heavily.

Another limitation concerns articulation and editability. HFGaussian integrates biomechanics through point-cloud-based pose regression and DensePose embeddings, but it does not provide explicit rigging or skinning of Gaussians for re-posing (Dey et al., 2024). By contrast, GauHuman, Animatable 3D Gaussian, AniGaussian, and PGHM are designed for animatable avatars through LBS or pose-guided deformation (Hu et al., 2023, Liu et al., 2023, Li et al., 24 Feb 2025, Peng et al., 7 Jun 2025). Thus, not all HumanGaussian methods are avatar models in the strong sense of novel-pose control.

Generative methods face their own issues. The text-driven HumanGaussian paper identifies over-saturation, floating artifacts, and unstable SDS gradients as central obstacles, and addresses them through negative prompt annealing and a prune-only phase (Liu et al., 2023). Single-image and multi-view-diffusion methods identify view inconsistency and facial distortion as persistent problems; MVD-HuGaS introduces a camera alignment module and a depth-based Facial Distortion Mitigation module to counter precisely these effects (Xiong et al., 11 Mar 2025).

Looking forward, the field is moving in at least four directions. First, integrated semantics and biomechanics, exemplified by HFGaussian, turn Gaussians into carriers of dense pose, keypoints, or other human features rather than RGB alone (Dey et al., 2024). Second, scalable latent generative modeling, exemplified by SIGMAN, uses millions of Gaussian assets and transformer-based latent generation (Yang et al., 9 Apr 2025). Third, stronger generalizable priors, exemplified by PGHM and Human Gaussian Graph models, attempt to amortize per-subject optimization while preserving animatability (Peng et al., 7 Jun 2025, Liu et al., 24 Jul 2025). Fourth, multi-person and crowd reconstruction are beginning to extend HumanGaussian beyond isolated subjects (Song et al., 18 Mar 2026).

Taken together, HumanGaussian now refers to a mature and rapidly diversifying research area: explicit Gaussian human representations anchored by human priors, rendered efficiently by splatting, and increasingly adapted to generalization, animation, semantics, and large-scale generation.