---
title: 'HumanGaussian: 3D Human Models via Gaussian Splatting'
url: https://www.emergentmind.com/topics/humangaussian
type: topic
---

# HumanGaussian: 3D Human Models via Gaussian Splatting

Searching arXiv for recent papers on HumanGaussian and closely related human Gaussian splatting methods.
HumanGaussian denotes a family of human-centered 3D Gaussian splatting formulations in which a person is represented by a set of 3D Gaussians and rendered by explicit splatting rather than volumetric ray marching. In the literature, the term appears both as the title of a specific text-driven generation method, "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" [2311.17061], and as a broader research direction encompassing generalizable sparse-view reconstruction, animatable avatars, and feature-aware human representations such as "Generalizable Human Gaussians for Sparse View Synthesis" [2407.12777], "Generalizable Human Gaussians from Single-View Image" [2406.06050], and "HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features" [2411.03086]. Across these works, the common premise is that humans can be modeled as explicit Gaussian primitives carrying geometry and appearance, often coupled with human priors such as SMPL or SMPL-X, and optimized or predicted for tasks including novel-view rendering, novel-pose animation, dense pose estimation, and text- or image-conditioned 3D generation.

## 1. Terminological scope and research lineage

The name "HumanGaussian" has two closely related uses in the literature. First, it names the text-to-3D method "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" [2311.17061], which generates 3D humans from text prompts by combining 3D Gaussian Splatting with Structure-Aware SDS, RGB-depth joint guidance, and Annealed Negative Prompt Guidance. Second, the term functions more broadly as a descriptor for human-focused Gaussian splatting systems, including generalizable reconstruction from sparse views [2407.12777], single-image human Gaussian prediction [2406.06050], integrated biomechanical feature prediction [2411.03086], animatable Gaussian avatars [2312.02973], and monocular or sparse-view avatar priors [2506.06645].

Within this broader usage, the central shift is from scene-specific Gaussian optimization toward human-specific structure. Several papers explicitly anchor Gaussians to SMPL or SMPL-X, use UV-aligned parameterizations, or deform canonical Gaussians through linear blend skinning. For example, "Generalizable Human Gaussians for Sparse View Synthesis" reformulates the learning of 3D Gaussian parameters into a regression process defined on the 2D UV space of a human template and introduces a multi-scaffold to represent offset details [2407.12777]. "Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modeling" uses a UV-aligned latent identity map and a disentangled Multi-Head U-Net to predict Gaussian attributes conditioned on pose and view [2506.06645].

This suggests that HumanGaussian research is best understood as the human-specialized branch of 3D Gaussian splatting. A plausible implication is that the term increasingly refers less to a single algorithm and more to a design space: explicit Gaussian primitives, human priors, and task-specific conditioning for rendering, reconstruction, or animation.

## 2. Core representation and rendering model

Most HumanGaussian methods inherit the standard 3D Gaussian representation. "HFGaussian" defines a 3D Gaussian as
\[
G(x) = \exp\left(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)\right),
\]
with mean \(\mu \in \mathbb{R}^3\), covariance \(\Sigma \in \mathbb{R}^{3\times3}\), color \(c\), and opacity \(\alpha\), and factorizes covariance as
\[
\Sigma = R S S^T R^T,
\]
where \(R \in SO(3)\) is rotation and \(S\) is diagonal with scales along principal axes [2411.03086]. Closely related formulations appear in GauHuman [2312.02973], Animatable 3D Gaussian [2311.16482], SIGMAN [2504.06982], and the sparse-view generalizable human Gaussian model [2407.12777].

Projection and rasterization also follow standard 3DGS practice. HFGaussian gives the 2D covariance as
\[
\Sigma' = J W \Sigma W^T J^T,
\]
where \(W\) is the view transform and \(J\) the Jacobian of the projective transform’s affine approximation, and uses ordered alpha compositing
\[
C = \sum_{i \in \mathcal{N}} c_i \alpha'_i \prod_{j=1}^{i-1}(1 - \alpha'_j)
\]
for pixels overlapped by depth-sorted Gaussians [2411.03086]. Comparable compositing equations are stated in GauHuman [2312.02973], Animatable 3D Gaussian [2311.16482], AniGaussian [2502.19441], and HumanGaussian [2311.17061].

What distinguishes HumanGaussian systems from generic 3DGS is not the renderer but the parameterization and conditioning of the Gaussian set. In "Generalizable Human Gaussians from Single-View Image," a human is represented by a set \(G=\{G_i\}\), where each \(G_i=\{x_i, s_i, r_i, \alpha_i, c_i\}\), and the model predicts \(65{,}536\) Gaussians per view at \(256\times256\) output resolution, or \(131{,}072\) Gaussians in the two-view refinement stage [2406.06050]. In "Generalizable Human Gaussians for Sparse View Synthesis," Gaussians are placed on SMPL-X scaffolds and their parameters are regressed in UV space [2407.12777]. In SIGMAN, a Gaussian primitive is \(g=(\mu, s, r, c, \alpha)\), but canonical parameters are defined as offsets from a SMPL-X template in T-pose [2504.06982].

A recurrent misconception is that Gaussian splatting alone provides human structure. The literature instead shows that explicit human structure usually enters through template anchoring, skinning, UV alignment, point-cloud regression, or feature supervision rather than the Gaussian primitive itself [2407.12777] [2411.03086] [2506.06645].

## 3. Human priors, canonical spaces, and articulation

A major organizing principle in HumanGaussian research is the use of canonical human space. GauHuman represents the 3D human in canonical space and transforms 3D Gaussians from canonical space to posed space with linear blend skinning [2312.02973]. For a canonical-space point \(\mathbf{p}^c\), its posed version is given by SMPL-based LBS, and GauHuman extends this to Gaussian means and covariances:
\[
\mathbf{p}^{t} = \mathbf{G}(\mathbf{J}^t, \boldsymbol{\theta}^t)\,\mathbf{p}^{c}
              + \mathbf{b}(\mathbf{J}^t, \boldsymbol{\theta}^t, \boldsymbol{\beta}^t),
\]
\[
\mathbf{\Sigma}^{t} = \mathbf{G}(\mathbf{J}^t, \boldsymbol{\theta}^t)\,\mathbf{\Sigma}^{c}\,\mathbf{G}(\mathbf{J}^t, \boldsymbol{\theta}^t)^{T}
\]
[2312.02973].

Animatable 3D Gaussian adopts the same canonical-versus-posed decomposition. It defines a skinned Gaussian
\[
P_{\text{skin}} = \{ x_0, R, S, \alpha_0, SH, \mathbf{w}, \delta_x \},
\]
with fixed skinning weights from a standard human model, a learned vertex displacement \(\delta_x\), and pose-driven LBS of Gaussian centers and orientations [2311.16482]. AniGaussian likewise decomposes motion into non-rigid local deformation and rigid SMPL-based deformation, with a pose guided deformation strategy that constrains the dynamic Gaussian avatar with SMPL pose guidance [2502.19441].

Other works embed the prior in UV space rather than directly in skeletal transforms. "Generalizable Human Gaussians for Sparse View Synthesis" anchors Gaussians on the SMPL-X mesh surface and multiple dilated scaffolds, with one Gaussian per foreground UV pixel per scaffold level [2407.12777]. PGHM defines UV maps for Gaussian position offsets, scale, and color, and maps them to posed space via SMPL-X LBS [2506.06645]. HGM uses an SMPL-guided branch that projects image features onto a densified SMPL mesh, propagates them through a sparse 3D volume, and fuses them with an image-based UNet branch to infer human Gaussians from a single image [2406.06050].

These designs separate the field into two broad classes. One class uses explicit deformation of canonical Gaussians via SMPL or SMPL-X for animatable avatars [2312.02973] [2311.16482] [2502.19441] [2506.06645]. The other class uses SMPL or SMPL-X as a structured prior for generalizable feed-forward Gaussian prediction [2407.12777] [2406.06050] [2411.03086]. This suggests that HumanGaussian methods treat the body model less as the final representation than as an organizing coordinate system for Gaussian placement, motion, or supervision.

## 4. Methodological paradigms

HumanGaussian methods now occupy several distinct paradigms. Optimization-based avatar systems reconstruct a subject from monocular or multi-view video by optimizing Gaussians directly. GauHuman reports training speed of 1–2 minutes per subject and rendering up to 189 FPS at 1024p resolution, with about \(13\)k 3D Gaussians [2312.02973]. Animatable 3D Gaussian reports fast reconstruction in seconds, real-time rendering around \(170\) FPS at \(540\times540\), and scalability to multi-human scenes [2311.16482]. AniGaussian targets animatable clothed human avatars from monocular RGB video, with approximately 30 minutes of training and about 45 FPS rendering on RTX 4090 [2502.19441].

Generalizable feed-forward reconstruction replaces per-subject optimization with learned Gaussian prediction. "Generalizable Human Gaussians for Sparse View Synthesis" learns Gaussian parameters from sparse views in a feed-forward manner and reports strong within-dataset and cross-dataset generalization, outperforming NHP and NIA in LPIPS and FID while remaining far faster than NeRF-like rendering [2407.12777]. "Generalizable Human Gaussians from Single-View Image" introduces a generate-then-refine pipeline with SMPL and diffusion priors, renders a coarse back view, refines it with a ControlNet-based diffusion model, and reconstructs refined Human Gaussians from the front view and refined back view [2406.06050]. On HuMMan, it reports PSNR \(23.86\), SSIM \(0.95\), LPIPS \(0.062\), and 382 FPS, compared with SHERF at PSNR \(20.83\), SSIM \(0.89\), LPIPS \(0.12\), and 1.33 FPS [2406.06050].

Feature-aware human representations extend Gaussian outputs beyond RGB. HFGaussian is a human-centric 3D Gaussian splatting framework that synthesizes novel-view RGB images, predicts 3D skeleton or keypoints, predicts 2D keypoints, and predicts dense pose from a few RGB views, all in real time at about 25 FPS and in a generalizable fashion [2411.03086]. It uses feature splatting, in which each Gaussian carries a latent feature vector rendered via the same alpha compositing rule as colors, and a DGCNN+PointNet hybrid pose regressor operating on a point cloud derived from depth maps [2411.03086].

Text-driven HumanGaussian methods form another paradigm. "HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting" uses SMPL-X initialization, a structure-aware SDS with RGB and depth branches, Annealed Negative Prompt Guidance, and a size-based prune-only phase to generate 3D humans from text in about one hour on a single A100 [2311.17061]. Its user study reports the highest mean opinion scores among compared methods for Texture Quality, Geometry Quality, and Text Alignment [2311.17061].

Large-scale latent generative paradigms have recently emerged. SIGMAN compresses multi-view images into Gaussian latents with a UV-structured VAE and uses a DiT-based conditional generator to produce 3D human Gaussians from an image or text condition. It is trained on HGS-1M, a dataset with \(1\) million 3D Gaussian assets, more than \(90\)M rendered views, and about \(110\)K identities [2504.06982]. This suggests a shift from direct regression toward distribution learning in a structured HumanGaussian latent space.

## 5. Representative systems and empirical profile

The literature spans several use cases and empirical regimes.

| System | Input/task | Key mechanism | Reported outcomes |
|---|---|---|---|
| HFGaussian [2411.03086] | Sparse RGB views | Generalizable Gaussian backbone, feature splatting, point-cloud pose regressor | 24.37 FPS; THuman2.0: PSNR 32.43, Dense pose MSE 0.0017, MPJPE 0.0704, PCK@0.2 0.8707 |
| GHG [2407.12777] | Sparse views | UV-space regression on SMPL-X scaffolds, inpainting, multi-scaffold output | Better LPIPS/FID than NHP and NIA; 4 FPS with inpainting, 15 FPS without |
| HGM [2406.06050] | Single image | UNet + SMPL-guided branch + ControlNet back-view refinement | HuMMan: PSNR 23.86, SSIM 0.95, LPIPS 0.062, 382 FPS |
| GauHuman [2312.02973] | Monocular video | Canonical-space Gaussians, LBS refinement, KL-guided split/clone and merge | ~13k Gaussians, 1–2 min training, up to 189 FPS |
| PGHM [2506.06645] | Monocular video | UV-aligned latent identity map, disentangled Multi-Head U-Net | ~20 min personalization; NeuMan: PSNR 31.85, SSIM 0.987, LPIPS 0.0171 |
| HumanGaussian [2311.17061] | Text prompt | Structure-Aware SDS, RGB+depth diffusion, annealed negative guidance | ~1 hour on one A100; best user-study scores among listed baselines |
| SIGMAN [2504.06982] | Image/text-to-3D generation | UV-structured VAE + MM-DiT + HGS-1M | HGS-1M with 1M Gaussian assets; image-to-3D PSNR 25.45, LPIPS 0.044, SSIM 0.939 |

Several cross-paper patterns are clear. Optimization-based systems still achieve very strong per-subject quality and control [2312.02973] [2502.19441], but feed-forward systems now offer broad generalization and near-real-time or real-time inference [2407.12777] [2406.06050] [2411.03086]. Parametric priors reduce adaptation time dramatically: PGHM reports about 20 minutes per subject, which is much lower than methods requiring hours or days of optimization [2506.06645]. Large-scale latent modeling pushes the field toward foundation-style HumanGaussian generation, with SIGMAN explicitly framing the task as latent distribution learning rather than direct 2D-to-3D regression [2504.06982].

It would be misleading to compare all metrics directly across papers because datasets, task formulations, and outputs differ. A plausible implication is that HumanGaussian research is no longer a single benchmark race; it is a collection of related representation strategies spanning rendering, pose estimation, generation, and animation.

## 6. Limitations, controversies, and open directions

Several limitations recur across the literature. Many methods remain single-person systems. HFGaussian is trained and evaluated on single-person scenes and identifies extension to multi-person scenes as an open direction [2411.03086]. CrowdGaussian directly targets multi-person 3D Gaussian crowd reconstruction from a single image, addressing occlusion, low clarity, and many diverse appearances with an occlusion-adapted large human model and Self-Calibrated Learning for diffusion refinement [2603.17779]. This suggests that crowd-scale HumanGaussian modeling is emerging but not yet standard.

Dependence on human priors is both a strength and a constraint. PGHM, AniGaussian, GauHuman, and related avatar models depend on reasonably accurate SMPL or SMPL-X fitting and often struggle with very loose or highly non-rigid clothing [2506.06645] [2502.19441]. HGM notes robustness to imperfect SMPL estimates because SMPL enters as a guiding feature volume rather than a hard runtime constraint, but still identifies the prior as important for plausible geometry [2406.06050]. A common misconception is that Gaussian splatting eliminates the need for parametric body priors; in practice, most high-performing human systems still exploit such priors heavily.

Another limitation concerns articulation and editability. HFGaussian integrates biomechanics through point-cloud-based pose regression and DensePose embeddings, but it does not provide explicit rigging or skinning of Gaussians for re-posing [2411.03086]. By contrast, GauHuman, Animatable 3D Gaussian, AniGaussian, and PGHM are designed for animatable avatars through LBS or pose-guided deformation [2312.02973] [2311.16482] [2502.19441] [2506.06645]. Thus, not all HumanGaussian methods are avatar models in the strong sense of novel-pose control.

Generative methods face their own issues. The text-driven HumanGaussian paper identifies over-saturation, floating artifacts, and unstable SDS gradients as central obstacles, and addresses them through negative prompt annealing and a prune-only phase [2311.17061]. Single-image and multi-view-diffusion methods identify view inconsistency and facial distortion as persistent problems; MVD-HuGaS introduces a camera alignment module and a depth-based Facial Distortion Mitigation module to counter precisely these effects [2503.08218].

Looking forward, the field is moving in at least four directions. First, integrated semantics and biomechanics, exemplified by HFGaussian, turn Gaussians into carriers of dense pose, keypoints, or other human features rather than RGB alone [2411.03086]. Second, scalable latent generative modeling, exemplified by SIGMAN, uses millions of Gaussian assets and transformer-based latent generation [2504.06982]. Third, stronger generalizable priors, exemplified by PGHM and Human Gaussian Graph models, attempt to amortize per-subject optimization while preserving animatability [2506.06645] [2507.18758]. Fourth, multi-person and crowd reconstruction are beginning to extend HumanGaussian beyond isolated subjects [2603.17779].

Taken together, HumanGaussian now refers to a mature and rapidly diversifying research area: explicit Gaussian human representations anchored by human priors, rendered efficiently by splatting, and increasingly adapted to generalization, animation, semantics, and large-scale generation.

Source: https://www.emergentmind.com/topics/humangaussian