SMPL+H: Integrated Body & Hand Model

• SMPL+H is a combined full-body and hand model that unifies SMPL and MANO to represent body pose, shape, wrist motion, and finger articulation in a single mesh.
• It uses a low-dimensional pose embedding for hand articulation, enabling robust performance capture despite occlusions, noise, and low-resolution scans.
• Positioned as an intermediary between SMPL and SMPL-X, SMPL+H facilitates detailed motion capture and paves the way for advanced generative modeling techniques.

SMPL+H is a combined full-body and detailed-hand parametric human model formed by attaching the MANO hand model to the SMPL body model, thereby representing body pose, body shape, wrist motion, and finger articulation within a single articulated mesh (Romero et al., 2022). It was introduced to address the mismatch between full-body capture pipelines, which historically modeled the body but assumed unrealistic hands, and hand-specific methods, which did not model the body jointly. In later taxonomies of human mesh recovery, SMPL+H is treated as the hand-extended variant of SMPL and as an intermediate representational level between body-only SMPL and the more comprehensive SMPL-X, which adds face modeling in addition to body and hands (Liu et al., 2024).

1. Origins and problem setting

SMPL+H was motivated by the observation that humans move their hands and bodies together, whereas many capture and modeling methods had treated bodies and hands separately (Romero et al., 2022). That separation is particularly problematic in full-body 4D capture. The original formulation emphasizes three acquisition difficulties: hands are small in full-body scans, they are often partially occluded by the body or by the other hand, and the scan data are noisy. The same discussion notes artifacts such as finger webbing, missing geometry, and even complete disappearance of the hand or arm in some frames. In that setting, a body-only model such as SMPL is insufficient because it assumes something like an open, rigid hand, while hand-only tracking methods do not provide a unified body-hand representation.

The resulting design goal was not merely to append extra joints, but to build a single model that could support realistic performance capture under low-resolution, occlusion, and noise. This yields a model intended for coordinated body-hand motion rather than separate subsystems for torso and fingers. A plausible implication is that SMPL+H is best understood as a representational response to coupled articulation, not simply as a larger parameter vector.

2. Constituent models and parametric formulation

The body component of SMPL+H comes from SMPL, which the original paper restates in the form

$$M(\beta,\theta)=W(T_P(\beta,\theta),J(\beta),\theta,\mathcal{W}),$$

with

$T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$

Here $M$ is the final posed mesh, $W$ is the linear blend skinning function, $T_P$ is the posed template, $\bar{T}$ is the mean template, $B_S(\beta)$ is the shape blend shape term, $B_P(\theta)$ is the pose blend shape term, $J(\beta)$ gives joint locations from shape, $\mathcal{W}$ are the skinning weights, $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$0 are shape coefficients, and $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$1 are pose parameters (Romero et al., 2022).

The hand component is MANO, introduced in the same work as a hand-only model learned from high-resolution scans. The abstract characterizes MANO as learned from around 1000 high-resolution 3D scans of hands of 31 subjects in a wide variety of hand poses, while the detailed acquisition description reports capture with a 3dMDhand system over about 2018 scans of 31 subjects, including both left and right hands and many poses, including grasp taxonomy poses and object interaction poses (Romero et al., 2022). MANO is described as realistic, low-dimensional, compatible with standard graphics packages, and able to capture pose-dependent non-rigid hand deformation. Its learned components include a template mesh, shape space, kinematic tree, blend weights, joint regressor, and pose blend shapes.

The hand model mirrors the SMPL decomposition into identity-dependent and pose-dependent geometry. The shape displacement is

$T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$2

where $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$3 are shape coefficients and $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$4 are PCA basis vectors of hand shape. The pose-dependent correction is

$T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$5

where $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$6 is the number of hand parts or joints, $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$7 are the elements of the concatenated rotation matrices, $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$8 is the zero pose, and $T_P(\beta,\theta)=\bar{T}+B_S(\beta)+B_P(\theta).$9 are the pose blend shape vectors (Romero et al., 2022). The stated role of these pose blend shapes is to correct the over-smoothing and collapse behavior of plain linear blend skinning, especially at knuckles and finger bends.

A central feature of MANO is its low-dimensional pose embedding. The paper frames this through hand synergies: although hands have many degrees of freedom, natural motion lies near a low-dimensional manifold. MANO therefore uses PCA on axis-angle pose parameters, and the experiments report that 6 pose components per hand are a good trade-off between accuracy and robustness (Romero et al., 2022). This matters operationally because it stabilizes fitting when hand data are noisy or missing.

SMPL+H is formed by combining these ingredients. The paper states that it takes the finger articulation components from MANO—blend weights, pose-dependent deformations, joint regressor, and kinematic tree or joints—while taking the rest of the body components from SMPL, including the wrist. It also uses the SMPL body shape space for the hand shape as well, so that hand and body morphology remain correlated (Romero et al., 2022). The reported dimensional change is from 66 DOF for SMPL to 78 DOF for SMPL+H, with optimization remaining manageable because each hand pose is represented in a 6D linear embedding.

3. Learning the model and fitting full-body 4D sequences

The learning procedure for MANO is described as an iterative registration-and-retraining pipeline. Hand scans are captured, cleaned and cropped, registered into correspondence with a template, and then used to learn the statistical hand model (Romero et al., 2022). The registration stage minimizes

$M$0

where $M$1 is a scan-to-mesh data term, $M$2 is a coupling term between optimized registration and model, $M$3 is a pose prior, and $M$4 is a shape prior. The shape prior uses a Mahalanobis-like penalty in PCA space, and the pose prior is refined during training. The paper further describes a bootstrapped procedure: start with a crude hand model, register scans, manually curate the good registrations, retrain a better model, and repeat. The stated reason is that hand scans are difficult to align because of finger similarity, self-occlusion, object occlusion, and noise.

After curated registrations are obtained, training proceeds in a SMPL-like iterative manner over the shape space $M$5, pose blend shapes $M$6, skinning weights $M$7, and joint regressor $M$8 (Romero et al., 2022). The paper also notes a locality prior for pose blend shapes based on geodesic distance on the template, penalizing distant joint-vertex dependencies and encouraging local corrections.

For full-body performance capture, SMPL+H is fit to 4D body scans from a 3dMDbody scanner at 60 fps. The paper reports that hands in such sequences are often noisy, partially missing, low-resolution, and sometimes absent for multiple frames. To cope with this, the fitting adapts a temporal registration method from DYNA/4Cap and adds a velocity prior so that the hand does not jump erratically when observations disappear (Romero et al., 2022).

The sequence fitting pipeline is two-stage. The first stage builds a subject-specific template by fitting the first frames with strong priors, then unposing and averaging aligned meshes. The second stage aligns that personalized template to all frames, optimizing pose and a coupled aligned mesh while using a zero-velocity regularizer (Romero et al., 2022). The reported outcome is a fully automatic fitting process that yields full-body models with natural movement and detailed hand motion.

The original paper also records limitations. Hand-object interaction is not explicitly modeled, and unusual hand poses may fall outside the low-dimensional pose space (Romero et al., 2022). These are not marginal caveats: they delimit the regime in which the synergy-based hand representation remains effective.

4. Position within the SMPL family and later taxonomy

Later survey literature places SMPL+H within a broader family of explicit template-based human models. The survey defines SMPL as a learnable skin-vertex model representing the human body as a 3D mesh with a topological structure, parameterized by pose parameters $M$9 and shape parameters $W$0; it then identifies MANO as the hand-inclusive extension referred to as SMPL+H, and contrasts it with SMPL-X, which combines body, face, and hands by incorporating FLAME and MANO (Liu et al., 2024). In that taxonomy, SMPL+H serves as the hand-aware extension of body-only SMPL.

Model	Coverage	Characterization in later literature
SMPL	Body	Pose $W$1 and shape $W$2 for a body mesh
SMPL+H / MANO-integrated SMPL	Body + hands	SMPL with hand representation
SMPL-X	Body + hands + face	Combines body, face, and hands
H4D	SMPL-based dynamic representation	SMPL-based, but not an SMPL+H model

Within template-based human mesh recovery, the survey organizes methods into template-based and template-free branches, with the template-based branch further subdivided into Naked, Detailed, With Clothes, With Hands, and Whole Body (Liu et al., 2024). SMPL+H is most directly relevant to the “Detailed → With Hands” branch, while many later explicit whole-body methods rely on SMPL-X rather than SMPL+H. The survey lists hand-inclusive and whole-body entries such as SGNify, Two-hands interaction, Hand-object interaction, PROX, ExPose, FrankMocap, PIXIE, Hand4Whole, PyMAF-X, OSX, and HybrIK-X.

The same survey attributes standard advantages and limitations of explicit templates to SMPL+H-style representations. Template-based methods can be more robust than non-parametric methods with the templates’ prior knowledge, and SMPL+H inherits the benefits of structured articulation, compactness, and compatibility with fitting objectives based on joints, silhouettes, shading, and temporal cues (Liu et al., 2024). At the same time, the survey states that explicit model-based methodologies such as SMPL and SMPL-X fall short of meeting the demands for detail that people expect. Applied to SMPL+H, this implies a persistent robustness-detail trade-off: explicit hand articulation improves over body-only SMPL, but hand detail, clothing, occlusion, and fine geometry remain difficult.

A recurrent misconception in adjacent literature is that any temporally structured SMPL-based method is therefore an SMPL+H method. H4D is an explicit counterexample. It is SMPL-based, but not an SMPL+H model: it uses SMPL parameters of shape and initial pose, motion and auxiliary latent codes, and evaluates on keypoints including hand-related keypoints, yet it has no separate articulated hand model, no explicit finger joints, and no SMPL+H parameterization (Jiang et al., 2022). Its gains come from temporal factorization and geometry compensation, not from MANO-style hand articulation.

5. SMPL-H as a motion representation in stochastic generative dynamics

A later line of work uses SMPL-H body parameters as a source representation for motion modeling rather than as the optimization variable of a mesh-fitting pipeline. In a McKean–Vlasov control framework for learning stochastic dynamics from endpoint and intermediate distributional observations, AMASS motion-capture data represented with SMPL-H body parameters are used as the raw motion source, and then converted into a reduced pose representation for learning (Boustany et al., 9 May 2026). The paper is explicit that it does not optimize over full mesh vertices.

In that study, SMPL-H is used for reconstructing body joints from motion files, rendering and visual inspection of sequences, extracting height, knee, torso, and contact-related motion statistics, and defining the motion representation fed to the stochastic dynamics model (Boustany et al., 9 May 2026). The learned object is a stochastic trajectory over articulated human pose sequences. The paper constructs a curated low-to-high posture transition subset of AMASS in a label-free way by scanning SMPL-H motion files, reconstructing joints, searching sliding windows satisfying a set of posture and motion criteria, and rejecting ambiguous or corrupted windows after rendering. The final curated set contains 460 clips total, with 414 training originals and 46 held-out originals, and all accepted sequences are resampled to 61 frames, yielding 59 intermediate frames.

The representation used for learning is a standardized 70-dimensional reduced vector retaining body-facing 6D global orientation, 63 body-pose coordinates, and body height; global horizontal translation and hand degrees of freedom are removed (Boustany et al., 9 May 2026). This means that the process preserves articulated body configuration together with coarse orientation and height, while discarding factors identified as nuisance for the task. The authors then model motion as a controlled diffusion

$W$3

and, for the AMASS experiment, specialize to

$W$4

The objective includes both terminal and intermediate distribution constraints,

$W$5

so that the model learns not merely to reach a final posture but to match intermediate pose distributions as well (Boustany et al., 9 May 2026).

The corresponding McKean–Vlasov FBSDE is

$W$6

In this interpretation, $W$7 is the pose state in reduced SMPL-H coordinates, $W$8 is the adjoint or control signal, and the forward motion is driven by $W$9 (Boustany et al., 9 May 2026). For KL penalties, the law-gradient term becomes a score-difference field,

$T_P$0

The AMASS experiment uses 59 intermediate training marginals with $T_P$1, $T_P$2, $T_P$3, full networks of hidden size 512 and 4 layers for both $T_P$4 and $T_P$5, Adam with learning rate $T_P$6, forward drift clipping at 30, and 1000 optimization steps (Boustany et al., 9 May 2026). On held-out clips, the paper reports that terminal + marginals improves over terminal only on path $T_P$7, height MAE, knee MAE, torso MAE, $T_P$8 MAE, $T_P$9 MAE, success, and foot slide. The interpretation given is that intermediate marginal supervision improves path-level realism and yields more human-like rising behavior. A common misconception would be to read this as direct mesh synthesis in SMPL+H space; the paper instead models stochastic trajectories over reduced pose representations derived from SMPL-H.

6. Direct mesh-level generation and representational shifts

A separate representational shift appears in text-to-motion generation, where SMPL-H is used not as a parametric hand-body control space but as a target mesh domain. In that work, the authors argue for absolute 3D coordinates in global space rather than local-relative HumanML3D-style encodings, and they extend the same idea to direct SMPL-H mesh vertex generation from text (Meng et al., 26 May 2025). The representation for each frame is

$\bar{T}$0

with each vertex expressed in absolute global XYZ coordinates.

The paper is explicit that it does not propose predicting SMPL-H pose parameters directly. Instead, it models SMPL-H mesh vertex trajectories through a pipeline consisting of global vertex-coordinate representation, compression with a Fully Convolutional Mesh Autoencoder, diffusion in a latent mesh space, and decoding back to full-resolution mesh vertices (Meng et al., 26 May 2025). For this mesh setting it uses a latent mesh size $\bar{T}$1, patch size $\bar{T}$2, a Transformer-based diffusion backbone, CLIP-B/32 text conditioning, and reports that AdaLN outperforms concatenation for conditioning while $\bar{T}$3-prediction performs best among $\bar{T}$4, $\bar{T}$5, and velocity targets. The diffusion loss is standard $\bar{T}$6, the autoencoder is trained with a smooth $\bar{T}$7 reconstruction loss, and classifier-free guidance is set to $\bar{T}$8 for text-to-SMPL-H mesh motion generation.

The main motivation for direct SMPL-H mesh generation is to avoid the conventional pipeline of generating joints, fitting SMPL or SMPL-H, and then obtaining meshes. The paper argues that such fitting is time-consuming, introduces reconstruction errors, and can amplify small joint errors into visible mesh artifacts, including shaky body parts and unnatural hand motion (Meng et al., 26 May 2025). Direct vertex generation is therefore presented as a way to better capture fine-grained surface motion, preserve subtle hand and flesh dynamics, reduce jitter and artifacts, and potentially reduce self-penetration issues.

On HumanML3D, which the paper describes as containing 14,616 motion sequences from AMASS and HumanAct12, each with three text captions and motions standardized to 20 FPS and capped at 10 seconds, direct text-to-SMPL-H mesh motion generation improves with scale (Meng et al., 26 May 2025). The reported XL model attains FID $\bar{T}$9, R-Prec@1 $B_S(\beta)$0, R-Prec@2 $B_S(\beta)$1, R-Prec@3 $B_S(\beta)$2, Matching $B_S(\beta)$3, CLIP $B_S(\beta)$4, and LSD $B_S(\beta)$5. The paper interprets the consistently low Laplacian Surface Distance as indicating good mesh structural preservation relative to the ground-truth T-pose.

Taken together, these later usages show two distinct afterlives of the SMPL+H lineage. One retains the articulated-model viewpoint but uses SMPL-H-derived reduced state spaces for stochastic path generation; the other abandons parametric hand-body control at inference and models full vertex trajectories directly in global coordinates (Boustany et al., 9 May 2026, Meng et al., 26 May 2025). This suggests that SMPL+H functions both as a canonical explicit hand-aware body model and as a source domain from which later generative systems derive reduced or fully geometric motion representations.