STG-Avatar: 3D Human Avatar Reconstruction

Updated 5 July 2026

STG-Avatar is a framework that reconstructs animatable human avatars from monocular RGB video by coupling spacetime Gaussians with linear blend skinning.
It employs explicit Gaussian representations and optical-flow-guided adaptive densification to effectively capture high-frequency dynamics like clothing wrinkles and rapid limb movements.
The method achieves superior performance with higher PSNR, improved SSIM, and faster FPS, inspiring broader applications in interactive avatar control and live performance.

STG-Avatar most directly denotes the framework introduced in "STG-Avatar: Animatable Human Avatars via Spacetime Gaussian," a 3D Gaussian Splatting method for reconstructing animatable human avatars from monocular RGB video by coupling Spacetime Gaussians with linear blend skinning (LBS) (Jiang et al., 25 Oct 2025). In adjacent literatures, however, closely related labels also designate FLAME-driven Gaussian head avatars, soft-binding temporal Gaussian head reconstruction, and even a mixed-stage theatrical toolchain for directing avatars in live performance. This suggests that STG-Avatar functions both as the name of a specific reconstruction method and as a broader point of convergence for research on controllable avatars, explicit dynamic representations, and operator-facing interfaces (Guo et al., 7 Mar 2025, Zhao et al., 25 Nov 2025, Gagneré et al., 2023).

1. Terminological scope

The literature represented here suggests that the label is not fully monosemous. It names one specific human-avatar reconstruction framework, while related papers use neighboring acronyms or explicitly describe "prototypes of an STG-Avatar system" in other domains.

Label	Meaning in the literature	Representative paper
STG-Avatar	Animatable human avatars via Spacetime Gaussian and LBS from monocular video	(Jiang et al., 25 Oct 2025)
STGA	Selective-Training Gaussian head Avatars for dynamic drivable 3D head avatars	(Guo et al., 7 Mar 2025)
STAvatar	Soft Binding and Temporal Density Control for monocular 3D head avatars reconstruction	(Zhao et al., 25 Nov 2025)
STG-Avatar system	AvatarStaging + AKN_Regie as tools and methods for staging and directing avatars in live performance	(Gagneré et al., 2023)

A further neighboring line, "StyleAvatar3D," addresses stylized 3D avatar generation rather than reconstruction, using diffusion-based synthetic multi-view data and an EG3D backbone (Zhang et al., 2023). The resulting landscape is therefore heterogeneous: some works emphasize monocular performance capture, some emphasize head-specific reconstruction under FLAME priors, some emphasize stylized generation, and some treat the avatar as a dramaturgical partner rather than a computer-graphics benchmark.

2. Mixed-stage and theatrical usage

Before the spacetime-Gaussian formulation, the term already had a broader conceptual life in theatre–game-engine research at Université Paris 8. Georges Gagneré and Cédric Plessiet describe AvatarStaging and AKN_Regie as systems that operationalize the "mixed stage," a configuration in which physical performers and digital avatars share a dramaturgical and spatial continuum. In that setting, the avatar may function as a marionnette virtuelle, avacteur, golem virtuel, or masque virtuel; the stage is extended into a real-time 3D virtual scenography; and spectators may simultaneously perceive the virtual scene, the mocap performer, and sometimes VR immersion (Gagneré et al., 2023).

AvatarStaging is defined as a "dispositif d’expérimentation en réalité mixte" using Unreal Engine 4, motion capture, and real-time control to support improvisation between human performers and virtual puppets or avatars. AKN_Regie, implemented heavily in Blueprints, provides a regie layer for avatar control, motion-capture inputs, cue sequences, camera changes, and environment changes. The architecture includes motion-capture input, motion retargeting, skeletal-mesh animation, Blueprint cue management, and rendering to projectors, VR headsets, and monitoring screens. This organization formalizes direction d’avatar: directing timing, gesture, proximity, gaze, and rhythm for avatars analogously to directing actors (Gagneré et al., 2023).

Theatre case studies such as ParOral, the Masques et Avatars workshops, Agamemnon Redux, and L’Ombre established a recurring methodological vocabulary: the mocapteur for gestural embodiment, the manipulacteur for translational and rotational control, cue systems as temporal structuring devices, and finite-state machines or behavior trees for limited autonomy. In this usage, an STG-Avatar system is not a head or body reconstruction method but a director-centric staging desk for avatar performance in mixed reality (Gagneré et al., 2023).

3. Spacetime Gaussian STG-Avatar for animatable human reconstruction

In its computer-graphics sense, STG-Avatar addresses animatable human avatar reconstruction from monocular RGB video. The target is a controllable avatar that can be driven by arbitrary 3D poses via SMPL/LBS, rendered in real time using 3D Gaussian Splatting, and made robust to high-frequency dynamic details such as clothing wrinkles, hair, and rapidly moving limbs. The method is motivated by two specific deficiencies in earlier 3DGS avatar systems: LBS-only deformation is piecewise rigid and therefore weak for cloth and soft tissue, while MLP-based non-rigid fields tend to be implicit, smooth, and computationally heavy (Jiang et al., 25 Oct 2025).

The pipeline begins from monocular video plus an external SMPL pipeline that provides shape parameters $\beta$ , per-frame pose parameters $\theta_t$ , and camera parameters. A canonical SMPL mesh defines the canonical space. The method initializes 50k Gaussians around the SMPL surface in canonical pose, then performs rigid–nonrigid co-optimization. The rigid component is conventional LBS,

$X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$

which maps a canonical Gaussian position $\mathbf{x}_c$ into posed space. The non-rigid component is a Spacetime-Gaussian polynomial correction,

$\mu_i(t)=X_L+\sum_{k=1}^{n_p} b_{i,k}(t-\mu_i^0)^k,$

with $n_p=2$ , together with a polynomial quaternion for orientation,

$q_i(t)=\sum_{k=0}^{n_q} c_{i,k}(t-\mu_i^\tau)^k,$

with $n_q=1$ . The covariance is then reconstructed in the usual Gaussian form $\Sigma_i(t)=R_i(t)S_i(t)S_i^\top(t)R_i^\top(t)$ (Jiang et al., 25 Oct 2025).

A critical conceptual point is that this is not a full 4D Gaussian in $(x,y,z,t)$ . Rather, each Gaussian has parameters that are functions of time. This yields an explicit temporal model with low-order polynomial trajectories shared across frames, which imposes smoothness without resorting to a global deformation MLP. The color head is likewise not SH-only: STG-Avatar introduces a Dynamic-Aware Color MLP,

$\theta_t$ 0

conditioning on current geometry, motion features, pose, and view direction. The total loss combines photometric, optical-flow, temporal, and regularization terms,

$\theta_t$ 1

These choices are directed at temporally stable, dynamically detailed, real-time rendering (Jiang et al., 25 Oct 2025).

4. Dynamic-region handling and empirical performance

The second major innovation is optical-flow-guided adaptive densification. STG-Avatar uses optical flow to identify persistent motion regions,

$\theta_t$ 2

and to sample new Gaussians anisotropically along the motion direction,

$\theta_t$ 3

A flow-weighted contribution measure then protects high-motion Gaussians from pruning. The intended effect is to allocate representational capacity to flapping clothing, fast limbs, and other regions that view-based or error-only densification under-samples (Jiang et al., 25 Oct 2025).

On ZJU-MoCap, the reported averages are as follows.

Method	PSNR / SSIM / LPIPS	Training / FPS
HumanNeRF	29.74 / 0.950 / 0.039	10 days / 0.2 FPS
GauHuman	27.52 / 0.933 / 0.171	4 min / 120 FPS
3DGS-Avatar	30.63 / 0.934 / 0.051	50 min / 50 FPS
STG-Avatar	31.60 / 0.954 / 0.038	25 min / 60 FPS

The paper also reports that PSNR in dynamic regions improved by about $\theta_t$ 4 dB versus 3DGS-Avatar, while LPIPS for clothing wrinkles decreases by $\theta_t$ 5 (Jiang et al., 25 Oct 2025). On THUman4.0, the method is described as preserving clothing textures, wrinkles, and facial features robustly across poses. The ablation study is especially revealing: removing flow-guided sampling yields PSNR $\theta_t$ 6, SSIM $\theta_t$ 7, LPIPS $\theta_t$ 8; removing STG and retaining only LBS drops performance to PSNR $\theta_t$ 9, SSIM $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 0, LPIPS $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 1; replacing the color MLP with SH gives PSNR $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 2, SSIM $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 3, LPIPS $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 4. The full model reaches PSNR $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 5, SSIM $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 6, LPIPS $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 7 in the ablation setting, indicating that the rigid–nonrigid coupling is the dominant performance contributor (Jiang et al., 25 Oct 2025).

A closely related head-avatar formulation is STGA, "Selective-Training Gaussian Head Avatars." STGA also uses FLAME as a rigged geometric prior and 3D Gaussian Splatting as the explicit appearance representation, but its key contribution is an optimization strategy rather than a new deformation family. Each Gaussian is embedded in a FLAME triangle; for each frame, only Gaussians in regions with strong expression changes are optimized, while others are frozen; and every 20 iterations all Gaussian splats parameters are temporarily unfrozen and jointly optimized to restore global consistency. The threshold is set to $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 8 after measuring triangle-center displacements, and the training design keeps GPU memory at about 3 GB regardless of dataset size. On the NeRSemble subjects 074, 104, and 218, STGA is reported to train in 10 minutes per head and to outperform both Gaussianavatars and a network-heavy Gaussian Head Avatar baseline under the same or shorter time budgets (Guo et al., 7 Mar 2025).

Another head-specific system is STAvatar, whose two central components are UV-Adaptive Soft Binding and Temporal Adaptive Density Control. It begins from FLAME tracking and triangle-bound Gaussians, but refines coarse LBS-driven parameters with per-Gaussian offsets sampled from a UV-space feature offset map. The final Gaussian parameters are obtained by adding position, opacity, and color offsets, multiplying scales elementwise, and composing rotations via quaternion multiplication. Temporal ADC then uses FLAME-Conditioned Temporal Clustering and a Fused Perceptual Error with Average–Peak criterion to decide cloning, with $X_L=\sum_{b=1}^{B} w_b(B_b\,\mathbf{x}_c),$ 9 and the top $\mathbf{x}_c$ 0 by peak error also selected. On INSTA, PointAvatar, NerFace, and HDTF, STAvatar reports the best SSIM and LPIPS on all four datasets, and the best PSNR on INSTA, NerFace, and HDTF; for example, INSTA reaches PSNR $\mathbf{x}_c$ 1, SSIM $\mathbf{x}_c$ 2, LPIPS $\mathbf{x}_c$ 3 (Zhao et al., 25 Nov 2025).

StyleAvatar3D occupies a different part of the design space. Rather than reconstructing a specific performer from video, it generates stylized 3D avatars by synthesizing a large multi-view image dataset with Stable Diffusion, ControlNet, view-specific prompts, and attribute prompts, then training an EG3D-based generator with a coarse-to-fine pose-aware discriminator. It further adds latent diffusion in StyleGAN $\mathbf{x}_c$ 4-space for image-conditioned 2D-to-3D avatar generation. On its stylized benchmark, the coarse-to-fine discriminator improves FID from 7.8 for baseline EG3D to 5.6 (Zhang et al., 2023). Taken together, these neighboring systems suggest that the contemporary STG-Avatar space is divided among three main strategies: explicit dynamic deformation for monocular performance capture, FLAME-embedded Gaussian heads with targeted optimization or temporal density control, and generative tri-plane pipelines for stylized avatar synthesis.

6. Significance, limitations, and research directions

Taken together, these works suggest a recurring design pattern: explicit Gaussian primitives are retained for rendering efficiency, while structured priors organize deformation and control. In the body-avatar setting this prior is SMPL with LBS; in the head-avatar setting it is typically FLAME; in the theatrical setting it is the cue architecture of Unreal Engine, Blueprint logic, and role-specific interfaces such as the mocapteur, manipulacteur, and director. The common technical problem is not merely how to render an avatar, but how to make avatar behavior controllable, temporally stable, and usable inside a larger operational system (Jiang et al., 25 Oct 2025, Gagneré et al., 2023).

The main limitations of STG-Avatar are also explicit. Its spacetime polynomials are time-indexed, so generalization to drastically different motion patterns is not established. The method depends on SMPL and therefore on pose and shape accuracy; errors in SMPL estimation propagate through the LBS backbone. Very loose garments and hair remain challenging because low-order polynomials may not fully capture irregular dynamics. The monocular setting also retains the usual depth and occlusion ambiguities, even if these are mitigated by SMPL priors (Jiang et al., 25 Oct 2025).

Future work in the surrounding literature points in several directions. For body avatars, the natural extension is more explicit pose-conditioned STG, better handling of extreme clothing and hair, stronger pose estimation, and possibly multi-human interaction (Jiang et al., 25 Oct 2025). For head avatars, the trajectory moves toward richer soft binding, temporal density control, and more expressive base meshes beyond standard FLAME support for hair and teeth (Zhao et al., 25 Nov 2025, Guo et al., 7 Mar 2025). In the mixed-stage lineage, future directions include a return to full immersion, richer autonomous golems virtuels, and networked virtual spaces (Gagneré et al., 2023). The cumulative picture is that STG-Avatar is less a single closed technique than a family of avatar systems organized around explicit Gaussian representations, structured deformation priors, and increasingly specialized mechanisms for direction, reenactment, and interactive control.