Gaussian Wardrobe: Composable 3D Avatars

• Gaussian Wardrobe is a 3D avatar framework that separates human bodies from reusable garment layers for realistic virtual try-on.
• It employs canonical space mapping and per-layer networks to learn garment-specific, pose-driven dynamics beyond rigid body constraints.
• The framework achieves photorealistic rendering with artifact correction and outperforms prior methods on metrics like PSNR, SSIM, and LPIPS.

Gaussian Wardrobe is a compositional, animatable 3D Gaussian avatar framework for free-form virtual try-on that decouples the human body from multiple garment layers and canonicalizes each garment into a shape-agnostic space, so that garments become reusable across subjects (Chen et al., 4 Mar 2026). It is trained from synchronized, calibrated multi-view videos and addresses a central limitation of monolithic neural avatars: prior NeRF- or Gaussian-based methods commonly treat body and clothing as one inseparable entity, which forces garments to move as if “attached” to the body mesh, limits expressiveness for loose or topologically distinct clothing such as skirts and open jackets, and prevents garment reuse. In response, Gaussian Wardrobe decomposes avatars into bodies and layers of neural garments, learns garment-specific pose-driven dynamics, and supports a practical 3D try-on pipeline with photorealistic rendering and inference-time artifact correction (Chen et al., 4 Mar 2026).

1. Problem formulation and design objectives

Gaussian Wardrobe is formulated around four explicit goals: learning a compositional avatar in which the body and each garment layer are separate neural assets; canonicalizing garments into a shape-agnostic space so that they can be stored and reused for different subjects; learning garment-specific, pose-driven dynamics from multi-view video without assuming garments follow body bones rigidly; and delivering a practical 3D try-on pipeline with photorealistic rendering and on-the-fly artifact correction (Chen et al., 4 Mar 2026).

The problem setting is motivated by the failure mode of monolithic avatars. When body and garments are encoded as a single entity, loose clothing is poorly represented, motion is overly constrained by the underlying body model, and garments cannot be transferred to other identities. Gaussian Wardrobe treats these as a representational issue rather than only a rendering issue: disentanglement must be built into the avatar parameterization, the training supervision, and the deformation model.

The framework uses SMPL-X as its subject and pose model, with shape $\beta \in \mathbb{R}^{10}$ and pose $\theta \in \mathbb{R}^{21+30+3+1}$. A single mesh template is reconstructed from the first frame and reposed to a canonical A-pose. This template is then segmented into layered components $L \in \{b,u,\ell,o\}$ for body, upper garment, lower garment, and optional outer garment, while hair and shoes are merged into the body layer to preserve detail (Chen et al., 4 Mar 2026).

2. Canonical layered representation

The representational core of Gaussian Wardrobe is a layered 3D Gaussian avatar in which each garment is canonicalized independently of subject shape. To remove subject-specific shape, each template vertex $v_i^t$ is mapped to a canonical vertex $v_i^c$ by removing SMPL-X blendshape offsets:

$v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$

where the blendshape offsets $o_{bi}$ are diffused into a $64^3$ voxel grid following the Fast-SNARF strategy and queried by trilinear interpolation (Chen et al., 4 Mar 2026).

Each layer is represented through orthographic front/back rasterizations. The coordinate maps

$C_L \in \mathbb{R}^{2\times H \times W \times 3}$

store canonical 3D points on the layer surface, while pose-conditioned positional maps

$$P_L(\theta_m) \in \mathbb{R}^{2\times H \times W \times 3}$$

are obtained by deforming the layer with Linear Blend Skinning under pose $\theta \in \mathbb{R}^{21+30+3+1}$0 (Chen et al., 4 Mar 2026).

For each layer, a network $\theta \in \mathbb{R}^{21+30+3+1}$1 takes $\theta \in \mathbb{R}^{21+30+3+1}$2 and predicts Gaussian maps

$\theta \in \mathbb{R}^{21+30+3+1}$3

Each Gaussian has parameters

$\theta \in \mathbb{R}^{21+30+3+1}$4

where the mean is initialized from the canonical coordinate and offset by $\theta \in \mathbb{R}^{21+30+3+1}$5, the covariance is

$\theta \in \mathbb{R}^{21+30+3+1}$6

$\theta \in \mathbb{R}^{21+30+3+1}$7 is opacity, and $\theta \in \mathbb{R}^{21+30+3+1}$8 is color (Chen et al., 4 Mar 2026). The method does not require SH color features; instead, it uses direct colors with view-dependent adjustment following Animatable Gaussians.

This layered construction is the mechanism by which the framework separates body geometry from garment geometry. Disentanglement is not left implicit. It is supported by explicit per-layer templates and U-Nets, per-layer segmentation supervision, inter-layer penetration regularization, and geometric priors favoring small, smooth offsets in canonical space (Chen et al., 4 Mar 2026).

3. Deformation, rendering, and occlusion handling

Although Gaussian Wardrobe uses SMPL-X and LBS, it is not restricted to pure skeleton-driven motion. Canonical Gaussians are first predicted in the identity-independent garment space and then mapped to the posed subject through a forward mapping $\theta \in \mathbb{R}^{21+30+3+1}$9. Conceptually, $L \in \{b,u,\ell,o\}$0 maps posed points to canonical space and $L \in \{b,u,\ell,o\}$1 maps canonical points back to posed space. In practice, the posed Gaussian mean and covariance are computed as

$L \in \{b,u,\ell,o\}$2

$L \in \{b,u,\ell,o\}$3

with per-bone transforms $L \in \{b,u,\ell,o\}$4 and diffused skinning weights $L \in \{b,u,\ell,o\}$5 (Chen et al., 4 Mar 2026). The learned offsets $L \in \{b,u,\ell,o\}$6 and local rotations $L \in \{b,u,\ell,o\}$7, predicted as functions of $L \in \{b,u,\ell,o\}$8, permit garment-specific deformation beyond pure LBS tied to the skeleton.

Rendering follows 3D Gaussian splatting. For a Gaussian with world mean $L \in \{b,u,\ell,o\}$9 and covariance $v_i^t$0, the camera-space transform is

$v_i^t$1

With perspective intrinsics $v_i^t$2 and $v_i^t$3, the projected center is

$v_i^t$4

and the 2D covariance is

$v_i^t$5

Per-pixel contribution is then defined by standard Gaussian Splatting alpha compositing:

$v_i^t$6

$v_i^t$7

(Chen et al., 4 Mar 2026).

At render time, posed Gaussians from body and garments are pooled into a single set and splatted jointly, so depth-aware blending resolves occlusion across layers automatically. The renderer also produces per-layer segmentation masks. In addition, Gaussian Wardrobe includes an inference-time penetration-aware correction step: after compositing, it renders per-layer masks, extracts closed contours, checks depth consistency, and, for a pixel enclosed by an outer garment region, applies a correction if

$v_i^t$8

In that case, the pixel color is replaced with that of the correct outermost layer (Chen et al., 4 Mar 2026).

4. Data pipeline, supervision, and optimization

Training requires synchronized multi-view capture with known intrinsics and extrinsics. The reported setup uses 12–14 synchronized views. Preprocessing reconstructs a mesh from the first frame, reposes it to canonical space by inverse LBS with diffused skinning fields, registers SMPL-X pose $v_i^t$9 and shape $v_i^c$0, segments the template into body and garment layers according to the 4D-DRESS protocol, and prepares 2D multi-class garment/body masks from an off-the-shelf method (Chen et al., 4 Mar 2026).

The loss design directly reflects the compositional objective. Given rendered RGB $v_i^c$1, ground truth $v_i^c$2, rendered masks $v_i^c$3, segmentation $v_i^c$4, body-only rendered mask $v_i^c$5, and body-template mask $v_i^c$6, the photometric loss is

$v_i^c$7

the face refinement term is

$v_i^c$8

and the segmentation loss is

$v_i^c$9

(Chen et al., 4 Mar 2026).

Adjacent-layer penetration is penalized by

$v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$0

and geometric regularizers are

$v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$1

The full objective is

$v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$2

Removing segmentation loss entangles body and garments and breaks try-on; removing penetration or geometric regularizers yields visible penetrations (Chen et al., 4 Mar 2026).

The architecture uses, for each layer, a StyleUNet-based design with two decoders for front and back and three separate StyleUNets per layer predicting color, offsets, and other Gaussian attributes. Inputs are positional maps at $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$3, and outputs are Gaussian maps at $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$4 (Chen et al., 4 Mar 2026). Optimization uses Adam with learning rate $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$5 and cosine annealing to $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$6, 300k iterations, batch size 1, $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$7, $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$8, and gradient clipping at $v_i^c = v_i^t - \sum_{b=1}^{10} \beta_b o_{bi},$9. Training proceeds in two stages: a coarse stage with $o_{bi}$0, $o_{bi}$1, and $o_{bi}$2 disabled, followed by a fine stage with the full loss. Typical training is approximately 2.5 days on an RTX 6000 (24 GB) for two-layer outfits, and outfits with an additional outer layer add approximately 1.5 days by freezing $o_{bi}$3 and fine-tuning the garment networks (Chen et al., 4 Mar 2026).

5. Digital wardrobe and virtual try-on

The reusable asset in Gaussian Wardrobe is the learned garment layer in canonical space. Each trained garment layer $o_{bi}$4 is stored as a shape-agnostic neural asset, while the target identity is defined by a body model $o_{bi}$5 and shape $o_{bi}$6; skin tone, hair, and face remain consistent (Chen et al., 4 Mar 2026).

Dressing a new subject consists of choosing garment layers such as $o_{bi}$7 and $o_{bi}$8, composing them with the target body, and applying the deformation map $o_{bi}$9 using the target pose $64^3$0 and shape $64^3$1 to obtain posed Gaussians:

$64^3$2

An optional body-Gaussian swapping step replaces body $64^3$3 and $64^3$4 inside garments with those from a compatible body model to reduce penetrations while keeping color, opacity, and scale to preserve identity (Chen et al., 4 Mar 2026).

This try-on pipeline is practical rather than purely conceptual because it includes explicit artifact handling at inference time. Per-layer masks, contour following, and depth checks are used to detect enclosed regions and correct pixels when inner and outer layers are rendered in an inconsistent order. A plausible implication is that the “digital wardrobe” is not only a representation of garments but also a compatibility mechanism for recombining neural clothing with new bodies under novel poses.

6. Evaluation, comparisons, and limitations

Gaussian Wardrobe is evaluated on 4D-DRESS and ActorsHQ. The 4D-DRESS evaluation uses 7 subjects, 16 garments, 12 views, 700–1100 training frames, and 300 test frames per subject. ActorsHQ uses 3 subjects, approximately 1000 training frames, 350 test frames, and 14 views. Reported metrics are PSNR, SSIM, and LPIPS, with mIoU, Recall, and F1 used in ablations on segmentation quality (Chen et al., 4 Mar 2026).

The reported benchmark averages are as follows:

Setting	Baselines	Gaussian Wardrobe
4D-DRESS novel pose	Animatable Gaussians 27.75/0.9577/0.0531; LayGA 27.58/0.9574/0.0543	28.06/0.9579/0.0527
ActorsHQ novel pose	Animatable Gaussians 27.92/0.9411/0.0418; LayGA 27.80/0.9413/0.0421	28.38/0.9436/0.0397
ActorsHQ novel view	Animatable Gaussians 30.51/0.9491/0.0332; LayGA 30.36/0.9482/0.0337	30.86/0.9489/0.0331

The ablation study reports segmentation quality of 0.848/0.898/0.924 without $64^3$5, 0.879/0.922/0.940 without $64^3$6, 0.883/0.927/0.942 without $64^3$7, and 0.893/0.936/0.947 for the full model in mIoU/Recall/F1 (Chen et al., 4 Mar 2026). These results are presented as evidence that segmentation supervision and regularization are necessary for disentanglement and try-on transfer.

Relative to prior methods, the paper positions Gaussian Wardrobe against three baselines. Versus NeRF-based avatars, Gaussians provide fast splatting and explicit per-primitive control, and layered compositionality is described as more practical to implement and render. Versus Animatable Gaussians, the framework adds canonicalization, per-layer networks, disentangling losses, and penetration handling, yielding sharper faces, better garment boundaries, and realistic free-form garment dynamics. Versus LayGA, it avoids tight coupling to the parametric mesh topology and better handles loose clothing and flaps whose motion is weakly correlated with bones; garments are shape-agnostic and reusable (Chen et al., 4 Mar 2026).

The principal limitations are equally explicit. The method requires multi-view, calibrated capture and reliable 2D segmentation, and scaling to in-the-wild monocular videos is future work. Extreme motions may still cause penetrations or dynamic inaccuracies. Thin structures and very loose layers remain challenging. Layered compositing increases memory and reduces FPS compared to monolithic avatars. On the reported hardware, the Animatable Gaussians baseline runs at approximately 1.5 FPS, whereas Gaussian Wardrobe renders at approximately 1.08 FPS with two garment layers and approximately 0.8 FPS with three, with rendering time growing approximately linearly with the number of layers (Chen et al., 4 Mar 2026).

A related direction is SemanticGarment, which also uses 3D Gaussian garments but is organized around semantic-controlled generation and editing from text or image prompts, structural human priors, and self-occlusion optimization rather than multi-view avatar capture (Wang et al., 21 Sep 2025). Taken together, these lines of work suggest a broader shift from monolithic human avatars toward reusable, semantically structured Gaussian garment assets, but Gaussian Wardrobe is specifically distinguished by multi-view training, layered body/garment disentanglement, and free-form virtual try-on under subject transfer (Chen et al., 4 Mar 2026, Wang et al., 21 Sep 2025).