---
title: 'MonoCloth: Monocular Cloth Capture'
url: https://www.emergentmind.com/topics/monocloth
type: topic
---

# MonoCloth: Monocular Cloth Capture

Searching arXiv for the named paper and closely related monocular cloth-capture work.
MonoCloth denotes both a specific 2025 method for monocular avatar reconstruction and a broader research target in monocular cloth capture. In the specific sense, "MonoCloth: Reconstruction and Animation of Cloth-Decoupled Human Avatars from Monocular Videos" reconstructs and animates clothed human avatars from monocular videos by explicitly decoupling body, face, hands, and clothing, and by assigning clothing a dedicated learned spatio-temporal module, CloSim, within an SMPL-X-anchored 3D Gaussian representation [2508.04505]. In the broader sense, the term aligns with a family of methods that attempt to infer garment geometry, appearance, and dynamics from single-view RGB observations despite severe geometric ambiguity, non-rigid motion, self-occlusion, and the mismatch between articulated body motion and history-dependent cloth deformation [2009.10711].

## 1. Research context and lineage

Monocular cloth capture developed through several partially overlapping paradigms. "MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video" addressed temporally coherent garment deformation without a pre-scanned personalized mesh template by building statistical PCA deformation models for T-shirt, short pants and long pants, and by combining silhouette, segmentation, texture, and normal cues in a differentiable-rendering pipeline [2009.10711]. "Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture" instead introduced a person-specific, learning-based monocular capture method with separate cloth geometry and a simulation layer embedded into training, claiming the first physics supervision in the weakly supervised deep monocular human performance capture setting [2011.12866].

Subsequent methods diversified the representation. "SCARF (Segmented Clothed Avatar Radiance Field)" used a hybrid model combining a mesh-based body with a segmented neural radiance field, optimized directly from monocular videos without any 3D supervision, and explicitly supported animation, novel-view synthesis, and clothing transfer [2210.01868]. "xCloth: Extracting Template-free Textured 3D Clothes from a Monocular Image" shifted toward template-free textured garment digitization from a single image via layered pixel-aligned depth, semantic, RGB, and normal maps, together with automatic UV atlas generation [2208.12934]. "REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos" formulated dynamic garment recovery as a joint optimization of explicit feature curves and implicit SDFs, with canonical-space template registration for open-boundary garment meshes [2305.14236]. "MOSS: Motion-based 3D Clothed Human Synthesis from Monocular Video" brought motion-conditioned Gaussian splatting into the area through Kinematic Gaussian Locating Splatting and a Surface Deformation Detector guided by matrix-Fisher modeling on $SO(3)$ [2405.12806].

Within that trajectory, MonoCloth combines explicit body–cloth decoupling, 3D Gaussian rendering, and a learned cloth-specific temporal module. A plausible implication is that it occupies an intermediate position between explicit simulation-based cloth modeling and purely appearance-driven monocular avatar methods: it does not solve cloth physics explicitly, but it also does not treat clothing as just another body surface [2508.04505].

## 2. Problem formulation and part-based decomposition

MonoCloth starts from the premise that monocular reconstruction is difficult because the input is fundamentally 2D and because loose garments, facial expressions, and articulated hands exhibit different reconstruction and deformation characteristics. The method therefore explicitly decouples the avatar into body, face, hands, and clothing, optimizing each component with a strategy matched to its motion and reconstruction complexity [2508.04505].

The underlying body prior is SMPL-X. MonoCloth uses an upsampled canonical mesh $v^{cano}$ and defines an identity-dependent mesh in canonical pose by learned per-vertex displacements,
$$
v^{id} = v^{cano} + \Delta x.
$$
3D Gaussians are placed at upsampled SMPL-X vertices, with attributes $\{x, c, s\}$ for position, color, and scale; opacity is fixed to 1 and the Gaussians are isotropic. The full Gaussian set is semantically decomposed as
$$
G = G_{face} \cup G_{hands} \cup G_{cloth} \cup G_{body}.
$$

The face and hands are anchored by strong parametric priors. $G_{face}$ is aligned to FLAME via vertex correspondences, while $G_{hands}$ is aligned to MANO/SMPL-X correspondences. Clothing is extracted through 2D clothing segmentation followed by a connectivity-based refinement, so that the cloth subset is not merely inferred as residual geometry but treated as an independently modeled component [2508.04505].

This decomposition is central to the method’s definition. The paper argues that a one-size-fits-all deformation model tends to produce flicker, drift, or implausible cloth behavior during animation because rigid, bone-driven body deformation differs qualitatively from soft, history-dependent cloth dynamics. MonoCloth’s architecture is therefore organized around the premise that clothing must be modeled separately if monocular avatars are to remain animatable and visually stable [2508.04505].

## 3. Canonical representation, triplane features, and dynamic deformation

MonoCloth uses a two-stage learning scheme. In Stage 1, a shared network is pretrained across multiple monocular subjects using a compact identity latent code $z_i \in \mathbb{R}^{64}$. In Stage 2, the model is fine-tuned per subject on the target monocular sequence. Canonical features are produced by a shared decoder
$$
T_i = D_\phi(z_i) = \{T^x_i, T^y_i, T^z_i\},
$$
where each triplane has shape $\mathbb{R}^{C \times H \times W}$ with $C=32$ and $H=W=128$ [2508.04505].

Per-vertex canonical features $feat^{cano}$ are obtained by bilinear sampling from the triplanes at the vertices of $v^{cano}$. MLP decoders then map these features to per-vertex Gaussian attributes: spatial displacement $\Delta x$, base color $c$, and scale $s$. The canonical avatar is thus a deformable Gaussian field anchored to a canonicalized SMPL-X mesh rather than an unstructured point cloud [2508.04505].

Dynamic modeling is expressed in canonical space and then transferred to posed frames through skinning:
$$
v_t = LBS(v^{id} + \Delta x_t, \theta_t).
$$
The paper emphasizes that offsets are predicted in canonical space and then skinned, which decouples offset learning from articulation and improves generalization across identities. Clothing is the main beneficiary of this design because it undergoes substantial non-rigid deformation, while exposed body parts such as forearms and calves are handled by a simpler pose-dependent module that predicts only small offsets [2508.04505].

The temporal conditioning window is
$$
P_T = \{\theta_{T-\Delta t}, \theta_T, \theta_{T+\Delta t}\},
$$
with $\Delta t = 0.2\,s$ and low sampling at 5 FPS. This explicit use of neighboring poses is intended to inject motion history into cloth deformation prediction rather than treating each frame as a purely static inverse problem [2508.04505].

## 4. CloSim, geometric supervision, and objective function

CloSim is the distinctive module of MonoCloth. The paper explicitly states that CloSim is a learned spatio-temporal simulator—not a physics solver. Its spatial component is a GCN over clothing Gaussians, and its temporal component is a GRU that encodes motion history [2508.04505].

For frame $t$, encoded clothing features are constructed as
$$
Z_t = GCN(Concat(feat^{cano}, feat^{pose}_t), E),
$$
where $feat^{cano} \in \mathbb{R}^{N \times 96}$, $feat^{pose}_t \in \mathbb{R}^{N \times 126}$, $N$ is the number of clothing Gaussians, and $Z_t \in \mathbb{R}^{N \times 128}$. Sequence modeling then predicts residuals for position, color, and scale:
$$
\{\Delta x_t, \Delta c_t, \Delta s_t\}_{t=1..T} = \Psi(GRU(\{Z_t\}_{t=1..T}, h_0)).
$$
During training, given $P_T$, CloSim outputs $\{\Delta x_{T-\Delta t}, \Delta x_{T+\Delta t}\}$, and a random supervision time $t \in [T-\Delta t, T+\Delta t]$ is chosen so that the target offset is linearly interpolated,
$$
\hat{\Delta x}_t = (1-\alpha)\Delta x_{T-\Delta t} + \alpha \Delta x_{T+\Delta t},
$$
with analogous interpolation for $\hat{\Delta c}_t$ and $\hat{\Delta s}_t$ [2508.04505].

MonoCloth uses a multi-target supervision regime. Rendered normals $\hat{N}_t$ and depth $\hat{D}_t$ are supervised against Sapiens-derived pseudo ground truth $N^{gt}_t$ and $D^{gt}_t$. The geometry loss is
$$
L_{geo} = \lambda_N L_N + \lambda_D L_D + \lambda_S L_S,
$$
with
$$
L_N = 1 - \langle \hat{N}^{pred}, N^{gt} \rangle,\quad
L_D = ||\hat{D}^{pred} - D^{gt}||_1,\quad
L_S = ||\hat{S}^{pred} - S^{gt}||_2^2,
$$
and $\lambda_N = 5$, $\lambda_D = 1$, $\lambda_S = 2$. Temporal smoothness is enforced through
$$
L_{temp} = \lambda_{temp}\sum_{t=1..T-1} \left( ||\Delta x_{t+1}-\Delta x_t||_2^2 + ||\Delta c_{t+1}-\Delta c_t||_2^2 + ||\Delta s_{t+1}-\Delta s_t||_2^2 \right),
$$
with $\lambda_{temp}=0.1$ [2508.04505].

RGB reconstruction combines pixel, structural, perceptual, and clothing-focused terms:
$$
L_{rgb} = ||I^{pred}-I^{gt}||_1,\quad
L_{ssim}=1-SSIM(I^{pred},I^{gt}),\quad
L_{lpips}=LPIPS(I^{pred},I^{gt}),\quad
L_{cloth}=||I^{mask}_{cloth}(I^{pred})-I^{mask}_{cloth}(I^{gt})||_1.
$$
Face and hands are regularized by correspondence losses,
$$
L_{face} = \sum_{i\in G_{face}} ||v_{FLAME,i}-v_{gauss,i}||_2^2,\quad
L_{hands} = \sum_{i\in G_{hands}} ||v_{MANO,i}-v_{gauss,i}||_2^2.
$$
The full training objective is
$$
L = L_{rgb} + \lambda_{ssim}L_{ssim} + \lambda_{lpips}L_{lpips} + \lambda_{cloth}L_{cloth} + L_{geo} + L_{temp} + \lambda_{reg}L_{reg} + \lambda_{face}L_{face} + \lambda_{hands}L_{hands}.
$$

Optimization uses Adam with initial learning rate $1e^{-3}$. The reported practical profile is approximately 5 minutes for 3D clothing segmentation and approximately 3.5 hours for per-subject reconstruction on an RTX 4090. The paper further notes that inference is fast once trained, with real-time rendering possible via 3DGS tooling [2508.04505].

## 5. Quantitative performance, animation, and clothing transfer

MonoCloth is evaluated on NeuMan and X-Humans. On NeuMan, using masked background and metrics over Seattle/Bike/Citron/Jogging, the full method reports PSNR 33.53, SSIM 0.986, and LPIPS $1.20\times10^{-2}$. On X-Humans, it reports PSNR 30.68, SSIM 0.976, and LPIPS $2.21\times10^{-2}$ [2508.04505].

| Setting | Metrics | Comparison note |
|---|---|---|
| NeuMan, Ours (full) | PSNR 33.53, SSIM 0.986, LPIPS $1.20\times10^{-2}$ | Higher than ExAvatar and 3DGS-Avatar |
| NeuMan, Ours (w/o pretrain) | PSNR 33.18, SSIM 0.985, LPIPS $1.28\times10^{-2}$ | Lower than full MonoCloth |
| NeuMan, ExAvatar | PSNR 31.70, SSIM 0.982, LPIPS $1.47\times10^{-2}$ | Baseline |
| NeuMan, 3DGS-Avatar | PSNR 29.75, SSIM 0.975, LPIPS $1.75\times10^{-2}$ | Baseline |
| NeuMan, GaussianAvatar | PSNR 28.90, SSIM 0.974, LPIPS $1.81\times10^{-2}$ | Baseline |
| NeuMan, Vid2Avatar-Pro | PSNR 32.71, SSIM 0.983, LPIPS $1.19\times10^{-2}$ | Multi-view pretrain baseline |
| X-Humans, Ours | PSNR 30.68, SSIM 0.976, LPIPS $2.21\times10^{-2}$ | Higher PSNR/SSIM than ExAvatar |
| X-Humans, ExAvatar | PSNR 29.41, SSIM 0.973, LPIPS $2.24\times10^{-2}$ | Baseline |

The ablations isolate the contribution of the cloth-specific dynamic model and of geometric pseudo-supervision. On NeuMan, removing CloSim offsets and retaining only canonical LBS reduces performance to PSNR 31.93, SSIM 0.983, LPIPS $1.40\times10^{-2}$. Removing temporal sampling gives PSNR 33.00, SSIM 0.985, LPIPS $1.29\times10^{-2}$. Removing $L_{geo}$ gives PSNR 32.55, SSIM 0.985, LPIPS $1.30\times10^{-2}$ [2508.04505]. These numbers support the paper’s claim that temporal modeling is especially beneficial on continuous sequences and that animations exhibit higher stability and robustness under drastic motions.

Animation is defined by driving the avatar with a new motion sequence $\{\theta_t\}$ and applying
$$
v_t = LBS(v^{id} + \Delta x_t, \theta_t),
$$
while rendering Gaussians with updated attributes $\{x+\Delta x_t, c+\Delta c_t, s+\Delta s_t\}$ [2508.04505]. Clothing transfer follows the same decoupled logic: extract $G_{cloth}$ from a source avatar, attach it to a target body through canonical attachment and skinning weights, estimate rest-shape offsets for the target identity, apply minor local attachment refinement, and then animate the transferred garment with the target’s $\theta_t$ [2508.04505]. The paper describes the resulting design as making garments plug-and-play across bodies.

## 6. Limitations, misconceptions, and position in the design space

A central clarification is that MonoCloth is not an explicit cloth-physics method. The paper states that CloSim does not model stretch/bend energies, friction, collisions, or self-collisions. Extreme motions, thin fabrics, or highly loose garments can therefore produce interpenetrations or underdamped behavior, and MonoCloth does not include explicit physics-based collision or self-collision energies [2508.04505]. A common misconception is to equate its cloth simulation module with a differentiable physics solver; this is incorrect in the terminology of the paper itself.

The method also depends on pseudo 3D cues. Normals and depth are taken from Sapiens, and failure modes arise under heavy occlusions or unusual lighting. Clothing mis-segmentation or label noise may cause incorrect decoupling or artifacts despite the connectivity-based refinement. Very fast motion or severe occlusions remain difficult even with temporal smoothing and GRU memory [2508.04505].

These limitations distinguish MonoCloth from earlier and parallel strands of monocular cloth research. The 2020 physics-aware method embeds a simulation layer into training and explicitly handles cloth–body collisions, reducing cloth–body intersections and improving physical plausibility under weak 2D multi-view supervision [2011.12866]. SCARF separates body and clothing through mesh-integrated volume rendering and segmentation-driven losses, but represents clothing as a canonical-space NeRF rather than as cloth-decoupled Gaussians [2210.01868]. REC-MV prioritizes explicit feature curves, multiple garment-type SDFs, and template registration for open-boundary garment meshes [2305.14236]. xCloth targets template-free textured 3D garment digitization from a single image through layered peelmaps and automatic UV atlas generation [2208.12934]. MOSS, by contrast, uses motion-aware Gaussian control through matrix-Fisher-guided orientation and density, plus a surface deformation detector, but does not organize the avatar around MonoCloth’s body/face/hands/clothing decomposition and CloSim formulation [2405.12806].

A plausible implication is that MonoCloth occupies a specific operating point in monocular cloth capture: it favors part-aware controllability, learned temporal clothing dynamics, and efficient 3D Gaussian rendering over explicit physical energies or category-specific garment templates. That choice gives it strong animation behavior, face and hand detail, and clothing transfer capability, while leaving unresolved the hardest cases of collision-rich, topology-changing, or strongly inertial cloth motion [2508.04505]. The paper does not state public release; its reproducibility is therefore tied to access to NeuMan and X-Humans and to reimplementation of 3D Gaussian splatting with triplane features, SMPL-X fitting, Sapiens supervision, and the CloSim GCN+GRU stack [2508.04505].

Source: https://www.emergentmind.com/topics/monocloth