ReMu: Layered 3D Clothed Human Reconstruction

• ReMu is a reconstruction pipeline that estimates separate 3D layers for the body, inner garments, and outer garments from progressive image captures.
• It uses inverse LBS for canonical alignment and explicit collision-aware penetration removal to ensure nearly interpenetration-free layers.
• The pipeline refines garment surfaces with neural unsigned distance fields, yielding high-quality meshes suitable for simulation and avatar creation.

ReMu is a training-free reconstruction pipeline for recovering a multi-layer 3D clothed human from Image Layers: a set of monocular RGB photographs of the same subject captured while progressively wearing different subsets of garments. Unlike single-surface clothed-human reconstruction, ReMu reconstructs the body, inner garments, and outer garments as separate layers, aligns them in a shared canonical body coordinate system, applies collision-aware penetration removal, and then refines surfaces with neural unsigned distance fields, yielding nearly penetration-free layered geometry for downstream tasks such as clothing simulation and avatar creation (Vuran et al., 2 Aug 2025).

1. Problem setting and capture model

ReMu addresses a reconstruction setting that prior clothed-human methods generally do not solve explicitly: recovering a multi-layer dressed human in which the body, inner garments, and outer garments are represented as distinct 3D layers rather than being fused into one surface. The input modality is a new capture setup called Image Layers, where a subject is photographed by a single static RGB camera multiple times while progressively adding clothing layers. One image may show only a lower garment, another the lower plus an inner garment, and another the full stack including an outer garment. The camera need not be calibrated, and the method does not use the camera position or matrix (Vuran et al., 2 Aug 2025).

This setting is designed to expose geometry that is otherwise hidden in a single image. Inner garments that become occluded in the final dressed state remain visible in earlier captures, which reduces the ambiguity that normally forces strong garment-category priors. At the same time, Image Layers introduces a coupled geometric problem: poses across captures are only approximately matched; independently reconstructed garments do not align naturally; and naive stacking of reconstructions produces collisions, misalignment, and broken boundaries. ReMu is formulated around these three difficulties.

The method is explicitly template-free and category-agnostic. It does not assume predefined garment templates, category-specific latent spaces, sewing-pattern priors, or synthetic-data priors tied to known classes. This is the basis for its reported behavior on clothing categories such as long coats and dresses, where category-specific methods are described as struggling while ReMu reconstructs the garments without retraining or category adaptation (Vuran et al., 2 Aug 2025).

2. Reconstruction pipeline and garment isolation

For each image layer $I_k$, ReMu first estimates a body model $B_k$ with SMPLer-X, then refines the body pose by minimizing 2D keypoint reprojection error. Given this body estimate, it uses SiTH as an off-the-shelf single-view clothed human reconstruction model to infer a watertight textured clothed mesh $M_k$. SiTH fits SMPL-X, hallucinates a back view with a diffusion model, and reconstructs a watertight clothed-human mesh from front/back image features and body guidance. At this stage, however, $M_k$ is still a fused clothed-human surface rather than an isolated garment layer (Vuran et al., 2 Aug 2025).

Garment separation is performed in 3D by combining mesh rendering, 2D segmentation, and back-projection. ReMu renders synthetic views of $M_k$, runs GroundedSAM on those views to segment the target clothing in 2D, and projects the labels back onto the mesh. In the implementation, the method renders 60 views, sampled circularly with $10^\circ$ horizontal spacing and $30^\circ$ spacing for upper and lower views. Vertex labels are assigned by majority vote over projected masks; disconnected fragments are filtered; and labels are smoothed to obtain a 3D garment region $S_k$ on $M_k$. This step converts a generic clothed-body reconstruction into an explicit layer-specific garment region (Vuran et al., 2 Aug 2025).

The overall pipeline then follows a fixed sequence: reconstruct a clothed mesh and parametric body for each image layer, isolate the garment of interest for that layer, warp all garments into a common canonical coordinate system, perform collision-aware penetration removal from inner to outer layers, and finally fit neural unsigned distance fields (UDFs) to refine boundaries and extract the final meshes. In the authors’ formulation, the central contribution is not a new monolithic network but a hybrid pipeline that combines explicit meshes with implicit neural fields.

3. Canonical-body alignment and layered collision handling

After garment extraction, the reconstructed meshes are still mutually inconsistent because the subject may have slightly different poses across captures. ReMu resolves this by defining a shared canonical body pose $B_c$, typically a T-pose, and using it as the common coordinate system for all layers. The body is represented by SMPL-X, while each garment initially remains a segmented region on a watertight reconstructed mesh. ReMu brings each mesh into canonical pose using inverse linear blend skinning (inverse LBS) (Vuran et al., 2 Aug 2025).

The forward LBS relation for a vertex $B_k$0 in observed pose and its canonical counterpart $B_k$1 is

$B_k$2

where $B_k$3 is the bone transformation matrix for joint $B_k$4 under pose $B_k$5, and $B_k$6 are skinning weights. The inverse mapping is

$B_k$7

The skinning weights are obtained by querying the nearest vertex on the SMPL-X body. This canonicalization is the mechanism that makes consistent geometric reasoning about “inner” and “outer” garments possible (Vuran et al., 2 Aug 2025).

ReMu then performs its central collision-aware optimization, implemented as an explicit geometric procedure rather than as a learned differentiable loss. The canonical body $B_k$8 is treated as layer $B_k$9, with $M_k$0, and garments are processed sequentially from inner to outer. For each garment $M_k$1, the goal is to obtain a corrected mesh $M_k$2 that lies outside $M_k$3 while preserving the original garment shape whenever possible.

For each vertex $M_k$4, the method finds the nearest point $M_k$5 on the body surface and uses the body normal $M_k$6 there as the only allowed displacement direction: $M_k$7 If $M_k$8, the vertex is moved onto the surface of the previous layer $M_k$9. If $M_k$0 and is not inside $M_k$1, it is left unchanged so that loose or draped geometry is preserved. If it lies inside $M_k$2, the method computes the intersection parameter $M_k$3 of $M_k$4 with $M_k$5 and moves the vertex just outside the previous layer by a small offset $M_k$6: $M_k$7 In implementation, the garment thickness parameter for this penetration-removal stage is fixed to 2 mm, and mesh queries are handled with NVIDIA Kaolin. The result is an explicitly nested layered geometry in which body, inner garments, and outer garments are ordered by construction rather than by soft penalty terms (Vuran et al., 2 Aug 2025).

4. Implicit UDF refinement and mesh extraction

The paper emphasizes two artifacts introduced by the explicit mesh stages. First, boundaries obtained by projecting 2D masks back to 3D can be noisy or fragmented. Second, penetration-removal shifts are discontinuous and can damage local smoothness. ReMu addresses both issues by re-representing each corrected garment as a neural implicit unsigned distance field and remeshing it afterward (Vuran et al., 2 Aug 2025).

For garment $M_k$8, the UDF is defined as

$M_k$9

In the main paper, the fitting objective is written

$M_k$0

The supplementary gives the training formulation actually used in implementation. For each corrected mesh $M_k$1 and garment region $M_k$2, points are sampled on the surface, near the surface by Gaussian perturbation, and uniformly in a bounding box. Points whose nearest surface point is not on $M_k$3 are discarded; the ground-truth unsigned distance $M_k$4 to $M_k$5 is then computed; and training uses a clamped loss,

$M_k$6

The field is unsigned rather than signed because garments are open or thin surfaces, and unsigned distance is described as more robust for such geometry (Vuran et al., 2 Aug 2025).

The network for each garment UDF is a 4-layer MLP with hidden dimensions 128, 256, 256, 128 and ReLU activations, trained with Adam at learning rate $M_k$7. The paper also studies positional encodings inspired by NeRF,

$M_k$8

and chooses $M_k$9 as a compromise between preserving wrinkles and avoiding noise reconstruction. After fitting, each garment is extracted with Marching Cubes. The supplementary states that the extraction threshold is 3 mm, producing a double-sided mesh with small thickness, and also describes a rendering-oriented postprocess that flips back-face normals for faces whose vertices lie sufficiently behind the body along the viewing direction (Vuran et al., 2 Aug 2025).

5. Evaluation, metrics, and ablation behavior

The evaluation uses the 4D-DRESS dataset, which contains multi-view images and 3D scans with semantic clothing labels for 32 clothed subjects. The authors randomly select 12 subjects to build a multilayer reconstruction benchmark, extract ground-truth lower, inner, and outer garment meshes from semantic labels, and deform lower and inner garments to the outer-garment pose for evaluation. For fairness, all compared methods are given ground-truth poses and labels, so the benchmark isolates clothing reconstruction rather than body fitting. ReMu is compared against SMPLicit, ClothWild, and ISP, together with adapted “+” variants that are allowed to use the same Image Layer inputs (Vuran et al., 2 Aug 2025).

The reported metrics are Chamfer Distance (CD) in millimeters, Normal Consistency (NC), and Intersection Ratio (IR). IR is introduced as a visibility-based measure of interpenetration: garments are rasterized from front and back views, $10^\circ$0 denotes the visible area of the garment alone, and $10^\circ$1 denotes the visible area after stacking all inner garments and body. Higher IR indicates that more of the garment is hidden by intersections with inner layers.

Quantitatively, ReMu achieves lower garments: CD $10^\circ$2 mm, NC $10^\circ$3, IR $10^\circ$4; inner garments: CD $10^\circ$5 mm, NC $10^\circ$6, IR $10^\circ$7; and outer garments: CD $10^\circ$8 mm, NC $10^\circ$9, IR $30^\circ$0. The paper stresses that these IR values remain below 0.5% for all garment categories, whereas original single-image baselines can show much higher penetration, including SMPLicit with outer IR $30^\circ$1 and ClothWild with inner IR $30^\circ$2. It also notes that even the inverse-LBS-aligned ground-truth 4D-DRESS meshes are not penetration-free, with reference IR values of $30^\circ$3 for lower, $30^\circ$4 for inner, and $30^\circ$5 for outer. The paper’s interpretation is that ReMu is not merely approximating aligned scan geometry; it is producing layered geometry that is more directly usable for simulation (Vuran et al., 2 Aug 2025).

The ablation study isolates three stages: inverse LBS only, inverse LBS plus penetration removal, and the full model including garment refinement. With inverse LBS only, penetrations are severe: lower IR $30^\circ$6, inner IR $30^\circ$7, outer IR $30^\circ$8. Adding penetration removal reduces them to lower $30^\circ$9, inner $S_k$0, and outer $S_k$1. Adding UDF refinement improves CD and NC but slightly raises IR: lower $S_k$2, inner $S_k$3, outer $S_k$4. The authors explicitly frame this as a tradeoff: explicit collision correction gives nearly perfect separation, while implicit refinement restores surface quality at the cost of a small reintroduction of intersections. They argue that the remaining IR is practical for Blender-based simulation (Vuran et al., 2 Aug 2025).

6. Relations to prior work, scope, and nomenclature

ReMu is positioned against earlier single-image multilayer clothing methods such as BCNet, SMPLicit, ClothWild, and ISP, which are described as relying on predefined garment templates, category-specific latent spaces, synthetic training data, or sewing-pattern priors. ReMu departs from this design space by reconstructing layers from actual observations across progressive dressing states. This is why the paper repeatedly describes it as template-free and category-agnostic, and why it highlights qualitative generalization to garments such as dresses and long coats without retraining (Vuran et al., 2 Aug 2025).

The method nonetheless has clear capture and modeling assumptions. It requires a set of image layers taken with a fixed monocular RGB camera, one image per garment layering state, and assumes the subject can be captured repeatedly while gradually adding clothing layers. In the reported implementation, the authors used an iPhone 13 Pro Max on a tripod, cropped images to $S_k$5, and removed backgrounds with Segment Anything. The current formulation is also described as tailored to subjects in roughly T/A pose. A practical limitation is that multi-state capture may be inconvenient in some settings; another is that the UDF refinement stage does not include explicit collision constraints, which is why small penetrations can reappear after refinement. The paper points to extending the idea to single-view videos as future work (Vuran et al., 2 Aug 2025).

A common misconception is to treat ReMu as a single learned network. The paper explicitly argues otherwise: its main contribution is a reconstruction formulation built from a shared canonical-body representation, explicit penetration-aware layering, and implicit UDF-based refinement. This suggests that ReMu is best understood as a hybrid geometric-neural system rather than as a category-conditioned predictor.

The name also requires disambiguation. “ReMu” in this context refers to “ReMu: Reconstructing Multi-layer 3D Clothed Human from Image Layers” (Vuran et al., 2 Aug 2025). It is distinct from “ReMU: Regional Minimal Updating for Model-Based Derivative-Free Optimization” (Xie et al., 4 Apr 2025) and from ReMU, “Response Matrix Utilities,” a Python package for response-matrix-centred forward folding in cross-section measurements (Koch, 2019). These are unrelated uses of a similar string, and only the first denotes the layered clothed-human reconstruction method described here.