MExECON: Multi-view Clothed Human Reconstruction

• MExECON is a multi-view 3D reconstruction pipeline that generates explicit clothed human avatars from sparse RGB images using a shared SMPL-X body prior.
• The method employs joint multi-view body optimization (JMBO) to reduce occlusion and depth ambiguity by enforcing consistency via silhouette, landmark, and normal losses across views.
• Its innovative integration of front/back normal maps and depth lifting recovers high-frequency clothing details without any network re-training.

Searching arXiv for papers on MExECON and closely related work. MExECON, short for Multi-view Extended Explicit Clothed humans Optimized via Normal integration, is a pipeline for 3D reconstruction of clothed human avatars from sparse multi-view RGB images. It extends the single-view method ECON to the multi-view setting by fitting a single SMPL-X body model jointly across all input views, using that optimized body as a low-frequency prior, and then recovering high-frequency clothing and hair detail through front/back normal map integration. Its central technical elements are Joint Multi-view Body Optimization (JMBO), multi-view consistency enforced through shared body parameters and calibrated cameras, reuse of a front normal estimator for back-view reconstruction through known camera rotation, and training-free multi-view gains obtained without any network re-training (Uğur et al., 21 Aug 2025).

1. Conceptual position and core design

MExECON is defined against a specific limitation of single-view clothed-human reconstruction. ECON reconstructs high-frequency clothing and hair detail from normal maps using a single frontal image, but this suffers from self-occlusion and depth ambiguity, including an unseen back side, flattened accessories, pose inaccuracies, and missing geometries on occluded regions. MExECON addresses that limitation by extending the pipeline to sparse multi-view input while retaining the explicit-mesh output and the pre-trained components of the original system (Uğur et al., 21 Aug 2025).

The method has four stated contributions. First, JMBO fits one SMPL-X body model to all available views and thereby reduces pose and shape ambiguity. Second, multi-view consistency is enforced by summing silhouette, landmark, and normal losses across views while sharing a single body parameterization. Third, front and back clothed normal maps are integrated, with the back map predicted by reusing the front normal estimator on the back RGB image and then rotating the predicted normals using a known camera transformation. Fourth, all gains are achieved without any network re-training; the improvements arise from optimization and normal-integration design rather than new learned components (Uğur et al., 21 Aug 2025).

A consequential distinction is that MExECON remains an explicit avatar-reconstruction method. SMPL-X provides the low-frequency body prior, but the final clothed mesh is not merely a direct offset of SMPL-X. Instead, it is formed by combining depth-lifted front and back partial surfaces with implicit completion and Poisson Surface Reconstruction, yielding a watertight explicit mesh suitable for downstream manipulation (Uğur et al., 21 Aug 2025).

2. Inputs, assumptions, and end-to-end pipeline

The pipeline takes sparse multi-view RGB images $\{I_i\}_{i=1}^N$ of a static subject, with known camera intrinsics and extrinsics for each view. In the reported experiments, $N \in \{2,8\}$, and front and back views are guaranteed among the inputs. The method assumes a static subject, calibrated cameras, negligible calibration error, and access to clothed silhouettes $S_i^c$ for each view; the technical description notes that, in practice, segmentation can be used to obtain these silhouettes (Uğur et al., 21 Aug 2025).

MExECON reuses several pre-trained components. PIXIE provides per-view SMPL-X estimates; ECON’s front normal estimator $G_F^N$ predicts clothed normal maps; MediaPipe supplies 2D landmarks and confidences; d-BiNI performs depth-aware bilateral normal integration; IF-Nets+ completes missing geometry; and PSR merges the partial reconstructions into a watertight mesh. No re-training of these components is required (Uğur et al., 21 Aug 2025).

The processing sequence is structured as follows. Each input view is first passed through PIXIE to obtain a per-view SMPL-X estimate. Those per-view estimates are averaged in shape and pose to initialize a single body model $M^b$. JMBO then jointly optimizes that shared body across all views. After body optimization, the method predicts a front clothed normal map from the frontal RGB view and a back clothed normal map from the back RGB view by reusing the same front estimator and applying a known camera rotation. d-BiNI lifts those front and back normal maps into depth maps constrained by body depth, yielding two partial surfaces. IF-Nets+ predicts an occupancy field to fill missing regions between those surfaces, and PSR stitches the front surface, back surface, and completed regions into the final explicit mesh $R$ (Uğur et al., 21 Aug 2025).

This organization implies a division of labor across stages. The multi-view body fit resolves global pose, scale, and limb placement, while the normal-integration stage injects clothing folds, wrinkles, hairstyles, and related surface detail. A plausible implication is that lateral views primarily improve the low-frequency body estimate through JMBO, because the final high-frequency integration is explicitly front/back rather than all-around (Uğur et al., 21 Aug 2025).

3. Joint Multi-view Body Optimization

JMBO is the central optimization stage. The body model is parameterized as

$M^b(\beta,\theta,\psi,t),$

where $\beta$ denotes shape parameters, $\theta$ full-body pose parameters including articulated hands, $\psi$ facial expression parameters, and $N \in \{2,8\}$0 global translation. These variables are jointly optimized and shared across all views (Uğur et al., 21 Aug 2025).

The JMBO objective is

$N \in \{2,8\}$1

with $N \in \{2,8\}$2, $N \in \{2,8\}$3, and $N \in \{2,8\}$4 (Uğur et al., 21 Aug 2025).

The silhouette term is

$N \in \{2,8\}$5

where $N \in \{2,8\}$6 is the rasterized body contour from the current SMPL-X model in view $N \in \{2,8\}$7 and $N \in \{2,8\}$8 is the clothed silhouette from the corresponding image. The normal term is

$N \in \{2,8\}$9

where $S_i^c$0 are body normals rendered from $S_i^c$1, and during JMBO the clothed normals are obtained through

$S_i^c$2

The landmark term is

$S_i^c$3

with MediaPipe landmarks $S_i^c$4, confidences $S_i^c$5, and projected SMPL-X joints $S_i^c$6. The head regularizer is

$S_i^c$7

where $S_i^c$8 is the head pitch vector from SMPL-X and $S_i^c$9 is the canonical forward-facing direction (Uğur et al., 21 Aug 2025).

The paper does not report explicit pose or shape priors beyond $G_F^N$0, nor does it report interpenetration or contact constraints in the JMBO loss. In the reported setting, plausibility is instead supported by the shared SMPL-X structure, calibrated multi-view supervision, and the combination of silhouette, landmark, and normal alignment (Uğur et al., 21 Aug 2025).

Empirically, JMBO improves body alignment substantially. In the ablation comparing front-view SMPL-X fitting, naïve multi-view averaging, and JMBO, the total loss per image decreases from 0.229 for front-view SMPL-X to 0.113 for naïve multi-view averaging and to 0.059 for JMBO. The paper summarizes this as a reduction of more than 74% versus the single-view baseline (Uğur et al., 21 Aug 2025).

4. Front/back normal estimation, integration, and explicit surface reconstruction

After body optimization, MExECON estimates clothed normal maps for the front and back. The front normal map follows the ECON-style front branch:

$G_F^N$1

The back normal map is produced without a separate back-view network:

$G_F^N$2

This reuse of the front estimator, together with the known camera rotation, is one of the method’s stated novelties (Uğur et al., 21 Aug 2025).

The predicted front and back normal maps are then converted to depth maps through d-BiNI:

$G_F^N$3

Here $G_F^N$4 and $G_F^N$5 are body depth maps rendered from the optimized SMPL-X prior. The role of d-BiNI is to jointly optimize front and back depth maps while constraining their junction with the body prior, thereby producing two partial surfaces $G_F^N$6 and $G_F^N$7 aligned to the low-frequency body estimate (Uğur et al., 21 Aug 2025).

Missing geometry between front and back is then completed by IF-Nets+, which predicts a full occupancy field using the optimized body prior together with the two depth maps. A mesh is extracted from that occupancy field, and PSR finally stitches the IF-Nets+ completion with the front and back partial reconstructions into a watertight explicit mesh:

$G_F^N$8

The output is therefore an explicit watertight clothed mesh rather than a purely implicit representation (Uğur et al., 21 Aug 2025).

The paper frames this stage as a decomposition into low- and high-frequency geometry. The optimized body $G_F^N$9 provides low-frequency shape and pose, fixes geometric scale, and stabilizes limb placement. High-frequency geometry—clothing folds, wrinkles, hairstyles, and accessories—is added through the integrated front/back normal maps. Using an actual back RGB image materially improves back-side detail, including hoods, belts, backpacks, and ponytails or buns (Uğur et al., 21 Aug 2025).

5. Evaluation protocol and empirical performance

The evaluation uses the THuman 2.1 test set with 20 clothed avatars, rendered from eight synthetic cameras in Blender at 1080×1920 resolution with known intrinsics and extrinsics. The reported metrics are Chamfer Distance (mm), P2S Accuracy (mm), P2S Completeness (mm), and Normal Consistency (deg), averaged over the 20 avatars (Uğur et al., 21 Aug 2025).

The final geometry comparison is summarized below.

Method	Geometry metrics	Normal error
ECON	Chamfer 40.56; P2S Acc 39.52; P2S Comp 37.21	45.79°
2-view MExECON	Chamfer 32.76; P2S Acc 29.72; P2S Comp 31.18	44.03°
8-view MExECON	Chamfer 23.12; P2S Acc 21.26; P2S Comp 22.84	39.54°
8-view VGGT	Chamfer 23.87; P2S Acc 22.21; P2S Comp 19.84	41.38°

Against ECON, the 8-view configuration reduces Chamfer Distance by 42% from 40.56 to 23.12 mm, P2S Accuracy by 46% from 39.52 to 21.26 mm, P2S Completeness by 38% from 37.21 to 22.84 mm, and Normal Consistency error by 13% from 45.79° to 39.54°. Compared with VGGT, MExECON is slightly worse in completeness but better in normal error; the paper associates this with VGGT producing smoother geometry and MExECON preserving finer surface detail (Uğur et al., 21 Aug 2025).

The body-optimization ablation is also quantitatively specific. Front-view SMPL-X fitting yields silhouette loss 0.103, landmark loss 1.259, normal loss 0.013, and total loss 0.229. Naïve multi-view averaging improves these to 0.092, 0.169, 0.012, and 0.113. JMBO further reduces them to 0.041, 0.153, 0.008, and 0.059 (Uğur et al., 21 Aug 2025).

The reported runtimes and memory footprints were obtained on a single NVIDIA A100 40 GB GPU. ECON requires about 45 s and about 8 GB peak memory; 2-view MExECON requires about 1 min and about 12 GB; 8-view MExECON requires about 2.5 min and about 24 GB; and 8-view VGGT requires about 5 s and about 14 GB. The paper attributes the higher memory demand of 8-view MExECON to per-iteration multi-view gradient computation and notes that batching views can reduce memory (Uğur et al., 21 Aug 2025).

Qualitatively, the method improves uprightness and pose stability relative to single-view ECON, which can lean or exaggerate the torso under frontal ambiguity. The reported visual gains are most pronounced on the back side and in accessories or hairstyles that are weakly constrained in a frontal-only setting (Uğur et al., 21 Aug 2025).

6. Operating conditions, failure modes, and practical use

MExECON is explicitly conditioned on camera calibration. Errors in intrinsics or extrinsics degrade multi-view consistency and normal alignment. It also assumes a static subject; motion across views would violate the shared-body formulation and, according to the technical description, would require per-view pose variables that the method does not include (Uğur et al., 21 Aug 2025).

View sparsity remains relevant even though the method works with as few as two views. Two views already improve markedly over single-view ECON, but additional views continue to improve accuracy. At the same time, lateral detail remains less directly enforced than front and back detail, because the final normal integration is front/back only. This means that side regions can remain underconstrained in very sparse capture settings (Uğur et al., 21 Aug 2025).

The paper identifies several failure cases. Arms or legs close to the torso can create ambiguous depths for d-BiNI and IF-Nets+, producing bulges or incomplete limbs; examples are reported for avatars 0083, 0023, and 0071. Loose clothing and topology changes can also be difficult for IF-Nets+ completion and PSR. Back-side hairstyles and accessories are generally improved, but lateral accessories remain harder because the integration design does not directly impose side-view surface normals. The output is geometry only; multi-view texturing is not part of the pipeline (Uğur et al., 21 Aug 2025).

For practical capture, the recommended minimum is a front and a back view, while the best reported results come from 6–8 evenly spaced views around the subject at consistent elevation. Accurate calibration, reliable silhouette extraction, and good MediaPipe landmark quality are treated as critical preprocessing requirements. The method is positioned for high-fidelity avatars in VR/AR, telepresence, content creation, gaming, and metaverse platforms, where an explicit mesh with an articulated SMPL-X prior is useful for animation and editing (Uğur et al., 21 Aug 2025).

In that sense, MExECON occupies a specific point in the clothed-human reconstruction landscape: it does not seek end-to-end retraining or fully general dense-view fusion, but rather combines a shared parametric body prior, calibrated multi-view optimization, and explicit front/back normal integration to obtain higher-fidelity clothed meshes from sparse RGB input (Uğur et al., 21 Aug 2025).