Learning Implicit Templates for Point-Based Clothed Human Modeling

Abstract: We present FITE, a First-Implicit-Then-Explicit framework for modeling human avatars in clothing. Our framework first learns implicit surface templates representing the coarse clothing topology, and then employs the templates to guide the generation of point sets which further capture pose-dependent clothing deformations such as wrinkles. Our pipeline incorporates the merits of both implicit and explicit representations, namely, the ability to handle varying topology and the ability to efficiently capture fine details. We also propose diffused skinning to facilitate template training especially for loose clothing, and projection-based pose-encoding to extract pose information from mesh templates without predefined UV map or connectivity. Our code is publicly available at https://github.com/jsnln/fite.

• The paper introduces the FITE framework, integrating implicit template learning via diffused skinning with explicit point sets to capture clothing dynamics.
• It employs a novel projection-based pose encoding strategy that consolidates multi-view position maps to manage complex deformations without predefined UV mapping.
• Empirical results highlight FITE's superior performance in representing diverse and loose clothing with fewer artifacts and improved visual coherence.

An Expert Overview of "Learning Implicit Templates for Point-Based Clothed Human Modeling"

The paper "Learning Implicit Templates for Point-Based Clothed Human Modeling" introduces a novel approach called FITE (First-Implicit-Then-Explicit) for modeling human avatars in clothing. The main novelty of this framework lies in integrating implicit surface templates with explicit point sets to capture the complexities of clothed human figures and the dynamic deformations caused by varying poses. This methodology leverages the strengths of both representations: implicit surfaces provide flexibility in topology management, while explicit point sets deliver efficiency in detailing.

Key Aspects and Methodology

The FITE framework operates in two distinct stages:

1. Implicit Template Learning: This initial phase focuses on capturing the general topology of clothing via implicit surface representations. Here, the authors introduce "diffused skinning," a novel method that effectively extends the SMPL model's skinning weights throughout 3D space. This technique ensures stability and accurate correspondences, which is especially essential for managing loose clothing. The authors emphasize addressing the topology challenge associated with traditional mesh-based approaches by decoupling the canonical shape optimization from skinning weight learning. The diffused skinning approach minimizes over-parameterization and enhances stability in capturing coarse geometries.
2. Point-Based Detailing: After forming the initial implicit templates, the second stage focuses on fine-tuning the model to encapsulate detailed and pose-specific clothing deformations. The authors implement a projection-based pose encoding strategy, an innovative mechanism that leverages rendered multi-view images, or "position maps," to consolidate pose information without pre-defined UV mapping or connectivity. This results in a continuous feature space more suited to managing variations across different attires.

Results and Observations

Empirical assessments demonstrate FITE's superior capacity to handle diverse clothing types compared to leading methods like POP, SNARF, and SCANimate. Specifically, FITE's performance excels at managing outfits that differ significantly from a baseline body model, such as dresses and flowing garments, where existing point-based approaches struggle. The paper reports competitive quantitative results using metrics such as Chamfer-$L_2$ distance and cosine similarity, along with qualitative observations suggesting better visual coherence and fewer artifacts, particularly around joints or complex clothing structures.

Implications and Future Directions

In practice, FITE's contributions are noteworthy for industries relying on high-fidelity virtual human representations, such as gaming, virtual reality, and animation, where accurately rendered clothing can significantly enhance realism. Theoretically, this research highlights a promising direction for combining implicit and explicit forms in 3D modeling, potentially influencing future methodologies in computer graphics and computational geometry.

Future research might extend FITE by consolidating multiple clothing items under unified representations or integrating more sophisticated reposing techniques, possibly through physics-based simulations alongside linear blend skinning. Another trajectory could involve refining disentanglement methods for better identifying and managing clothing versus pose variations in the generated templates.

In conclusion, this paper presents a methodologically robust framework with clear practical and theoretical advancements in the domain of clothed human modeling, reflecting a substantial contribution to computer vision and graphics.