SMPLicit: Topology-aware Generative Model for Clothed People (2103.06871v2)

Abstract: In this paper we introduce SMPLicit, a novel generative model to jointly represent body pose, shape and clothing geometry. In contrast to existing learning-based approaches that require training specific models for each type of garment, SMPLicit can represent in a unified manner different garment topologies (e.g. from sleeveless tops to hoodies and to open jackets), while controlling other properties like the garment size or tightness/looseness. We show our model to be applicable to a large variety of garments including T-shirts, hoodies, jackets, shorts, pants, skirts, shoes and even hair. The representation flexibility of SMPLicit builds upon an implicit model conditioned with the SMPL human body parameters and a learnable latent space which is semantically interpretable and aligned with the clothing attributes. The proposed model is fully differentiable, allowing for its use into larger end-to-end trainable systems. In the experimental section, we demonstrate SMPLicit can be readily used for fitting 3D scans and for 3D reconstruction in images of dressed people. In both cases we are able to go beyond state of the art, by retrieving complex garment geometries, handling situations with multiple clothing layers and providing a tool for easy outfit editing. To stimulate further research in this direction, we will make our code and model publicly available at http://www.iri.upc.edu/people/ecorona/smplicit/.

• The paper introduces a unified model that consolidates various garment topologies into one system for flexible 3D clothing representation.
• It employs an implicit function conditioning mechanism using SMPL human-body parameters to facilitate realistic garment deformations.
• Experimental results show that SMPLicit outperforms specialized models in both 3D scan fitting and image reconstruction tasks.

SMPLicit: Topology-aware Generative Model for Clothed People

The paper "SMPLicit: Topology-aware Generative Model for Clothed People" introduces an innovative approach to modeling clothed human figures in 3D, extending beyond the conventional garment representation that primarily focuses on basic displacements from a standard human mesh. The core of the paper lies in the development of a unified model capable of accommodating a wide range of garment topologies using a low-dimensional, semantically interpretable latent space. Unlike previous methods that necessitate separate training for each type of garment, SMPLicit consolidates different garment styles and anatomical configurations into a singular framework.

Key Contributions

The paper outlines several contributions of SMPLicit:

1. Unified Representation: The model represents various garment topologies, such as sleeveless tops, hoodies, and open jackets, within a single generative system. This flexibility enables efficient manipulation of garment characteristics such as size and fit.
2. Implicit Function Conditioning: SMPLicit employs a conditioning mechanism using SMPL human-body parameters paired with a latent space that aligns with clothing attributes. This implicit function network facilitates the generation of garment deformations accurately reflecting body pose and shape.
3. Generation and Editing Capabilities: Beyond the basic representation, SMPLicit offers tools for garment editing, which includes switching garments and repositioning body posture, thus supporting applications like virtual try-on solutions.
4. Differentiable Integration: The model’s differentiable nature allows seamless integration into deep learning systems, fostering broader applications in digital animation and 3D reconstruction from image data.

Results and Demonstrations

The paper provides experimental demonstrations wherein SMPLicit exceeds current state-of-the-art methodologies in two primary scenarios: fitting to 3D scans and reconstructing 3D figures from images. Within the experimental setup:

• 3D Scan Fitting: The model achieves comparable or superior fitting results relative to specialized models trained for each garment type, such as TailorNet and CAPE, without requiring distinct models for each clothing type.
• 3D Image Reconstruction: SMPLicit showcases robust 3D reconstruction capabilities, adapting to complex garment geometries and multiple clothing layers present in real-world images.

Implications and Future Directions

The implications of SMPLicit are multifaceted, impacting both theoretical and practical domains. Theoretically, the integration of topology flexibility into generative garment modeling advances the understanding of model capabilities and limitations. Practically, SMPLicit's application in areas like virtual fitting, gaming, and film production holds potential for significant innovation, offering nuanced control over visual assets.

Future research might explore expanding the dimensionality of garment control features or integrating more diverse data types to further refine the model's accuracy and applicability. Additionally, advancements in computational efficiency and real-time processing could enhance its utility in interactive and responsive environments.

In conclusion, SMPLicit represents a significant step forward in generative clothing models, unifying various garments within a single robust framework, and providing flexible 3D content generation and manipulation capabilities. This paper lays a foundation for further exploration into clothing representation and manipulation in virtual environments.