TryOnGAN: Body-Aware Try-On via Layered Interpolation (2101.02285v2)

Abstract: Given a pair of images-target person and garment on another person-we automatically generate the target person in the given garment. Previous methods mostly focused on texture transfer via paired data training, while overlooking body shape deformations, skin color, and seamless blending of garment with the person. This work focuses on those three components, while also not requiring paired data training. We designed a pose conditioned StyleGAN2 architecture with a clothing segmentation branch that is trained on images of people wearing garments. Once trained, we propose a new layered latent space interpolation method that allows us to preserve and synthesize skin color and target body shape while transferring the garment from a different person. We demonstrate results on high resolution 512x512 images, and extensively compare to state of the art in try-on on both latent space generated and real images.

• The paper presents a novel GAN-based approach using layered latent space interpolation to synthesize photorealistic garment try-on images.
• It employs a pose-conditioned StyleGAN2 and segmentation branch to disentangle pose from style while preserving personal identity.
• Extensive experiments reveal improved FID scores and user preference, demonstrating superior performance over existing virtual try-on methods.

Overview of "TryOnGAN: Body-Aware Try-On via Layered Interpolation"

The paper "TryOnGAN: Body-Aware Try-On via Layered Interpolation" presents an approach to virtual clothes try-on using GANs, with a focus on personalizing garment fitting to different body shapes, preserving skin color, and achieving seamless integration. Virtual try-on systems aim to computationally visualize garments on a person, potentially revolutionizing the apparel shopping experience by offering high-quality visualizations that faithfully represent body shape and garment details.

Methodology

The authors develop a model based on StyleGAN2, a well-regarded generative adversarial network architecture known for producing high-fidelity images. Key components of their method include:

• Pose-conditioned Model: The use of a pose-conditioned StyleGAN2 architecture allows the method to disentangle pose from style, crucial for maintaining the person's identity while altering the garment.
• Segmentation Branch: By incorporating a segmentation branch, the model can segment garments from images, aiding the localization of garment regions for targeted style transfer.
• Layered Latent Space Interpolation: The heart of TryOnGAN lies in the interpolation of layers within the StyleGAN2 network. By optimizing interpolation coefficients in latent space, the method can adaptively synthesize the desired garment over the target person while preserving identity and body shape.

Their approach does not require paired training data, instead relying on unpaired images, which is advantageous since such data is abundant and diverse garment-identity combinations can be learned.

Experimental Evaluation

The authors report extensive experiments on high-resolution images of 512×512 pixels, showcasing the superiority of TryOnGAN over existing methods such as ADGAN and CPVITON. Quantitatively, they demonstrate this through improved FID scores, indicating enhanced photorealism, and qualitative evaluations showing better detail preservation and garment shape continuity. Furthermore, human participant studies corroborate these improvements, preferring TryOnGAN results for their quality.

Strong Results and Claims

The paper asserts significant advances over state-of-the-art in handling variations in body shape and garment texture complexities, particularly excelling in maintaining photorealism and detail in synthesized images.

Implications and Future Directions

The implications of this research span both academic and commercial realms. From a theoretical perspective, it introduces a novel interpolation-based method for garment synthesis, promising a pathway to more sophisticated image editing and synthetic image generation. Practically, developing such virtual try-on systems could drastically improve the consumer experience in online retail by offering a more accurate depiction of clothing fit and style on individual bodies.

Despite the advancements, the authors acknowledge limitations, particularly in realism when synthesizing images with extreme poses or rare garment attributes not present in the training data. Future work could focus on these atypical cases, improving projection methods into the GAN latent space, and refining layer interpolation techniques.

In conclusion, this paper represents a solid contribution to the AI field, advancing the capabilities of virtual personal garment try-on through innovative GAN-based interpolation methods. It sets a foundation for continued efforts enhancing personalization in fashion technology applications.