ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes (1803.10562v2)

Abstract: Recent studies on face attribute transfer have achieved great success. A lot of models are able to transfer face attributes with an input image. However, they suffer from three limitations: (1) incapability of generating image by exemplars; (2) being unable to transfer multiple face attributes simultaneously; (3) low quality of generated images, such as low-resolution or artifacts. To address these limitations, we propose a novel model which receives two images of opposite attributes as inputs. Our model can transfer exactly the same type of attributes from one image to another by exchanging certain part of their encodings. All the attributes are encoded in a disentangled manner in the latent space, which enables us to manipulate several attributes simultaneously. Besides, our model learns the residual images so as to facilitate training on higher resolution images. With the help of multi-scale discriminators for adversarial training, it can even generate high-quality images with finer details and less artifacts. We demonstrate the effectiveness of our model on overcoming the above three limitations by comparing with other methods on the CelebA face database. A pytorch implementation is available at https://github.com/Prinsphield/ELEGANT.

• The paper proposes a novel exemplar-based model that exchanges latent encodings between face images to transfer multiple attributes simultaneously.
• It employs disentangled latent partitions and residual learning with multi-scale discriminators to achieve high-fidelity, diverse image generation.
• Experimental results on the CelebA database show improved attribute transfer quality and identity preservation compared to existing methods.

ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes

The paper, titled "ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes," introduces a novel model aimed at addressing several limitations prevalent in face attribute transfer methods. Previous approaches have made significant advances in this field but still face notable challenges, such as the inability to generate images by exemplars, limitations in transferring multiple face attributes simultaneously, and reduced image quality. The proposed ELEGANT framework is designed to effectively tackle these issues.

Key Contributions and Methodology

1. Exemplar-Based Image Generation: The ELEGANT model excels in generating target face attributes as present in reference images by exchanging latent encodings between images. This approach contrasts with previous GAN-based methods that either rely on standard labels without accommodating the inherent diversity of attributes like "bangs" or "smiling." ELEGANT successfully incorporates the specific styles of these attributes from reference images, thus enriching variability.
2. Multiple Attribute Transfer: Unlike several existing methodologies that can only manipulate a single attribute at a time, ELEGANT encodes multiple attributes in a disentangled manner. This allows the model to transfer several attributes simultaneously. This capacity is achieved by dividing latent encodings into different parts, each corresponding to a unique attribute, facilitating the manipulation of multiple attributes without interference.
3. Enhanced Image Quality: To improve the generation quality, ELEGANT employs residual learning strategies and utilizes multi-scale discriminators, which allow the capture of both fine details and holistic image features. The generator emphasizes learning residual images, which simplifies training by focusing changes only on necessary regions of the image. Furthermore, multi-scale discriminators enhance the adversarial training process by focusing on both broad content and intricate details.

Experimental Evaluation and Results

The authors demonstrate the effectiveness of ELEGANT using the CelebA database. The model surpasses existing frameworks like UNIT, CycleGAN, StarGAN, and DNA-GAN in generating exemplar-based images with higher fidelity and varied attribute styles. Specific experiments detail the successful transfer of attributes such as "bangs," "smiling," "eyeglasses," and more. The results highlight ELEGANT's capability in maintaining identity consistency while modifying attributes significantly.

Quantitatively, the paper employs Fréchet Inception Distance (FID) to evaluate image realism and diversity. ELEGANT produces competitive FID scores across multiple attribute tasks, indicating high-quality generation performance compared to other models in the field.

Theoretical and Practical Implications

The introduction of ELEGANT provides a robust platform for improving face attribute transfer tasks. The disentangled representation in latent space not only enhances the control over multiple attributes but also paves the way for further research into more complex and nuanced attribute manipulation. Practically, this model could enhance applications in areas such as digital content creation, virtual avatars, and entertainment media where precise attribute generation and personalization are crucial.

Overall, the ELEGANT model signifies a thoughtful integration of generative adversarial and encoding strategies to resolve pervasive challenges in style variance, attribute multiplicity, and image quality in face attribute transfer. Future research directions could explore expanding this methodology to diverse datasets and refining the disentanglement process for a broader range of applications.