RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes (1908.07269v1)

Abstract: Multi-domain image-to-image translation has gained increasing attention recently. Previous methods take an image and some target attributes as inputs and generate an output image with the desired attributes. However, such methods have two limitations. First, these methods assume binary-valued attributes and thus cannot yield satisfactory results for fine-grained control. Second, these methods require specifying the entire set of target attributes, even if most of the attributes would not be changed. To address these limitations, we propose RelGAN, a new method for multi-domain image-to-image translation. The key idea is to use relative attributes, which describes the desired change on selected attributes. Our method is capable of modifying images by changing particular attributes of interest in a continuous manner while preserving the other attributes. Experimental results demonstrate both the quantitative and qualitative effectiveness of our method on the tasks of facial attribute transfer and interpolation.

• The paper introduces relative attributes that enable fine-grained control, overcoming binary limitations in traditional image translation.
• It employs a single generator with three discriminators to ensure output realism, attribute accuracy, and smooth interpolation.
• Experimental results on datasets like CelebA show improved visual quality and attribute classification over methods such as StarGAN and AttGAN.

RelGAN: Multi-Domain Image-to-Image Translation via Relative Attributes

The paper presents a comprehensive methodology for advancing multi-domain image-to-image translation leveraging generative adversarial networks (GANs), specifically through the introduction of RelGAN. The novelty in RelGAN lies in its utilization of relative attributes to facilitate attribute modifications across multiple domains, as opposed to traditional methods that assume binary-valued attributes. This relative attribute mechanism directly addresses the constraints found in earlier models which lack fine-grained control and require full attribute specifications even when only minor changes are desired.

Key Contributions

1. Relative Attributes for Fine-Grained Control: The concept of relative attributes allows users to specify changes in a continuous manner, thereby overcoming the binary limitations and facilitating realistic interpolations between attribute states. This approach starkly contrasts with the target-attribute-based formulations of previous methods.
2. Generator and Discriminator Design: The RelGAN model comprises a single generator and three bespoke discriminators. Each discriminator is tailored to ensure various aspects of quality and realism in generated images—covering the realism of output images, accuracy of translations as per relative attributes, and quality of interpolations.
3. Improved Interpolations: The model incorporates an interpolation discriminator to improve the quality of attribute interpolation. This allows RelGAN to provide a more authentic and smoothly-varying transition between the input and the desired output across a spectrum of attribute intensity.

Experimental and Quantitative Evidence

RelGAN's capabilities are demonstrated through extensive experimentation on datasets such as CelebA, CelebA-HQ, and FFHQ. Notably, the experimental results underline RelGAN's superiority over state-of-the-art methods like StarGAN and AttGAN:

• Visual Quality: The use of Fréchet Inception Distance (FID) as a metric underscores RelGAN's ability to produce visually convincing images, with lower FID scores compared to its counterparts across different datasets and attribute categories.
• Attribute Classification: By employing a pre-trained classifier on generated images, the paper measures the classification accuracy of facial attributes. RelGAN exhibited higher accuracy in preserving and modifying attributes correctly, thereby indicating more effective attribute transfer.
• Image Reconstruction and Interpolation: For facial image reconstruction, RelGAN achieves notable performance, indicative of its prowess in preserving unchanged attributes. In interpolation tasks, standard deviation of SSIM is utilized to evaluate smoothness, with RelGAN achieving superior performance through less abrupt changes as compared to comparable frameworks.

Implications and Future Work

RelGAN advances the field of image synthesis by addressing fundamental limitations in attribute modification, paving the way for applications needing nuanced control over image alterations. The development of relative attribute vectors uniquely positions RelGAN to serve practical use cases where precise control over image attributes and a high degree of realism are pivotal.

Future explorations could further optimize adversarial mechanisms and leverage mask techniques to enhance attribute manipulations' reliability and visual fidelity. Additionally, broader applications in synthetic data generation for training machine learning models, enhanced image editing tools, and augmented reality applications stand to benefit from these foundational advances.

This paper marks a substantive contribution to the domain of generative models, particularly in managing multiple attribute changes within images in a user-friendly and semantically meaningful manner.