Learning to Discover Cross-Domain Relations with Generative Adversarial Networks

Abstract: While humans easily recognize relations between data from different domains without any supervision, learning to automatically discover them is in general very challenging and needs many ground-truth pairs that illustrate the relations. To avoid costly pairing, we address the task of discovering cross-domain relations given unpaired data. We propose a method based on generative adversarial networks that learns to discover relations between different domains (DiscoGAN). Using the discovered relations, our proposed network successfully transfers style from one domain to another while preserving key attributes such as orientation and face identity. Source code for official implementation is publicly available https://github.com/SKTBrain/DiscoGAN

• The paper introduces DiscoGAN, a framework that learns bidirectional mappings between unpaired domains without explicit supervision.
• It employs reconstruction loss to ensure one-to-one correspondences, effectively preventing mode collapse in GAN training.
• Experimental results demonstrate successful translations in varied applications such as image stylization, face attribute conversion, and sketch-to-photo synthesis.

Learning to Discover Cross-Domain Relations with Generative Adversarial Networks

The paper entitled "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks" by Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim introduces a novel framework for discovering relations between unpaired data from different domains using Generative Adversarial Networks (GANs). The method, referred to as DiscoGAN, is groundbreaking in its ability to learn mappings between domains without relying on explicitly paired datasets.

Overview of the Research

The intrinsic challenge addressed by this research is the automatic discovery of relations between different data domains, a task that humans can perform intuitively but poses significant challenges for machines, traditionally requiring extensive supervision. The proposed methodology leverages GANs to map instances between domains and relies on unsupervised learning, eliminating the need for costly paired data.

Core Contributions

1. DiscoGAN Architecture: The core of the proposed DiscoGAN architecture leverages two intertwined GAN models, each dedicated to learning the mapping function between two domains, ensuring that the translation is bidirectional and covers both directions comprehensively.
2. Reconstruction Loss: By incorporating a reconstruction loss, DiscoGAN ensures that each generated image in the target domain accurately represents the corresponding image in the source domain. This bijective mapping safeguards against mode collapse and maintains a one-to-one relation.
3. Applicability Without Explicit Pairing: The model operates effectively on datasets collected independently without any need for explicit labeling or annotated supervision, making the approach more scalable and adaptable to various applications.

Experimental Validation

The paper presents a thorough experimental validation encompassing toy domains and real-world image datasets, highlighting the robustness and versatility of DiscoGAN.

Toy Experiment

The toy experiment with 2D Gaussian mixtures validates the theoretical model's ability to avoid mode collapse and discover meaningful correspondences between modes in different domains. The colored background visualizes discriminator outputs, demonstrating that unlike baseline models, DiscoGAN maps each mode in domain A to a distinct mode in domain B without collapsing multiple domain A modes into one domain B mode.

Real-World Experiments

1. Car to Car & Face to Face Translation: These experiments demonstrate DiscoGAN's ability to translate images in domains where the primary shared feature is azimuth rotation. The results highlight a strong correlation between the predicted angles of input and translated images, underscoring DiscoGAN's proficiency in capturing and translating orientation features.
2. Face Attribute Conversion: Applying DiscoGAN to the CelebA dataset for tasks such as gender and hair color transformation demonstrates the model's capability to change specific attributes while preserving other facial features. This also extends to sequential and repeated attribute modifications, showcasing the model's stability and consistency.
3. Cross-Domain Visual Feature Translation: In experiments involving visually distinct domains (e.g., chairs to cars, cars to faces), DiscoGAN consistently captures and translates shared orientation features, despite significant visual differences between the source and target domains.
4. Edges-to-Photos: The model excels in translating sketches to colored images, showing its ability to handle 1-to-N mappings effectively by generating realistic photos with diverse colorization from a single edge image.
5. Handbags to Shoes: This represents the application of DiscoGAN to domains with less obvious shared features, yet the model successfully captures and translates fashion styles, maintaining formality and patterns across the domains.

Implications and Future Directions

The proposed DiscoGAN framework holds substantial practical implications for cross-domain image translation tasks, including applications in computer vision, creative industries, and pattern recognition. The ability to perform these translations without explicit pairing opens avenues for more robust and flexible AI systems.

Theoretical Impact

The theoretical contributions underscore the importance of bidirectional mapping and reconstruction loss in avoiding mode collapse in GANs, particularly in unsupervised settings. The work points towards future explorations in mixed-modal translations, such as linking textual descriptions with visual images, enhancing the model's versatility and application breadth.

In conclusion, the paper introduces DiscoGAN, a GAN-based model uniquely capable of discovering cross-domain relations without supervised pairing. The method's proficiency in maintaining feature integrity across translations, combined with its robust experimental validation, underscores its potential to advance the state of cross-domain generative models.