MineGAN: effective knowledge transfer from GANs to target domains with few images (1912.05270v3)

Abstract: One of the attractive characteristics of deep neural networks is their ability to transfer knowledge obtained in one domain to other related domains. As a result, high-quality networks can be trained in domains with relatively little training data. This property has been extensively studied for discriminative networks but has received significantly less attention for generative models. Given the often enormous effort required to train GANs, both computationally as well as in the dataset collection, the re-use of pretrained GANs is a desirable objective. We propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs. This is done using a miner network that identifies which part of the generative distribution of each pretrained GAN outputs samples closest to the target domain. Mining effectively steers GAN sampling towards suitable regions of the latent space, which facilitates the posterior finetuning and avoids pathologies of other methods such as mode collapse and lack of flexibility. We perform experiments on several complex datasets using various GAN architectures (BigGAN, Progressive GAN) and show that the proposed method, called MineGAN, effectively transfers knowledge to domains with few target images, outperforming existing methods. In addition, MineGAN can successfully transfer knowledge from multiple pretrained GANs. Our code is available at: https://github.com/yaxingwang/MineGAN.

• The paper introduces a novel miner network that efficiently refines the latent space to transfer pretrained GAN knowledge to target domains with few images.
• The paper addresses mode collapse by steering the generator towards productive latent regions, ensuring diverse and high-fidelity outputs.
• The paper validates its approach with superior FID metrics across various pretrained architectures, proving its scalable application to conditional and multiple GAN integrations.

MineGAN: Knowledge Transfer from GANs to New Domains with Limited Data

This paper introduces MineGAN, a novel method for transferring knowledge from pretrained Generative Adversarial Networks (GANs) to target domains with limited available data. The foundational idea behind MineGAN is to effectively leverage existing GANs to minimize the need for extensive data retraining, which is usually computationally expensive and resource-intensive. This approach is particularly relevant in scenarios where only a small amount of target domain images is available.

Key Contributions

1. Miner Network Concept: MineGAN introduces a miner network, which modifies the sampling distribution of the input latent space to steer the pretrained GAN towards generating images that closely resemble a specific target domain. The miner methodically identifies productive regions within the latent space of a pre-trained GAN, facilitating efficient knowledge transfer.
2. Avoiding Mode Collapse: Traditional methods of fine-tuning GANs can suffer from mode collapse, where the generative model outputs repetitive, low-diversity results. MineGAN effectively addresses this by utilizing the miner network to guide the generator to appropriate zones in the latent space, thus preserving diversity and improving image quality.
3. Integration of Multiple Pretrained GANs: MineGAN extends beyond singular models by enabling the transfer of knowledge from multiple pretrained GANs to a single target domain. This integration is managed by assigning sampling probabilities to different generators based on their relevance to the target distribution.
4. Quantitative Performance: The paper provides comprehensive experimental validation across various datasets and GAN architectures, such as Progressive GAN and BigGAN. MineGAN exhibits superior performance over TransferGAN and other baseline methods in terms of Frechet Inception Distance (FID), which quantifies how closely the generated images resemble real sample distributions.
5. Applications to Conditional GANs: The method is also applicable to conditional GANs, demonstrating flexibility in generating domain-specific images in both on-manifold and off-manifold scenarios.

Implications

MineGAN offers a robust framework for efficient knowledge transfer in generative models, significantly reducing the demand for large target datasets. Practically, this could benefit fields such as digital content creation, where synthesizing realistic imagery in niche domains is desired but lacks extensive data.

Future Directions

The framework opens pathways for further exploration into the aggregation of GANs, potentially employing self-supervised learning methodologies to refine target sampling distributions without labeled data. Moreover, enhancing the miner network's complexity could further aid in the synthesis of even more challenging off-manifold targets.

In conclusion, the MineGAN approach presents a significant advancement in the field of generative modeling through its innovative mechanism for transferring knowledge using minimalized data, offering practical and scalable solutions for industries relying on customized image generation.