2000 character limit reached
Towards Mode Balancing of Generative Models via Diversity Weights (2304.11961v3)
Published 24 Apr 2023 in cs.LG and cs.AI
Abstract: Large data-driven image models are extensively used to support creative and artistic work. Under the currently predominant distribution-fitting paradigm, a dataset is treated as ground truth to be approximated as closely as possible. Yet, many creative applications demand a diverse range of output, and creators often strive to actively diverge from a given data distribution. We argue that an adjustment of modelling objectives, from pure mode coverage towards mode balancing, is necessary to accommodate the goal of higher output diversity. We present diversity weights, a training scheme that increases a model's output diversity by balancing the modes in the training dataset. First experiments in a controlled setting demonstrate the potential of our method. We discuss connections of our approach to diversity, equity, and inclusion in generative machine learning more generally, and computational creativity specifically. An implementation of our algorithm is available at https://github.com/sebastianberns/diversity-weights
- 2018. Discriminator Rejection Sampling. In ICLR.
- 2018. A Note on the Inception Score. In Proceedings of the ICML Workshop on Theoretical Foundations and Applications of Deep Generative Models.
- 2020. Bridging Generative Deep Learning and Computational Creativity. In Proceedings of ICCC.
- 2021. Automating Generative Deep Learning for Artistic Purposes: Challenges and Opportunities. In Proceedings of ICCC.
- Boden, M. A. 2004. The Creative Mind: Myths and Mechanisms. Routledge, second edition.
- 2016. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Advances in NeurIPS, volume 29.
- 2014. Parameters Tell the Design Story: Ideation and Abstraction in Design Optimization. In Proceedings of the Symposium on Simulation for Architecture & Urban Design.
- 2021. Active Divergence with Generative Deep Learning - A Survey and Taxonomy. In Proceedings of ICCC.
- Colton, S. 2022. Towards Educating Artificial Neural Systems. In Proceedings of the International Workshop on Neuro-Symbolic Learning and Reasoning.
- 2022. The Vendi Score: A Diversity Evaluation Metric for Machine Learning. ArXiv:2210.02410v1.
- 2019. Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting. In Advances in NeurIPS, volume 32.
- 2017. Improved Training of Wasserstein GANs. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in NeurIPS, volume 30.
- 2018. Women also Snowboard: Overcoming Bias in Captioning Models. In Proceedings of ECCV.
- 2017. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Advances in NeurIPS, volume 30.
- 2022. Deduplicating Training Data Mitigates Privacy Risks in Language Models. In Proceedings of ICML.
- 2017. Apocrita - High Performance Computing Cluster for Queen Mary University of London. DOI:10.5281/zenodo.438045.
- 2015. Adam: A Method for Stochastic Optimization. In ICLR.
- 2023. Large-scale text-to-image generation models for visual artists’ creative works. In International Conference on Intelligent User Interfaces.
- Kolmogorov, A. 1933. Grundbegriffe der Wahrscheinlichkeitsrechnung. Springer-Verlag.
- 2019. Improved Precision and Recall Metric for Assessing Generative Models. In Advances in NeurIPS.
- 2023. The Role of ImageNet Classes in Fréchet Inception Distance. In ICLR.
- 2021. Self-Diagnosing GAN: Diagnosing Underrepresented Samples in Generative Adversarial Networks. In Advances in NeurIPS, volume 34.
- 2014. Computational Game Creativity. In Proceedings of ICCC.
- 2017. Application Domains Considered in Computational Creativity. In Proceedings of ICCC.
- Loughran, R. 2022. Bias and Creativity. In Proceedings of ICCC.
- 2023. Large language models generate functional protein sequences across diverse families. Nature Biotechnology.
- 2022. On Aliased Resizing and Surprising Subtleties in GAN Evaluation. In Proceedings of CVPR.
- 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of ICML.
- 2021. Zero-Shot Text-to-Image Generation. In Proceedings of ICML.
- 2022. Impact of Pretraining Term Frequencies on Few-Shot Reasoning. ArXiv:2202.07206v2.
- Ritchie, G. 2007. Some Empirical Criteria for Attributing Creativity to a Computer Program. Minds and Machines 17(1).
- 2012. The Standard Definition of Creativity. Creativity Research Journal 24(1).
- 2022. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. In Advances in NeurIPS, volume 35.
- 2016. Improved Techniques for Training GANs. In Advances in NeurIPS, volume 29.
- Smith, G. 2017. Computational Creativity and Social Justice: Defining the Intellectual Landscape. In Proceedings of the Workshop on Computational Creativity and Social Justice at ICCC.
- 2015. Why Greatness Cannot Be Planned: The Myth of the Objective. Springer.
- 2018. ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases. In Proceedings of ECCV.
- 2016. Rethinking the Inception Architecture for Computer Vision. In Proceedings of CVPR.
- 2019. Systematizing Creativity: A Computational View. Computational Creativity: The Philosophy and Engineering of Autonomously Creative Systems 1–19.
- 2023. “An Adapt-or-Die Type of Situation”: Perception, Adoption, and Use of Text-To-Image-Generation AI by Game Industry Professionals. ArXiv:2302.12601v3.
- 2018. Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network. In Proceedings of GECCO.
- 2020. Inclusive GAN: Improving Data and Minority Coverage in Generative Models. In Vedaldi, A.; Bischof, H.; Brox, T.; and Frahm, J.-M., eds., Proceedings of ECCV. Springer International Publishing.
- 2017. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
- 2019. Rethinking Generative Mode Coverage: A Pointwise Guaranteed Approach. In Advances in NeurIPS, volume 32.