Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Fair Sampling in Diffusion Models through Switching Mechanism (2401.03140v5)

Published 6 Jan 2024 in cs.LG and cs.CY

Abstract: Diffusion models have shown their effectiveness in generation tasks by well-approximating the underlying probability distribution. However, diffusion models are known to suffer from an amplified inherent bias from the training data in terms of fairness. While the sampling process of diffusion models can be controlled by conditional guidance, previous works have attempted to find empirical guidance to achieve quantitative fairness. To address this limitation, we propose a fairness-aware sampling method called \textit{attribute switching} mechanism for diffusion models. Without additional training, the proposed sampling can obfuscate sensitive attributes in generated data without relying on classifiers. We mathematically prove and experimentally demonstrate the effectiveness of the proposed method on two key aspects: (i) the generation of fair data and (ii) the preservation of the utility of the generated data.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (40)
  1. Uncovering and mitigating algorithmic bias through learned latent structure. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, 289–295.
  2. Anderson, B. D. 1982. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3): 313–326.
  3. ediffi: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324.
  4. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22563–22575.
  5. Sega: Instructing diffusion using semantic dimensions. arXiv preprint arXiv:2301.12247.
  6. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11472–11481.
  7. Fair generative modeling via weak supervision. In International Conference on Machine Learning, 1887–1898. PMLR.
  8. Diffedit: Diffusion-based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11427.
  9. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34: 8780–8794.
  10. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, 214–226.
  11. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 259–268.
  12. Fair diffusion: Instructing text-to-image generation models on fairness. arXiv preprint arXiv:2302.10893.
  13. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29.
  14. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770–778.
  15. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840–6851.
  16. Classifier-Free Diffusion Guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
  17. Polarity sampling: Quality and diversity control of pre-trained generative networks via singular values. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10641–10650.
  18. Adversarial examples are not bugs, they are features. Advances in neural information processing systems, 32.
  19. Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, 1548–1558.
  20. FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 1548–1558.
  21. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems, 35: 26565–26577.
  22. On the fairness of generative adversarial networks (gans). In 2021 International Conference” Nonlinearity, Information and Robotics”(NIR), 1–7. IEEE.
  23. Variational diffusion models. Advances in neural information processing systems, 34: 21696–21707.
  24. DiffWave: A Versatile Diffusion Model for Audio Synthesis. In International Conference on Learning Representations.
  25. Learning multiple layers of features from tiny images.
  26. Diffusion Models Already Have A Semantic Latent Space. In The Eleventh International Conference on Learning Representations.
  27. Deep Learning Face Attributes in the Wild. In Proceedings of International Conference on Computer Vision (ICCV).
  28. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, 15(2018): 11.
  29. Learning transferable features with deep adaptation networks. In International conference on machine learning, 97–105. PMLR.
  30. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35: 5775–5787.
  31. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125.
  32. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684–10695.
  33. Fairness GAN: Generating datasets with fairness properties using a generative adversarial network. IBM Journal of Research and Development, 63(4/5): 3–1.
  34. D2c: Diffusion-decoding models for few-shot conditional generation. Advances in Neural Information Processing Systems, 34: 12533–12548.
  35. Denoising Diffusion Implicit Models. In International Conference on Learning Representations.
  36. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations.
  37. Fairness definitions explained. In Proceedings of the international workshop on software fairness, 1–7.
  38. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers.
  39. Fairgan: Fairness-aware generative adversarial networks. In 2018 IEEE International Conference on Big Data (Big Data), 570–575. IEEE.
  40. Fairgan+: Achieving fair data generation and classification through generative adversarial nets. In 2019 IEEE International Conference on Big Data (Big Data), 1401–1406. IEEE.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yujin Choi (10 papers)
  2. Jinseong Park (13 papers)
  3. Hoki Kim (11 papers)
  4. Jaewook Lee (44 papers)
  5. Saerom Park (3 papers)
Citations (4)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets