Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
107 tokens/sec
Gemini 2.5 Pro Premium
58 tokens/sec
GPT-5 Medium
29 tokens/sec
GPT-5 High Premium
25 tokens/sec
GPT-4o
101 tokens/sec
DeepSeek R1 via Azure Premium
84 tokens/sec
GPT OSS 120B via Groq Premium
478 tokens/sec
Kimi K2 via Groq Premium
213 tokens/sec
2000 character limit reached

Tackling the Singularities at the Endpoints of Time Intervals in Diffusion Models (2403.08381v2)

Published 13 Mar 2024 in cs.CV

Abstract: Most diffusion models assume that the reverse process adheres to a Gaussian distribution. However, this approximation has not been rigorously validated, especially at singularities, where t=0 and t=1. Improperly dealing with such singularities leads to an average brightness issue in applications, and limits the generation of images with extreme brightness or darkness. We primarily focus on tackling singularities from both theoretical and practical perspectives. Initially, we establish the error bounds for the reverse process approximation, and showcase its Gaussian characteristics at singularity time steps. Based on this theoretical insight, we confirm the singularity at t=1 is conditionally removable while it at t=0 is an inherent property. Upon these significant conclusions, we propose a novel plug-and-play method SingDiffusion to address the initial singular time step sampling, which not only effectively resolves the average brightness issue for a wide range of diffusion models without extra training efforts, but also enhances their generation capability in achieving notable lower FID scores.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. Brian D.O. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, pages 313–326, 1982.
  2. AUTOMATIC1111. Stable diffusion web ui. https://github.com/AUTOMATIC1111/stable-diffusion-webui, 2022.
  3. Blended diffusion for text-driven editing of natural images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18208–18218, 2022.
  4. A continuous time framework for discrete denoising models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
  5. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems (NeurIPS), pages 8780–8794, 2021.
  6. Score-based generative modeling with critically-damped langevin diffusion. In International Conference on Learning Representations (ICLR), 2022.
  7. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12873–12883, 2021.
  8. Generative adversarial nets. In Advances in Neural Information Processing Systems, 2014.
  9. Nicholas Guttenberg. Diffusion with offset noise. https://www.crosslabs.org/blog/diffusion-with-offset-noise, 2023.
  10. Prompt-to-prompt image editing with cross-attention control. In International Conference on Learning Representations (ICLR), 2023.
  11. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems (NeurIPS), 2017.
  12. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
  13. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), pages 6840–6851, 2020.
  14. Video diffusion models. In ICLR Workshop on Deep Generative Models for Highly Structured Data, 2022.
  15. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems (NeurIPS), pages 26565–26577, 2022.
  16. Imagic: Text-based real image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6007–6017, 2023.
  17. Variational diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), pages 21696–21707, 2021.
  18. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations (ICLR), 2021.
  19. Common diffusion noise schedules and sample steps are flawed. arXiv preprint arXiv:2305.08891, 2023.
  20. Microsoft coco: Common objects in context. In European Conference on Computer Vision (ECCV), pages 740–755, 2014.
  21. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
  22. Mathematical analysis of singularities in the diffusion model under the submanifold assumption. arXiv preprint arXiv:2301.07882, 2023.
  23. Srflow: Learning the super-resolution space with normalizing flow. In European Conference on Computer Vision (ECCV), page 715–732, 2020.
  24. David McAllester. On the mathematics of diffusion models. arXiv preprint arXiv:2301.11108, 2023.
  25. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations (ICLR), 2022.
  26. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning (ICML), pages 16784–16804, 2022.
  27. Sdxl: improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023.
  28. Grad-tts: A diffusion probabilistic model for text-to-speech. In International Conference on Machine Learning (ICML), pages 8599–8608, 2021.
  29. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), pages 8748–8763, 2021.
  30. Zero-shot text-to-image generation. In International Conference on Machine Learning (ICML), pages 8821–8831, 2021.
  31. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022.
  32. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), pages 234–241, 2015.
  33. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22500–22510, 2023.
  34. Palette: Image-to-image diffusion models. In ACM SIGGRAPH Conference Proceedings, 2022a.
  35. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems (NeurIPS), pages 36479–36494, 2022b.
  36. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations (ICLR), 2022.
  37. LAION-5b: An open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems (NeurIPS), pages 25278–25294, 2022.
  38. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning (ICML), pages 2256–2265, 2015.
  39. Denoising diffusion implicit models. In International Conference on Learning Representations (ICLR), 2021a.
  40. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations (ICLR), 2021b.
  41. Dual diffusion implicit bridges for image-to-image translation. In International Conference on Learning Representations (ICLR), 2023.
  42. Diffusers: State-of-the-art diffusion models. https://github.com/huggingface/diffusers, 2022.
  43. Paint by example: Exemplar-based image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18381–18391, 2023.
  44. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, 2023a.
  45. Exploring dual-task correlation for pose guided person image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7713–7722, 2022.
  46. Formulating discrete probability flow through optimal transport. In Advances in Neural Information Processing Systems (NeurIPS), 2023b.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets