Observation-Guided Diffusion Probabilistic Models (2310.04041v2)
Abstract: We propose a novel diffusion-based image generation method called the observation-guided diffusion probabilistic model (OGDM), which effectively addresses the tradeoff between quality control and fast sampling. Our approach reestablishes the training objective by integrating the guidance of the observation process with the Markov chain in a principled way. This is achieved by introducing an additional loss term derived from the observation based on a conditional discriminator on noise level, which employs a Bernoulli distribution indicating whether its input lies on the (noisy) real manifold or not. This strategy allows us to optimize the more accurate negative log-likelihood induced in the inference stage especially when the number of function evaluations is limited. The proposed training scheme is also advantageous even when incorporated only into the fine-tuning process, and it is compatible with various fast inference strategies since our method yields better denoising networks using the exactly the same inference procedure without incurring extra computational cost. We demonstrate the effectiveness of our training algorithm using diverse inference techniques on strong diffusion model baselines. Our implementation is available at https://github.com/Junoh-Kang/OGDM_edm.
- Estimating the optimal covariance with imperfect mean in diffusion probabilistic models. In ICML, 2022a.
- Analytic-DPM: An analytic estimate of the optimal reverse variance in diffusion probabilistic models. In ICLR, 2022b.
- Diffusion models beat GANs on image synthesis. In NeurIPS, 2021.
- GENIE: Higher-order denoising diffusion solvers. In NeurIPS, 2022.
- Generative adversarial nets. In NeurIPS, 2014.
- Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
- Denoising diffusion probabilistic models. In NeurIPS, 2020.
- Equivariant diffusion for molecule generation in 3D. In ICML, 2022.
- Diff-TTS: A denoising diffusion model for text-to-speech. In INTERSPEECH, 2021.
- Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080, 2021.
- Training generative adversarial networks with limited data. In NeurIPS, 2020.
- Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022.
- Refining generative process with discriminator guidance in score-based diffusion models. In ICML, 2023.
- On fast sampling of diffusion probabilistic models. In ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021.
- DiffWave: A versatile diffusion model for audio synthesis. In ICLR, 2021.
- Learning multiple layers of features from tiny images. 2009.
- Improved precision and recall metric for assessing generative models. 2019.
- Pseudo numerical methods for diffusion models on manifolds. In ICLR, 2022.
- Deep learning face attributes in the wild. In ICCV, 2015.
- Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388, 2021.
- Diffusion probabilistic models for 3D point cloud generation. In CVPR, 2021.
- On distillation of guided diffusion models. In CVPR, 2023.
- Improved denoising diffusion probabilistic models. In ICML, 2021.
- Permutation invariant graph generation via score-based generative modeling. In AISTATS, 2020.
- High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
- Progressive distillation for fast sampling of diffusion models. In ICLR, 2022.
- Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
- Denoising diffusion implicit models. In ICLR, 2021a.
- Score-based generative modeling through stochastic differential equations. In ICLR, 2021b.
- Consistency models. In ICML, 2023.
- An Introduction to Numerical Analysis. Cambridge University Press, 1 edition, 2003.
- Rethinking the inception architecture for computer vision. In CVPR, 2016.
- Sauer Timothy. Numerical Analysis. Pearson, 3 edition, 2017.
- Diffusion-GAN: Training GANs with diffusion. 2023.
- Learning fast samplers for diffusion models by differentiating through sample quality. In ICLR, 2022.
- Tackling the generative learning trilemma with denoising diffusion GANs. In ICLR, 2022.
- LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015.
- LION: Latent point diffusion models for 3D shape generation. In NeurIPS, 2022.