Global Well-posedness and Convergence Analysis of Score-based Generative Models via Sharp Lipschitz Estimates
Abstract: We establish global well-posedness and convergence of the score-based generative models (SGM) under minimal general assumptions of initial data for score estimation. For the smooth case, we start from a Lipschitz bound of the score function with optimal time length. The optimality is validated by an example whose Lipschitz constant of scores is bounded at initial but blows up in finite time. This necessitates the separation of time scales in conventional bounds for non-log-concave distributions. In contrast, our follow up analysis only relies on a local Lipschitz condition and is valid globally in time. This leads to the convergence of numerical scheme without time separation. For the non-smooth case, we show that the optimal Lipschitz bound is O(1/t) in the point-wise sense for distributions supported on a compact, smooth and low-dimensional manifold with boundary.
- Convex viscosity solutions and state constraints. Journal de mathématiques pures et appliquées, 76 (3):265–288, 1997.
- B. Anderson. Reverse-time diffusion equation models. Stoch. Process. Appl., 12(3):313–326, 1982.
- V. De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Transactions on Machine Learning Research, 2022.
- G. Brigati and F. Pedrotti. Heat flow, log-concavity, and Lipschitz transport maps. arXiv preprint arXiv:2404.15205, 2024.
- Verifying the union of manifolds hypothesis for image data. ICLR, 2023.
- Deep conditional generative learning: Model and error analysis. arXiv preprint arXiv:2402.01460, 2024.
- Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In International Conference on Machine Learning, pages 4735–4763. PMLR, 2023.
- Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. arXiv preprint arXiv:2209.11215v3, 2023.
- P. Dhariwal and A. Nichol. Diffusion models beat GANs on image synthesis. NeurIPS, 34:8780–8794, 2021.
- Masked diffusion transformer is a strong image synthesizer. ICCV, 2023.
- Deep Learning. MIT press, 2016.
- Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 3(47):1–33, 2022.
- Convergence analysis of probability flow ODE for score-based generative models. arXiv preprint arXiv:2404.09730, 2024.
- N. Ikeda and S. Watanabe. Stochastic differential equations and diffusion processes. Elsevier, 2014.
- Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. ICML, 162:11201–11228, 2022.
- A. Kolmogorov. Zur Umkehrbarkeit der statistischen Naturgesetze (Engl.: On the reversibility of the statistical laws of nature). Math. Ann., 113:766–772, 1937.
- Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Processing Systems, 35:22870–22882, 2022.
- Universal approximation for log-concave distributions using well-conditioned normalizing flows. arXiv preprint arXiv:2107.02951, 2021.
- J. Pidstrigach. Score-based generative models detect manifolds. NeurIPS, 2022.
- Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. NeurIPS, 2019.
- Score-based generative modeling through stochastic differential equations. ICLR, 2021.
- T. Strömberg. Semiconcavity estimates for viscous Hamilton–Jacobi equations. Archiv der Mathematik, 94(6):579–589, 2010.
- A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000.
- Q. Zhang and Y. Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204.13902, 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
