Score-Based Generative Modeling with Critically-Damped Langevin Diffusion (2112.07068v4)

Abstract: Score-based generative models (SGMs) have demonstrated remarkable synthesis quality. SGMs rely on a diffusion process that gradually perturbs the data towards a tractable distribution, while the generative model learns to denoise. The complexity of this denoising task is, apart from the data distribution itself, uniquely determined by the diffusion process. We argue that current SGMs employ overly simplistic diffusions, leading to unnecessarily complex denoising processes, which limit generative modeling performance. Based on connections to statistical mechanics, we propose a novel critically-damped Langevin diffusion (CLD) and show that CLD-based SGMs achieve superior performance. CLD can be interpreted as running a joint diffusion in an extended space, where the auxiliary variables can be considered "velocities" that are coupled to the data variables as in Hamiltonian dynamics. We derive a novel score matching objective for CLD and show that the model only needs to learn the score function of the conditional distribution of the velocity given data, an easier task than learning scores of the data directly. We also derive a new sampling scheme for efficient synthesis from CLD-based diffusion models. We find that CLD outperforms previous SGMs in synthesis quality for similar network architectures and sampling compute budgets. We show that our novel sampler for CLD significantly outperforms solvers such as Euler--Maruyama. Our framework provides new insights into score-based denoising diffusion models and can be readily used for high-resolution image synthesis. Project page and code: https://nv-tlabs.github.io/CLD-SGM.

• The paper introduces CLD, which improves score-based generative models by simplifying the denoising objective using a joint data-velocity approach.
• The methodology employs a novel score matching objective and the SSCS integrator, demonstrating superior performance and reduced sampling time on CIFAR-10.
• The proposed framework opens new research directions for high-resolution image synthesis and integration with advanced generative modeling techniques.

An Analytical Exploration of Score-Based Generative Modeling with Critically-Damped Langevin Diffusion

This paper presents an innovative approach to Score-Based Generative Models (SGMs) by introducing Critically-Damped Langevin Diffusion (CLD). The authors identify limitations in existing SGMs which rely on overly simplistic diffusion processes, resulting in complex denoising tasks. Drawing from the principles of statistical mechanics, the paper proposes CLD as a solution—a novel methodology that augments traditional data with auxiliary velocity variables to enhance generative modeling performance through more efficient data transformation processes.

Key Propositions and Methodological Enhancements

The central thesis of this paper builds on the critique that current SGMs use diffusion processes that do not optimally support the denoising task essential for generating novel data from perturbed states. The proposed CLD aims to rectify this by introducing a diffusion process that leverages a joint data-velocity space akin to Hamiltonian systems. Such systems inherently provide computational economies through shared dynamical insights pertinent to both data and velocity.

The authors propose a new score matching objective tailored to CLD, which only necessitates learning the score function of the velocity given data. This adjustment is posited as a simplification over learning scores for the data directly, thus potentially reducing the complexity involved in the learning process.

Numerical Results and Implementation

The experiments conducted in this paper demonstrate CLD's capabilities across several benchmarks, particularly the CIFAR-10 image modeling task, where it outperforms existing models. The authors attribute the model's improved synthesis quality and computational efficiency to the CLD's design, which facilitates smoother transitions and permits effective, scalable training.

SSCS, a novel SDE integrator specifically devised for CLD, is highlighted as a significant contribution. Compared to the commonly used Euler--Maruyama solver, SSCS exhibits superior performance in generating samples while allowing non-trivial reductions in sampling time without compromising on sample quality.

Implications and Future Directions

The introduction of CLD holds both theoretical and practical implications. Theoretically, it offers a fresh lens to examine the existing challenges within SGMs and diffusion models, paving the way for further exploration of statistical mechanical concepts in AI model development. Practically, CLD's enhanced capability to model complex data distributions more efficiently presents a myriad of applications, notably in high-resolution image synthesis and beyond.

The work suggests several future research directions, including the adaptation of CLD to diverse generative tasks beyond imaging, integration with other accelerated sampling methods, and optimizations towards maximum likelihood training paradigms. The potential fusion of CLD with latent space models, like Latent SGMs (LSGMs), could further elevate its application scope and efficiency.

Overall, this paper contributes a significant advancement in SGM methodologies, emphasizing the gains of adopting principles from statistical mechanics to computational models in artificial intelligence. The proposition of CLD and its accompanying tools is a promising step towards more robust and efficient generative modeling frameworks.