Mean-Field Langevin Diffusions with Density-dependent Temperature

Abstract: In the context of non-convex optimization, we let the temperature of a Langevin diffusion to depend on the diffusion's own density function. The rationale is that the induced density reveals to some extent the landscape imposed by the non-convex function to be minimized, such that a density-dependent temperature can provide location-wise random perturbation that may better react to, for instance, the location and depth of local minimizers. As the Langevin dynamics is now self-regulated by its own density, it forms a mean-field stochastic differential equation (SDE) of the Nemytskii type, distinct from the standard McKean-Vlasov equations. Relying on Wasserstein subdifferential calculus, we first show that the corresponding (nonlinear) Fokker-Planck equation has a unique solution. Next, a weak solution to the SDE is constructed from the solution to the Fokker-Planck equation, by Trevisan's superposition principle. As time goes to infinity, we further show that the density induced by the SDE converges to an invariant distribution, which admits an explicit formula in terms of the Lambert $W$ function.