---
title: 'MALA: Metropolis-adjusted Langevin Algorithm'
url: https://www.emergentmind.com/topics/metropolis-adjusted-langevin-algorithm-mala-283267ae-b692-431f-8f74-68d770e1f14d
type: topic
---

# MALA: Metropolis-adjusted Langevin Algorithm

The Metropolis-adjusted Langevin Algorithm (MALA) is a Markov chain Monte Carlo (MCMC) method that constructs reversible Markov chains targeting a chosen distribution on high-dimensional Euclidean space or manifolds. MALA integrates information from the gradient of the log-target density into proposal moves and corrects for discretization bias by a Metropolis–Hastings accept-reject mechanism. Variants based on convex analysis, subdifferential calculus, local or global adaptation, and geometric preconditioning have broadened the scope of MALA to non-smooth, high-dimensional, or non-Euclidean sampling regimes.

## 1. Foundations and Algorithmic Structure

MALA is derived from the Euler–Maruyama discretization of the overdamped Langevin diffusion targeting a distribution $\pi(x)\propto\exp(-U(x))$:
\[
dX_t = -\nabla U(X_t)\,dt + \sqrt{2}\,dW_t,
\]
where $W_t$ denotes Brownian motion. A single MALA proposal, with step size $h>0$, is given by
\[
X' = X - \frac{h}{2}\nabla U(X) + \sqrt{h}Z,\qquad Z\sim\mathcal{N}(0, I_d).
\]
The proposed $X'$ is accepted with probability
\[
\alpha(X, X') = 1 \wedge \frac{\pi(X')q(X' \to X)}{\pi(X)q(X \to X')},
\]
where $q(x\to y)$ is the Gaussian proposal density centered at $x-(h/2)\nabla U(x)$, with covariance $hI_d$. The Metropolis–Hastings correction ensures that $\pi$ is the unique invariant distribution.

The optimal acceptance rate for high-dimensional product targets or suitable non-product targets is approximately $0.574$, and optimal step-size scaling is $h\sim cN^{-1/3}$ in stationarity for a $N$-dimensional system; $h\sim cN^{-1/2}$ out of stationarity [1103.0542][1608.08379][2204.10793].

## 2. Theoretical Performance and High-dimensional Scaling

The efficiency and mixing time of MALA have been rigorously analyzed in both product and non-product settings. For targets with i.i.d. structure and sufficiently smooth potentials, running MALA in stationarity with step-size $h\sim \ell N^{-1/3}$ yields an exploration time of order $\mathcal{O}(N^{1/3})$ for the invariant measure [1103.0542][2204.10793].

If the target is strongly log-concave with $\alpha I \preceq \nabla^2 V(x)\preceq \beta I$, non-asymptotic mixing time analysis yields an optimal rate of $\widetilde{\Theta}(d^{1/2})$ as the dimension $d\to\infty$, up to polylogarithmic factors, for any $\chi^2$, total variation, KL, or Wasserstein metric [2012.12810]. This bridges the gap between previous diffusion-limit intuition ($\mathcal{O}(d^{1/3})$) and prior non-asymptotic $O(d)$ bounds.

For Bayesian posteriors approximating multivariate Gaussians, a refined $s$-conductance profile method demonstrates that, after warmup, MALA achieves optimal dimension and condition number dependence of order $d^{1/3}\kappa$ (where $\kappa$ is the condition number) for the mixing time [2206.06491].

When the log-target admits only weak higher-order regularity (incoherence-type assumptions, not global operator norm bounds), sublinear-in-dimension mixing for weakly log-concave and certain nonconvex targets is achievable [1902.08452].

## 3. Algorithmic Variants and Extensions

Several sophisticated extensions of MALA handle challenges arising from non-smooth potentials, sparsity, geometry, and adaptivity.

- **Proximal MALA:** Suitable for log-concave, possibly non-smooth $U$. Replaces $\nabla U$ in the proposal by the proximity operator $\text{prox}_{\lambda U}(x) = \arg\min_{y} U(y) + \frac{1}{2\lambda}\|y-x\|^2$. Accept/reject is performed using Gaussian proposals shifted to $\text{prox}_{\lambda U}(x)$ [1306.0187][2204.10793]. Proximal MALA achieves robust geometric ergodicity and matches the optimal scaling asymptotics of gradient MALA.

- **Metropolis-adjusted Subdifferential Langevin Algorithm (MASLA):** For locally Lipschitz or nonconvex, non-differentiable $U$. Replaces gradients by elements of a measurable conservative field (set-valued extension of subgradient), ensuring stationarity and reversibility almost everywhere. Empirical results confirm geometric ergodicity and ability to sample from densities inaccessible to classical or proximal MCMC [2507.06950].

- **Shrinkage–Thresholding MALA (STMALA):** Designed for sparse Bayesian variable selection, combines MALA with a coordinate-wise soft-thresholding operator to effect variable-inclusion moves. Achieves V-geometric ergodicity and superior exploration compared to both basic MALA and reversible-jump MCMC in sparse high-dimensional regimes [1312.5658].

- **Fisher-adaptive and Geometric Preconditioning:** Fisher-adaptive MALA (FaMALA) and related approaches employ the inverse Fisher information matrix as an optimal preconditioner, learning this matrix adaptively using streaming estimates from the history of gradients. This preconditioning maximizes the expected squared jump distance and leads to dimension-robust efficiency superior to standard adaptive MALA and pCN [2305.14442][2503.09374]. Geometric and manifold variants use local metric tensors or stochastic development for sampling in non-Euclidean geometries [1309.2983][2201.08072].

- **Backpropagation-Free MALA:** Uses forward-mode automatic differentiation and randomly sampled search directions to obviate the need for reverse-mode gradient computation in machine learning settings, while maintaining detailed balance via modified proposals and acceptance rules [2505.18081].

- **Locally Adaptive Step Size (autoMALA):** Adjusts the proposal step size at each iteration based on local characteristics of the target, preserving invariance by augmenting state with random thresholds and enforcing reversibility through acceptance interval tests [2310.16782].

## 4. Non-Asymptotic Mixing, Conductance, and Robustness

Recent analyses have established quantitative non-asymptotic convergence guarantees for MALA under relaxed smoothness, log-concavity, and isoperimetric assumptions. In particular, mixing time can be bounded as:
\[
T_\text{mix}(\epsilon) = O\left(\frac{\sqrt{L\Upsilon}}{\psi_\mu^2}\log\frac{1}{\epsilon}\right)
\]
where $L$ is the operator-norm Hessian bound, $\Upsilon$ its trace, and $\psi_\mu$ isoperimetric constant of the target [2304.04095]. The dependence on $\Upsilon$ can yield improved dimension scaling in models with low-rank or sparse curvature.

For targets that are perturbations of high-dimensional Gaussians, sharp Wasserstein contraction rates for (semi-implicit Euler) MALA are shown to be uniform in $d$ under mild smoothness and convexity; optimal diffusion-limit rates are recovered as $h\to0$ [1210.1180].

When applied to non-smooth or composite log-densities, subdifferential MASLA and proximal MALA retain exponential convergence under broader conditions, while empirical studies confirm practical superiority to standard MALA, unadjusted Langevin, or non-gradient MCMC in the presence of non-differentiability [1306.0187][2507.06950].

## 5. Geometric, Manifold, and Preconditioned Approaches

- **Position-dependent and Riemannian MALA:** Allows local matrix-valued preconditioning for anisotropic and curved targets. Requires careful incorporation of divergence (or connection/Christoffel) terms in the drift to guarantee stationarity under Lebesgue measure [1309.2983]. In many statistical models, using the Fisher information metric or Hessian of the negative log-likelihood as the local metric delivers robust adaptation to geometry and mixing times substantially superior to identity-preconditioned MALA.

- **Geometric Adaptive Langevin Dynamics (GALA):** Formulates the Langevin equation on a Riemannian manifold and discretizes with the correct drift and diffusion, as prescribed by stochastic development and Itô calculus. Empirical studies show order-of-magnitude gains in mixing, acceptance, and statistical efficiency over Euclidean MALA and existing Riemannian variants, especially in ill-conditioned or high-dimensional regimes [2201.08072].

- **Fisher-adaptive MALA:** Online estimation of the Fisher information allows scaling proposals optimally with respect to the statistical geometry of the posterior, substantially improving effective sample size and autocorrelation, particularly in inverse problems and ill-conditioned posteriors [2305.14442][2503.09374].

## 6. Applications and Practical Considerations

MALA and its variants are widely used in Bayesian statistical inference, computational imaging, hierarchical and high-dimensional models, nonparametric regression, machine learning, and inverse problems. Proximal and subdifferential MALA extend applicability to convex and nonconvex composite posteriors (e.g., Bayesian lasso, total-variation image deconvolution, low-rank matrix factorization, neural network weight posteriors with ReLU activations) [1306.0187][2507.06950].

Efficient computation of the proximity operator or Fisher matrix is crucial for practical deployment. For smooth potentials, gradient-based MALA is computationally simpler, while for non-smooth or structurally sparse models, proximal or thresholded variants offer orders-of-magnitude better mixing and credible region estimation [2204.10793][1312.5658].

In high dimensions, careful tuning of the step size to target the optimal acceptance probability, preconditioning (either geometric or Fisher-adaptive), and possible use of locally adaptive step size rules (autoMALA) are empirically validated strategies for maximizing sampling efficiency [2310.16782][2305.14442][2503.09374].

## 7. Limitations, Open Problems, and Future Directions

While the theory for MALA’s efficiency and optimal scaling is mature in the context of smooth, strongly log-concave, or high-dimensional product targets, less is known for general non-log-concave, multimodal, or manifold-constrained sampling [2012.12810][1902.08452][2302.11971]. Recent work extends MALA to locally Lipschitz and non-differentiable settings, but full non-asymptotic mixing time bounds and robustness guarantees are yet incomplete for these regimes [2507.06950].

Further research directions include: rigorous mixing time analysis for MALA with hard constraints and in the presence of support restrictions [2302.11971]; adaptive and online strategies for learning geometric and Fisher preconditioners at scale; principled extension of local adaptation (autoMALA) to ensure irreducibility across all regimes; integration of backpropagation-free techniques to enable scalable Bayesian neural network inference [2505.18081]; and hybridization with Hamiltonian Monte Carlo for high-curvature or multimodal posteriors [2312.14027].

---

**References:**  
[1103.0542], [2204.10793], [2012.12810], [1306.0187], [1309.2983], [1312.5658], [2305.14442], [2503.09374], [2310.16782], [2507.06950], [2505.18081], [1210.1180], [1902.08452], [2201.08072], [2304.04095], [2206.06491], [1608.08379], [2312.14027], [2302.11971]

Source: https://www.emergentmind.com/topics/metropolis-adjusted-langevin-algorithm-mala-283267ae-b692-431f-8f74-68d770e1f14d