Geometry-Aware MALA Proposals

• The paper introduces geometry-aware adaptations in MALA by replacing isotropic covariance with local, position-dependent metrics to better capture posterior curvature and constraints.
• It details multiple formulations—including Riemannian, mirror, and barrier methods—and derives rigorous mixing-time bounds based on the chosen geometric structure.
• Empirical results highlight that these proposals improve effective sample sizes and computational efficiency in high-dimensional and constrained sampling scenarios.

Searching arXiv for the cited works and closely related geometry-aware MALA papers. Geometry-aware Metropolis-adjusted Langevin proposals are Metropolis–Hastings kernels obtained by discretizing a Langevin diffusion whose drift and noise are adapted to a non-Euclidean geometry, typically through a position-dependent metric, a mirror map, a barrier Hessian, or a Fisher–Rao construction. The resulting proposals replace the isotropic covariance of standard MALA by a local covariance determined by geometry and then apply an accept–reject correction to restore exact invariance of the target distribution. Across the literature, this framework appears in unconstrained Riemannian formulations, position-dependent and Hessian-based MALA, mirror and barrier methods for convex constraints, and more recent Gaussian-invariant or learned-metric variants (Xifara et al., 2013, Roy et al., 2021, Srinivasan et al., 2023).

1. Conceptual framework and defining ingredients

A geometry-aware Metropolis-adjusted Langevin proposal starts from a smooth target density $\pi$, or equivalently a potential $U$ or $f$, together with a state-dependent notion of local geometry. In the Riemannian setting this geometry is a positive-definite matrix $G(x)$, interpreted as a metric tensor; in mirror formulations it is induced by a Legendre-type mirror map $\psi$; in barrier-based constrained methods it is typically the Hessian of a log-barrier; and in Fisher–Rao constructions it is induced from the statistical manifold of densities (Roy et al., 2021, Srinivasan et al., 2023, Srinivasan et al., 2024, Roy, 2024).

The common pattern is a local Gaussian or Gaussian-like proposal whose mean contains a geometry-preconditioned gradient term and whose covariance is proportional to the inverse metric. For example, on $\mathbb{R}^n$ with position-dependent metric $G(x)$, one Euler–Maruyama discretization gives

$$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$

followed by the Metropolis–Hastings correction

$$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$

This is the basic position-dependent MALA template described in several formulations of manifold or preconditioned MALA (Roy et al., 2021, Xifara et al., 2013).

What makes these proposals “geometry-aware” is not merely the use of gradients, but the replacement of Euclidean scaling by a local tensor that encodes anisotropy, curvature, or boundary structure. In the unconstrained case this can align proposal covariances with posterior curvature. In constrained spaces it can make proposals shrink or deform near the boundary in a way that respects the feasible region. A plausible implication is that geometry is used simultaneously for stability, acceptance, and state-space exploration, rather than only for local preconditioning (Mamajiwala et al., 2022, Chok et al., 6 Oct 2025).

2. Diffusion foundations: Riemannian, mirror, and barrier geometries

The continuous-time foundation is a Langevin diffusion whose invariant law is the target. On a Riemannian manifold with metric $G(x)$, one formulation uses the Stratonovich SDE

$U$0

with $U$1; equivalently, in Itô form the diffusion admits $U$2 as invariant law (Roy et al., 2021). Closely related expressions appear in PMALA and GALA, where the extra geometry-dependent drift is written as an Itô correction drift or Christoffel drift and is required for the continuous diffusion to preserve the desired invariant density with respect to Lebesgue measure or the manifold volume form (Xifara et al., 2013, Mamajiwala et al., 2022).

Mirror formulations recast the dynamics in dual coordinates. For a compact convex set $U$3 and a Legendre-type mirror map $U$4, mirror Langevin dynamics are written in dual space as

$U$5

where $U$6, while in primal variables

$U$7

This identifies the dynamics as a Langevin diffusion on the Riemannian manifold $U$8 (Srinivasan et al., 2023).

Barrier-based constrained samplers make the geometry explicit through interior-point metrics. In MAPLA the manifold is $U$9 with inner product $f$0, and the continuous-time dynamics with stationary $f$1 are

$f$2

where the divergence term ensures detailed balance (Srinivasan et al., 2024). In the Dikin–Langevin construction for polyhedra, the geometry is given by the inverse Hessian $f$3 of the log-barrier $f$4, and the continuous-time SDE includes both $f$5 and $f$6 (Chok et al., 6 Oct 2025).

These formulations show that “geometry-aware” can refer to several distinct but related objects: a local metric tensor, a dual coordinate system, or an interior-point Hessian. The shared principle is local adaptation of both drift and diffusion.

3. Proposal construction and Metropolis correction

Discretization converts the continuous diffusion into a proposal kernel. In the simplest Riemannian case, the proposal is Gaussian with mean $f$7 and covariance $f$8, and exact $f$9-invariance is recovered through the standard Metropolis ratio (Roy et al., 2021). PMALA writes the proposal as

$G(x)$0

with

$G(x)$1

Its defining feature is that the diffusion is constructed directly in $G(x)$2 so that $G(x)$3 is invariant with respect to Lebesgue measure, rather than relying on a manifold-volume formulation (Xifara et al., 2013).

GALA uses an Itô SDE with a Christoffel drift $G(x)$4, followed by an Euler–Maruyama step

$G(x)$5

and accepts with

$G(x)$6

The proposal therefore adapts both the drift and the noise to the local metric (Mamajiwala et al., 2022).

Mirror constructions implement the proposal in dual coordinates. Starting from $G(x)$7, the mirror Langevin algorithm performs the dual update

$G(x)$8

then maps back through

$G(x)$9

Because the unadjusted MLA chain is biased, MAMLA applies an MH filter with acceptance

$\psi$0

where $\psi$1 is the MLA proposal density. The paper emphasizes a Bregman change-of-variable formula under which many Jacobian factors cancel between forward and reverse proposals, leaving a simple closed-form ratio (Srinivasan et al., 2023).

Hessian-based proposals form another prominent subclass. In the Gaussian-invariant construction, one defines

$\psi$2

and uses

$\psi$3

with the usual MH accept–reject step. The paper describes this as a “second-order” or manifold MALA proposal and notes that in a fully correct Riemannian formulation one would add a divergence drift, but for the Hessian-based proposal this term is dropped and absorbed approximately into the Metropolis correction (Titsias et al., 26 Jun 2025).

A recurrent misconception is that any position-dependent Gaussian proposal with a local covariance is automatically the discretization of a correct geometric diffusion. The PMALA/MMALA comparison shows this is not generally true: the specific drift correction determines which reference measure is preserved and whether the resulting continuous process is invariant for $\psi$4 or for a density relative to the manifold volume $\psi$5 (Xifara et al., 2013).

4. Constrained spaces and self-concordant geometry

Geometry-aware MALA proposals are especially prominent for constrained sampling. In compact convex domains, the mirror or barrier geometry can encode the boundary directly. MAMLA targets distributions supported on a compact and convex set and uses a self-concordant mirror function, often a self-concordant barrier, so that the metric is $\psi$6. The method is unbiased relative to the target, whereas known discretizations of the mirror Langevin dynamics including MLA have an asymptotic bias (Srinivasan et al., 2023).

Its fast-mixing analysis is explicitly tied to self-concordant geometry. When $\psi$7 is a self-concordant barrier with parameter $\psi$8, and $\psi$9 is $\mathbb{R}^n$0-strongly convex and $\mathbb{R}^n$1-smooth relative to $\mathbb{R}^n$2 and $\mathbb{R}^n$3-Lipschitz in the local $\mathbb{R}^n$4-norm, the MH-adjusted chain is reversible with respect to $\mathbb{R}^n$5, and conductance is lower bounded using one-step overlap between nearby states in the $\mathbb{R}^n$6 metric together with isoperimetric inequalities on $\mathbb{R}^n$7 (Srinivasan et al., 2023).

MAPLA generalizes the same philosophy through a preconditioned Langevin algorithm with metric $\mathbb{R}^n$8 on a proper convex subset of $\mathbb{R}^n$9. The one-step proposal is

$G(x)$0

or, equivalently, with a projection,

$G(x)$1

followed by

$G(x)$2

For standard self-concordance, the Dikin ellipsoids $G(x)$3 stay inside $G(x)$4 for $G(x)$5, and proposals adapt to local curvature of $G(x)$6. Under the stronger self-concordant$G(x)$7 property, tighter control of $G(x)$8 and the log-determinant leads to larger allowable $G(x)$9 and thus fewer iterations (Srinivasan et al., 2024).

The Dikin–Langevin method for polyhedra is a closely related barrier-aware construction. Here $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$0, $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$1, and $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$2. The continuous-time diffusion has a boundary no-flux property: by a Fokker–Planck argument, trajectories started in the interior remain in $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$3 almost surely, so feasibility is maintained by construction. In discrete time, the method uses an Euler–Maruyama proposal with state-dependent covariance and an MH correction; if a proposal lies outside $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$4, it has $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$5 and is rejected (Chok et al., 6 Oct 2025).

These constrained formulations make geometry-aware MALA distinct from reflection- or projection-based approaches. The geometry itself contracts the local covariance near the boundary, often through a barrier Hessian or Dikin ellipsoid, and this is the principal mechanism by which feasibility is respected (Srinivasan et al., 2023, Chok et al., 6 Oct 2025).

5. Ergodicity, reversibility, and mixing theory

The Metropolis correction has two related theoretical roles: it restores exact stationarity and it enables reversible-kernel arguments for convergence rates. In position-dependent MALA on $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$6, geometric ergodicity is obtained under uniform ellipticity, local boundedness of the proposal mean, asymptotic acceptance control, and a drift-dominating condition on the proposal center. Under these assumptions, if $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$7 is bounded away from $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$8 and $$Q_h(x,\cdot)=\mathcal N(m(x),\,h\,G(x)^{-1}), \qquad m(x)=x+\tfrac12 h\,G(x)^{-1}\nabla\log\pi(x),$$9 on compacts, the Metropolis-adjusted Langevin chain is geometrically ergodic: $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$0 The proof uses the Lyapunov function $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$1, a drift inequality, and compact-set minorization via Meyn–Tweedie theory (Roy et al., 2021).

The same paper also gives converse statements. If the mean increment is uniformly bounded, geometric ergodicity forces $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$2 to have some exponential moments, and if the asymptotic drift is too large relative to the current state, the chain cannot be geometrically ergodic (Roy et al., 2021). This places geometry-aware proposals inside the same drift/minorization framework as other MH chains, but with conditions expressed through the state-dependent covariance structure.

For constrained mirror MALA, the central result is a nonasymptotic mixing-time bound derived from conductance. When $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$3 is a self-concordant barrier with parameter $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$4,

$$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$5

and standard Cheeger–Lovász bounds imply

$$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$6

In the merely convex case, using a symmetric barrier property with parameter $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$7,

$$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$8

By choosing $$\alpha(x,y)=\min\Bigl\{1,\frac{\pi(y)\,Q_h(y,x)}{\pi(x)\,Q_h(x,y)}\Bigr\}.$$9 up to $G(x)$0, the method achieves a mixing time of order $G(x)$1, independent of the domain’s condition number (Srinivasan et al., 2023).

MAPLA provides analogous high-accuracy bounds for constrained sampling with a metric $G(x)$2. Under standard self-concordance, if

$G(x)$3

then

$G(x)$4

Under self-concordant$G(x)$5, the allowable step-size bound improves accordingly (Srinivasan et al., 2024).

A consistent theme is the dependence on error tolerance. MAMLA states that the reversibility induced by the Metropolis–Hastings filter yields an exponentially better dependence on the error tolerance for approximate constrained sampling, while MAPLA describes itself as a high-accuracy sampler due to the polylogarithmic dependence on the error tolerance in its mixing-time upper bounds (Srinivasan et al., 2023, Srinivasan et al., 2024).

6. Variants, implementation trade-offs, and empirical behavior

Several major variants differ mainly in how they choose the metric and how much geometry they compute per step. Standard MALA uses $G(x)$6. PCMALA uses a constant positive-definite $G(x)$7, which adapts proposal scale to known posterior covariance. MMALA uses a state-dependent $G(x)$8, often chosen from Fisher information or minus Hessian plus prior curvature, and therefore adapts locally to posterior geometry (Roy et al., 2021). PMALA modifies the drift correction so that the invariant density is exactly $G(x)$9 with respect to Lebesgue measure, and it is algebraically simpler because it uses $U$00 rather than $U$01 (Xifara et al., 2013).

The computational burden is correspondingly heterogeneous. For MMALA, one step requires a factorization of $U$02 and, in the fully geometric version, the gradient of $U$03; the cited GLMM study summarizes this as $U$04 per step, whereas PCMALA with nearly constant $U$05 factorizes once (Roy et al., 2021). In the Dikin–Langevin polyhedral setting, each step costs one Hessian of the log-barrier and one $U$06 linear solve, while the authors contrast this with the cost of a general Riemann-manifold MALA that also requires metric gradients, Cholesky factorizations, and divergence terms (Chok et al., 6 Oct 2025).

Empirical studies consistently report gains when the metric captures the dominant local geometry, but they also show that more geometry does not automatically imply better time-normalized performance. In spatial GLMMs, PCMALA with a “diagonal plus block” inverse Fisher at MAP, denoted PCMALA$U$07, outperforms MMALA in ESS per CPU-second by a factor $U$08, despite ignoring local curvature variations (Roy et al., 2021). In the PMALA/MMALA comparison, when the equivalence condition holds the two methods have essentially identical ESS per iteration, but PMALA is slightly cheaper per iteration; when the condition fails, PMALA attains substantially higher ESS (Xifara et al., 2013).

Constrained samplers show a similar pattern. For uniform sampling on polytopes and ellipsoids, MAMLA shows mixing times scaling like $U$09, agnostic to affine-conditioning, matching the theory. For Dirichlet sampling on the simplex, using the simplex log-barrier mirror gives rapid convergence in the 2-Wasserstein distance, significantly faster than unadjusted MLA. The reported acceptance-rate experiments indicate that step sizes $U$10, much larger than the pessimistic $U$11, can still maintain $U$12 acceptance up to $U$13, suggesting the worst-case one-step overlap analysis is conservative (Srinivasan et al., 2023).

MAPLA reports, for truncated-Dirichlet sampling on the simplex with the log-barrier Hessian metric, empirical mixing time measured in $U$14 or energy distance growing $U$15, with MAPLA slope $U$16 versus DikinWalk $U$17; at $U$18, to reach $U$19, MAPLA requires about $U$20 steps versus about $U$21 for DikinWalk, and acceptance under $U$22 is about $U$23 versus about $U$24 (Srinivasan et al., 2024). The modified Dikin–Langevin sampler likewise reports strong split-$U$25 diagnostics on constrained anisotropic Gaussians and more cross-well transitions on a bimodal target than the Dikin random walk (Chok et al., 6 Oct 2025).

Other recent variants extend the same geometry-aware template. GALA uses a metric and Christoffel drift and reports strong empirical performance on anisotropic and high-dimensional targets, including logistic regression (Mamajiwala et al., 2022). Gaussian-invariant manifold MALA chooses a Hessian-based local covariance and, for Gaussian targets, becomes exactly reversible in a way that supports explicit Poisson-equation control variates for variance reduction (Titsias et al., 26 Jun 2025). A neural-network variant models the preconditioner $U$26 as a random-matrix output of a small neural network and adapts it on the fly; for each fixed $U$27, the Metropolized discretization satisfies detailed balance, and under mild conditions on the decay of the learning rate the overall chain still satisfies ergodicity (Zarezadeh et al., 22 Mar 2025). A distinct Fisher–Rao approach constructs proposals by moving in the statistical manifold of densities via geodesics and then applying a Jacobian-corrected Metropolis ratio, thereby embedding Langevin-type proposals into a geometric MH framework that is not restricted to local covariance adaptation (Roy, 2024).

One persistent controversy concerns how much of the continuous geometric drift must be retained in discrete time. Some constructions include explicit divergence or Christoffel terms in the proposal mean (Xifara et al., 2013, Mamajiwala et al., 2022), while others omit them and rely on the Metropolis correction to absorb the discrepancy approximately or exactly at the level of the invariant distribution (Srinivasan et al., 2023, Titsias et al., 26 Jun 2025, Chok et al., 6 Oct 2025). This suggests that “geometry-aware MALA” is best understood as a family of Metropolized discretizations sharing a common geometric principle rather than a single canonical algorithm.