---
title: Proximal Sampler for Non-smooth Distributions
url: https://www.emergentmind.com/topics/proximal-sampler
type: topic
---

# Proximal Sampler for Non-smooth Distributions

A Proximal Sampler is a Markov chain Monte Carlo (MCMC) algorithm designed for sampling from probability distributions whose potential functions are non-smooth, weakly smooth, or even non-convex, and which may only be required to satisfy a logarithmic Sobolev or Poincaré inequality, rather than strong convexity or smoothness. The core mechanism is an alternating Gibbs sampler, closely analogous to the proximal point algorithm in convex optimization, but adapted to the sampling setting. Unlike discretizations of Langevin diffusions—which are known to degrade in the presence of non-smooth or weakly smooth targets—the proximal sampler leverages the regularization properties of proximal mappings, replacing gradient-based Markov transitions with randomized, regularized steps. This approach enables state-of-the-art sampling complexity in a broad range of settings, including strongly log-concave, log-concave, and certain classes of non-log-concave targets [2202.13975][2302.10081].

## 1. Algorithmic Definition and Alternating Sampling Framework

The proximal sampler operates on a joint augmented density 
$$
\Pi(x, y) \propto \exp\left(-f(x) - \frac{1}{2\eta}\|x - y\|^2\right)
$$
for $x, y \in \mathbb{R}^d$ and stepsize $\eta>0$. The Markov chain alternates the following steps:

1. **Gaussian randomization:** $y_{k} \sim \mathcal{N}(x_k, \eta I)$
2. **Proximal update (restricted Gaussian oracle):** $x_{k+1} \sim \pi^{X|Y}(x | y_k) \propto \exp\left(-f(x) - \frac{1}{2\eta}\|x-y_k\|^2\right)$

This chain has $\pi(x) \propto \exp(-f(x))$ as its stationary marginal. The key nontrivial operation is efficiently sampling from the conditional $\pi^{X|Y}(\cdot|y)$, referred to as the Restricted Gaussian Oracle (RGO) [2202.13975][2202.06386][2302.10081].

## 2. Restricted Gaussian Oracle: Realizations and Rejection Sampling

Implementation of the RGO is central to the efficiency of the proximal sampler. When $f$ is convex (possibly non-smooth), the RGO can be realized via rejection sampling using a carefully constructed Gaussian proposal. The proposal is centered at a suitably chosen stationary point—typically, a (possibly approximate) minimizer of the regularized functional $f(x) + \frac{1}{2\eta}\|x-y\|^2$. Specifically, a quadratic envelope (derived via a local quadratic or linearization argument based on the semi-smoothness of $f$) is used to sandwich the target, allowing control of acceptance probability. With careful selection of $\eta$, the expected number of proposals is $O(1)$ per sample, even in high dimensions [2202.13975][2302.10081][2110.04597].

In the non-convex case, provided that the target measure $\pi$ satisfies either a log-Sobolev (LSI) or Poincaré (PI) inequality and that the sum of semi-smoothness parameters and constants is appropriately controlled, rejection sampling remains efficient and maintains rigorous approximation guarantees [2205.10188][2202.13975].

## 3. Convergence Analysis and Complexity Guarantees

Assume that the target $\pi$ satisfies LSI with constant $C_{\mathrm{LSI}}$, or PI with constant $C_{\mathrm{PI}}$, and that $f$ is a (possibly composite) sum of functions with semi-smoothness parameters $\alpha_i \in [0,1]$. For suitable $\eta$ (dependent on $L_\alpha$, $d$, and target accuracy), the proximal sampler achieves:

- **LSI (in KL):**
  $$
  K = \widetilde{O}\left(C_{\mathrm{LSI}} \sum_i L_{\alpha_i}^{2/(\alpha_i+1)} d \log(1/\epsilon)\right)
  $$
  iterations to reach KL divergence $\leq \epsilon$ [2202.13975][2302.10081].

- **PI (in $\chi^2$):**
  $$
  K = \widetilde{O}\left(C_{\mathrm{PI}} \sum_i L_{\alpha_i}^{2/(\alpha_i+1)} d \log(1/\epsilon)\right)
  $$
  for $\chi^2$ divergence $\leq \epsilon$.

- **Log-concave (convex $f$):**
  $$
  K = \widetilde{O}\left(W_2^2(\rho_0, \pi) \sum_i L_{\alpha_i}^{2/(\alpha_i+1)} d / \epsilon \right)
  $$
  to drive KL to $\epsilon$ [2202.13975]. No polynomial-in-$1/\varepsilon$ dependence in the non-convex, semi-smooth regime is required.

In the strongly log-concave and smooth case, the proximal sampler achieves optimal dimension dependence (up to logs), improving over Langevin-based methods:
$$
K = \widetilde{O}\left(\kappa \sqrt{d} \log(1/\delta)\right),\quad \kappa = L/\alpha
$$
for TV error $\leq \delta$ [2302.10081].

## 4. Theoretical Extensions: General Potentials, Heavy Tails, and Inexactness

### Non-convex and Composite Potentials

The proximal sampler also applies to non-convex, composite, or sum-of-semi-smooth potentials, requiring only that $\pi$ satisfy LSI or PI. Key to this extension is the realization that the regularized functional is "nearly" locally convex, so the rejection step remains efficient at scale [2202.13975][2205.10188].

### Heavy Tail Regimes

Proximal samplers using a Gaussian kernel (standard RGO) suffer a low-accuracy barrier for heavy-tailed targets, only achieving polynomial-in-$1/\epsilon$ convergence rates. However, recent advances show that substituting the Gaussian with an $\alpha$-stable heat kernel (in both forward and backward Gibbs steps) overcomes this barrier, achieving $O(\log(1/\epsilon))$ mixing times when the target and kernel tail indices match [2405.16736].

### Inexact RGO and Accumulated Error

Controlled inexactness in RGO can be tolerated. By new concentration inequalities for semi-smooth functions, the accumulated total variation error scales linearly with the number of steps, so with per-step tolerance $\zeta=\delta/K$ one achieves overall TV error $\delta$ [2302.10081]. Similar logic holds for $W_2$ and $\chi^2$ distances.

## 5. Practical Realizations and Comparison with Other Methods

Efficient practical realization of the RGO is demonstrated for numerous classes:

- **Non-smooth convex $f$:** Proximal-bundle methods combined with rejection sampling facilitate efficient sampling even without explicit proximal maps [2110.04597].
- **Constrained or composite sampling:** Epigraph lifting and cutting-plane methods implement RGOs using only minimal oracles (separation for the constraint, subgradient for $f$) [2602.14478].
- **Stochastic or high-dimensional regimes:** Stochastic gradient variants achieve optimal (or near-optimal) dependence on dimension and error [2405.16734].
- **Non-Euclidean geometry or manifold support:** Proximal samplers generalize via the log-Laplace transform or Riemannian constructs [2502.07265][2302.06085].

Compared to Langevin and Metropolis-adjusted methods, the proximal sampler strictly improves complexity in the semi-smooth or non-smooth regime and does not require strong convexity, dissipativity, or smoothness [2202.13975][2302.10081][2110.04597].

## 6. Proof Techniques and Theoretical Foundations

The convergence analysis of proximal samplers relies

Source: https://www.emergentmind.com/topics/proximal-sampler