DC-LA: Difference-of-Convex Langevin Algorithm

Abstract: We study a sampling problem whose target distribution is $π\propto \exp(-f-r)$ where the data fidelity term $f$ is Lipschitz smooth while the regularizer term $r=r_1-r_2$ is a non-smooth difference-of-convex (DC) function, i.e., $r_1,r_2$ are convex. By leveraging the DC structure of $r$, we can smooth out $r$ by applying Moreau envelopes to $r_1$ and $r_2$ separately. In line of DC programming, we then redistribute the concave part of the regularizer to the data fidelity and study its corresponding proximal Langevin algorithm (termed DC-LA). We establish convergence of DC-LA to the target distribution $π$, up to discretization and smoothing errors, in the $q$-Wasserstein distance for all $q \in \mathbb{N}^*$, under the assumption that $V$ is distant dissipative. Our results improve previous work on non-log-concave sampling in terms of a more general framework and assumptions. Numerical experiments show that DC-LA produces accurate distributions in synthetic settings and reliably provides uncertainty quantification in a real-world Computed Tomography application.

• The paper introduces DC-LA, a Langevin-based stochastic sampler equipped for non-smooth, non-weakly convex regularizers, providing convergence guarantees without requiring strong convexity at infinity.
• DC-LA outperformed traditional methods in sampling tasks and CT reconstruction, achieving lowest bin KL divergence and higher SSIM/PSNR scores, respectively, while providing valuable uncertainty quantification.
• DC-LA's practicality aligns with real-world problem and has demonstrated robust performance across a broad range of hyperparameters, supporting both theoretical applications and scalable, real-world computations.

Problem setting and motivation

The paper addresses sampling from a Gibbs distribution $\pi \propto e^{-V}$ on $\mathbb{R}^d$ where the potential admits a composite DC structure $V = f + r = f + (r_1 - r_2)$, with $f$ Lipschitz smooth and $r_1, r_2$ convex but not necessarily differentiable. This structure is common in Bayesian imaging and inverse problems: $f$ arises from a smooth likelihood (e.g., $\|Ax-b\|^2$ under Gaussian noise) while $r$ is a nonsmooth prior. The class of DC regularizers includes widely used sparsity-promoting penalties such as $\ell_1-\ell_2$, $\ell_1-\ell_{\sigma_q}$, capped-$\mathbb{R}^d$0, PiL, and data-driven priors built from differences of input-convex neural networks (DICNNs). Crucially, several of these regularizers are not weakly convex—the appendix proves this for $\mathbb{R}^d$1, $\mathbb{R}^d$2, capped-$\mathbb{R}^d$3, and leaky-ReLU DICNNs—so existing proximal Langevin analyses that require weak convexity do not apply.

The paper also makes a pointed criticism of the closest prior work [renaud2025stability]: its Assumption 3 requires the Moreau envelope $\mathbb{R}^d$4 to be strongly convex at infinity with modulus at least $\mathbb{R}^d$5, while $\mathbb{R}^d$6 can be shown to be at most $\mathbb{R}^d$7-smooth. The authors prove in their appendix that these two requirements are jointly contradictory for small step sizes, i.e., that assumption cannot be satisfied. This motivates a new analysis built only on distant dissipativity of $\mathbb{R}^d$8 itself.

Algorithm design

DC-LA follows the DC programming principle of redistributing the concave component: the forward (gradient) step applies to $\mathbb{R}^d$9, the backward (proximal) step to $V = f + r = f + (r_1 - r_2)$0. Because both $V = f + r = f + (r_1 - r_2)$1 and $V = f + r = f + (r_1 - r_2)$2 may be nonsmooth, each is replaced by its Moreau envelope with parameter $V = f + r = f + (r_1 - r_2)$3, yielding the augmented potential $V = f + r = f + (r_1 - r_2)$4. The iteration reads

$V = f + r = f + (r_1 - r_2)$5

The paper shows that applying a single Moreau envelope to the full DC function $V = f + r = f + (r_1 - r_2)$6 would be ill-posed—the proximal operator of a DC function can be multivalued and discontinuous (demonstrated via $V = f + r = f + (r_1 - r_2)$7)—whereas the difference-of-Moreau-envelope construction preserves tractability. Using an identity connecting $V = f + r = f + (r_1 - r_2)$8 to $V = f + r = f + (r_1 - r_2)$9, DC-LA decomposes entirely into elementary operations: gradients of $f$0 and proximal operators of $f$1 and $f$2, all of which are available in closed form for standard sparsity penalties. A simplified variant, DC-LA-S, applies directly when $f$3 is smooth (covering Geman, log-sum, Laplace, MCP, and SCAD penalties), removing the need to smooth $f$4.

Convergence guarantees

The main assumptions are: (i) $f$5 is distant dissipative in a subdifferential sense; (ii) $f$6 is Lipschitz continuous (equivalently, bounded subgradients); (iii) $f$7 is either Lipschitz, or differentiable with $f$8-Hölder gradient for $f$9, or smooth. A useful reduction shows that under (ii)–(iii), distant dissipativity of $r_1, r_2$0 holds if and only if $r_1, r_2$1 alone is distant dissipative—a checkable condition since $r_1, r_2$2 is typically quadratic from Gaussian likelihoods.

The central result establishes, for every $r_1, r_2$3,

$r_1, r_2$4

with $r_1, r_2$5, under explicit step-size conditions scaling as $r_1, r_2$6. The bound decays stage-wise: geometric contraction in $r_1, r_2$7, discretization error $r_1, r_2$8, and smoothing bias $r_1, r_2$9. The proof strategy writes the auxiliary sequence $f$0 as a general ULA with drift $f$1, verifies that this drift is $f$2-Lipschitz and distant dissipative, then combines stability results for general-drift Langevin chains with a novel bound $f$3 obtained by controlling the density ratio between smoothed and unsmoothed targets using sub-Gaussian tails implied by dissipativity. The same rate carries over to DC-LA-S with modified constants. To the authors' knowledge, this is the first Langevin-type convergence guarantee covering non-weakly-convex DC regularizers such as $f$4 and DICNNs.

One caveat the paper acknowledges: the constants $f$5 behave like $f$6 and the contraction rate degrades as $f$7, so the theoretical bound does not prescribe how to balance $f$8 against $f$9 optimally; the rates are qualitative rather than sharp.

Experiments

Two settings are evaluated. First, sampling with an $\|Ax-b\|^2$0 prior against Gaussian targets in 2D ($\|Ax-b\|^2$1, correlated covariances), comparing DC-LA against ULA, Moreau ULA, and PSGLA over 5000 chains of length 1000. DC-LA faithfully reproduces the target density's axis-promoting sparsity structure and achieves the lowest binned KL divergence across all bin resolutions and all tested means and covariances, whereas ULA and Moreau ULA are blurry and PSGLA overemphasizes coordinate axes. An ablation over $\|Ax-b\|^2$2 grids shows a broad stable region, indicating robustness to hyperparameter choice, with $\|Ax-b\|^2$3 on the order of $\|Ax-b\|^2$4 (but smaller) performing well.

Second, computed tomography reconstruction on Mayo Clinic abdominal CT scans ($\|Ax-b\|^2$5, 350 parallel-beam projections over $\|Ax-b\|^2$6) with a pretrained DICNN prior trained via adversarial regularization. With 100 chains of length 2000, the DC-LA posterior mean achieves SSIM 0.8493 / PSNR 26.23, exceeding the MAP estimate from the proximal subgradient method at matched iteration count (SSIM 0.7586 / PSNR 24.28); PSM matches DC-LA only after roughly 10,000 iterations and saturates thereafter. Notably, DC-LA additionally yields pixel-wise posterior variance maps that concentrate uncertainty around boundaries and fine textures—an uncertainty quantification capability absent from MAP-only pipelines. The paper honestly reports a failure mode: samples miss a small lower-left structure and the variance map fails to flag it, suggesting either insufficient chain mixing or genuine posterior overconfidence.

Limitations and open questions

Several limitations are stated explicitly. The CT experiment leaves the ill-conditioned likelihood unaugmented rather than adding the radial confining term the theory suggests, so Lemma on dissipativity does not strictly apply there. Proximal operators of the ICNNs are computed approximately via a single fixed-point iteration; extending the analysis to inexact proximal operators (in the spirit of known results on biased proximal Langevin) remains open. The dependence of the constants on $\|Ax-b\|^2$7 is analyzed only for $\|Ax-b\|^2$8 and only asymptotically. Finally, whether the produced variance images are useful in downstream tasks is left as an open question.

Conclusion

DC-LA provides a provably convergent forward-backward sampler for potentials with nonsmooth difference-of-convex regularizers under distant dissipativity, removing both the weak-convexity requirement and the unsatisfiable strong-convexity-at-infinity condition of prior work. Its guarantees cover the practical class of non-weakly-convex sparsity penalties and neural DC priors, and experiments confirm accurate synthetic sampling and usable posterior uncertainty quantification in large-scale CT reconstruction.