Proximal nested sampling for high-dimensional Bayesian model selection (2106.03646v3)

Abstract: Bayesian model selection provides a powerful framework for objectively comparing models directly from observed data, without reference to ground truth data. However, Bayesian model selection requires the computation of the marginal likelihood (model evidence), which is computationally challenging, prohibiting its use in many high-dimensional Bayesian inverse problems. With Bayesian imaging applications in mind, in this work we present the proximal nested sampling methodology to objectively compare alternative Bayesian imaging models for applications that use images to inform decisions under uncertainty. The methodology is based on nested sampling, a Monte Carlo approach specialised for model comparison, and exploits proximal Markov chain Monte Carlo techniques to scale efficiently to large problems and to tackle models that are log-concave and not necessarily smooth (e.g., involving l_1 or total-variation priors). The proposed approach can be applied computationally to problems of dimension O(10^6) and beyond, making it suitable for high-dimensional inverse imaging problems. It is validated on large Gaussian models, for which the likelihood is available analytically, and subsequently illustrated on a range of imaging problems where it is used to analyse different choices of dictionary and measurement model.

• The paper introduces proximal nested sampling, a novel method leveraging proximal MCMC within the nested sampling framework to enable efficient high-dimensional Bayesian model selection.
• This approach scales to dimensions up to O(10^6) and handles non-smooth priors such as \u03b9_1 and total variation using proximal operators and advanced sampling.
• Validating against known solutions, the method was applied to imaging problems like denoising and reconstruction, showing potential for robust model selection in high-dimensional applications.

Proximal Nested Sampling for High-Dimensional Bayesian Model Selection

The paper "Proximal Nested Sampling for High-Dimensional Bayesian Model Selection" introduces a novel methodology to address the computational challenges associated with Bayesian model selection in high-dimensional settings, particularly in the context of inverse imaging problems. Traditionally, Bayesian model selection is impeded by the difficulty of computing the marginal likelihood, also known as the Bayesian evidence, especially in situations involving high-dimensional parameter spaces. This research proposes a method that leverages nested sampling combined with proximal Markov chain Monte Carlo (MCMC) techniques to efficiently scale to such large-scale problems.

Methodology

The proposed proximal nested sampling approach is designed to handle not only high-dimensional models but also non-smooth priors, such as those involving $\ell_1$ or total-variation regularizers, which are frequently encountered in Bayesian imaging applications. The methodology builds on the existing framework of nested sampling, which transforms the problem of computing the marginal likelihood into a one-dimensional integral over the prior volume, thus simplifying the problem significantly. However, a major challenge is sampling from the prior distribution under a hard constraint imposed by the likelihood, which is managed through advanced sampling techniques.

The critical innovation in this work is the integration of proximal MCMC, specifically designed for log-concave models that are potentially non-smooth. This is achieved using Moreau-Yosida approximations of non-differentiable terms, allowing efficient sampling from complex constrained distributions. The proximal nested sampling method involves:

1. Proximal MCMC Techniques: Utilizing unadjusted Langevin algorithms with Metropolis-Hastings corrections to ensure hard constraints are respected, while ensuring the approach scales well with problem dimensions.
2. Handling Non-Smooth Priors: The method accommodates non-smooth models by relying on proximal operators, which are computationally feasible to evaluate for many practical functions.
3. Dimensional Scalability: The proposed methodology demonstrates the ability to tackle problems with dimensions reaching up to $\mathcal{O}(10^6)$, a significant expansion over traditional nested sampling techniques which are typically limited to lower dimensional settings.

Validation and Applications

The effectiveness of proximal nested sampling was validated against scenarios with known analytical solutions, demonstrating convergence and accuracy even in very high-dimensional problems. Furthermore, the approach was applied to two canonical imaging problems: image denoising and image reconstruction. These applications illustrate the potential of proximal nested sampling to inform decisions related to model choice, such as selection of the best sparsifying dictionary or tuning of regularization parameters.

Implications and Future Work

The implications of this research are substantial for fields reliant on high-dimensional Bayesian inference, such as astrophysics, medical imaging, and machine learning, where model selection is critical yet challenging due to computational constraints. By providing a scalable and flexible framework, this work enables more thorough exploration of model space than was previously feasible.

Future directions could include further theoretical exploration of convergence properties, extensions to models including multi-modal posterior distributions, and adaptations to incorporate likelihood-informed priors in an efficient manner. Additionally, while the current approach is tailored for log-concave models, future work might explore adaptations for more complex non-convex prior landscapes.

This paper represents a significant stride in making high-dimensional Bayesian model selection more accessible and computationally feasible, opening new avenues for robust scientific inference in data-intensive applications.