Automatic Posterior Transformation for Likelihood-Free Inference (1905.07488v1)

Abstract: How can one perform Bayesian inference on stochastic simulators with intractable likelihoods? A recent approach is to learn the posterior from adaptively proposed simulations using neural network-based conditional density estimators. However, existing methods are limited to a narrow range of proposal distributions or require importance weighting that can limit performance in practice. Here we present automatic posterior transformation (APT), a new sequential neural posterior estimation method for simulation-based inference. APT can modify the posterior estimate using arbitrary, dynamically updated proposals, and is compatible with powerful flow-based density estimators. It is more flexible, scalable and efficient than previous simulation-based inference techniques. APT can operate directly on high-dimensional time series and image data, opening up new applications for likelihood-free inference.

• The paper introduces APT as a novel sequential neural posterior estimation method that directly targets the posterior without relying on explicit likelihood functions.
• APT demonstrates robust performance by accurately inferring parameters in high-dimensional simulations such as two-moons, Lotka-Volterra, and SPDE-RPS models.
• The method integrates dynamic proposal distributions and flexible neural architectures, bridging classical Bayesian computation with modern machine learning techniques.

Exploring Automatic Posterior Transformation for Likelihood-free Inference

The paper "Automatic Posterior Transformation for Likelihood-free Inference" introduces a novel approach to simulation-based inference when explicit likelihoods are unavailable or computationally prohibitive. The technique, known as Automatic Posterior Transformation (APT), addresses the challenges of linking complex mechanistic models with empirical measurements and offers a promising alternative to existing likelihood-free inference methods.

Core Contributions

APT is a sequential neural posterior estimation methodology designed to directly target the posterior distribution without relying on the likelihood function. It leverages powerful flow-based density estimators and dynamically updated proposal distributions for improved flexibility and performance over earlier simulation-based inference techniques. Notably, APT advances the field with the following contributions:

• Posterior Transformation: It efficiently transforms posterior estimates by maximizing the probability of simulation parameters using dynamically chosen proposals. This allows for the use of arbitrary proposals without necessitating importance weights or post-hoc corrections, which are limitations in previous methods like SNPE-A and SNPE-B.
• High-dimensional Data Handling: APT is capable of operating on high-dimensional time series and image data, thus broadening its applicability in domains where traditional methods falter.
• Algorithmic Flexibility: The workflow accommodates various neural network architectures, such as recurrent and convolutional networks, and mixes classical Bayesian computation with modern machine learning paradigms.

Key Numerical Findings and Claims

APT demonstrates strong numerical performance across a range of challenging inference tasks. Some key findings include:

• On the "two-moons" simulation, APT efficiently identified both global and local structures of the posterior distribution, where methods like SNPE-B were impeded by high variance due to importance weights.
• In high-dimensional scenarios, such as the Lotka-Volterra model, APT outperformed classical methods by producing accurate posterior estimates around true parameters with fewer simulations.
• For SPDE-RPS models with image-like outputs, APT, utilizing a CNN, successfully inferred posterior distributions that closely matched the ground truth parameters, outperforming methods like SNPE-A/B.

Theoretical Implications

From a theoretical standpoint, APT’s independence from likelihood allows it to sidestep complexities associated with likelihood evaluation and focuses directly on feature-learning pertinent to posterior estimation. It integrates features of likelihood-free methods and posterior density approaches, bridging the gap with adaptable proposal strategies.

Implications for Future AI Developments

APT's ability to efficiently infer parameters without sacrificing accuracy presents a robust tool for real-world applications, such as in systems biology, economics, and computational neuroscience. Given its adaptability to high-dimensional and complex data outputs, future advancements in AI could see APT being incorporated within models that require rapid, robust parameter estimation without the conventional burdens of tuning likelihood functions.

In conclusion, APT marks a significant step forward in the domain of likelihood-free inference, showcasing enhanced flexibility and scalability. It opens avenues for researchers to enhance inference strategies while leveraging the power of neural networks tailored for complex simulations. As AI expands to more intricate systems with latent parameters, methods such as APT will be invaluable for advancing our understanding and predictive capabilities in various scientific fields.