ABC random forests for Bayesian parameter inference (1605.05537v5)

Abstract: This preprint has been reviewed and recommended by Peer Community In Evolutionary Biology (http://dx.doi.org/10.24072/pci.evolbiol.100036). Approximate Bayesian computation (ABC) has grown into a standard methodology that manages Bayesian inference for models associated with intractable likelihood functions. Most ABC implementations require the preliminary selection of a vector of informative statistics summarizing raw data. Furthermore, in almost all existing implementations, the tolerance level that separates acceptance from rejection of simulated parameter values needs to be calibrated. We propose to conduct likelihood-free Bayesian inferences about parameters with no prior selection of the relevant components of the summary statistics and bypassing the derivation of the associated tolerance level. The approach relies on the random forest methodology of Breiman (2001) applied in a (non parametric) regression setting. We advocate the derivation of a new random forest for each component of the parameter vector of interest. When compared with earlier ABC solutions, this method offers significant gains in terms of robustness to the choice of the summary statistics, does not depend on any type of tolerance level, and is a good trade-off in term of quality of point estimator precision and credible interval estimations for a given computing time. We illustrate the performance of our methodological proposal and compare it with earlier ABC methods on a Normal toy example and a population genetics example dealing with human population evolution. All methods designed here have been incorporated in the R package abcrf (version 1.7) available on CRAN.

• The paper introduces an ABC Random Forests method for Bayesian inference that bypasses summary statistic selection and tolerance calibration.
• The method provides superior accuracy in parameter point estimation and credible interval determination compared to existing ABC methods.
• This method is computationally efficient, robust, and simplifies complex model analysis for researchers, especially in fields like genomics.

ABC Random Forests for Bayesian Parameter Inference

This paper introduces an innovative use of Random Forests (RF) within the framework of Approximate Bayesian Computation (ABC) for Bayesian parameter inference, particularly targeting situations where likelihood functions are intractable. The approach tackles two significant challenges within ABC frameworks: the preliminary selection of summary statistics components and the calibration of the tolerance level, which traditionally dictates the acceptance or rejection of simulated parameter values.

The authors propose a method that utilizes Random Forests to perform likelihood-free Bayesian inferences without these two prerequisites. By leveraging the RF methodology introduced by Breiman (2001), the authors formulate a regression setting where a new random forest is derived for each component of the parameter vector of interest. Key to this approach is its robustness to the choice of summary statistics, independence from tolerance level specification, and a beneficial balance between point estimator precision and computational efficiency. The paper benchmarks the proposed method against existing ABC solutions using two examples: a simplistic Normal distribution model and a more complex population genetics case pertaining to human population evolution.

Strong numerical results are demonstrated, with RF-augmented ABC providing superior approximation to the expected values and credible intervals of posterior distributions over traditional methods. Notably, the approach handles an extensive feature space of potential summary statistics, which might traditionally introduce noise, without requiring manual reduction—a key benefit over classic K-nearest neighbor techniques within ABC.

Theoretical and Practical Implications:

• Non-parametric Regression Application: The integration of RF within ABC circumvents the manual selection of summary statistics, effectively automating the extraction of relevant features. This adds significant versatility and efficiency in tackling large datasets and complex models.
• Parameter Estimation Quality: The method demonstrates superior accuracy in point estimation of parameters and credible interval determination compared to alternative ABC approaches, with applications showing reasonable accuracy even as datasets grow in complexity.
• Computational Efficiency: The approach, as integrated within the R package abcrf, promotes robust parameter inference without extensive reliance on computationally intensive calibration, such as tuning tolerance levels.
• Ease of Use & Implementation: With the RF mechanism, users can significantly ease their analyses of comprehensive datasets in fields like population genetics without sacrificing inferential reliability.

While promising, further research into multidimensional parameter applications using RF within the ABC framework is warranted. This includes enhancing the approach's applicability to fully capture posterior covariance estimations and extending it to non-conjugate models that exhibit greater complexity and non-linearity in parameter relationships.

The paper's findings could propel advancements in statistical methodologies applied to disciplines with intricate data patterns and require sophisticated inferential models, showing particular promise for genomics and broader biological data analytics. These developments underscore the RF-inclusion's potential in realizing scalable, reliable, and interpretable Bayesian inference processes.