FBMS R Package: Bayesian Model Selection

• FBMS R Package is an R toolkit that enables Bayesian model selection and averaging by leveraging advanced MJMCMC and GMJMCMC techniques.
• It is designed for complex regression settings, supporting both generalized linear and nonlinear models while efficiently exploring vast candidate model spaces.
• The package integrates automated nonlinear feature construction, prior-based complexity control, and robust model averaging for predictive inference and uncertainty quantification.

Searching arXiv for the exact FBMS package paper and related methodological context. FBMS is an R package for Bayesian model selection and Bayesian model averaging in regression settings where the candidate model space is large, multimodal, or enriched with nonlinear constructed features. The package implements Monte Carlo model-exploration methods centered on Mode Jumping Markov Chain Monte Carlo (MJMCMC) and genetically modified MJMCMC (GMJMCMC), with the stated aim of improving exploration in Bayesian generalized linear models and in the broader class of Bayesian generalized nonlinear models (BGNLMs) (Frommlet et al., 31 Aug 2025). Its scope includes inferential and predictive modeling for Gaussian, binomial, Poisson, and gamma responses, together with extensibility to mixed-effects models, survival/Cox regression, and other custom likelihood-based settings.

1. Identity, purpose, and domain of application

FBMS, expanded as “Flexible Bayesian Model Selection,” is presented as an R toolkit for Bayesian model selection and Bayesian model averaging in regression problems where exhaustive search is infeasible and ordinary local search can fail because of posterior multimodality (Frommlet et al., 31 Aug 2025). The package is designed to answer several core questions in Bayesian regression: which predictors or transformed predictors matter, what posterior probability should be assigned to candidate models, what posterior inclusion probabilities should be assigned to features, how predictions should be averaged over model uncertainty, and how very large model spaces can be explored without becoming trapped in local posterior modes.

The package is described as supporting Bayesian generalized linear models, including Gaussian, binomial, Poisson, and gamma families, and Bayesian generalized nonlinear models in which predictors may be nonlinear transformed or composite features built recursively from the original covariates. It also permits extension through user-defined likelihood or posterior evaluators to mixed-effects models, mixed-effects Poisson regression, survival/Cox regression, and custom model classes. This supports the characterization of FBMS as a framework for searching over model structures rather than a package restricted to linear variable selection.

A common misunderstanding is to treat FBMS as only a variable-selection package for additive Gaussian models. The package is instead positioned for complex regression settings with large discrete model spaces, strong feature correlations, multiple posterior modes, nonlinear feature construction, and predictive inference under model uncertainty. This suggests that the defining object of computation is not merely a coefficient vector within a fixed design matrix, but the model space itself.

2. Statistical basis: BGNLMs and posterior model probabilities

The conceptual core of FBMS is the class of Bayesian generalized nonlinear models. For Gaussian responses, the paper gives the model

$$Y_i = \beta_0 + \sum_{j=1}^{q} \gamma_j \beta_j F_j(\bm{x}_i) + \epsilon_i, \qquad \epsilon_i \sim \mathcal{N}(0,\phi),$$

where $Y_i$ is the response, $\bm{x}_i$ is the vector of original covariates, $F_j(\bm{x}_i)$ is the $j$-th candidate feature, $\gamma_j \in \{0,1\}$ indicates inclusion of feature $j$, $\beta_j$ is its regression coefficient, and $\phi = \sigma^2$ is the dispersion parameter (Frommlet et al., 31 Aug 2025). The full candidate feature set is denoted $\mathcal{F}$. If there are $Y_i$0 candidate features, the number of candidate models is $Y_i$1, which is the immediate source of the combinatorial difficulty addressed by the package.

The posterior probability of a model $Y_i$2 is written as

$Y_i$3

with the proportional form $Y_i$4. Since summation over all models is generally impossible, FBMS approximates the posterior using the visited model set $Y_i$5:

$Y_i$6

This approximation underlies posterior summaries, posterior inclusion probabilities, and prediction.

For Gaussian models with Zellner’s $Y_i$7-prior and $Y_i$8, the marginal likelihood is given as

$Y_i$9

For Jeffreys prior, the paper uses a Laplace/BIC-style approximation,

$\bm{x}_i$0

These formulas show that FBMS is structured around marginal-likelihood-based Bayesian model comparison rather than around single-model penalized estimation.

3. Nonlinear feature construction and model-space exploration

A central feature of FBMS is that the candidate predictors need not be the original covariates. Features are generated from a user-specified transformation set $\bm{x}_i$1 using three operators: nonlinear modifications, nonlinear projections, and interactions (Frommlet et al., 31 Aug 2025). The package also uses mutation, meaning that original covariates can be reintroduced after having been removed from the current feature population.

For nonlinear projections, the paper gives the feature form

$\bm{x}_i$2

where $\bm{x}_i$3 is a chosen nonlinear transform and $\bm{x}_i$4 are internal projection parameters. This construction allows the feature space to represent ordinary linear effects, interactions, fractional polynomials, logic regression terms, symbolic-regression-type expressions, and neural-network-like nonlinear projection features.

The package uses MJMCMC to address the fact that nonlinear model spaces are often multimodal. The rationale is explicit: multiple correlated features can explain the same signal, different nonlinear expressions can be nearly equivalent, and feature engineering induces redundancy and strong dependence. MJMCMC therefore combines local Metropolis–Hastings moves with occasional large jumps, followed by local optimization and randomization, to permit direct transitions between distant posterior modes rather than traversing many low-probability intermediate states.

GMJMCMC extends this logic to the case where the feature set itself is not fixed. The algorithm alternates between searching through models from a current population of features $\bm{x}_i$5 and updating that population using posterior feedback. The four population-update operators are interaction, modification, nonlinear projection, and mutation. Features with higher estimated marginal posterior probabilities are more likely to be retained or used as parents, while low-probability features are more likely to be removed. This suggests that FBMS operates simultaneously as a model-search engine and as a Bayesian feature-engineering mechanism.

4. Priors, complexity control, and Bayesian model averaging

FBMS exposes priors on both model parameters and model structure. The generic factorization is

$\bm{x}_i$6

Among the parameter priors listed are Jeffreys prior, Zellner’s $\bm{x}_i$7-prior,

$\bm{x}_i$8

and mixtures of $\bm{x}_i$9-priors based on a tCCH prior on $F_j(\bm{x}_i)$0 (Frommlet et al., 31 Aug 2025). The paper lists g-prior, Jeffreys, robust, hyper-g, hyper-g-n, intrinsic, empirical Bayes local/global, JZS, and AIC/BIC/Jeffreys-BIC among the available prior choices. For Gaussian models with unknown variance, the dispersion prior is $F_j(\bm{x}_i)$1, while for binomial, Poisson, and gamma models $F_j(\bm{x}_i)$2.

The default model prior penalizes complexity:

$F_j(\bm{x}_i)$3

where $F_j(\bm{x}_i)$4, the default is $F_j(\bm{x}_i)$5, and the default complexity measure $F_j(\bm{x}_i)$6 is operation count (oc). Alternative complexity descriptors include width and depth. The paper also gives a custom logic-regression prior,

$F_j(\bm{x}_i)$7

These choices formalize the package’s attempt to balance expressive feature construction against structural regularization.

Bayesian model averaging in FBMS is implemented procedurally by weighting model-specific predictions by approximate posterior model probabilities from visited models. The returned predictions include weighted mean predictions and weighted quantiles or credible intervals. Posterior inclusion probabilities are central summaries, defined as the marginal posterior probabilities $F_j(\bm{x}_i)$8 estimated from the explored model set. A plausible implication is that the package treats posterior uncertainty about feature existence and posterior uncertainty about prediction as two aspects of the same visited-model approximation.

5. Software architecture, interfaces, and workflow

The principal high-level interface is fbms(), which supports formula-based model specification analogous to lm() or glm() (Frommlet et al., 31 Aug 2025). Lower-level samplers include mjmcmc(), gmjmcmc(), mjmcmc.[parallel](https://www.emergentmind.com/topics/additive-parallel-correction)(), and gmjmcmc.parallel(). The helper generators gen.probs.mjmcmc(), gen.probs.gmjmcmc(), gen.params.mjmcmc(), and gen.params.gmjmcmc() expose the search-kernel probabilities and the associated algorithmic control parameters. Downstream tools include summary(), plot(), predict(), diagn_plot(), get.best.model(), and get.mpm.model().

Typical usage specifies a formula, a data frame, a method such as "mjmcmc" or "gmjmcmc", a response family such as "gaussian", "binomial", "poisson", "gamma", or "custom", and a vector of transformation names. If no formula is supplied, the package assumes that the first column is the response and the remaining columns are predictors. Built-in transformations include activation-type functions such as sigmoid, relu, and gelu; smooth transforms such as sin_deg, cos_deg, gauss, erf, and arcsinh; root and log-related transforms such as sqroot, troot, and p0; fractional polynomial transforms such as pm2, pm1, pm05, p05, p2, p3, and second-order forms like p0p1 and p0p2; and the logical transform not.

The params$$feat</code> sublist governs feature generation, including maximum feature depth <code>D</code>, maximum number of features in a model <code>L</code>, projection strategy <code>alpha</code>, maximum population size <code>pop.max</code>, whether to retain original covariates, preselection controls, minimum retained proportion, minimum posterior inclusion threshold for generation, collinearity checking, and maximum projection size. For nonlinear projections, internal coefficients$$F_j(\bm{x}_i)$9 can be handled by "unit", "deep", or "random" strategies. The "unit" strategy fixes all $j$0 values to $j$1, the "deep" strategy optimizes them across layers with gradient-free optimization, and the "random" strategy samples them from priors.

The paper is explicit that the default settings are often illustrative rather than sufficient for substantive analyses. It recommends increasing the number of populations P, the number of MJMCMC iterations per population N, and the final-population iterations N.final, and it notes that multiple independent parallel runs often outperform a single long chain. This indicates that practical use of FBMS typically requires deliberate tuning of exploration intensity rather than reliance on nominal defaults.

6. Applications, diagnostics, strengths, and limitations

The package is demonstrated across a broad range of model classes. The Gaussian nonlinear regression example based on exoplanet data is used to recover an approximation to Kepler’s third law,

$j$2

illustrating symbolic-regression-like recovery of scientific structure (Frommlet et al., 31 Aug 2025). Other examples include Gaussian linear variable selection, interaction models, Bayesian fractional polynomial selection, nonlinear projection models, high-dimensional gene-expression screening, logic regression, logistic regression for spam classification, linear mixed models, mixed-effects Poisson regression, subsampling for tall-data logistic regression, and Cox regression with nonlinear effects and fractional polynomials.

The results reported in the examples are used primarily to show behavior rather than to define universal performance guarantees. The exoplanet example indicates that default single-chain settings may miss the true structure, that increasing the number of populations and iterations improves results, and that 40 parallel chains can recover the correct nonlinear feature with posterior probability greater than $j$3. The linear example shows that FBMS does not automatically overfit with nonlinear terms and that mjmcmc may be preferable when only linear effects are expected. The interaction and fractional-polynomial examples show how the generation probabilities can restrict the effective model class. The Cox-regression example indicates that model averaging can improve predictive C-index relative to linear-only alternatives.

Diagnostics are handled through diagn_plot(), which summarizes stability of log posterior summaries over populations, across chains or within a chain, and can display statistics such as max, median, mean, and variance with confidence bands based on sliding windows. The paper is careful about the interpretation of these diagnostics: stable trends suggest convergence or mixing, but do not guarantee that the global mode has been reached, and apparent convergence may still correspond to all chains being trapped in a strong local mode. This is one of the main methodological caveats surrounding the package.

The paper also states several limitations. Computational cost can be high when nonlinear feature spaces are rich, when custom likelihoods or mixed-effects back ends such as INLA or RTMB are used, or when many populations and chains are required. Results may be sensitive to settings such as P, N, N.final, generation probabilities, and population size. Multicollinearity and posterior dilution can complicate interpretation in rich nonlinear spaces. These caveats do not negate the package’s intended role; rather, they define its operating regime as a flexible but tuning-dependent Bayesian model-space exploration system.

A final misconception addressed by the package design is that model uncertainty can be handled adequately after selecting a single best model. FBMS is organized around posterior model probabilities, posterior inclusion probabilities, and model-averaged prediction, so the package’s logic is fundamentally Bayesian model averaging rather than single-model reporting. This suggests that its most characteristic contribution is not only efficient search, but also the retention and use of visited-model uncertainty as an inferential object in its own right.