Basis Function BIC (BF-BIC)

• BF-BIC is a refinement of the Bayesian Information Criterion that adjusts penalties using the real log-canonical threshold (RLCT) to address singular model structures.
• It leverages algebraic geometry and asymptotic Laplace integrals to accurately select models in settings like latent trees, basis expansions, and mixture models.
• BF-BIC incorporates additional corrections, such as log-log terms, to ensure theoretical consistency and improved performance over traditional BIC in complex applications.

Basis Function BIC (BF-BIC) is a refinement of the Bayesian Information Criterion (BIC) tailored to account for complex model structures—especially those involving basis expansions and singular parameter spaces—where classical regularity conditions underpinning traditional BIC break down. It modifies the penalty for model complexity by precisely quantifying the underlying geometric or algebraic singularities via the real log-canonical threshold (RLCT), generalizing the classical BIC to settings such as latent tree graphical models, basis-expansion regressions, nonlinear causal models, mixture models, clustering, and models with order constraints. Theoretical foundations are grounded in asymptotic Laplace integral analysis and algebraic geometry, providing consistency even in singular regimes. BF-BIC therefore supports robust model selection for applications spanning phylogenetics, Bayesian networks, regression with basis functions, causal discovery, and high-dimensional mixture modeling.

1. Mathematical Foundations: Laplace Integrals and RLCT

BF-BIC arises from the asymptotic behavior of the marginal likelihood integral

$$Z(N) = \int_{\Theta} L(\theta; X^N) \varphi(\theta)\,d\theta$$

where $L(\theta; X^N)$ is the likelihood and $\varphi(\theta)$ the prior over the parameter space $\Theta$. Traditionally, BIC uses a Laplace approximation valid when the MLE is an isolated point: $$\log Z(N) = \hat{\ell}_N - \frac{d}{2} \log N + O(1)$$ with dimension $d$. However, for singular models (e.g., Bayesian networks with hidden variables on trees), the MLE is non-isolated and Laplace's approximation fails. By employing resolution of singularities from algebraic geometry, the exact asymptotic expansion is

$$\log Z(N) = \hat{\ell}_N - (\mathrm{rlct})\log N + (\mathrm{mult} - 1)\log\log N + O(1)$$

where $\mathrm{rlct}$ (real log-canonical threshold) encodes effective model complexity as a rational number, and $\mathrm{mult}$ is its multiplicity (see Theorem 2.7, (Zwiernik, 2010)). In the regular case, $\mathrm{rlct}=d/2$.

The computation of RLCT proceeds by reparameterizing the model (e.g., tree cumulant coordinates for Markov trees) and "monomializing" the log-likelihood difference $$f(\theta) = \ell(\hat{p}; \hat{p}) - \ell(p(\theta); \hat{p})$$, then solving for the smallest pole of the zeta function

$$\zeta(z) = \int f(\theta)^{-z} \varphi(\theta)\,d\theta$$

This analytic approach forms the foundation of BF-BIC: instead of penalizing by $d/2 \log N$ as BIC does, the penalty is $\mathrm{rlct}\log N + (\mathrm{mult}-1)\log\log N$.

2. Application to Graphical Models with Hidden Variables

The original impetus for BF-BIC was model selection in Bayesian networks over rooted trees with hidden inner nodes and binary observed leaves. Using tree cumulants, the likelihood is recast as a sum of squares, and RLCT is computed via the geometry of the zero set of $f(\theta)$. For general Markov models,

$$\log Z(N) = \hat{\ell}_N - \frac{1}{4}(3n + l_2 + 5l_3 - \delta)\log N + O(1)$$

where $n$ is the number of leaves, $l_2$ counts degenerate inner nodes, $l_3$ counts fully connected inner nodes, and $\delta$ is a correction term for root degeneracy ((Zwiernik, 2010), Theorems 5.4, 5.8). This penalty term precisely captures structural singularities, yielding an information criterion—BF-BIC—that adapts to model geometry and avoids over- or under-penalizing model complexity.

The workflow for BF-BIC in this setting:

1. Compute the marginal likelihood integral and define $f(\theta)$.
2. Reparameterize to facilitate singularity analysis.
3. Resolve singularities and compute RLCT and multiplicity.
4. Use the expansion above to obtain the correct BF-BIC penalty and select models accordingly.

BF-BIC is particularly impactful in phylogenetics and hidden Markov model settings, where naive BIC leads to inconsistent or biased selection due to hidden variable-induced singularities.

3. Basis Expansion Models and Occam's Razor

In regression and classification problems, basis expansions allow flexible modeling: $y(x, w) = \sum_j w_j \phi_j(x)$ BF-BIC is relevant in Bayesian model selection contexts that integrate over coefficients $w$ using priors and likelihoods. The Laplace approximation yields a marginal likelihood penalized by the Hessian determinant ("Occam factor"), automatically integrating Occam's razor. Compared to classical BIC—where penalty is solely in terms of number of parameters $M$ and sample size $N$ ($M\log N$)—the full Bayesian treatment using basis functions more precisely penalizes according to the shape and spread of the posterior, as seen in (Delgado et al., 2015).

For misspecified models, and to interpolate between AIC and BIC behaviors, a robust BF-BIC approach is achieved by modeling both signal and noise spaces using noncentral Gamma priors and penalizing according to observed signal strength $F^2$. The resulting criterion adapts its penalty based on the degree of model misspecification and signal-to-noise ratio (Kock et al., 2017), outperforming classical BIC/AIC in weak signal settings.

4. Extensions to Order Constraints, Clustering, and Mixture Models

Order-constrained models, energy‐based clustering, and mixture models routinely expose basis-induced singularities and irregular parameter spaces, requiring modification of the standard BIC penalty. For order constraints (e.g., $\beta_1 > \beta_2 > 0$), BF-BIC employs truncated priors (unit information and local unit information) and additional penalty terms reflecting constraint-induced reduced parameter volume (Mulder et al., 2018). Closed-form corrections involving multivariate probabilities yield significant reductions in error probabilities for model selection under constraints.

In clustering problems with normally distributed data, the standard BIC (using a Laplace approximation) is often invalid for small cluster sizes. BF-BIC resolves this by exact marginalization over cluster centers, leading to penalty terms that scale with cluster-specific sizes and model partition combinatorics (Webster, 2020, Teklehaymanot et al., 2017). The result is a separation between data-fidelity (sum-of-squares fit) and a precisely tailored penalty that remain valid even for singleton clusters.

In mixture models, consistency of order selection via BIC requires strong regularity not always available. Small logarithmic tweaks (ν-BIC, ε-BIC) to the penalty term—minor for practical sample sizes—allow BF-BIC to provide consistency guarantees under substantially weaker assumptions, including for non-differentiable Laplace mixtures and mixtures of regression models (Nguyen et al., 25 Jun 2025).

5. Algorithmic and Practical Implications

BF-BIC is algorithmically implementable via the following steps:

• Define a suitable set of basis functions or coordinate transformations that expose model singularities.
• Calculate the normalized log-likelihood difference and corresponding Laplace integral.
• Apply algebraic-geometric tools (resolution of singularities, Newton diagrams) to resolve fibers and compute RLCT and multiplicity.
• Evaluate the BF-BIC score: maximize penalized likelihood with RLCT/log-log correction, select the optimal model structure.

For structure learning in Bayesian networks, entropy-based pruning rules (leveraging conditional and marginal entropies) can be combined with BF-BIC scoring to significantly shrink the candidate search space while retaining optimality (Campos et al., 2017), leading to up to 50% practical reduction in computation.

In clustering, BF-BIC computation via exact marginalization or adapted Laplace methods enables robust enumeration of clusters, particularly sensitive to cluster composition. Combined algorithms (e.g., EM followed by BF-BIC scoring) facilitate accurate cluster count estimation on synthetic and real data sets (Teklehaymanot et al., 2017).

Implementations of BF-BIC and its extensions are available in statistical and machine learning toolkits, including specialized packages for order-constrained model selection (R’s BICpack), scalable causal discovery suites (BOSS, Python/R/Java implementations), and standard clustering workflows.

6. Impact in Causal Learning and High-Dimensional Discovery

In modern causal inference for nonlinear, continuous, or mixed data, BF-BIC is leveraged for scalable score-based and hybrid conditional independence searches. By truncating additive basis expansions (e.g., Legendre polynomials) for each variable and embedding categorical variables through degenerate-Gaussian representations, BF-BIC remains robust to moderate interaction nonlinearity and supports high-dimensional variable sets (Ramsey et al., 5 Oct 2025). This method outperforms costly kernel-based approaches (KCI, RFCI) in both accuracy and runtime for causal graph structure recovery in nonlinear neural causal models.

In practical applications, BF-BIC is shown to recover interpretable structures—such as wildfire risk causality in complex meteorological datasets—by efficiently capturing nonlinear dependencies and combinatorial constraints of large-scale data.

7. Relevance, Limitations, and Theoretical Guarantees

BF-BIC generalizes classical BIC, precisely quantifying model complexity under singularities via RLCT, order constraints, clustering composition, and basis function dimensionality. It achieves large-sample consistency in model selection even in irregular regimes where standard tools fail. Remaining limitations include the need for algebraic-geometric expertise to compute RLCT in highly singular or intricate models, and the computational burden can grow in high-dimensional or deeply nested expansion settings. However, for many modern statistical applications across graphical models, regression, clustering, and causal inference, BF-BIC provides a theoretically sound, empirically validated, and pragmatic criterion for high-fidelity model selection.