Bayesian Network LLM Fusion

• Bayesian Network LLM Fusion is a framework that integrates large language models with Bayesian networks to enhance interpretability and robustness through explicit probabilistic reasoning.
• It employs ensemble methods, prompt-based parameter extraction, and iterative structure discovery to efficiently fuse linguistic context with statistical inference in domains like sentiment analysis and trading.
• BNLF improves predictive accuracy and transparency while reducing resource demands, offering clear advantages over standalone LLMs or data-driven models.

Bayesian Network LLM Fusion (BNLF) is a family of frameworks that integrate LLMs with Bayesian networks (BNs) to combine the linguistic, contextual, or probabilistic reasoning strengths of LLMs with the explicit, interpretable, and mathematically robust inference provided by BNs. BNLF methodologies have demonstrated utility in domains such as sentiment analysis, data-efficient BN parameterization, and automated BN structure discovery. BNLF approaches can operate in data-free, data-aware, or ensemble settings, and frequently enable improved interpretability, data efficiency, and robustness compared to individual LLMs or purely data-driven statistical fusion.

1. Methodological Foundations of BNLF

BNLF denotes the process of leveraging LLMs at different stages of Bayesian network construction, learning, and inference. The fusion can take several forms:

• Late fusion of LLM predictions (ensemble-style): BN nodes represent discrete model outputs from multiple LLMs, with the BN capturing conditional dependencies and aggregating predictions probabilistically (Amirzadeh et al., 30 Oct 2025).
• Parameterization via probabilistic extraction: LLMs are queried to elicit empirical conditional probability estimates for BN Conditional Probability Tables (CPTs), either in the absence of data or as informative priors combined with observed data (Nafar et al., 21 May 2025).
• BN structure induction via LLM reasoning: LLMs construct BN structures (the directed acyclic graph or DAG) from metadata or iteratively refine these structures in combination with scoring criteria when data are available (Zhang et al., 1 Nov 2025).
• Hybrid architectures for decision-making: LLMs are used to contextualize complex environments, construct context-specific BNs, and parameterize CPTs by data selection, as in algorithmic trading (Kuang et al., 30 Nov 2025).

These approaches all employ a probabilistic graphical framework to integrate LLM outputs, using explicit inference, model explainability, and data fusion properties unique to BNs.

2. BNLF for Model Ensemble and Prediction Aggregation

The canonical BNLF application in model ensemble is realized as a late fusion mechanism that aggregates discrete LLM sentiment predictions via a BN (Amirzadeh et al., 30 Oct 2025). The design involves:

• Discrete random variables $M_1, M_2, M_3$ representing predictions from LLMs (e.g., FinBERT, RoBERTa, BERTweet).
• A class variable $S$ encoding ground-truth sentiment, and a corpus variable $C$ capturing dataset source.
• Directed edges $C \rightarrow M_i$ and $M_i \rightarrow S$ capture, respectively, corpus effects on each LLM and LLM prediction influence on final sentiment, with $C \rightarrow S$ encoding direct corpus label bias.

The joint distribution factorizes as:

$$P(C,M_1,M_2,M_3,S) = P(C)\prod_{i=1}^3 P(M_i|C) \cdot P(S|M_1,M_2,M_3,C).$$

Inference produces $P(S|M_1, M_2, M_3, C)$, combining hard evidence ($m_i$) or optionally soft evidence (posterior probabilities). Parameter learning is performed via maximum likelihood with optional Dirichlet smoothing, eliminating any need for LLM fine-tuning. This modular approach yields accuracy improvements of approximately 6% over the best constituent models and supports interpretable “scenario” analysis (Amirzadeh et al., 30 Oct 2025).

3. BNLF for Data-Efficient BN Parameterization

When populating CPTs in a BN for probabilistic reasoning, BNLF enables extraction of expert priors from LLMs and fuses these with observed data (Nafar et al., 21 May 2025). The “Extracting Probabilistic Knowledge” (EPK) method uses carefully designed prompts to elicit $P(X = x | Pa(X) = pa)$ for each BN variable and its parent configuration. Prompts are constructed per CPT row:

These nodes are related to the question inside a Bayesian Network: * Xi: description * Parent variables (with descriptions) Given $Pa(X_i) = pa$, what is the probability that $X_i = x_j$?

The returned values $P_j$ are normalized to form conditional distributions $\tilde P(X_i = x_j | pa)$.

For empirical fusion, BNLF applies one of:

• Linear pooling:

$$p_i = \alpha q_i + (1- \alpha) r_i, \;\;\; \alpha = 1/N,$$

where $q_i$ is the LLM prior, $r_i$ is the relative frequency from $N$ observed samples.

• Pseudocount fusion:

$$p_i = \frac{C q_i + n_i }{ C + \sum_j n_j }, \quad C = 1/N,$$

with $n_i$ the count for $X_i = x_i$ under $Pa(X) = pa$.

Experiments over 80 BNs from the bnRep corpus show that EPK priors reduce BN-KL divergence by 20–50% versus data-only MLE from the same number of samples. At $N = 3$, EPK+data achieves BN-KL ≈ 0.95 nats (MLE-30 = 0.80; Uniform+N always higher). The standard deviation of BN-KL is halved, indicating more robust parameter estimation, especially for rare configurations. Downstream marginal inference errors decrease substantially (MAE = 0.11 for GPT-4o+30, versus 0.18 for pure MLE-30) (Nafar et al., 21 May 2025).

4. BNLF for Structure Discovery and Automated Graph Construction

BNLF frameworks also enable structure learning either in a data-free (“PromptBN”) or data-aware (“ReActBN”) setting (Zhang et al., 1 Nov 2025):

• PromptBN: The LLM is prompted with variable descriptions and tasked with returning node parent lists or edge lists (showing directionality and justifications). Output is parsed and validated for acyclicity and internal consistency.
• ReActBN: Starting from a PromptBN-elicited skeleton, the LLM and a statistical scorer (BIC) interact in a loop. At each iteration, possible graph modifications (add, delete, reverse edges) are enumerated; the LLM is presented with moves and corresponding $\Delta$BIC, and selects the best next action, subject to a tabu list to prevent cycling.

This BNLF approach achieves state-of-the-art structural recovery (e.g., SHD=0 on the Asia network) and matches or exceeds traditional constraint- or score-based algorithms in the low- or no-data regime. Query complexity is greatly reduced relative to baseline LLM-only approaches (Zhang et al., 1 Nov 2025).

5. Hybrid BNLF Architectures in Application Domains

BNLF supports flexible, application-specific architectures. In the financial trading domain, an LLM is used for:

• Contextual analysis: Extract current market and sentiment features.
• Structure construction: Generate a context-specific BN structure (node/edge list via JSON).
• Parameterization: Select historical trades similar to current context to populate CPTs.
• Transparent inference: Compute explicit posterior distributions over decision variables, with user-facing risk metrics (Expected Loss, Value-at-Risk, Conditional VaR).
• Iterative feedback: Systematic post-trade updates to structure and parameters, enabling learning from recent outcomes (exponential decay parameter update).

This design achieves 15.3% annualized return with a Sharpe ratio of 1.08, maximum drawdown −8.2%, and crucially supports explainability with an average of 27 decision factors tracked per trade (Kuang et al., 30 Nov 2025).

6. Interpretability, Computational Efficiency, and Trade-offs

BNLF frameworks offer several desirable characteristics (Amirzadeh et al., 30 Oct 2025, Zhang et al., 1 Nov 2025, Kuang et al., 30 Nov 2025, Nafar et al., 21 May 2025):

• Interpretability: Graphical explanation of dependencies, scenario/sensitivity analysis, and justification for inference.
• Environmental efficiency: No LLM fine-tuning or re-training required; inference is performed with pre-trained LLMs augmented by lightweight BN inference (e.g., on CPU for typical BN sizes).
• Modularity and extensibility: LLMs, domains, and context variables can be added or removed by modifying BN structure and retraining CPDs.
• Computational cost: Inference time scales with the number of input LLMs (in ensemble settings) and the complexity of the BN. The increase over a single model prediction remains modest for most practical settings.

A trade-off exists between the marginal computational cost of querying multiple LLMs and the substantial gains in accuracy, robustness, and transparency. BNLF enables orders-of-magnitude reductions in resource demands compared to fine-tuning or large-scale ensembles.

7. Limitations, Assumptions, and Research Directions

Across empirical studies, several limitations and assumptions have been identified:

• Conditional independence assumptions: Ensemble BNLF models often assume conditional independence of LLM outputs given context or label, reducing parameter count but ignoring potential correlations among LLMs (Amirzadeh et al., 30 Oct 2025).
• Brittleness and hallucinations: Structure induction via LLM can yield spurious or misformatted outputs; strict validation and output parsing are necessary (Zhang et al., 1 Nov 2025).
• Support for variable types: Current implementations predominantly handle discrete variables, with extension to continuous or mixed variables as future work.
• Prompt design sensitivity: PromptBN and EPK require carefully curated prompt templates for robustness across domains and LLM versions.
• Data regime effect: The benefit of LLM-derived priors diminishes in large-data regimes where empirical counts dominate, but is maximal for small $N$.

Future research directions called out in the literature include extension to continuous-variable BNs, integration of conditional independence tests within the LLM loop, and development of multi-agent (committee-based) BNLF systems to further reduce bias and variance (Zhang et al., 1 Nov 2025, Nafar et al., 21 May 2025).

---

The Bayesian Network LLM Fusion paradigm delivers a data- and computation-efficient mechanism for unifying the contextual strengths of LLMs with the structured probabilistic reasoning of Bayesian networks, affording improvements in accuracy, interpretability, and environmental impact across diverse AI workflows (Amirzadeh et al., 30 Oct 2025, Zhang et al., 1 Nov 2025, Nafar et al., 21 May 2025, Kuang et al., 30 Nov 2025).