BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models (2404.12494v3)

Abstract: Predictive models often need to work with incomplete information in real-world tasks. Consequently, they must provide reliable probability or confidence estimation, especially in large-scale decision-making and planning tasks. Current LLMs are insufficient for accurate estimations, but they can generate relevant factors that may affect the probabilities, produce coarse-grained probabilities when the information is more complete, and help determine which factors are relevant to specific downstream contexts. In this paper, we make use of these capabilities of LLMs to provide a significantly more accurate probabilistic estimation. We propose BIRD, a novel probabilistic inference framework that aligns a Bayesian network with LLM abductions and then estimates more accurate probabilities in a deduction step. We show BIRD provides reliable probability estimations that are 30% better than those provided directly by LLM baselines. These estimates further contribute to better and more trustworthy decision making.

• The paper introduces BIRD, a novel Bayesian framework that enhances LLM decision-making through integrated abductive reasoning and deductive modeling.
• It employs a three-stage process involving factor generation, entailment mapping, and probabilistic modeling to achieve superior performance and interpretability.
• Experimental results reveal a 35% improvement over GPT-4 in preference evaluations and a 1.3% average gain in cross-domain performance.

BIRD: A Trustworthy Bayesian Inference Framework for LLMs

The paper introduces BIRD (Bayesian Inference from Abduction and Deduction), a novel Bayesian inference framework designed to enhance the reliability of LLM decision-making by integrating abductive reasoning, LLM entailment, and deductive Bayesian modeling. The framework addresses the limitations of LLMs, which primarily rely on inductive reasoning, often leading to unreliable decisions in real-world scenarios with incomplete information. BIRD aims to provide controllable and interpretable probability estimations for model decisions, thereby improving their trustworthiness.

Core Components of BIRD

Figure 1: Two examples of temporal reasoning and planning. GPT 4 estimates the same probabilities for two different conditions for both examples, while BIRD (ours) successfully distinguishes the difference and can thus help the user make a more proper decision.

BIRD operates through three main stages: abductive factor generation, LLM entailment for context-factor mapping, and deductive Bayesian probabilistic modeling.

1. Abductive Factor Generation: LLMs conceptualize the input query into relevant factors, creating an intermediate symbolic representation. This involves generating sentences that describe situations increasing the likelihood of different outcomes and summarizing these sentences into structured factors with corresponding values.
2. LLM Entailment: LLMs map the given context, including a scenario and additional conditions, to the factors identified in the previous stage. This process uses LLM entailment to determine which factors are implied by the provided information, ensuring consistent mapping to the same factor structure.
3. Deductive Bayesian Probabilistic Modeling: An external, learnable text-based Bayesian model is employed to align LLM decisions and estimate outcome probabilities based on the identified factors. This model uses the law of total probability to differentiate between world modeling and observations, enhancing the reliability of probability estimations.

   Figure 2: An overview of BIRD. Given a scenario, we first conduct abductive factor generation, followed by LLM classification for factor-outcome mapping. The factor values in blue support outcome 1 and those in yellow support outcome 2. We learn an algorithm with LLM coarse classification decisions for the Bayesian modeling to enable better alignment. During inference, given an additional condition, we use LLM entailment for context-factor mapping and then estimate probabilities through the trained Bayesian model using the complete information space.

Mathematical Formulation

The framework's mathematical formulation centers on estimating the probability of an outcome $O_i$ given a context $C$, which includes a scenario $S$ and additional condition $U$. Instead of direct induction ($X \rightarrow Y$), BIRD employs abduction ($X \rightarrow Z$) to conceptualize input queries into intermediate factors, followed by deduction ($X, Z \rightarrow Y$) to fit a Bayesian model.

The predictive probability is derived by marginalizing over the complete information space $\mathcal{F}$:

$$\mathbb{P}(O_i|C) = \sum_{f\in \mathcal{F}} \mathbb{P}(O_i|f)\mathbb{P}(f|C)$$

where $f$ represents a specific instance in the information space, and $\mathbb{P}(O_i|f)$ and $\mathbb{P}(f|C)$ denote the world preferences and observations, respectively.

Experimental Results

The effectiveness of BIRD was evaluated across three datasets focused on reasoning and planning: Com2Sense, a temporal reasoning dataset, and PlaSMa. Experiments were conducted using the Llama-2-70b-instruct model. The framework's performance was assessed intrinsically through the reliability of estimated probabilities and its ability to make hard-label decisions, and extrinsically via its utility in generating training signals and follow-up questions.

BIRD demonstrated superior alignment with human preferences, achieving an F1 score of 65%, surpassing GPT-4 by 35% in preference-based pairwise evaluations. The framework also showed comparable performance to chain-of-thought (CoT) methods in decision-making tasks, while providing enhanced interpretability and controllability. Furthermore, BIRD's probability estimations served as reliable training signals, leading to a 1.3% average performance increase on cross-domain datasets when fine-tuning a T5-large model. The framework also facilitated the generation of more humanly preferable follow-up questions, indicating its potential for interactive agent systems.

Ablation Studies

Ablation studies were conducted to assess the impact of different components within the BIRD framework. The results indicated that the proposed abductive sampling method outperformed direct factor generation, improving accuracy by 4.4% and reducing the unknown answer rate by 16.9%. Additionally, the learnable Bayesian modeling component, particularly when trained with reliable signals from LLMs, contributed positively to uncertainty calibration, enhancing the overall performance.

Implications and Future Directions

BIRD offers a significant advancement in enhancing the trustworthiness and reliability of LLM decision-making. By integrating abductive and deductive reasoning, the framework provides interpretable and controllable probability estimations, addressing the limitations of purely inductive approaches. The results suggest that BIRD can be effectively applied in real-world applications requiring robust decision-making under uncertainty. Future research directions include exploring improved factor generation techniques, expanding the framework to handle more complex scenarios, and integrating BIRD into interactive agent systems for more efficient and controlled decision-making processes.

Conclusion

The BIRD framework presents a valuable approach for improving the reliability and trustworthiness of LLMs in decision-making contexts. By combining abductive factor generation, LLM entailment, and deductive Bayesian modeling, BIRD achieves state-of-the-art performance in aligning with human judgments and generating reliable probability estimations. The framework's modular design and promising experimental results highlight its potential for broader applications in AI and NLP.