Getting Wiser from Multiple Data: Probabilistic Updating according to Jeffrey and Pearl (2405.12700v1)

Abstract: In probabilistic updating one transforms a prior distribution in the light of given evidence into a posterior distribution, via what is called conditioning, updating, belief revision or inference. This is the essence of learning, as Bayesian updating. It will be illustrated via a physical model involving (adapted) water flows through pipes with different diameters. Bayesian updating makes us wiser, in the sense that the posterior distribution makes the evidence more likely than the prior, since it incorporates the evidence. Things are less clear when one wishes to learn from multiple pieces of evidence / data. It turns out that there are (at least) two forms of updating for this, associated with Jeffrey and Pearl. The difference is not always clearly recognised. This paper provides an introduction and an overview in the setting of discrete probability theory. It starts from an elementary question, involving multiple pieces of evidence, that has been sent to a small group academic specialists. Their answers show considerable differences. This is used as motivation and starting point to introduce the two forms of updating, of Jeffrey and Pearl, for multiple inputs and to elaborate their properties. In the end the account is related to so-called variational free energy (VFE) update in the cognitive theory of predictive processing. It is shown that both Jeffrey and Pearl outperform VFE updating and that VFE updating need not decrease divergence - that is correct errors - as it is supposed to do.

• The paper’s main contribution is its rigorous analysis and formal comparison of Jeffrey and Pearl’s probabilistic updating methods.
• It demonstrates how independent versus conjoint Bayesian updates affect evidence validity and model precision in AI applications.
• The findings offer actionable insights for developing advanced learning algorithms and refining cognitive decision-making theories.

Insights on Probabilistic Updating According to Jeffrey and Pearl

Introduction

The paper "Getting Wiser from Multiple Data: Probabilistic Updating according to Jeffrey and Pearl" by Bart Jacobs explores the intricate process of probabilistic updating from multiple pieces of evidence. This work addresses a critical aspect of learning within the domain of discrete probability theory, specifically comparing and contrasting the updating mechanisms proposed by Jeffrey and Pearl. Given the complexities inherent in the evidential updating process and its significant implications for AI and cognitive science, this paper provides a rigorous formal foundation for understanding these methodologies.

Key Concepts and Methodological Framework

The essence of probabilistic updating lies in transforming a prior distribution within the light of new evidence, producing a posterior distribution. Typically, this process is understood through the lens of Bayesian updating, yet complexities arise when multiple pieces of evidence are introduced. The paper meticulously expounds on two distinct updating paradigms: Jeffrey's and Pearl's methods.

• Jeffrey's Updating: This method deals with evidence by independently updating with each piece of evidence and then aggregating these updates through a convex combination. Mathematically, if we denote the evidence as a multiset $\psi$ containing factors $p_i$, Jeffrey's update is represented as:

$$\omega \underset{J}{|} \psi = \sum_{p_i \in \psi} (\psi)(p_i) \cdot \omega \underset{B}{|} p_i$$

where $\omega \underset{B}{|} p_i$ denotes the Bayesian update with the single factor $p_i$.

• Pearl's Updating: This approach treats multiple pieces of evidence jointly by forming a conjunction of the factors and performing a single Bayesian update. Formally:

$$\omega \underset{P}{|} \psi = \omega \underset{B}{|} \left(\bigwedge_{p_i \in \psi} p_i\right)$$

Here, $\bigwedge$ denotes the pointwise multiplication (conjunction) of the predicates.

Analysis of Validity and Updates

The paper rigorously characterizes the validity of evidence and how updates according to Jeffrey and Pearl affect these validities. The following are some salient theoretical advances:

1. Validity Definitions:
   • Jeffrey Validity: For evidence $\psi$ and distribution $\omega$, Jeffrey's validity acts independently on each factor:

   $$\omega \underset{J}{\models} \psi = \prod_{p_i \in \psi} (\omega \models p_i)^{(\psi)(p_i)}$$

• Pearl Validity: Integrates all factors into a single combined factor for updating:

  $$\omega \underset{P}{\models} \psi = \omega \models \left(\bigwedge_{p_i \in \psi} p_i\right)$$

1. Updating Mechanisms:
   • Jeffrey Update: Increases Jeffrey validity, thus indicating that post-update, the plausible truth of evidence is higher.
   • Pearl Update: A single conjunctive update increases Pearl validity significantly.

Practical Implications and Future Directions

The rigorous mathematical treatment of updating mechanisms presented in the paper serves as a foundational framework applicable in numerous practical scenarios, particularly in fields such as AI and cognitive science. Several implications emerge:

• Medical Diagnosis: The choice between Jeffrey and Pearl updates can influence diagnostic tools and the interpretation of test results, affecting treatment pathways.
• Predictive Processing: Comparisons with variational free energy (VFE) updates reveal that both Jeffrey and Pearl outperform VFE updates, emphasizing the need for deeper insights into the most appropriate methodologies for cognitive models.

Speculations on Future Developments

The results in this paper suggest several avenues for future research and development in AI and cognitive science:

• Enhanced Learning Algorithms: Leveraging the formal foundations provided, one could develop more robust and accurate algorithms for learning from multiple data sources.
• Cognitive Theories: Refining cognitive models to better align with Jeffrey or Pearl updating mechanisms could enhance our understanding of human inference and decision-making processes.

In conclusion, Jacobs' exploration into probabilistic updating offers a thorough mathematical grounding that distinguishes between independent and dependent methods of evidence incorporation. The elucidation of Jeffrey and Pearl's approaches provides crucial insights into improving learning mechanisms in both theoretical and applied contexts. This paper sets the stage for further research into optimizing Bayesian updates and their applications in technology and cognitive science.