Bayesian ILP: A Probabilistic Logic Approach

• Bayesian Inductive Logic Programming is a probabilistic framework that treats logic programs as hypotheses by integrating Bayesian inference with ILP.
• It leverages minimum message length and Bayesian reasoning to optimize rule complexity and parameter estimation in relational models.
• The approach unifies logical reasoning with probabilistic semantics across settings including constraint-based, proof-theoretic, and hybrid models.

Bayesian Inductive Logic Programming is a form of inductive logic programming in which logic programs are treated as hypotheses equipped with priors, examples are assigned a probabilistic likelihood, and learning is cast as posterior maximization or, equivalently in a minimum message length formulation, total message-length minimization (Sharma et al., 8 Aug 2025). In a broader probabilistic-logic sense, it is concerned with learning probabilistic, relational models that combine ILP-style relational structure with Bayesian uncertainty and inference, and it includes target languages in which Bayesian-network structure, conditional probability tables, and logic programming are tightly integrated (Costa et al., 2012).

1. Bayesian formulation of the ILP problem

A direct Bayesian formulation specifies data $D$ as examples $E=(E^+,E^-)$ of a target predicate $p/a$ together with background knowledge $B$, where each example is a ground atom $p(\vec{c})$ labelled true or false (Sharma et al., 8 Aug 2025). Hypotheses $H$ are probabilistic logic programs built from a set of definite clauses $R$ defining $p$, together with two Problog-style probabilistic rules parameterized by $\theta_+$ and $\theta_-$. If $E=(E^+,E^-)$0 entails $E=(E^+,E^-)$1, the “true” classifier rule with parameter $E=(E^+,E^-)$2 applies; if $E=(E^+,E^-)$3 does not entail $E=(E^+,E^-)$4, the “false” classifier rule with parameter $E=(E^+,E^-)$5 applies. In this setting, the hypothesis is essentially $E=(E^+,E^-)$6, and the learning objective is

$E=(E^+,E^-)$7

The hypothesis space is restricted by a syntactic bias: a maximum number of literals per rule, a maximum number of variables per rule, a maximum number of clauses per program, and non-recursive definite clauses (Sharma et al., 8 Aug 2025). Complexity is measured through code length rather than by a purely syntactic size penalty: the program must encode how many literals are used, how they are partitioned into rules, which predicate symbols appear in rule bodies, how variables are reused, how the continuous parameters $E=(E^+,E^-)$8 and $E=(E^+,E^-)$9 are transmitted, and how example-group sizes are represented.

This formulation makes explicit what is only implicit in many deterministic ILP systems. Classical systems such as FOIL, Progol, Aleph, and MaxSynth typically optimize heuristic or combinatorial objectives, whereas the Bayesian formulation treats a logic program as a probabilistic object with a prior and evaluates it by a likelihood defined over logical entailments and label noise (Sharma et al., 8 Aug 2025).

2. Priors, likelihoods, and the MML criterion

Minimum Message Length formalizes Bayesian inference as coding: the first part of the message encodes the hypothesis $p/a$0, and the second part encodes the examples $p/a$1 given $p/a$2 (Sharma et al., 8 Aug 2025). With

$p/a$3

the total message length is

$p/a$4

Minimizing $p/a$5 is equivalent to maximizing $p/a$6. In this sense, the MML cost is a computable surrogate for $p/a$7.

The prior over logic programs is structurally decomposed. A key component is a generality prior over predicates constructed from the Herbrand Base $p/a$8: for a predicate symbol $p/a$9, if $B$0 is the number of atoms in $B$1 containing $B$2, then

$B$3

For a rule $B$4, the probability of its body predicate multiset is multinomial, so predicates that occur often in the Herbrand Base receive shorter code lengths (Sharma et al., 8 Aug 2025). This yields an explicit probabilistic version of the ILP bias toward simple, general rules. Additional code-length terms encode the partition of literals into clauses, variable-frequency assignments, variable sequences, the continuous parameters $B$5 and $B$6, and the size of the predicted-positive subset.

The parameters $B$7 and $B$8 are given beta priors $B$9, and their message lengths use the standard MML approximation for Bernoulli parameters (Sharma et al., 8 Aug 2025). The resulting finite-precision coding acts as automatic regularization, because extreme parameter values incur extra cost unless they are strongly supported by data.

The likelihood is also structured. One part encodes which ground literals are observed among those entailed by the “true” and “false” sub-hypotheses; another part encodes their truth values through Bernoulli factors. The literal term penalizes hypotheses that entail many possible examples but for which only a small subset is observed. This is the mechanism that allows the Bayesian ILP formulation to discriminate against trivial “always positive” programs in positive-only learning (Sharma et al., 8 Aug 2025).

The contrast with the MDL-style cost used in MaxSynth is explicit. The C-MDL score is

$p(\vec{c})$0

whereas the MML objective specifies priors, noise models, and parameter coding directly (Sharma et al., 8 Aug 2025). On this account, Bayesian ILP is not merely “logic plus a penalty term”; it is a full probabilistic model of structure, parameters, and examples.

3. Target languages and probabilistic semantics

Several probabilistic logic languages serve as targets or substrates for Bayesian ILP. In CLP(BN), Bayesian networks are expressed within constraint logic programming by attaching constraints of the form {V = Sk with p(Domain, Table, Parents)} to clauses, where Sk is a Skolem term, Domain is a finite discrete domain, Table is a conditional probability table, and Parents is a list of parent Skolem terms (Costa et al., 2012). Each ground Skolem term is treated as a random variable, and the program defines a joint distribution over all ground Skolem terms. Operationally, proof search accumulates Bayesian-network fragments; unification merges corresponding BN nodes; and a successful proof yields both a logical answer substitution and an attached BN whose probabilities agree with the global BN semantics.

CLP(BN) is directly positioned as a target language for Bayesian ILP because it combines first-order logical representation, Bayesian-network semantics, and relational modeling in a single formalism (Costa et al., 2012). It subsumes probabilistic relational models, differs from Bayesian Logic Programs in that probabilities are defined over ground Skolem terms, and can represent recursion, function symbols, and aggregation through ordinary Prolog mechanisms such as setof and findall.

A different proof-theoretic route is probLO, which extends Linear Objects in multiplicative-additive linear logic with probabilistic annotations on multi-head methods (Acclavio et al., 19 Jan 2026). In probLO, Bayesian networks are encoded by tables of weighted methods; proof states carry accumulated probabilities; expansion multiplies weights; and additive branching sums alternatives. The paper emphasizes that internal numerical probability computation is achieved through slicing rather than by relying on an external semantic interpretation. Although this work focuses on representation and inference rather than learning, it supplies a logic-programming substrate on which Bayesian ILP could be built (Acclavio et al., 19 Jan 2026).

Hybrid probabilistic logic programming extends this picture to discrete and continuous random variables. A recent thesis introduces context-specific likelihood weighting, the hybrid language DC#, the scalable first-order inference algorithm FO-CS-LW, and DiceML for learning the structure and parameters of hybrid probabilistic logic programs and for relational autocompletion (Kumar, 2023). This places Bayesian ILP within a hybrid setting in which Bayesian semantics, relational structure, and induction are all present simultaneously.

4. Learning procedures and optimization strategies

The 2025 MML-based Bayesian ILP work presents two learners (Sharma et al., 8 Aug 2025). RandomLearner generates a large pool of random rules under the syntactic bias, samples random programs from that pool, computes $p(\vec{c})$1 on the training set, evaluates exact MML cost $p(\vec{c})$2 and the C-MDL baseline, and selects the best programs. Its purpose is to compare scoring criteria without conflating them with search effects.

ApproxLearner is structurally similar to MaxSynth but uses cost-function independent constraints for rule generation and CP-SAT as the constraint solver (Sharma et al., 8 Aug 2025). Because MML is non-linear, the additive syntax cost $p(\vec{c})$3 is kept linear and the remaining terms are approximated by a piecewise linear objective in aggregated statistics such as $p(\vec{c})$4 and the number of predicted positives. This yields a practical, though not globally optimal, optimization route for Bayesian ILP.

CLP(BN) adopts a different learning strategy. Because CLP(BN) programs are logic programs, existing ILP systems can be adapted to learn them with only simple modifications (Costa et al., 2012). The paper demonstrates this with ALEPH: clauses are learned independently, scored by the Bayesian Information Criterion, and then post-processed to remove cycles introduced by independent clause induction. Here, induction of clauses corresponds to Bayesian-network structure generation, while parameter fitting corresponds to CPT learning.

These procedures indicate that Bayesian ILP is not tied to a single optimizer. Exact random search, constraint-based approximation, ILP clause search with BIC, and proof-theoretic probabilistic inference all appear as compatible components, provided the hypothesis language and scoring semantics are explicit.

5. Empirical behavior and comparative findings

The MML-based study evaluates four families of datasets—Alzheimer, Zendo, 1D-ARC, and Trains—for a total of 30 tasks, using 10 repetitions, balanced test accuracy, and Wilcoxon signed-rank tests with Benjamini–Hochberg correction at $p(\vec{c})$5 (Sharma et al., 8 Aug 2025). On the 30 standard tasks, there is no significant difference between MML and C-MDL. Under varying class proportions for fixed training sizes 20 and 50, MML has higher mean balanced accuracy in 11 of 12 tested conditions, with significance at $p(\vec{c})$6. On data-efficiency experiments with training sizes $p(\vec{c})$7, MML significantly outperforms C-MDL at all sizes; improvements exceed 4% mean accuracy on 5 of 8 sizes, and at size 1 the reported improvement is $p(\vec{c})$8. With single training example, Herbrand-based generality priors improve accuracy by $p(\vec{c})$9, while no significant difference is observed at larger training sizes. On 10–50% injected noise and the real noisy Alzheimer datasets, MML and C-MDL show no significant difference in accuracy. In ApproxLearner, the piecewise-linear MML approximation outperforms C-MDL by $H$0 on average on 100% positive datasets and by $H$1 on 100% negative datasets, with no significant difference on standard noiseless tasks (Sharma et al., 8 Aug 2025).

CLP(BN) provides a different kind of empirical evidence. On a synthetic school database derived from a hand-crafted probabilistic relational model, ALEPH-based learning recovers the original structure almost perfectly with enough data, up to directionality of some arcs (Costa et al., 2012). On a metabolic-activity classification task with 4,346 entries on 862 genes and many missing values, the learned CLP(BN) model yields an ROC curve slightly, though not significantly, better than the single-point performance of the ordinary logic program learned by ALEPH (Costa et al., 2012). These results are feasibility results rather than a direct Bayesian-versus-non-Bayesian comparison, but they establish that probabilistic relational models can in fact be induced in a first-order logic-programming setting.

6. Related directions, limitations, and open questions

Bayesian ILP sits next to several neighboring lines rather than exhausting the landscape of probabilistic or coding-based ILP. “Neuro-Logic Lifelong Learning” is not explicitly probabilistic, but it formalizes lifelong ILP through a shared knowledge base $H$2 of intermediate predicates and task-specific modules, and its own exposition gives a hierarchical Bayesian reinterpretation in terms of shared parameters $H$3, task-specific parameters $H$4, and sequential posteriors over tasks (He et al., 16 Nov 2025). This suggests a route from symbolic Bayesian ILP toward continual and multi-task neuro-symbolic learning.

Streaming and revision are also active adjacent themes. “Incremental Learning of Event Definitions with Inductive Logic Programming” is explicitly not a Bayesian paper, yet it introduces a support set described as behaving almost exactly like sufficient statistics or counts for clauses in a probabilistic model, and its single-pass revision mechanism is presented as a deterministic analogue of online Bayesian updating over a discrete hypothesis space of Event Calculus theories (Katzouris et al., 2014). A plausible implication is that scalable online Bayesian ILP may require similarly compressed summaries of clause behavior.

Coding-based ensemble methods constitute another neighboring direction. “Symbolic Snapshot Ensembles” observes that most ILP algorithms learn a single hypothesis from a single training run, then trains an ILP algorithm only once, saves intermediate hypotheses, and combines them using a minimum description length weighting scheme; on benchmarks including game playing and visual reasoning, the reported gain is 4% predictive accuracy with less than 1% computational overhead (Liu et al., 28 Oct 2025). This is not itself a Bayesian formulation, but it underscores the importance of multi-hypothesis reasoning in ILP.

The current Bayesian ILP literature also identifies concrete limitations. The MML-based approach assumes finite domains in order to construct the Herbrand Base; for infinite domains, truncated Herbrand bases based only on observed ground terms are suggested (Sharma et al., 8 Aug 2025). ApproxLearner does not guarantee globally optimal MML programs, and MML-specific pruning constraints analogous to those of MaxSynth remain open. The beta hyperparameters $H$5 are chosen heuristically, and performance in high-noise settings is sensitive to these choices. Exact MML scoring is computationally expensive, while the approximate search remains heavy; future directions explicitly mentioned include MCMC over programs, variational approximations, and guided search using learnable surrogates (Sharma et al., 8 Aug 2025). In the hybrid PLP setting, the introduction of DC#, FO-CS-LW, and DiceML indicates that support for discrete and continuous random variables, scalable inference, and structure learning with missing values remain central research problems for Bayesian ILP (Kumar, 2023).

Bayesian Inductive Logic Programming, in this sense, is best understood not as a single algorithm but as a family of approaches defined by a common commitment: logic programs are hypotheses with structured priors, examples are evaluated by probabilistic likelihoods that respect relational and logical semantics, and learning is a form of Bayesian model selection or approximation carried out in a first-order hypothesis space (Sharma et al., 8 Aug 2025).