Logical Boltzmann Machine (LBM) Overview

• Logical Boltzmann Machines (LBMs) are neurosymbolic systems that encode propositional logic in strict disjunctive normal form within a Restricted Boltzmann Machine, ensuring a one-to-one mapping between satisfying assignments and global energy minima.
• They leverage a bipartite architecture and energy-based inference methods, such as Gibbs sampling and free energy minimization, to efficiently find satisfying completions even in high-dimensional search spaces.
• LBMs support both sound reasoning and parameter optimization for learning, achieving state-of-the-art performance on several ILP benchmarks while offering a transparent, rule-based logical framework.

A Logical Boltzmann Machine (LBM) is a neurosymbolic system that encodes arbitrary propositional logic formulae in strict disjunctive normal form (SDNF) within the structure of a Restricted Boltzmann Machine (RBM). By construction, LBMs provide an exact correspondence between logical satisfiability and energy minimization—global minima of the network energy are in one-to-one correspondence with satisfying assignments of the encoded logical theory. With appropriately constructed connections and biases, LBMs enable both sound reasoning (identifying satisfying assignments) and learning from data (optimization of parameters for conditional likelihood), demonstrated to exceed or match state-of-the-art symbolic, neurosymbolic, and neural approaches on standard Inductive Logic Programming (ILP) datasets (Tran et al., 2021).

1. Formal Definition and Architectural Principles

An LBM consists of two principal types of binary units:

• Visible units $x_i \in \{0,1\}$ represent Boolean variables ($\pr{x}_i$).
• Hidden (clause) units $h_j \in \{0,1\}$, with one associated to each conjunctive clause of the logical knowledge base in SDNF.

Connectivity is strictly bipartite: each clause unit directly connects only to the visible units corresponding to variables present (positively or negatively) in its clause. The construction proceeds by:

• Writing the target formula $\varphi$ in strict DNF, i.e., as a disjunction of clauses, with at most one clause being true under any assignment:

$$\varphi \equiv \bigvee_{j=1}^L \left( \bigwedge_{t \in S_{T_j}} x_t \wedge \bigwedge_{k \in S_{K_j}} \neg x_k \right)$$

• Each clause is encoded as a hidden unit $h_j$ connected according to the presence (positive/negative) of variables.

The LBM’s energy function is that of a standard RBM:

$E(\vt{x},\vt{h}) = -\sum_{i,j} w_{ij}x_ih_j - \sum_i a_i x_i - \sum_j b_j h_j$

Weights are set as $w_{ij}=1$ if $x_i$ is positive in clause $j$, $\pr{x}_i$0 if negative, and zero otherwise. Clause hidden biases are $\pr{x}_i$1 for small $\pr{x}_i$2.

The effect is that minimizing the energy with respect to $\pr{x}_i$3 for fixed $\pr{x}_i$4 activates exactly those hidden units for satisfied clauses, and the number of satisfied clauses translates directly into energy.

2. Energy-Logical Equivalence and Theoretical Guarantees

LBMs provide a strictly sound mapping between logical models and energy minima via their definition:

• For SDNF, at most one clause is true, so the Boolean value of the formula under assignment $\pr{x}_i$5 is

$\pr{x}_i$6

• By construction,

$\pr{x}_i$7

so global energy minima align exactly with satisfying assignments of $\pr{x}_i$8. This property guarantees that inference by energy minimization is logically sound.

An immediate consequence is that energy-based probabilistic (e.g., Gibbs) sampling, conditioned on partial assignments, will focus sampling on satisfying completions of the logical constraints.

3. Encoding and Representing Propositional Logic

General propositional formulae are first compiled into strict DNF:

• Strict DNF (SDNF) is required so that disjunctions are pairwise exclusive—i.e., at most one clause true per assignment.
• Conversion merges overlapping DNF clauses to ensure exclusivity by redundancy elimination, with practical linear-time procedures. The size of the resulting SDNF is typically tractable for rule-based knowledge bases.

Illustrative examples:

• XOR equivalence:

$\pr{x}_i$9

yields an LBM with four hidden units.

• Horn rule: $h_j \in \{0,1\}$0 is encoded as four SDNF clauses, thus four hidden units.

LBMs therefore admit lossless, transparent encoding of propositional theories at clause level, supporting compositional and rule-based reasoning.

4. Reasoning and Inference via Energy Minimization

Two principal inference regimes are supported:

• Gibbs Sampling: For a partial assignment $h_j \in \{0,1\}$1, alternate sampling hidden and visible states using sigmoid activation:

$h_j \in \{0,1\}$2

$h_j \in \{0,1\}$3

Sampling locates satisfying completions efficiently due to the logical-energy equivalence.

• Free Energy Minimization: For limited unassigned variable sets, posterior probabilities can be computed explicitly over all such assignments, using the free energy:

$h_j \in \{0,1\}$4

Minimizing $h_j \in \{0,1\}$5 identifies maximally probable/satisfying completions.

Empirical reasoning performance: LBMs discovered all satisfying assignments for complex formulas (up to 30 variables, $h_j \in \{0,1\}$6 assignments) using less than 0.75% of the full search space and runtime below one minute, indicating high efficiency for large-scale logical search (Tran et al., 2021).

5. Learning and Parameter Optimization

LBM parameter learning targets conditional distributions, typically $h_j \in \{0,1\}$7, supporting discriminative tasks. The protocol encompasses:

1. Initialization: Encode background knowledge as fixed hidden units.
2. Expansion: Add new (learnable) hidden units initialized randomly to capture extra patterns not present in the background knowledge.
3. Training: Apply contrastive divergence (CD-1) or exact gradient descent to update weights $h_j \in \{0,1\}$8 and biases $h_j \in \{0,1\}$9 via stochastic gradient descent.

Comparative empirical assessment on seven ILP benchmarks (Mutagenesis, KRK, UW-CSE, and four Alzheimer’s-related tasks) showed LBM achieving top accuracy in five out of seven datasets, outperforming symbolic ILP (Aleph), neurosymbolic ILP (CILP++), and standard RBMs with no background knowledge.

Benchmark	Aleph (sym)	CILP++ (neuro.)	Plain RBM	LBM (ours)
Mutagenesis	0.84	0.85	0.87	0.89
KRK	0.85	0.85	0.94	0.96
UW-CSE	0.92	0.87	0.92	0.95
Alzheimer’s-Amine	0.89	0.88	0.93	0.93
Alzheimer’s-Acetyl	0.83	0.72	0.90	0.92
Alzheimer’s-Memory	0.90	0.82	0.89	0.89
Alzheimer’s-Toxic	0.72	0.77	0.68	0.82

Bold denotes the highest reported accuracy per dataset.

6. Strengths, Limitations, and Prospects

LBMs offer strong guarantees and empirical efficiency:

• Soundness: Energy minima are exactly the logical models of the knowledge base.
• Efficiency: High-dimensional logical search is tractable for large classes of formulas, with substantial state space pruning via energy-guided sampling.
• Flexibility: Weighted clauses and mixed learning objectives are supported, enabling both unsupervised model search and supervised learning.

Limitations include:

• DNF blow-up: SDNF conversion can incur exponential clause growth in the worst case.
• Hidden-unit scaling: One hidden unit per clause imposes burdens for large or highly redundant theories.
• Partition function $\varphi$0: Exact evaluation remains intractable for large RBMs, though not needed for typical training or inference routines.

Future research avenues comprise:

• Extensions to first-order logic using parameter-tying and pattern-based clauses.
• Efficient SDNF compilation methods, including merging and knowledge compilation.
• Integration with classical SAT-pruning for hybrid enumeration-based solvers.
• Expansion to probabilistic-logic and multi-modal deep pipelines, linking perception and symbolic reasoning in unified architectures (Tran et al., 2021).