---
title: 'Predicate Networks: Relational Models'
url: https://www.emergentmind.com/topics/predicate-networks
type: topic
---

# Predicate Networks: Relational Models

Predicate networks are a family of machine learning and probabilistic models in which the fundamental units of representation and inference are predicates—structures that capture relational or invariant properties—rather than solely propositional or feature-level variables. This entry surveys three principal classes of predicate networks: neural predicate networks that discover and operate over latent generative structures via distributed assemblies and oscillatory dynamics; probabilistic predicate networks that generalize Bayesian networks by representing knowledge as a static graph of predicates and handle cyclic causal dependencies; and theory-reconstruction-based predicate networks that formulate predicate invention as an autoencoder over relational data. Additionally, a specific neural class—predicate networks for predicate-argument structure analysis—implements pooling and attention over multi-predicate interactions in sequence models.

## 1. Neural Predicate Networks: Architecture and Learning Principles

Neural predicate networks, as exemplified by Martin & Doumas [1810.01127], are structured around the following architectural motifs:

- **Layered Banks:** The network contains three main banks—Focal Bank (FB) for current input/attentional state, Active-Memory Bank (AMB) as a working buffer, and Long-Term-Memory Bank (LTMB) for persistent storage of learned predicates and role representations.
- **Feature Pool:** All representations are distributed over a shared pool of $F$ real-valued feature units. Assemblies in the memory banks project to and receive feedback from this pool.
- **Predicate and Role-Binding Assemblies:** Predicate assemblies encode invariant properties as distributed vectors. Role-binding assemblies ('RBs') bind predicates to arguments through phase-locked or phase-lagged oscillatory activity without conflating their content.
- **Connectivity:** Projections to and from the feature pool are predominantly excitatory and weighted; competition and sparsity are enforced within banks via lateral inhibition and “yoked” inhibitory interneurons, which also mediate synchronous or desynchronous activation for compositional binding.

**Learning Protocol:** Learning is an unsupervised two-stage process:

1. **Intersection Discovery:** Given two input activations $a, b \in \{0,1\}^F$, their intersection $r = a \land b$ captures invariant features.
2. **Hebbian Predicate Learning:** A new LTMB predicate node $P$ is recruited and its weights are updated according to
   $$
   \Delta w_k = \eta \cdot r \odot (1 - w_k)
   $$
   or variations thereof. This consolidates the discovered invariant as a reusable predicate assembly.

**Bank Mapping:** Synaptic mappings between FB and AMB are formed Hebbian-style to allow flexible reactivation of predicates in varying input contexts:
$$
\Delta M_{ij} = \eta_{map} \, x_i^{(\text{FB})} \, y_j^{(\text{AMB})}
$$

This enables the network to represent and retrieve structured knowledge via dynamic assemblies, rather than static feature conjunctions [1810.01127].

## 2. Rhythmic Predicate Binding and Generative Compositionality

Predicate networks achieve compositional generalization through oscillatory binding mechanisms:

- **Phase-Lag-1 Binding:** Predicate and argument assemblies are activated sequentially in distinct sub-phases within an oscillatory cycle, maintaining their separation while enabling relational binding. For predicate $P$ and argument $A$, firing likelihoods are given by
  $$
  u_P(t) = \sin(\omega t + \phi_0), \quad u_A(t) = \sin(\omega t + \phi_0 + \Delta\phi)
  $$
  with $\Delta\phi \approx \pi/2$.
- **Phase-Lag-0 Binding:** To encode an entire proposition as a unit (e.g., $(P, A_1, A_2, ...)$), all constituent assemblies fire synchronously, allowing recursive packaging.
- **Generative Composition:** Compositional inference or action is achieved by co-activating predicate $P$, argument $A$, and their associated role-binding assembly $RB$ in the correct phase relation, formally
  $$
  z = w_P + w_{RB}(P \rightarrow A) + w_A
  $$
  enabling the network to instantiate new propositions from previously learned components [1810.01127].

## 3. Probabilistic Predicate Networks

Probabilistic predicate networks, introduced by Lin [1303.5415], extend Bayesian networks by constructing a single static graph whose nodes are unary predicate templates and whose edges represent type inheritance or causal/structural features, with cycles permitted at the schema level.

- **Formulation:**  
  - Events $E$ provide a domain for unary predicates $C = \{c_1, ... c_N\}$.
  - Features $F$ encode causal or structural functions $f: c_i \rightarrow c_j$.
  - Priors $\pi_i$ and conditionals $\theta_f$ parameterize occurrence and feature instantiation probabilities.
- **Joint Distribution:** Defined through the chain rule over all ground atoms:
  $$
  P(\{c_k(e)\}) = \prod_{e \in E} \prod_{i=1}^N P(c_i(e) \mid \text{Pa}(c_i(e)))
  $$
- **Handling Cycles and Recursion:**  
  - Type-level cycles (e.g., contagious-disease spread) are valid as long as each ground instantiation terminates.
  - The approach supports recursive processes (e.g., plan recognition, list processing) without dynamic network re-assembly.
- **Inference and Learning:**  
  - Abductive explanations are generated by assembling 'scenarios'—grounded causal trees scored by product of local probabilities.
  - Learning parameters involves frequency-based estimation or EM for hidden structure.

**Illustrative Applications:**
- Complex diagnoses, plan recognition, and recursive causal explanation are handled within the same static predicate template graph [1303.5415].

## 4. Predicate Networks in Predicate-Argument Structure Analysis

In PAS analysis, particularly for Japanese, predicate networks denote deep bidirectional RNNs with explicit pooling and attention mechanisms capturing cross-predicate interactions [1806.03869]:

- **Model Structure:**  
  - Each sentence is processed by a stacked bi-RNN; for each predicate-word pair, a vector $h^0_{i,t}$ encodes embeddings and predicate-position indicators.
  - Multi-predicate (MP) input flags further encode all predicate positions, increasing representational power.
- **Interaction Layers:**  
  - **Pooling:** Max-pooling combines hidden states $h^K_{i,t}$ across predicates.
  - **Cross-Predicate Attention:** Learns weighted relations over distant predicate-argument pairs, insensitive to token distances.
  - **Self-Attention:** Captures intra-predicate dependencies, optionally stacked after pooling for enhanced expressivity.
- **Loss and Training:**  
  - Negative log-likelihood over gold argument labels, dropouts, and Adam optimizer.
- **Empirical Results:**  
  - MP+Pool-SelfAtt model achieves a single-model F$_1$ of 83.94 on the NTC 1.5 test set, outperforming pooled or self-attention models alone. Zero (indirect) F$_1$ increases by 1.18, indicating strong gains in long-range dependency resolution.

Pooling and attention-based predicate networks successfully capture “distance-free” dependencies, facilitating argument-sharing and long-range role assignment in language tasks [1806.03869].

## 5. Predicate Networks via Theory Reconstruction and Predicate Invention

Predicate invention can be formalized as a theory reconstruction problem using an autoencoder-like network over relational data [1606.08660]:

- **Framework:**  
  - Input is a set of formulas $T$ (from language $\mathcal{L}$) true in a knowledge base $\mathcal{KB}$.
  - Invented predicates $Q$ (not in $P$) are assigned definitions $\mathcal{F}$, and a new hidden theory $T_Q$ is induced.
  - Encoder $\mathcal{E}$ computes activations in hidden space by applying $\mathcal{F}$ to $T$.
  - Decoder $\mathcal{D}$ reconstructs $T$ from $T_Q$.
  - Objective:
    $$
    \min_{Q, \mathcal{F}} \left\| T \;\triangle\; \mathcal{D} (\mathcal{E}(T)) \right\|_\Delta - q(Q, \mathcal{F}, \mathcal{KB})
    $$
    where $q(\cdot)$ regularizes for sparsity or brevity.
- **Network Parallels:**  
  - Input: one unit per formula in $\mathcal{L}$.
  - Hidden: one unit per instantiation of each hidden predicate $h \in Q$.
  - Connections are unweighted; structure learning seeks minimal definitions capturing the theory's content.
- **Significance:**  
  - Provides a unified formalism for both predicate invention and theory revision as autoencoding problems.
  - Opens avenues for multi-layer (stacked) predicate discovery and integration with neural and probabilistic models [1606.08660].

## 6. Compositional Generalization, Inference, and Empirical Metrics

Predicate network models are characterized by strong compositionality and the ability to extrapolate structure across novel domains:

- **Neural Predicate Networks:** Achieve >90% one-shot transfer accuracy from raw pixels across transformed Atari game environments, without supervised labels, due to abstract predicate representation [1810.01127].
- **Probabilistic Predicate Networks:** Efficiently explain indirect observations via abductive scenario assembly, scoring explanations by product of priors and local conditionals [1303.5415].
- **Evaluation Metrics:**  
  - Compositionality is assessed by test accuracy of composed propositions $(P, A_1, A_2, ...)$.
  - Predicate vocabulary size and convergence rates empirically support a small set of reusable predicates sufficing for generalization:
    $$
    \text{accuracy(test)} \approx 1 - \exp(-\alpha \cdot N_\text{predicates})
    $$
- **PAS Predicate Networks:** Enhanced $F_1$ on indirect-argument (Zero) prediction reflects increased capacity for multi-predicate generalization through pooling/attention [1806.03869].

## 7. Comparative Summary and Outlook

Predicate networks constitute a central paradigm for bridging symbolic reasoning and neural or probabilistic modeling. Their core attributes are summarized in the table below:

| Approach                       | Representation              | Learning/Inference                    |
|---------------------------------|-----------------------------|---------------------------------------|
| Neural predicate networks       | Assemblies in feature pool; oscillatory binding | Hebbian intersection; phase-based composition [1810.01127]     |
| Probabilistic predicate networks| Static schema of unary predicates with causal features | Abductive scenario construction; frequency/EM parameter learning [1303.5415] |
| Theory reconstruction networks  | Binary units for formulas and invented predicates | Autoencoder minimization of theory reconstruction loss [1606.08660] |
| PAS predicate networks          | Bi-RNNs with cross-predicate pooling and attention | End-to-end supervised learning for multi-predicate structure [1806.03869]     |

Predicate networks overcome the structural limits of propositional models, enable relational abstraction and one-shot generalization, and realize soft unification of logic-based representations with numerical optimization. Ongoing extensions include incorporation of continuous phase-coding, differentiable definition learning, and scaling to complex, uncertain domains.

Source: https://www.emergentmind.com/topics/predicate-networks