---
title: 'γILP: Visual Logic Rule Learning'
url: https://www.emergentmind.com/topics/ilp
type: topic
---

# γILP: Visual Logic Rule Learning

Searching arXiv for the target paper and a few related methods named in the provided data.
γILP, spelled yILP in the paper, is a framework for learning first-order logic rules directly from images without relying on symbolic labels for objects or relations, while also addressing predicate invention when relations are not explicitly defined [2604.07897]. It is presented as a fully differentiable pipeline from image constant substitution to rule structure induction, designed to bridge visual perception and conceptual rule induction in a unified setting spanning symbolic relational datasets, relational image datasets, and pure image datasets such as Kandinsky patterns [2604.07897]. Its core formulation combines image and relation encoders, differentiable clustering, a latent knowledge base, differentiable ground substitution, and a neural inductive logic programming module that extracts first-order rules from trained parameters [2604.07897].

## 1. Definition and problem formulation

The framework is situated in inductive rule learning over a first-order language $L = (R, F, C, V)$ with predicate symbols $R$, function symbols $F$, constants $C$, and variables $V$ [2604.07897]. A term is a constant, variable, or $f(t_1,\ldots,t_n)$, an atom is $p(t_1,\ldots,t_n)$ with $p \in R$, and a rule is a definite clause of the form $Q_0 \leftarrow Q_1,\ldots,Q_n$ [2604.07897]. A logic program $P$ is a set of rules; $\mathrm{Ground}(P)$ denotes all ground instances of $P$, the Herbrand base is $B_P$, and the immediate consequence operator is
$$
T_P(I) = \{\mathrm{head}(r)\mid r\in \mathrm{ground}(P),\ \mathrm{body}(r)\subseteq I\}.
$$
The learning objective is to induce a logic program $P$ such that $B \cup P$ entails positive examples and not negative ones [2604.07897].

The paper identifies two obstacles for this setting when the input is visual rather than symbolic. The first is symbol grounding without label leakage: visual inputs must be mapped to logical constants and predicates without supervised labels for those symbols. The second is predicate invention: the system must discover new relational concepts that are not explicitly provided in the data [2604.07897]. FO rules are described as more expressive than propositional rules and essential for interpreting relations between objects in images [2604.07897].

The rule space is represented through generic Horn clauses,
$$
\forall \mathbf{x}\; \Big(\bigwedge_i P_i(\mathbf{x}_i)\to Q(\mathbf{x}')\Big),
$$
with example clauses such as
$$
\forall x,y\; (\mathrm{LeftOf}(x,y)\land \mathrm{Red}(x)\to \mathrm{Target}(y)).
$$
For binary targets, the paper also uses a forward-chaining bias of the form
$$
pt(X,Y) \leftarrow p_1(X,V_3),\; p_2(V_3,Y)
$$
to reduce search complexity [2604.07897].

## 2. Architecture and data flow

γILP is defined by a visual-to-logical pipeline in which raw inputs are first embedded and then transformed into latent logical structures [2604.07897]. The encoders map images and, when available, textual relations to embeddings. For image constants, the experiments use ViT or VAE encoders producing $e \in \mathbb{R}^d$; for textual relations, the system uses VAE/BERT-like encoders producing relation embeddings in the same latent space [2604.07897].

A differentiable clustering module, denoted by the generalization function $g$, maps constant embeddings to $K$ latent centroids $C=\{c_1,\ldots,c_K\}$ that serve as variables or abstract constants [2604.07897]. The clustering objective is given as
$$
\mathcal{L}_{\mathrm{cluster}} = \sum_{e\in E}\sum_{i=1}^{K} f(h(e), c_i)\cdot G_{i,f}(h(e), \alpha; C),
$$
with
$$
G_{i,f}(h(e), \alpha; C)=\frac{\exp(-\alpha f(h(e), c_i))}{\sum_{i'}\exp(-\alpha f(h(e), c_{i'}))},
$$
where $h$ is the encoder, $f$ is a distance such as squared Euclidean distance, and $\alpha>0$ controls the soft or hard character of the assignment [2604.07897]. In effect, constants are generalized to centroids in a way that remains differentiable.

The latent knowledge base depends on whether relations are defined. If the relations are provided, the knowledge base is
$$
KB = \{ r \oplus g(e_1)\oplus g(e_2)\mid r(e_1,e_2)\in B\},
$$
where $\oplus$ denotes concatenation [2604.07897]. If relations are undefined, as in pure image settings, the knowledge base becomes
$$
KB = \{g(e)\mid e\in B\},
$$
that is, a set of centroids for the constants present in the instance [2604.07897].

Differentiable substitution then constructs batches of ground substitutions by replacing variables with centroid embeddings and generates inputs $x$ encoding body-atom truth values together with labels $y$ [2604.07897]. The labels represent target atom truth values in relational tasks and instance-level class labels in pure image tasks. The neural rule learner receives these tensors and induces first-order rules over unary and binary predicates, including placeholder predicates when relations are not defined [2604.07897].

## 3. Differentiable semantics and rule induction

The rule learning component is a multi-layer fuzzy-logic network that learns the analogue of $T_P$ and supports program extraction through thresholding [2604.07897]. Truth values are continuous, with $\mu(\cdot)\in [0,1]$ for atoms computed by lookup functions from the latent knowledge base [2604.07897]. The target truth value is predicted by
$$
y = \mathrm{RuleNetwork}(x)= \bigvee\big(f_m(f_{m-1}(\cdots f_1(x)\cdots))\big),
$$
where fuzzy disjunction is implemented as
$$
\bigvee(x_1,\dots,x_n)=1-\prod_i(1-x_i).
$$
Each intermediate layer is
$$
f(x)=\mathrm{ReLU}(Mx-d),
$$
with trainable matrix $M$ and fixed bias $d$ controlling the activation threshold [2604.07897].

A row-softmax normalization is imposed on the trainable matrices:
$$
M_i[j,k]=\frac{\exp(\hat M_i[j,k])}{\sum_{k'}\exp(\hat M_i[j,k'])},\quad \forall j.
$$
This ensures that each row forms a convex combination over inputs, which the paper describes as a soft selection of body atoms [2604.07897]. Conjunction is therefore approximated by a linear mixture followed by thresholding,
$$
\mu_{A\land B}\approx \mathrm{ReLU}(w_A\mu_A+w_B\mu_B-d),
$$
while disjunction is implemented by the product $s$-norm [2604.07897]. Implication and negation are not explicitly parameterized; instead, clause evaluation is modeled by composition of mixtures and thresholds [2604.07897].

The global loss combines predictive fit and clustering quality:
$$
\mathcal{L}=\mathrm{MSE}\big(y,\mathrm{RuleNetwork}(x)\big)+\gamma\cdot \mathcal{L}_{\mathrm{cluster}}.
$$
The parameter $\gamma$, called $y$ in the paper’s ablation and denoted here as $\gamma$, controls the trade-off between rule satisfaction and generalization quality [2604.07897]. Larger $\gamma$ emphasizes cluster assignments and smaller $\gamma$ prioritizes fitting rule semantics. The paper states that fixed values are used in experiments and ablations, although scheduling such as warm-up or annealing is possible [2604.07897].

After training, clause extraction proceeds through the program tensor
$$
M_P = \prod_i M_i.
$$
Each row corresponds to a clause, and entries above a threshold determine which body atoms belong to that clause [2604.07897]. Rule precision and recall are then computed over induced substitutions. The paper gives
$$
\mathrm{Precision}(r)=\frac{\#\theta\ \text{s.t. body}(r)\ \text{and head}(r)\ \text{true}}{\#\theta\ \text{s.t. body}(r)\ \text{true}}
$$
and
$$
\mathrm{Recall}(r)=\frac{\#\ \text{ground-truth head positives covered by }r}{\#\ \text{ground-truth head positives}
}
$$
as the clause-scoring routine [2604.07897].

## 4. Predicate invention and weakly supervised grounding

A central feature of γILP is predicate invention from percepts rather than from predefined symbolic vocabularies [2604.07897]. When relations are undefined, the system attaches placeholder predicates to term lists, unary for $V_i$ and binary for $(V_i,V_j)$, while constraining the number of variables to equal the number of clusters and associating each variable $V_i$ with centroid $c_{v_i}$ [2604.07897]. The semantics of these predicates are then inferred post hoc by analyzing the constants assigned to the corresponding variables and, for binary placeholders, their order [2604.07897].

The paper explicitly distinguishes learning from translation. LLMs are used only to translate inferred semantics into natural language; they are not used for learning the predicates themselves [2604.07897]. This design is presented as a way to avoid label leakage while still allowing human-readable interpretation of the induced program [2604.07897].

For undefined relations, truth lookup is defined by conjunction over variable assignments:
$$
L(a^\theta)=\bigwedge_{V_i\in \mathrm{terms}(a)} L(V_i^\theta).
$$
In other words, all variable-centroid assignments for the atom’s variables must be present in the knowledge base [2604.07897]. This differs from the relational case, where lookup is based on whether a grounded atom belongs to the latent knowledge base.

The paper provides an explicit semantics-induction routine. For binary atoms $(V_i,V_j)$, the procedure retrieves constants associated with the variables and issues the query “What is the relation between the two ordered sets of images?” For unary atoms $(V_i)$, it queries “What is the common property of the set of images?” The resulting descriptions are then used to generalize the constrained logic program by replacing placeholders with induced predicates [2604.07897]. A plausible implication is that interpretability is produced in two stages: a latent structural stage during optimization and a linguistic stage after rule induction.

## 5. Experimental scope and reported results

The experimental evaluation spans three regimes: classical symbolic ILP datasets, relational image datasets, and pure image datasets [2604.07897]. The symbolic tasks include arithmetic problems such as Predecessor, Odd, Even, Lessthan, Fizz, and Buzz; list tasks such as Member and Length; family-tree tasks such as Son, Grandparent, Husband, Uncle, Relatedness, and Father; and graph tasks such as Undirected Edge, Adjacent to Red, Two Children, Graph Coloring, Connectedness, and Cyclic [2604.07897].

On the symbolic relational task Predecessor, γILP learns
$$
\mathrm{pre}(X,Y)\leftarrow \mathrm{succ}(Y,X),
$$
with precision $1$ and recall $1$ according to the paper’s appendix table [2604.07897]. More broadly, the paper states that yILP matches DFORL across most classical ILP tasks, while struggling with Fizz and Buzz because they require $4$–$6$ variables and therefore produce a search-space explosion without templates [2604.07897].

For relational image datasets, the setup replaces symbolic constants with MNIST images while keeping relations defined as text embeddings and hiding image labels from the learner [2604.07897]. The atom embedding is written as
$$
a^\theta = r \oplus V_1^\theta \oplus V_2^\theta \oplus \cdots \oplus V_n^\theta,
$$
with lookup $L(a^\theta)=1$ if the atom is in the knowledge base and $0$ otherwise [2604.07897]. The paper reports that yILP learns complete rules in most tasks and gives examples from temporal MNIST sequences:
$$
\mathrm{target}(X)\leftarrow \mathrm{succ}(X,Y)\land \mathrm{before}_2(X,Y)\land \mathrm{target}(Y),
$$
and
$$
\mathrm{target}(X)\leftarrow \mathrm{before}_8(X,Y)\land \mathrm{before}_{10}(X,Y)\land \mathrm{target}(Y).
$$
It also states that LLMs degrade if relation semantics are obfuscated, whereas yILP still induces rules, which the paper attributes to the absence of label leakage [2604.07897].

For pure image datasets, the paper studies Kandinsky patterns under the tasks two-pair (TP), one-red (OR), and one-triangle (OT) [2604.07897]. In the TP setting, placeholder rules
$$
\mathrm{Positive}\leftarrow p_1(V,R), \quad \mathrm{Positive}\leftarrow p_2(X,Y)
$$
are interpreted via cluster analysis and LLM translation as approximately “same shape, different color,” yielding the generalized rule
$$
\mathrm{Positive}\leftarrow \mathrm{same\_shape\_and\_different\_color}(X,Y).
$$
For OR, the induced rule
$$
\mathrm{Positive}\leftarrow p_1(T), \quad \mathrm{Positive}\leftarrow p_2(U)
$$
is translated and generalized to
$$
\mathrm{Positive}\leftarrow \mathrm{color\_in\_red}(U),
$$
and for OT,
$$
\mathrm{Positive}\leftarrow p(Z)\leadsto \mathrm{Positive}\leftarrow \mathrm{shape\_in\_triangle}(Z)
$$
[2604.07897]. Reported accuracies are yILP-ViT $=1.00$ on OR and OT and $0.75$ on TP, and yILP-VAE $=0.77$ on OR and OT and $0.64$ on TP [2604.07897].

The paper summarizes these results as strong performance on symbolic ILP benchmarks, relational image datasets, and pure image datasets, and specifically notes that RIPPER and C4.5 perform worse and are less interpretable on Kandinsky patterns [2604.07897].

## 6. Optimization, implementation, and computational characteristics

The optimization procedure is end-to-end and uses AdamW [2604.07897]. Gradients propagate from the MSE term at the rule-network output through the differentiable substitution-generated inputs and then into encoders and cluster centroids through the clustering term [2604.07897]. No explicit sparsity or length penalties are imposed; instead, structure sparsity arises from the ReLU thresholds and row-softmax mixtures [2604.07897]. Predicate invention is regularized only implicitly, through variable constraints equating the number of variables and the number of clusters [2604.07897].

The design choices reported for the experiments are specific. Constant encoders are ViT or VAE; textual relations are encoded by VAE; clustering uses differentiable $k$-means with learnable centroids and a hardness parameter $\alpha$; and the rule network is layered linear-ReLU with row-softmax normalization and final fuzzy disjunction [2604.07897]. The paper explicitly states that no GNNs or slot attention are used and that integration proceeds through concatenation and tensorized lookups [2604.07897].

Typical hyperparameters are also provided. The number of clusters $K$ is $8$–$10$ for Kandinsky and $10$ in MNIST-digit experiments; rule-network learning rates are $0.05$–$0.5$; clustering learning rates are $0.1$–$0.5$; $\alpha \approx 20$; and $\gamma$ lies in the range $1$–$4$ in the ablations [2604.07897]. The bias $d$ is fixed per layer, and training continues until convergence or a time limit, with symbolic ILP experiments using, for example, $5$ minutes [2604.07897].

The input dimension for predefined relations is described by
$$
N = |R_b|\cdot P(d,2) + |R_u|\cdot d - 1,\quad P(d,2)=d(d-1),
$$
so the dimensionality grows quadratically in the number of variables $d$ and linearly in the number of predicates [2604.07897]. Batch substitution and tensorized lookup are described as GPU-friendly, and training times on A100 are said to be practical, although they increase with $d$ and knowledge-base size [2604.07897].

The pseudocode in the paper makes the training regime explicit. Differentiable substitution first updates clustering, obtains $g(e)$ for all constant embeddings, and then samples positive and negative substitutions either from known target facts or from image instances [2604.07897]. The training loop constructs $x$ by knowledge-base lookup, sets $y$, computes
$$
\mathcal{L} = \mathrm{MSE}(y,\hat y)+\gamma\cdot \mathcal{L}_{\mathrm{cluster}},
$$
updates encoders, centroids, and rule parameters using AdamW, and finally extracts rules from $M_P$ through thresholding [2604.07897].

## 7. Position in the literature, limitations, and open directions

The paper positions γILP against several strands of ILP and neurosymbolic research [2604.07897]. Classical ILP systems such as FOIL and Aleph are described as symbolic and non-differentiable, whereas yILP is presented as end-to-end and GPU-based through differentiable substitution [2604.07897]. SILP/OILP are characterized as differentiable rule-template methods over symbolic inputs, and yILP is described as extending this style of induction to images without label leakage and with predicate invention from percepts [2604.07897]. NeuralILP, DeepProbLog, and SATNet are said to combine neural perception with symbolic reasoning but typically require symbolic labels for constants or predicates, while yILP operates in latent space with clustering-based grounding [2604.07897]. NeurRL is described as extending differentiable rule induction to raw sequences, whereas yILP focuses on images and relations, adding predicate invention and LLM-based translation [2604.07897].

The paper’s qualitative analysis states that invented predicates often correspond to consistent, human-meaningful concepts, such as same shape and different color [2604.07897]. It also notes that many rules reach precision $1$, although in the two-pair Kandinsky task precision can drop when only a subset of required pairs is captured [2604.07897].

Several failure modes are identified. The approach is sensitive to detection and segmentation errors because poor encodings or unstable clustering degrade grounding [2604.07897]. Spurious correlations may arise when clustering groups objects by nuisance features rather than semantically relevant ones, making $\gamma$ tuning important [2604.07897]. The method can also overfit or underfit on long-chain tasks such as Fizz and Buzz when stronger language bias or templates are absent [2604.07897]. This suggests that, despite the differentiable design, search control remains a significant issue in higher-arity or longer-range rule induction.

Future directions listed in the paper include richer quantifiers and longer rules through stronger language bias or meta-interpretive priors, temporal and spatial rules with explicit relation grounding, multimodal integration of text and images, coupling with LLMs for promptable rule priors and constrained search, and extensions to GNNs or relational transformers for structured perception [2604.07897]. In that sense, γILP is presented not as a replacement for symbolic ILP or perception-heavy architectures, but as a framework for unifying differentiable symbol grounding, rule induction, and post hoc predicate semantics within a single visual-relational learning system [2604.07897].

Source: https://www.emergentmind.com/topics/ilp