---
title: 'StableRule: Robust KG Reasoning'
url: https://www.emergentmind.com/topics/stable-rule-learning-stablerule
type: topic
---

# StableRule: Robust KG Reasoning

Searching arXiv for the directly relevant StableRule paper and supporting work on stable rule learning, SIRUS, MOSS, and related stability theory.
Stable Rule Learning, often abbreviated StableRule and named **StaR** in its direct knowledge-graph formulation, denotes learning procedures in which the stability of the learned rules is treated as a primary objective rather than a by-product of predictive training. In the most direct contemporary usage, StableRule is the framework introduced for **knowledge graph reasoning under agnostic distribution shift**, where feature decorrelation is integrated with a rule learning network to improve out-of-distribution generalization of learned logical rules [2507.05110]. More broadly, the term connects to a wider literature on stable and interpretable rule sets, including frequency-based rule extraction from random forests, multi-objective optimization of sparsity–accuracy–stability trade-offs, and stability-based generalization theory [2004.14841], [2506.08030], [1903.05457].

## 1. Definition and problem setting

In the StaR formulation, the underlying task is standard knowledge graph reasoning or completion. A knowledge graph is written as
$$
G=(E,R,\mathcal{F}_o),
$$
where \(E\) is the entity set, \(R\) is the relation set, and \(\mathcal{F}_o\) is the set of observed facts used for training. A fact is a triple \((e_h,r,e_t)\), equivalently a query–answer pair with query \(q=(e_h,r,?)\) and answer \(a=e_t\). The prediction target is
$$
P(a\mid G,q).
$$
Rules are chain-like Horn clauses of the form
$$
r_{b_1}(x,z_1)\wedge \cdots \wedge r_{b_l}(z_{l-1},y)\rightarrow r_h(x,y),
$$
with body \(\mathbf{r_b}=[r_{b_1},\dots,r_{b_l}]\) and head \(r_h\) [2507.05110].

The distinguishing problem addressed by StableRule is **out-of-distribution knowledge graph reasoning**. The paper formalizes this as training on observed facts from one graph while evaluating on multiple heterogeneous test environments whose query distributions differ, especially through changes in the distribution of subject entities \(P(e_h)\) for each relation [2507.05110]. Two motivating shift types are highlighted. **Density shift** arises when local subgraph density differs between train and test queries. **Pathway shift** arises when dominant reasoning paths differ across query groups. The paper’s key analytical claim is that query shift induces covariate shift in the rule-body or relation-path distribution \(P(\mathbf{r_b})\), which then distorts estimation of rule plausibility scores such as \(P(r_h\mid \mathbf{r_b})\) [2507.05110].

This emphasis on **rule stability under changing environments** distinguishes StableRule from earlier KG rule learners that assume I.I.D. training and test conditions. It also aligns StableRule with a broader interpretability argument already made in stable rule-set learning: stability is not merely low-variance prediction, but reproducibility of the selected symbolic structures themselves under perturbation or resampling [2004.14841].

## 2. StableRule (StaR) architecture

StaR is an end-to-end rule learning framework with two coupled components: an encoder–decoder rule learning network and a feature-decorrelation reweighting mechanism [2507.05110]. The encoder maps rule bodies into latent embeddings,
$$
\phi_e:\mathcal{R}_b\rightarrow \mathbf{Z},
$$
and the decoder maps those embeddings to rule heads,
$$
\phi_d:\mathbf{Z}\rightarrow \mathcal{R}_h.
$$
The learned head-prediction probability acts as the rule plausibility score \(S(\mathbf{r_b},r_h)\), that is, the model estimate of \(P(r_h\mid \mathbf{r_b})\) [2507.05110].

The central stable-learning mechanism is **feature decorrelation in latent rule-body space**. If \(\mathbf{Z}_{n*}\) denotes the embedding of sample \(n\) and \(\mathbf{Z}_{*i}\) the \(i\)-th latent dimension across a batch, the intended target is independence across dimensions, written in the paper as \(\mathbf{Z}_{*i}\perp \mathbf{Z}_{*j}\) for \(i\neq j\). The associated moment factorization condition is
$$
\mathbb{E}[\mathbf{Z}_{*i}^a\mathbf{Z}_{*j}^b]
=
\mathbb{E}[\mathbf{Z}_{*i}^a]\mathbb{E}[\mathbf{Z}_{*j}^b],
\qquad a,b\in\mathbb{N}.
$$
Because all moments are impractical, StaR optimizes a truncated moment-matching decorrelation loss \(L_d\) under learned sample weights \(\mathbf{W}=\{w_n\}_{n=1}^N\), with truncation order \(O_d\) [2507.05110].

The training objective for the rule learner itself is weighted cross-entropy:
$$
\hat{\phi_e},\hat{\phi_d}
=
\underset{\phi_e,\phi_d}{\arg\min}
\sum_{i=1}^{n} w_i\,L(\phi_d(\phi_e(\mathbf{r_b}^i)),r_h^i).
$$
Optimization alternates between an inner loop that adjusts \(\mathbf{W}\) to reduce latent dependence and an outer step that updates \(\phi_e,\phi_d\) for weighted rule-head prediction [2507.05110].

Rule instances are obtained by **backtracking-enhanced random walks** over the training graph. For each sampled fact, the method starts at source entity \(e_h\), walks with fixed length \(l\), and chooses the next entity uniformly from the current neighborhood,
$$
P(e_i\mid e_{i-1})=\frac{1}{|N(e_{i-1})|}.
$$
Whenever there is a direct relation from the starting entity \(e_h\) to the current entity \(e_i\), the traversed relation path becomes a positive rule instance with head \(r_h\); otherwise a negative sample with head “None” is created [2507.05110]. The appendix hyperparameters reported for StaR include \(O_d=2\), weight-learning rate \(\alpha_d=0.01\), and maximum rule length \(l=3\) [2507.05110].

A formal justification is given for the reweighting stage. The paper proves that if the number of covariates \(d\) is fixed, the bandwidths \(h_i\to 0\), and \(Nh_1\cdots h_d\to\infty\), then there exists a nonnegative weighting scheme such that \(L_d\to 0\) with probability 1. A constructive asymptotic solution is
$$
w_n^\star
=
\frac{\prod_{i=1}^{d}\hat f(Z_{ni})}
{\hat f(Z_{n1},\dots,Z_{nd})},
$$
namely the density ratio between the product of marginals and the joint density [2507.05110]. The paper does not present this as an end-to-end generalization theorem for KG reasoning, but as a principled justification for dependence-reducing reweighting in latent rule space.

## 3. Empirical behavior in out-of-distribution KG reasoning

StaR is evaluated on seven benchmarks: **Family, Kinship, UMLS, WN18RR, FB15K-237, YAGO3-10, and DBLP** [2507.05110]. The OOD construction differs by setting. For statistical relation learning on DBLP, the paper uses the **Author** relation and partitions test queries by **country affiliation** into US, CN, DE, FR, and JP, producing naturally heterogeneous environments [2507.05110]. For six knowledge-graph-completion datasets, it constructs synthetic test environments by clustering **path-describing vectors** that quantify path counts between query–answer pairs; clustering is done per relation [2507.05110]. A stronger inductive setting is also considered, with train/validation/test splits that are entity-disjoint and further partitioned into subgroups in the same way [2507.05110].

Evaluation uses standard filtered link-prediction metrics: **Hit@1, Hit@10, and MRR**. Results are averaged over five runs, and subgroup results are summarized as mean \(\pm\) standard deviation [2507.05110]. Across six shifted KGC datasets, StaR is reported as the best among rule-learning methods on all six. The reported MRR values include **88.88\%** on Family, **66.01\%** on Kinship, **75.90\%** on UMLS, **51.01\%** on WN18RR, **41.16\%** on FB15K-237, and **40.26\%** on YAGO3-10 [2507.05110]. Against NCRL, the paper reports gains such as **51.01 vs 49.02** on WN18RR, **41.16 vs 38.76** on FB15K-237, and **40.26 vs 38.33** on YAGO3-10 [2507.05110]. Even without synthetic shift, StaR improves over rule-based baselines on FB15K-237 and YAGO3-10 [2507.05110]. In inductive link prediction, the reported MRR values are **48.53** on WN18RR, **38.86** on FB15K-237, and **38.18** on YAGO3-10, again ahead of NCRL [2507.05110].

Ablation analysis isolates the contribution of decorrelation and sampling. On FB15K-237 and YAGO3-10, the full StaR obtains **41.16 / 40.26** MRR, compared with **38.91 / 38.29** for **NoDecor**, **40.37 / 39.40** for **FirstMoment**, and **39.86 / 39.12** for **NoBack** [2507.05110]. The paper summarizes these as an average **5.18\% decrease in MRR** when decorrelation is removed and **2.99\%** when backtracking is removed [2507.05110]. A correlation visualization shows the latent feature correlation matrix becoming much closer to blank after reweighting, supporting the intended mechanism [2507.05110].

Interpretability remains explicit because final reasoning is still rule-based. The paper reports learned YAGO3-10 rules such as
$$
\text{isLocatedIn}(x,y)\leftarrow \text{hasNeighbor}(x,z)\wedge \text{isLocatedIn}(z,y),
$$
$$
\text{isCitizenOf}(x,y)\leftarrow \text{isMarriedTo}(x,z)\wedge \text{isCitizenOf}(z,y),
$$
$$
\text{livesIn}(x,y)\leftarrow \text{worksAt}(x,z)\wedge \text{isLocatedIn}(z,y),
$$
and
$$
\text{graduatedFrom}(x,y)\leftarrow \text{hasAcademicAdvisor}(x,z)\wedge \text{worksAt}(z,y).
$$
The paper contrasts these with less semantically coherent rules learned by NCRL and argues that StaR produces more general and semantically valid rules [2507.05110].

## 4. Antecedents in stable symbolic rule-set learning

The most direct methodological precursors to StableRule are stability-driven rule extraction methods from tabular ML, especially the SIRUS line and later multi-objective rule-set optimization. SIRUS, introduced as **“Stable and Interpretable RUle Set”**, is explicitly presented as “a stable rule learning algorithm” for regression and, in an earlier companion paper, for classification [2004.14841], [1908.06852]. Its core idea is to use a random forest not as the final predictor but as a generator of candidate rules, then keep only those paths that recur frequently across randomized trees. For regression, path frequency is
$$
\hat p_{M,n}(P)=\frac{1}{M}\sum_{\ell=1}^{M}\mathds{1}_{P\in T(\Theta_\ell,\mathscr D_n)},
$$
and rules are retained when \(\hat p_{M,n}(P)>p_0\) [2004.14841]. In classification, the selected elementary rules are averaged; in regression, the final predictor uses a nonnegative ridge aggregation of the surviving rules [1908.06852], [2004.14841].

SIRUS is notable because it makes an explicit design choice that later StableRule formulations inherit conceptually: **stability should be defined at the level of selected rules, not only predictions**. The paper measures rule-list reproducibility with the Dice–Sorensen overlap and proves an asymptotic stability theorem stating that, under its sampling and discretization assumptions and for \(p_0\) outside the set of population path frequencies, the rule-set overlap converges to 1 in probability [2004.14841]. It also argues that quantile discretization of split thresholds is essential; without discretization, exact path recurrence across perturbed samples is effectively unavailable for continuous features [2004.14841]. The classification paper likewise frames stability as structural reproducibility of short rule lists under data perturbation and reports strong empirical improvements over CART, RuleFit, Node Harvest, and other interpretable baselines [1908.06852].

MOSS extends the same agenda by treating stability as one axis of a **multi-objective optimization problem**. It defines an in-sample proxy for stability from rule selection proportions \(\Pi_i\),
$$
H_1(z)=\sum_{i=1}^{m}\Pi_i z_i,
$$
and combines this with a ridge-regularized predictive loss \(H_2(z)\) under a sparsity cap \(\sum_i z_i\le k\) [2506.08030]. Rather than returning a single model, MOSS computes a Pareto frontier between stability and accuracy. Empirically, it is positioned between SIRUS and highly accurate but unstable baselines such as FIRE or RuleFit. The paper reports that on **24 out of 31** datasets, **MOSS-\(\epsilon\)-H** and SIRUS have empirical stability within one standard error, while MOSS-\(\epsilon\)-H has about a **10\% increase in out-of-sample \(R^2\)**; relative to FIRE, it gives about **190\%** more empirical stability for only about **2\%** lower out-of-sample \(R^2\) [2506.08030].

Taken together, SIRUS and MOSS establish a stable symbolic-rule perspective in which ensemble recurrence, discretized rule spaces, and explicit accuracy–stability trade-offs are central design elements. StaR transfers that perspective into a KG-reasoning setting by stabilizing not split thresholds or rule-set overlaps directly, but latent path representations under agnostic distribution shift [2507.05110].

## 5. Stability notions and theoretical context

Stable rule learning also sits within a broader theory of algorithmic stability. A central theoretical reference is the \(L_q\)-stability framework, which shows that **exponential generalization bounds** can be derived from a notion of stability that is weaker than uniform stability, distribution-dependent, and more computationally tractable [1903.05457]. For a deterministic learner \(A\), the paper defines
$$
\beta_q^2(A,\ell,P,n)
=
\frac1n\sum_{i=1}^n
\left\|
\ell(A(S_n),X)-\ell(A(S_n^{-i}),X)
\right\|_q^2,
$$
with \(X\sim P\) independent of \(S_n\) [1903.05457]. The same work emphasizes that deleted or leave-one-out estimates are more naturally controlled by stability than resubstitution error, and introduces a separate \(\gamma_q\) quantity for the resubstitution case [1903.05457]. For stable rule learning, this supports a deletion-sensitivity view of robustness rather than a purely worst-case, distribution-free one.

A complementary perspective comes from online learning. The paper on online learnability introduces **online stability**,
$$
|f(A(S^{\backslash m}),z_m)-f(A(S),z_m)|
\le
\epsilon_{\mathrm{on\text{-}stable}}(m),
$$
and shows that this type of stability is sufficient for no-regret learning for online stable Regularized ERM and related classes such as Follow-the-Regularized-Leader, Mirror Descent, gradient-based methods, and randomized algorithms like Weighted Majority and Hedge [1108.3154]. In binary classification, the paper establishes that online learnability is equivalent to the existence of a potentially randomized uniform-LOO stable RERM [1108.3154]. This suggests that a sequential StableRule system could, in principle, seek no-regret behavior by controlling rule loss sensitivity to the newest example, although that extension is an interpretation rather than a theorem about rule induction itself.

The theoretical landscape therefore distinguishes several meanings of “stable.” In SIRUS, stability is convergence of the selected rule set itself [2004.14841]. In \(L_q\) stability, it is deletion sensitivity of the learner’s loss [1903.05457]. In online stability, it is the effect of adding or removing the latest observation in an adversarial sequence [1108.3154]. StableRule, in the StaR sense, adds an OOD component: stability is tied to reducing spurious dependence in latent rule representations so that rule plausibility scores remain more reliable under heterogeneous test environments [2507.05110].

## 6. Adjacent usages, limitations, and open directions

The expression “stable rule learning” is not confined to symbolic rule-set induction. It also appears in adjacent literatures where the “rules” are reward rules, update rules, or first-order logic programs learned from non-symbolic inputs.

| Usage | Representative description | Paper |
|---|---|---|
| KG StableRule / StaR | Feature decorrelation with a rule learning network for OOD KG reasoning | [2507.05110] |
| Stable symbolic rule sets | Frequent-rule extraction or multi-objective stability–accuracy optimization | [2004.14841], [2506.08030] |
| Rule-based RL for LMs | Stable convergence under deterministic format and answer rewards | [2502.14768], [2505.12504] |
| Visual first-order rule learning | Differentiable rule induction from images with clustering-based abstraction | [2604.07897] |
| Belief-theoretic stability rule | Strongest stable proposition under Bayesian conditioning | [2509.02495] |

In rule-based RL for language models, “stable” refers to optimization stability rather than symbolic rule reproducibility. Logic-RL uses deterministic **format reward** and **answer reward** on synthetic Knights-and-Knaves puzzles and reports that a strong system prompt, strict output-format checking, and a simple fixed-hyperparameter recipe produce stable convergence under modified REINFORCE++ [2502.14768]. CPGD sharpens that point by arguing that deterministic rule-based rewards do not by themselves guarantee stable learning; instead, stable post-training requires a clipped log-policy-ratio objective and an explicit policy-drift constraint [2505.12504]. These works are adjacent to StableRule in spirit but do not learn human-readable symbolic rule sets.

A different adjacent direction is visual first-order rule induction. The paper on **γILP** proposes a fully differentiable pipeline from image embeddings and differentiable clustering to first-order rule induction and reports empirical stability across random seeds on Kandinsky tasks, as well as sensitivity to the number of clusters, rule learning rate, and \(\lambda\) [2604.07897]. This suggests a perceptual extension of stable rule learning in which abstraction through clustering is a source of stability. By contrast, the paper on **probabilistically stable revision** is not a learning algorithm at all: it studies Leitgeb’s stability rule for categorical belief, where accepted propositions are those entailed by the strongest probabilistically stable set under Bayesian conditioning [2509.02495].

Several limitations remain explicit across the literature. StaR reduces dependence only approximately because the practical decorrelation objective truncates moment order at finite \(O_d\), introduces alternating optimization overhead, and does not exploit environment labels even when they might be available [2507.05110]. SIRUS is tied to shallow, axis-aligned, quantile-discretized rules and provides asymptotic rather than finite-sample stability guarantees [2004.14841], [1908.06852]. MOSS optimizes an empirical stability proxy based on selection proportions rather than a population-level stability criterion [2506.08030]. γILP struggles when correct rules require many variables, as in Fizz and Buzz, and its predicate-invention semantics partly depend on external LLM translation [2604.07897]. RLIE shows that natural-language rules can be globally weighted by logistic regression, but it does not measure selection stability across runs or environments [2510.19698]. CPGD, finally, stabilizes rule-based RL for language models, but its evaluation is limited to on-policy training and 7B-scale models [2505.12504].

The common open direction is clear. Stable rule learning is no longer only about extracting readable rules; it is about making those rules reproducible under perturbation, reliable under shift, and operationally robust when embedded in larger learning systems. The direct StableRule framework for KG reasoning provides one concrete answer by stabilizing latent rule-body representations under agnostic distribution shift [2507.05110]. The surrounding literature suggests that future progress will likely combine structural recurrence, explicit stability objectives, deletion- or environment-based theory, and broader rule languages than the current axis-aligned or fixed-path forms [2004.14841], [2506.08030], [1903.05457].

Source: https://www.emergentmind.com/topics/stable-rule-learning-stablerule