StableRule: Robust KG Reasoning

• Stable Rule Learning (StableRule) is a framework for knowledge graph reasoning that prioritizes the reproducibility of symbolic rules under varying data distributions.
• It employs a coupled encoder-decoder network with latent feature decorrelation to address covariate shifts and improve rule plausibility estimation.
• Empirical evaluations on multiple benchmarks demonstrate that StableRule enhances out-of-distribution generalization and yields more interpretable rules.

Searching arXiv for the directly relevant StableRule paper and supporting work on stable rule learning, SIRUS, MOSS, and related stability theory. Stable Rule Learning, often abbreviated StableRule and named StaR in its direct knowledge-graph formulation, denotes learning procedures in which the stability of the learned rules is treated as a primary objective rather than a by-product of predictive training. In the most direct contemporary usage, StableRule is the framework introduced for knowledge graph reasoning under agnostic distribution shift, where feature decorrelation is integrated with a rule learning network to improve out-of-distribution generalization of learned logical rules (Liu et al., 7 Jul 2025). More broadly, the term connects to a wider literature on stable and interpretable rule sets, including frequency-based rule extraction from random forests, multi-objective optimization of sparsity–accuracy–stability trade-offs, and stability-based generalization theory (Bénard et al., 2020, Liu et al., 2 Jun 2025, Abou-Moustafa et al., 2019).

1. Definition and problem setting

In the StaR formulation, the underlying task is standard knowledge graph reasoning or completion. A knowledge graph is written as

$G=(E,R,\mathcal{F}_o),$

where $E$ is the entity set, $R$ is the relation set, and $\mathcal{F}_o$ is the set of observed facts used for training. A fact is a triple $(e_h,r,e_t)$, equivalently a query–answer pair with query $q=(e_h,r,?)$ and answer $a=e_t$. The prediction target is

$P(a\mid G,q).$

Rules are chain-like Horn clauses of the form

$$r_{b_1}(x,z_1)\wedge \cdots \wedge r_{b_l}(z_{l-1},y)\rightarrow r_h(x,y),$$

with body $\mathbf{r_b}=[r_{b_1},\dots,r_{b_l}]$ and head $E$0 (Liu et al., 7 Jul 2025).

The distinguishing problem addressed by StableRule is out-of-distribution knowledge graph reasoning. The paper formalizes this as training on observed facts from one graph while evaluating on multiple heterogeneous test environments whose query distributions differ, especially through changes in the distribution of subject entities $E$1 for each relation (Liu et al., 7 Jul 2025). Two motivating shift types are highlighted. Density shift arises when local subgraph density differs between train and test queries. Pathway shift arises when dominant reasoning paths differ across query groups. The paper’s key analytical claim is that query shift induces covariate shift in the rule-body or relation-path distribution $E$2, which then distorts estimation of rule plausibility scores such as $E$3 (Liu et al., 7 Jul 2025).

This emphasis on rule stability under changing environments distinguishes StableRule from earlier KG rule learners that assume I.I.D. training and test conditions. It also aligns StableRule with a broader interpretability argument already made in stable rule-set learning: stability is not merely low-variance prediction, but reproducibility of the selected symbolic structures themselves under perturbation or resampling (Bénard et al., 2020).

2. StableRule (StaR) architecture

StaR is an end-to-end rule learning framework with two coupled components: an encoder–decoder rule learning network and a feature-decorrelation reweighting mechanism (Liu et al., 7 Jul 2025). The encoder maps rule bodies into latent embeddings,

$E$4

and the decoder maps those embeddings to rule heads,

$E$5

The learned head-prediction probability acts as the rule plausibility score $E$6, that is, the model estimate of $E$7 (Liu et al., 7 Jul 2025).

The central stable-learning mechanism is feature decorrelation in latent rule-body space. If $E$8 denotes the embedding of sample $E$9 and $R$0 the $R$1-th latent dimension across a batch, the intended target is independence across dimensions, written in the paper as $R$2 for $R$3. The associated moment factorization condition is

$R$4

Because all moments are impractical, StaR optimizes a truncated moment-matching decorrelation loss $R$5 under learned sample weights $R$6, with truncation order $R$7 (Liu et al., 7 Jul 2025).

The training objective for the rule learner itself is weighted cross-entropy:

$R$8

Optimization alternates between an inner loop that adjusts $R$9 to reduce latent dependence and an outer step that updates $\mathcal{F}_o$0 for weighted rule-head prediction (Liu et al., 7 Jul 2025).

Rule instances are obtained by backtracking-enhanced random walks over the training graph. For each sampled fact, the method starts at source entity $\mathcal{F}_o$1, walks with fixed length $\mathcal{F}_o$2, and chooses the next entity uniformly from the current neighborhood,

$\mathcal{F}_o$3

Whenever there is a direct relation from the starting entity $\mathcal{F}_o$4 to the current entity $\mathcal{F}_o$5, the traversed relation path becomes a positive rule instance with head $\mathcal{F}_o$6; otherwise a negative sample with head “None” is created (Liu et al., 7 Jul 2025). The appendix hyperparameters reported for StaR include $\mathcal{F}_o$7, weight-learning rate $\mathcal{F}_o$8, and maximum rule length $\mathcal{F}_o$9 (Liu et al., 7 Jul 2025).

A formal justification is given for the reweighting stage. The paper proves that if the number of covariates $(e_h,r,e_t)$0 is fixed, the bandwidths $(e_h,r,e_t)$1, and $(e_h,r,e_t)$2, then there exists a nonnegative weighting scheme such that $(e_h,r,e_t)$3 with probability 1. A constructive asymptotic solution is

$(e_h,r,e_t)$4

namely the density ratio between the product of marginals and the joint density (Liu et al., 7 Jul 2025). The paper does not present this as an end-to-end generalization theorem for KG reasoning, but as a principled justification for dependence-reducing reweighting in latent rule space.

3. Empirical behavior in out-of-distribution KG reasoning

StaR is evaluated on seven benchmarks: Family, Kinship, UMLS, WN18RR, FB15K-237, YAGO3-10, and DBLP (Liu et al., 7 Jul 2025). The OOD construction differs by setting. For statistical relation learning on DBLP, the paper uses the Author relation and partitions test queries by country affiliation into US, CN, DE, FR, and JP, producing naturally heterogeneous environments (Liu et al., 7 Jul 2025). For six knowledge-graph-completion datasets, it constructs synthetic test environments by clustering path-describing vectors that quantify path counts between query–answer pairs; clustering is done per relation (Liu et al., 7 Jul 2025). A stronger inductive setting is also considered, with train/validation/test splits that are entity-disjoint and further partitioned into subgroups in the same way (Liu et al., 7 Jul 2025).

Evaluation uses standard filtered link-prediction metrics: Hit@1, Hit@10, and MRR. Results are averaged over five runs, and subgroup results are summarized as mean $(e_h,r,e_t)$5 standard deviation (Liu et al., 7 Jul 2025). Across six shifted KGC datasets, StaR is reported as the best among rule-learning methods on all six. The reported MRR values include 88.88\% on Family, 66.01\% on Kinship, 75.90\% on UMLS, 51.01\% on WN18RR, 41.16\% on FB15K-237, and 40.26\% on YAGO3-10 (Liu et al., 7 Jul 2025). Against NCRL, the paper reports gains such as 51.01 vs 49.02 on WN18RR, 41.16 vs 38.76 on FB15K-237, and 40.26 vs 38.33 on YAGO3-10 (Liu et al., 7 Jul 2025). Even without synthetic shift, StaR improves over rule-based baselines on FB15K-237 and YAGO3-10 (Liu et al., 7 Jul 2025). In inductive link prediction, the reported MRR values are 48.53 on WN18RR, 38.86 on FB15K-237, and 38.18 on YAGO3-10, again ahead of NCRL (Liu et al., 7 Jul 2025).

Ablation analysis isolates the contribution of decorrelation and sampling. On FB15K-237 and YAGO3-10, the full StaR obtains 41.16 / 40.26 MRR, compared with 38.91 / 38.29 for NoDecor, 40.37 / 39.40 for FirstMoment, and 39.86 / 39.12 for NoBack (Liu et al., 7 Jul 2025). The paper summarizes these as an average 5.18\% decrease in MRR when decorrelation is removed and 2.99\% when backtracking is removed (Liu et al., 7 Jul 2025). A correlation visualization shows the latent feature correlation matrix becoming much closer to blank after reweighting, supporting the intended mechanism (Liu et al., 7 Jul 2025).

Interpretability remains explicit because final reasoning is still rule-based. The paper reports learned YAGO3-10 rules such as

$(e_h,r,e_t)$6

$(e_h,r,e_t)$7

$(e_h,r,e_t)$8

and

$(e_h,r,e_t)$9

The paper contrasts these with less semantically coherent rules learned by NCRL and argues that StaR produces more general and semantically valid rules (Liu et al., 7 Jul 2025).

4. Antecedents in stable symbolic rule-set learning

The most direct methodological precursors to StableRule are stability-driven rule extraction methods from tabular ML, especially the SIRUS line and later multi-objective rule-set optimization. SIRUS, introduced as “Stable and Interpretable RUle Set”, is explicitly presented as “a stable rule learning algorithm” for regression and, in an earlier companion paper, for classification (Bénard et al., 2020, Bénard et al., 2019). Its core idea is to use a random forest not as the final predictor but as a generator of candidate rules, then keep only those paths that recur frequently across randomized trees. For regression, path frequency is

$q=(e_h,r,?)$0

and rules are retained when $q=(e_h,r,?)$1 (Bénard et al., 2020). In classification, the selected elementary rules are averaged; in regression, the final predictor uses a nonnegative ridge aggregation of the surviving rules (Bénard et al., 2019, Bénard et al., 2020).

SIRUS is notable because it makes an explicit design choice that later StableRule formulations inherit conceptually: stability should be defined at the level of selected rules, not only predictions. The paper measures rule-list reproducibility with the Dice–Sorensen overlap and proves an asymptotic stability theorem stating that, under its sampling and discretization assumptions and for $q=(e_h,r,?)$2 outside the set of population path frequencies, the rule-set overlap converges to 1 in probability (Bénard et al., 2020). It also argues that quantile discretization of split thresholds is essential; without discretization, exact path recurrence across perturbed samples is effectively unavailable for continuous features (Bénard et al., 2020). The classification paper likewise frames stability as structural reproducibility of short rule lists under data perturbation and reports strong empirical improvements over CART, RuleFit, Node Harvest, and other interpretable baselines (Bénard et al., 2019).

MOSS extends the same agenda by treating stability as one axis of a multi-objective optimization problem. It defines an in-sample proxy for stability from rule selection proportions $q=(e_h,r,?)$3,

$q=(e_h,r,?)$4

and combines this with a ridge-regularized predictive loss $q=(e_h,r,?)$5 under a sparsity cap $q=(e_h,r,?)$6 (Liu et al., 2 Jun 2025). Rather than returning a single model, MOSS computes a Pareto frontier between stability and accuracy. Empirically, it is positioned between SIRUS and highly accurate but unstable baselines such as FIRE or RuleFit. The paper reports that on 24 out of 31 datasets, MOSS-$q=(e_h,r,?)$7-H and SIRUS have empirical stability within one standard error, while MOSS-$q=(e_h,r,?)$8-H has about a 10\% increase in out-of-sample $q=(e_h,r,?)$9; relative to FIRE, it gives about 190\% more empirical stability for only about 2\% lower out-of-sample $a=e_t$0 (Liu et al., 2 Jun 2025).

Taken together, SIRUS and MOSS establish a stable symbolic-rule perspective in which ensemble recurrence, discretized rule spaces, and explicit accuracy–stability trade-offs are central design elements. StaR transfers that perspective into a KG-reasoning setting by stabilizing not split thresholds or rule-set overlaps directly, but latent path representations under agnostic distribution shift (Liu et al., 7 Jul 2025).

5. Stability notions and theoretical context

Stable rule learning also sits within a broader theory of algorithmic stability. A central theoretical reference is the $a=e_t$1-stability framework, which shows that exponential generalization bounds can be derived from a notion of stability that is weaker than uniform stability, distribution-dependent, and more computationally tractable (Abou-Moustafa et al., 2019). For a deterministic learner $a=e_t$2, the paper defines

$a=e_t$3

with $a=e_t$4 independent of $a=e_t$5 (Abou-Moustafa et al., 2019). The same work emphasizes that deleted or leave-one-out estimates are more naturally controlled by stability than resubstitution error, and introduces a separate $a=e_t$6 quantity for the resubstitution case (Abou-Moustafa et al., 2019). For stable rule learning, this supports a deletion-sensitivity view of robustness rather than a purely worst-case, distribution-free one.

A complementary perspective comes from online learning. The paper on online learnability introduces online stability,

$a=e_t$7

and shows that this type of stability is sufficient for no-regret learning for online stable Regularized ERM and related classes such as Follow-the-Regularized-Leader, Mirror Descent, gradient-based methods, and randomized algorithms like Weighted Majority and Hedge (Ross et al., 2011). In binary classification, the paper establishes that online learnability is equivalent to the existence of a potentially randomized uniform-LOO stable RERM (Ross et al., 2011). This suggests that a sequential StableRule system could, in principle, seek no-regret behavior by controlling rule loss sensitivity to the newest example, although that extension is an interpretation rather than a theorem about rule induction itself.

The theoretical landscape therefore distinguishes several meanings of “stable.” In SIRUS, stability is convergence of the selected rule set itself (Bénard et al., 2020). In $a=e_t$8 stability, it is deletion sensitivity of the learner’s loss (Abou-Moustafa et al., 2019). In online stability, it is the effect of adding or removing the latest observation in an adversarial sequence (Ross et al., 2011). StableRule, in the StaR sense, adds an OOD component: stability is tied to reducing spurious dependence in latent rule representations so that rule plausibility scores remain more reliable under heterogeneous test environments (Liu et al., 7 Jul 2025).

6. Adjacent usages, limitations, and open directions

The expression “stable rule learning” is not confined to symbolic rule-set induction. It also appears in adjacent literatures where the “rules” are reward rules, update rules, or first-order logic programs learned from non-symbolic inputs.

Usage	Representative description	Paper
KG StableRule / StaR	Feature decorrelation with a rule learning network for OOD KG reasoning	(Liu et al., 7 Jul 2025)
Stable symbolic rule sets	Frequent-rule extraction or multi-objective stability–accuracy optimization	(Bénard et al., 2020, Liu et al., 2 Jun 2025)
Rule-based RL for LMs	Stable convergence under deterministic format and answer rewards	(Xie et al., 20 Feb 2025, Liu et al., 18 May 2025)
Visual first-order rule learning	Differentiable rule induction from images with clustering-based abstraction	(Gao et al., 9 Apr 2026)
Belief-theoretic stability rule	Strongest stable proposition under Bayesian conditioning	(Mierzewski, 2 Sep 2025)

In rule-based RL for LLMs, “stable” refers to optimization stability rather than symbolic rule reproducibility. Logic-RL uses deterministic format reward and answer reward on synthetic Knights-and-Knaves puzzles and reports that a strong system prompt, strict output-format checking, and a simple fixed-hyperparameter recipe produce stable convergence under modified REINFORCE++ (Xie et al., 20 Feb 2025). CPGD sharpens that point by arguing that deterministic rule-based rewards do not by themselves guarantee stable learning; instead, stable post-training requires a clipped log-policy-ratio objective and an explicit policy-drift constraint (Liu et al., 18 May 2025). These works are adjacent to StableRule in spirit but do not learn human-readable symbolic rule sets.

A different adjacent direction is visual first-order rule induction. The paper on γILP proposes a fully differentiable pipeline from image embeddings and differentiable clustering to first-order rule induction and reports empirical stability across random seeds on Kandinsky tasks, as well as sensitivity to the number of clusters, rule learning rate, and $a=e_t$9 (Gao et al., 9 Apr 2026). This suggests a perceptual extension of stable rule learning in which abstraction through clustering is a source of stability. By contrast, the paper on probabilistically stable revision is not a learning algorithm at all: it studies Leitgeb’s stability rule for categorical belief, where accepted propositions are those entailed by the strongest probabilistically stable set under Bayesian conditioning (Mierzewski, 2 Sep 2025).

Several limitations remain explicit across the literature. StaR reduces dependence only approximately because the practical decorrelation objective truncates moment order at finite $P(a\mid G,q).$0, introduces alternating optimization overhead, and does not exploit environment labels even when they might be available (Liu et al., 7 Jul 2025). SIRUS is tied to shallow, axis-aligned, quantile-discretized rules and provides asymptotic rather than finite-sample stability guarantees (Bénard et al., 2020, Bénard et al., 2019). MOSS optimizes an empirical stability proxy based on selection proportions rather than a population-level stability criterion (Liu et al., 2 Jun 2025). γILP struggles when correct rules require many variables, as in Fizz and Buzz, and its predicate-invention semantics partly depend on external LLM translation (Gao et al., 9 Apr 2026). RLIE shows that natural-language rules can be globally weighted by logistic regression, but it does not measure selection stability across runs or environments (Yang et al., 22 Oct 2025). CPGD, finally, stabilizes rule-based RL for LLMs, but its evaluation is limited to on-policy training and 7B-scale models (Liu et al., 18 May 2025).

The common open direction is clear. Stable rule learning is no longer only about extracting readable rules; it is about making those rules reproducible under perturbation, reliable under shift, and operationally robust when embedded in larger learning systems. The direct StableRule framework for KG reasoning provides one concrete answer by stabilizing latent rule-body representations under agnostic distribution shift (Liu et al., 7 Jul 2025). The surrounding literature suggests that future progress will likely combine structural recurrence, explicit stability objectives, deletion- or environment-based theory, and broader rule languages than the current axis-aligned or fixed-path forms (Bénard et al., 2020, Liu et al., 2 Jun 2025, Abou-Moustafa et al., 2019).