Relational Neurosymbolic Markov Models

• Relational Neurosymbolic Markov Models are probabilistic sequential models that fuse neural representations with symbolic logic to enforce hard and soft constraints.
• They employ lifted inference, differentiable surrogates, and dynamic programming to achieve scalable, constraint-compliant reasoning in complex structured tasks.
• Empirical results demonstrate that NeSy-MMs provide interpretable proof traces and robust performance in tasks like MiniHack reconstruction and knowledge graph completion.

Relational Neurosymbolic Markov Models (NeSy-MMs) are a class of probabilistic sequential models designed to provide end-to-end differentiable integration of relational logical constraints within deep architectures. By combining neural representation learning with first-order logic reasoning under the Markov property, NeSy-MMs directly address the challenge of scalable, interpretable, and constraint-compliant learning and inference in structured and sequential problems (Smet et al., 2024, Jiao et al., 11 Nov 2025, Marra et al., 2020).

1. Formal Structure and Definitions

NeSy-MMs are built on hybrid state spaces that combine subsymbolic neural variables $N_t$ (e.g., embeddings, feature vectors) and relational symbolic states $S_t$ (truth valuations over ground atoms). The composite state at each time step is $X_t = (N_t, S_t)$. The domain consists of constants $\mathcal{C}$ and relational predicate symbols of varying arity. First-order logic rules $\mathcal{R}$ and constraints $\mathcal{C}$ are defined over this vocabulary.

A core property is the first-order Markov assumption:

$$P(X_{0:T}, Z_{0:T}) = P(X_0)P(Z_0|X_0) \prod_{t=1}^T P(X_t|X_{t-1}) P(Z_t|X_t)$$

where each conditional transition probability $P(X_t|X_{t-1})$ is factored over relational and neural components:

$$P(N_t, S_t | N_{t-1}, S_{t-1}) = \prod_{f \in \mathcal{F}} \psi_f(N_{t-1}, S_{t-1}, N_t, S_t)$$

Relational logical constraints $C \in \mathcal{C}$ are encoded with “constraint potentials” $S_t$0, where $S_t$1 if $S_t$2 is violated, $S_t$3 otherwise. These potentials enforce hard logical requirements directly in the generative process.

A generalized abstract form of a NeSy-MM is $S_t$4, encapsulating:

• $S_t$5: relational logic program
• $S_t$6: embedding function for symbolic goals
• $S_t$7: neural compatibility scorer
• $S_t$8: Markov Decision Process (MDP) components for proof/derivation semantics (Jiao et al., 11 Nov 2025).

2. Integration of Neural and Symbolic Reasoning

NeSy-MMs operate on both sub-symbolic neural information and symbolic logical structure in a unified probabilistic graphical model. Transitions can be purely neural (deep nets), symbolic (conditional probability tables over logical atoms), or hybrid.

Symbolic constraints enter as factors in the transition model—either as hard constraints via indicator potentials or soft, differentiable surrogates (e.g., t-norm fuzzy logic for relaxed inference (Marra et al., 2020)). This design enables the system to guarantee constraint satisfaction during both learning and inference.

In Deep Stochastic Logic Program (DSLP) instantiations, proof search is interpreted as an MDP, with states corresponding to logical goals and actions to clause applications. Neural scoring functions guide clause selection and proof trajectories, which are then optimized using dynamic programming or reinforcement learning algorithms (Jiao et al., 11 Nov 2025).

3. Inference and Learning Algorithms

A principal challenge addressed by NeSy-MMs is scalable inference. Exact inference is intractable in general ($S_t$9P-hard), but NeSy-MMs employ several strategies:

• Piecewise and pseudo-likelihood approximations (for RNM-style undirected models) (Marra et al., 2020)
• Rao-Blackwellised Particle Filtering (RBPF) for differentiable sequential inference, maintaining differentiability without nondifferentiable resampling (Smet et al., 2024)
• Dynamic programming for value function computation in MDP semantics, where tractable state spaces admit exact solutions (Jiao et al., 11 Nov 2025)
• Policy-gradient reinforcement learning for large or intractable goal spaces, using algorithms such as PPO and gradient estimators such as RLOO

Gradient computation leverages both pathwise automatic differentiation (for neural components) and score-function estimators for symbolic variables, with unbiased variance control.

Joint training employs expectation-maximization-style alternation. Neural parameters and logic rule weights are tuned end-to-end, exploiting closed-form updates for certain logic weights and standard gradient ascent/backpropagation for neural modules (Marra et al., 2020).

4. Scalability Mechanisms

Scalability in NeSy-MMs is achieved through several key techniques:

• Lifted inference: Symbolic constraints and state are factorized into clusters, compiled, and cached, allowing tractable evaluation even in domains with large numbers of entities and relations (Smet et al., 2024).
• Derivation-based semantics: By modeling distributions over proof (derivation) steps rather than possible worlds, DSLP-based NeSy-MMs circumvent the exponential cost of model counting (Jiao et al., 11 Nov 2025).
• Avoidance of grounding: Constraints are only grounded as needed, often remaining in lifted form and reused across particles and time.
• Cluster factorization: Independent clusters allow inference to scale linearly with sequence length and number of particles.

Complexity analysis demonstrates that, for modest cluster sizes (e.g., $X_t = (N_t, S_t)$0 logical variables per cluster), inference grows polynomially in domain size and linearly in both number of clusters and time steps, in contrast to exponential growth in propositional approaches (Smet et al., 2024).

Table: Selected Scalability and Performance Metrics

Model	Sequential Scale	Proof/Clause Evaluation	Performance Highlights
DPrL-DP/PG (Jiao et al., 11 Nov 2025)	Up to $X_t = (N_t, S_t)$1 digits	$X_t = (N_t, S_t)$2	MRR=0.986 (Family KG), Hits@10=0.918 (WN18RR)
NeSy-MM (Smet et al., 2024)	$X_t = (N_t, S_t)$3 up to 20, $X_t = (N_t, S_t)$4 up to 15	$X_t = (N_t, S_t)$5	89.6% location accuracy (MiniHack gen.)

5. Empirical Evaluation and Case Studies

NeSy-MMs have been evaluated in generative and discriminative sequential tasks, knowledge graph completion, and classical relational reasoning benchmarks.

• Sequence modeling (MiniHack): NeSy-MM achieves 89.6% reconstruction accuracy for agent location, compared to 30.6% for transformers and 1.5% for Deep-HMM (Smet et al., 2024).
• Discriminative classification: Balanced accuracy and F1 remain robust under out-of-distribution settings, outperforming pure transformer baselines, with minimal degradation as grid size or horizon increases (Smet et al., 2024).
• MNIST addition: DPrL-DP variant scales to 500 digits, facilitating proof-guided training at scales infeasible for prior NeSy systems (Jiao et al., 11 Nov 2025).
• Knowledge graph completion: DPrL-PG matches or exceeds neurosymbolic state-of-the-art, facilitating depth-3 proofs with radically reduced grounding and explicit proof traces (Jiao et al., 11 Nov 2025).
• Citation classification (Citeseer): Relational Neural Machines (RNM) outperform pure neural nets and static rule-weight systems, demonstrating the advantage of joint learning of neural and logic weights (Marra et al., 2020).

6. Interpretability and Test-Time Adaptation

All NeSy-MM instantiations produce interpretable proof traces, with symbolic states serving as explicit explanations for predictions and decisions. The enforcement of hard constraints guarantees logical consistency; constraint violations are not sampled. Constraint potentials $X_t = (N_t, S_t)$6 can be added or removed at test time, facilitating zero-shot adaptation to new out-of-distribution scenarios without retraining. Empirical demonstrations include on-the-fly enforcement of previously unseen logical restrictions in generative trajectory modeling (Smet et al., 2024, Jiao et al., 11 Nov 2025).

A plausible implication is that NeSy-MMs offer strong guarantees of reasoning reliability and invariance under logical constraint modifications, a property not attainable by standard deep learning architectures.

7. Theoretical Context and Comparative Analysis

NeSy-MMs generalize and unify prior approaches in neurosymbolic learning:

• When only symbolic constraints are present, NeSy-MM reduces to Markov Logic Networks.
• When only neural-supervised classification is present, it reduces to standard neural networks.
• For fixed logical rule weights, it subsumes Logic Tensor Networks and Semantic-Based Regularization (Marra et al., 2020).

Exact inference remains $X_t = (N_t, S_t)$7P-hard in general; practical deployment leverages approximations and tractable special cases. Proposition 2.1 (Jiao et al., 11 Nov 2025) demonstrates universal approximation for neural scoring functions over derivations, establishing expressive equivalence with classical stochastic logic programs.

Research in this area is motivated by the need for trustworthy, constraint-satisfying sequential AI systems, especially in domains such as automated planning, knowledge reasoning, and robust generative modeling.

---

Relational Neurosymbolic Markov Models establish a rigorous framework for integrating relational logic and deep sequential learning, supporting scalable and interpretable inference while providing strong guarantees with respect to logical constraints. This synthesis is validated empirically on tasks spanning generative modeling, relational reasoning, and knowledge graph completion, and forms the basis for future developments in constraint-aware AI (Smet et al., 2024, Jiao et al., 11 Nov 2025, Marra et al., 2020).