Neuro-symbolic Automata (NeSyA)

• NeSyA is a neuro-symbolic architecture that fuses neural perception with deterministic symbolic automata for temporal logic grounding in sequence tasks.
• It employs a differentiable model that computes weighted model counting over symbolic transition guards to process sequential data efficiently.
• Empirical benchmarks on tasks like synthetic driving and grid navigation demonstrate NeSyA’s superior performance in accuracy and scalability compared to fuzzy logic baselines.

Neuro-symbolic Automata (NeSyA) constitute a neuro-symbolic architecture that integrates neural perception with the temporal reasoning capabilities of symbolic automata. The framework is designed for sequential and temporal domains, especially sequence classification and sequence tagging, where temporal logic constraints and symbolic knowledge play a crucial role. NeSyA employs deterministic symbolic finite automata (s-FA) for temporal logic grounding, combined with a neural network for perceptual grounding, jointly forming an end-to-end differentiable probabilistic model for structured sequence learning (Manginas et al., 2024).

1. Symbolic Automaton Definition and Temporal Reasoning

At the core of NeSyA is the deterministic symbolic finite automaton (s-FA). Formally, an s-FA is a tuple

$\mathcal{A} = (P, Q, q_0, \delta, F)$

where:

• $P = \{p_1, \dots, p_{|P|}\}$: finite set of propositional Boolean atoms (the alphabet),
• $Q = \{q_0, q_1, \dots, q_{N-1}\}$: finite set of states,
• $q_0$: designated initial state,
• $F \subseteq Q$: set of accepting (final) states,
• $\delta: Q \times Q \to B(P)$: transition function, mapping a pair of states $(q, q')$ to a propositional formula $\phi_{q \to q'}$ (the transition guard) over $P$.

Determinism is enforced such that for each state $q \in Q$ and any truth assignment $P = \{p_1, \dots, p_{|P|}\}$0, exactly one outgoing transition guard $P = \{p_1, \dots, p_{|P|}\}$1 evaluates to true: $P = \{p_1, \dots, p_{|P|}\}$2 Any finite trace $P = \{p_1, \dots, p_{|P|}\}$3, with $P = \{p_1, \dots, p_{|P|}\}$4, is accepted by $P = \{p_1, \dots, p_{|P|}\}$5 if there is a unique run $P = \{p_1, \dots, p_{|P|}\}$6, $P = \{p_1, \dots, p_{|P|}\}$7, $P = \{p_1, \dots, p_{|P|}\}$8, such that at each step the corresponding guard is satisfied. Temporal properties specified in LTL$P = \{p_1, \dots, p_{|P|}\}$9 can be compiled into such automata, supporting complex temporal specifications for sequences [De Giacomo & Vardi ’13].

2. Hybrid Neural–Symbolic Architecture

NeSyA operationalizes perception-to-symbol grounding through a neural network, termed the grounding network, which serves as the perception module: $Q = \{q_0, q_1, \dots, q_{N-1}\}$0 For each observation $Q = \{q_0, q_1, \dots, q_{N-1}\}$1, the grounding network produces $Q = \{q_0, q_1, \dots, q_{N-1}\}$2, where $Q = \{q_0, q_1, \dots, q_{N-1}\}$3 approximates the posterior probability of atom $Q = \{q_0, q_1, \dots, q_{N-1}\}$4 being true given input $Q = \{q_0, q_1, \dots, q_{N-1}\}$5.

These probabilities induce a probabilistic transition matrix for the automaton: $Q = \{q_0, q_1, \dots, q_{N-1}\}$6 meaning each symbolic transition guard is evaluated in a probabilistic sense, yielding the probability of transitioning between states given perceptual input.

3. Probabilistic Semantics and Inference

NeSyA models the automaton as a time-inhomogeneous Markov chain with transition probabilities determined by neural perceptual grounding. Key components:

• Transition probability: The likelihood of transitioning from state $Q = \{q_0, q_1, \dots, q_{N-1}\}$7 to $Q = \{q_0, q_1, \dots, q_{N-1}\}$8 at time $Q = \{q_0, q_1, \dots, q_{N-1}\}$9 is given by evaluating the guard with the current grounding vector.
• Forward recursion: For $q_0$0 the one-hot vector for $q_0$1, unnormalized filtered state probabilities evolve as

$q_0$2

such that $q_0$3.

• Sequence acceptance probability: The likelihood that a sequence is accepted is

$q_0$4

or equivalently by summing over runs reaching final states.

• Tagging probability: At each time step,

$q_0$5

All operations, including weighted model counting (WMC) over guards and matrix-based filtering, are differentiable in the grounding vector.

4. Differentiability and Learning Procedure

NeSyA is inherently end-to-end differentiable. WMC computations on compiled circuits (d-DNNF or BDDs) are multilinear in their arguments and differentiable with respect to the neural network parameters. The matrix products for state filtering remain within the domain of standard differentiable linear algebra.

• Loss for sequence classification:

$q_0$6

• Loss for sequence tagging:

$q_0$7

Gradients are backpropagated through both the WMC computations and the neural grounding network. The crucial property is that no approximate relaxation is needed; all operations are exactly differentiable.

5. Applications, Empirical Benchmarks, and Scalability

NeSyA targets sequence classification and tagging tasks that involve temporal structure or logical constraints. Two benchmarks illustrate its capabilities:

Synthetic Driving (Sequence Classification):

• Sequences consist of noisy image patches with encoded propositional events.
• NeSyA employs a small convolutional network backbone and is compared to the fuzzy-LTL grounding approach.
• Results indicate NeSyA outperforms the baseline both in accuracy (e.g., achieving $q_0$8 accuracy for $q_0$9 at various sequence lengths) and in training efficiency (e.g., $F \subseteq Q$0 min vs. $F \subseteq Q$1 min for $F \subseteq Q$2, sequence length 20).

Grid Navigation (Sequence Tagging):

• Trajectories are labeled according to temporal specifications such as reaching a key or door.
• NeSyA shows rapid generalization in low-data regimes, with Macro-F1 of $F \subseteq Q$3 with 30 examples compared to $F \subseteq Q$4 for a CNN-LSTM.

Scalability:

• NeSyA exhibits much lower update times per batch compared to DeepStochlog, e.g., 0.08 s vs. 30 s per batch for the most complex formulas, representing a nearly 400× speed advantage.

Seq Len	Method	$F \subseteq Q$5 Acc	Time (min)	$F \subseteq Q$6 Acc	Time	$F \subseteq Q$7 Acc	Time
10	NeSyA	1.00	0.8	0.98±0.03	1.1	1.00	2.3
	Fuzzy	0.91±0.06	10.8	0.70±0.13	22.5	0.78±0.04	29.9
20	NeSyA	1.00	1.2	0.99±0.01	1.7	0.99±0.01	3.0
	Fuzzy	0.77±0.22	21.4	0.69±0.14	43.7	0.70±0.10	57.7
30	NeSyA	1.00	1.7	0.94±0.11	2.3	0.97±0.03	3.8
	Fuzzy	0.98±0.01	31.7	0.55±0.10	55.9	0.50	86.6

Macro-F1 Generalization:

Examples	NeSyA F₁	CNN–LSTM F₁
30	0.98±0.03	0.66±0.07
50	0.99±0.01	0.83±0.08
100	0.95±0.09	0.94±0.04

6. Implementation and Training Details

NeSyA utilizes a convolutional neural network for visual grounding (3×(Conv3×3→ReLU→MaxPool2×2), 64 feature maps, followed by fully connected layers to a sigmoid activation for probability grounding). Optimization is performed using Adam with typical hyperparameters (e.g., learning rate $F \subseteq Q$8, batch sizes 16–32, early stopping based on validation loss). For symbolic transition guards, WMC is computed over compiled circuits (such as d-DNNF or BDD), supporting exact differentiability.

The system is trained under weak supervision at either the sequence level (classification) or per-step (tagging), with targets defined by the automaton acceptance of the perceptually grounded symbolic events.

7. Limitations and Prospects

Current limitations include restriction to propositional symbolic automata, excluding variables and data-parameters or richer first-order logics. Compilation from LTL$F \subseteq Q$9 can entail worst-case exponential blowups, mitigated but not eliminated by symbolic representations such as BDDs. The automaton structure (states, transition guards) must be provided and is not jointly learned with neural parameters. The method is currently limited to finite-horizon logic; infinite-horizon temporal logics (standard LTL with Büchi acceptance) are not supported. Future directions include joint structural learning, first-order automata, on-the-fly automaton compilation, and integration as a temporal logic shield or reward shaping mechanism in reinforcement learning.

NeSyA demonstrates that a deterministic symbolic automaton, linked to neural perception through weighted model counting and matrix-based filtering, enables efficient, scalable, and generalizable neuro-symbolic sequence modeling. Empirical results show superior performance and scalability relative to fuzzy-logic and neuro-probabilistic baselines, especially in settings with logical supervision and limited data (Manginas et al., 2024).