THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture

Abstract: We present THEIA, a modular neural architecture that learns complete Kleene three-valued logic (K3) end-to-end without any external symbolic solver, and investigate what architectural prior enables compositional generalization under uncertainty. THEIA processes four mathematical domains (arithmetic, order, set membership, propositional logic) through dedicated engines that converge in a final logic module. Trained on a 2M-sample dataset with input space ~3.4 x 10^13, it achieves 12/12 Kleene K3 rule coverage across 5 seeds in 7.93 +/- 1.40 minutes (6.5x faster under matched settings; ~3.6x under Transformer-standard tuning, App. G). A mod-3 sequential composition experiment generalizes from 5-step training to 500-step evaluation at 99.97% +/- 0.02% -- a result requiring a structured backbone: replacing the four-engine backbone with a flat MLP collapses length generalization to chance by 50 steps at both tested capacities (0.80M and parameter-matched 2.75M), while a pre-LN TF8LTuned Transformer baseline (3,582,147 params) trained under the identical protocol reaches 99.24% at 500 steps (Appendix F). Mechanistic probing reveals that modularity induces a delayed verdict: upstream engines encode domain-specific variables without committing to the final truth value (probe accuracy <= 74% uncertainty-only ceiling), with the verdict emerging only at the Logic Engine boundary -- causally confirmed by activation patching (100% flip rate on 986 matched OR pairs, replicated across n=5 seeds; 100.0% aggregate on 4,898 pairs; generalized to AND with 100% flip rate on 4,719 pairs). The Transformer baseline reaches equivalent correctness through a qualitatively different representational trajectory (contraction then expansion), suggesting that modular and monolithic architectures implement distinct compositional strategies.

• The paper demonstrates that a pure-neural architecture can learn complete Kleene three-valued logic, achieving 99.96% accuracy on core rules.
• It introduces a modular design with separate domain engines that encode arithmetic, order, set, and logic operations without symbolic post‐processing.
• The study highlights robust compositional generalization and faster convergence compared to Transformer and flat MLP architectures.

Modular Neural Learning of Kleene Three-Valued Logic: The THEIA Architecture

Introduction and Motivation

The paper "THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture" (2604.11284) presents a systematic investigation into end-to-end neural induction of Kleene's strong three-valued logic (K3) algebra. Addressing a core challenge in neuro-symbolic AI, the work asks whether a fully neural architecture—without any explicit symbolic logic or external solver—can both learn and generalize the compositional principles of K3, including sensitivity to the "Unknown" value and non-trivial short-circuit/absorption rules essential across domains such as database querying (SQL NULL), incomplete medical diagnosis, and legal reasoning under uncertainty.

Related Work and Novelty

Prior work in neuro-symbolic systems (e.g., DeepProbLog, NeurASP, Scallop) invariably delegates three-valued inference to explicit symbolic solvers or hardcoded gates. Previous explorations of multi-valued neural logic either encode logical connectives at the neuron level or rely on external symbolic components, circumventing the challenge of end-to-end neural acquisition of K3 operational semantics from data. THEIA is distinct in its construction: all reasoning and propagation of unknown/uncertain values are learned internally via generic MLP-based modules, with no hand-engineered logical gates or symbolic post-processing.

Architectural Design

The core of THEIA is a modular sequence-parallel architecture, with reasoning partitioned over four discrete domains:

• Arithmetic Engine: MLP stack processes numerical inputs and basic operators.
• Order Engine: Encodes relational queries and ordering predicates.
• Set Engine: Models set membership with dedicated treatment of uncertainty.
• Logic Engine: Aggregates higher-level logical composition (AND, OR, implication, biconditional) from upstream domain representations.

Information flow is tightly channeled: inter-domain communication is enforced through explicit, learned bridge MLPs, and each engine maintains strict parameter separation. Each domain engine output, together with operator embeddings and specialized treatment for the Unknown token, are mapped into a unified feature space. The final output is derived as the cosine-classified nearest among three orthogonal prototypes representing True, False, and Unknown.

Experimental Regimen

A substantial synthetic evaluation suite is constructed to ensure both intra-rule generalization and stress-test compositionality:

• Training on 2 million samples spanning all combinations across a dense input space ($\sim 3.4 \times 10^{13}$ instances) with systematic injection of Unknowns ($P_\text{unk} = 0.15$).
• Per-domain diagnostic tests for coverage of all 12 core K3 absorption/short-circuit rules (complemented by a full 39-rule truth table analysis).
• Architectural ablation—modular backbone vs. flat MLP and Transformer-based architectures.
• Sequential composition experiments: training on short (5-step) reasoning chains and evaluating up to 500-step chains, critical for assessing length generalization and error accumulation in non-trivial state propagation.

All experiments control for optimizer, batch size, and parameter count to isolate the effect of architectural priors on generalization.

Main Results

Coverage and Generalization:

THEIA achieves 99.96% accuracy (mean) across the primary 12 K3 rules with 100% rule-seed coverage, reliably learning both trivial and edge-case algebraic behaviors from end-to-end data. In the key sequential composition task (mod-3 propagation with no absorbing states), THEIA generalizes from 5-step training to 500-step evaluation at 99.97% ± 0.02% accuracy, with the modular design sustaining near-perfect performance even as test chains scale $100\times$ beyond training depth.

Backbone Ablation and Contradictory Evidence:

Replacing the modular backbone with flat MLPs, even with parameter matching (0.8M and 2.75M), results in catastrophic failure of compositional generalization: test performance collapses to chance after 50 steps. Only structured architectures—either explicitly modular (THEIA) or attention-based (Transformer)—sustain length generalization. A parameter-matched 8-layer Transformer, trained under an identical protocol, achieves 99.24% at 500 steps, but with a notable wall-clock inefficiency.

Efficiency and Convergence:

THEIA demonstrates a substantial convergence speed advantage: median wall-clock time to full Kleene rule coverage is $7.9 \pm 1.4$ minutes, compared to $51.5 \pm 11.0$ minutes for the Transformer, corresponding to a $6.5\times$ improvement under matched settings. Hyperparameter tuning narrows but does not eliminate this gap.

Mechanistic Probing and Interpretability:

Intensive probe-based analyses (linear and 2-layer MLP probes at each domain boundary) establish that THEIA enforces "delayed verdict" computation: upstream modules encode only their assigned domain variable, with the final logical verdict only emergent at the logic head. Upstream hidden states do not contain decodable True/False information beyond the uncertainty baseline (max probe accuracy $<74\%$ before the logic module). Activation patching experiments causally confirm the location of verdict commitment—perturbations upstream of the logic engine flip the network's final output on all tested pairs, but not before. This demonstrates a true modular, localist information-processing phenotype.

Comparative Architectural Strategies

Despite reaching equivalent functional correctness, THEIA and Transformer architectures utilize fundamentally distinct representational dynamics. THEIA's modular pathway implements monotonic, late-stage commitment; the Transformer, in contrast, exhibits an initial contraction and later expansion of class-separating information, with verdict-relevant features distributed and mixed earlier in the stack. This dichotomy illustrates divergent paths to compositional generalization: explicit domain decomposition vs. learned self-attending structure.

Implications and Future Directions

The results underscore that architectural prior—specifically modular domain separation or attention-based compositional pathways—is necessary for robust neural learning and propagation of Kleene three-valued logic under uncertainty. Flat, monolithic architectures cannot sustain generalization under discretized, error-resilient chain composition, even with scale or capacity boosts.

THEIA's approach yields practical benefits: faster convergence in the neural theorem-proving regime, more interpretable intermediate representations (delayed and causally localized verdicts), and improved error resilience in long-chain compositional reasoning. This has clear implications for the design of neurosymbolic systems where handling formal Unknowns is critical, such as knowledge bases, formal verification, and question-answering with missing information. The modular design is especially compatible with mechanistic interpretability and introspectable neural execution, offering attractive properties for integrating learned reasoning with future neuro-symbolic hybrids.

Unresolved challenges include extending compositional generalization to truly global computations (beyond local chain or GNN message passing), scaling local state representations (e.g., beyond mod-3), and systematic validation on naturalistic benchmarks. Furthermore, understanding the limits of modularity versus attention, especially as neural operators grow in expressive capacity, remains a pertinent research direction.

Conclusion

THEIA establishes that it is possible for a pure neural architecture to learn complete, not partially hand-encoded, Kleene three-valued logic through end-to-end data-driven optimization. The architectural prior—modular decomposition—fundamentally enables compositional generalization across reasoning steps and unseen depths where monolithic networks fail. THEIA sets a new baseline for neural architectures performing uncertainty-aware logical reasoning, with significant implications for both mechanistic interpretability and the practical engineering of scalable, reliable neuro-symbolic systems.