Lattice Deduction Transformers

Abstract: We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic, abstract-interpretation-based approximation of the set of solution candidates. An $800$K-parameter LDT achieves $100\%$ accuracy on Sudoku-Extreme and Snowflake Sudoku, at a fraction of the training cost of prior small recurrent reasoners, while remaining empirically sound: the model returns a correct answer or abstains. A $1.8$M-parameter variant reaches $99.9\%$ accuracy on Maze-Hard. Frontier LLMs score $0\%$ on all three benchmarks.

• The paper presents a recurrent transformer model that projects latent states through a structured lattice to ensure sound logical deductions using abstract interpretation.
• The methodology achieves 100% accuracy on Sudoku variants and 99.9% on maze problems, outperforming baseline models with significantly lower training costs.
• It integrates lattice encoding, dual conflict detection, and stochastic search to maintain logical consistency and efficiently handle combinatorial reasoning tasks.

Lattice Deduction Transformers: Sound and Efficient Neuro-Symbolic Deduction

Introduction and Motivation

"Lattice Deduction Transformers" (2605.08605) presents a highly principled neural architecture for sound and efficient logical deduction, formulated within the framework of abstract interpretation and realized through recurrent transformer models. Motivated by the observation that chain-of-thought (CoT) LLMs fail on search-hard structured reasoning tasks, the paper proposes the Lattice Deduction Transformer (LDT), which projects transformer latent states through structured lattices between forward passes, explicitly mirroring symbolic deduction in constraint solvers.

The key technical insight is that by operating over abstract domains with well-characterized soundness properties, LDT can guarantee that all reasoning steps maintain logical correctness, either yielding a valid solution or abstaining. This stands in contrast to the non-robust, opaque inferences typical in neural approaches to symbolic reasoning and combinatorial search.

Abstract Interpretation and Lattice Formulation

The theoretical foundation is abstract interpretation. Here, the concrete search space is the powerset lattice $C = 2^S$ over candidate solutions $S$, and the abstract domain $A$ is a join-semilattice—such as the per-variable candidate sets in CSP-style (e.g., Sudoku) problems. A Galois connection links the concrete and abstract domains via abstraction ($\alpha$) and concretization ($\gamma$) operators.

Deduction becomes a process of descending this lattice from the least informative top element $T$ towards maximally informative (singleton) states, with the guarantee that the model's deduction cannot rule out true solutions. The deduction operator in the abstract domain is the most precise sound approximation of concrete deduction, ensuring that no false facts are derived and distinguishing completeness from soundness in the model's updates.

Lattice Deduction Transformer Architecture

The core LDT architecture consists of a recurrent transformer with explicit projection mechanisms to translate between the transformer's latent space and the structured abstract lattice. Between each transformer forward-pass, the latent state is projected into the lattice domain, refined by the abstract deduction step, and re-encoded.

Specific architectural choices include:

• Lattice Encoding: Multi-hot sigmoid representation, with one logit per possible value per cell (e.g., $9 \times 9 \times 9$ for Sudoku). Each cell's candidate set evolves toward singleton assignments through deduction.
• Dual Conflict Representation: An explicit CLS token is used for global conflict detection, complementing the per-cell candidate set encoding.
• Recurrent Structure: The model employs $16$ unrolled transformer steps per "Solve" iteration, each acting as a sound deduction step, with residual injection of the lattice-encoded input signal.
• Search and Branching: When deduction cannot progress, a cell with multiple candidates is randomly chosen and pinned to a value probabilistically, implementing stochastic search over the abstract lattice.

Training Procedure and Supervision

Training is conducted on-policy; the model's own deductive process defines the state distribution. The loss functions are carefully engineered to penalize irrecoverable errors (false eliminations) more heavily than incomplete deduction, balancing the fundamental soundness/completeness tension. In settings where multiple solutions are possible (as in shortest-path maze problems), supervision utilizes the intersection of multiple sampled correct solutions via the abstraction operator, ensuring fidelity to the feasible solution space without confusion over multiple optima.

A key contribution is the "alpha" operator, which computes the abstraction of all surviving solutions consistent with the current partial assignment—a domain-agnostic supervisory signal that generalizes beyond single-groundtruth training, and aligns learning closely with the underlying search geometry.

The model's search is empirically sound: It either yields a provably correct answer or abstains, a property not held by baseline LLMs or prior neural reasoners.

Empirical Results

LDT is rigorously evaluated on three challenging benchmarks:

• Sudoku-Extreme: Hard instances requiring deep search, where LDT with 800K parameters achieves $100\%$ accuracy and empirical soundness using only $4$ minutes of training on a single GPU, surpassing prior recurrent reasoners (TRM, HRM, Sotaku) both in accuracy and training efficiency. State-of-the-art LLMs (Claude Opus, DeepSeek V4-Pro, GPT-5.4) achieve $S$0 accuracy, demonstrating the domain gap between sequence generation models and search-based symbolic deduction.
• Snowflake Sudoku: A challenging variable-topology puzzle (hex-based grid), where LDT generalizes without architectural modification and retains $S$1 accuracy, handling variable-arity constraints and masking irrelevant positions.
• Maze-Hard: Large-scale ($S$2) shortest-path problems with millions of solutions per instance. LDT with 1.8M parameters and multi-solution (K=512) supervision attains $S$3 accuracy and significant search efficiency, outperforming prior specialized architectures.

LDT is shown to require orders-of-magnitude less train-time and inference cost than other neuro-symbolic methods for comparable accuracy, with training budgets as low as $S$4 examples. Crucially, the empirical soundness threshold is reached with sufficient training iterations, after which the model never returns an incorrect output.

Theoretical and Practical Implications

This work demonstrates that the explicit integration of lattice-based reasoning and learning yields neural models that maintain logical consistency throughout deduction, a property indispensable for mission-critical, search-intensive deployments. The iterative, symbolic-deduction-aligned stepwise descent mirrors well-established paradigms in symbolic AI (SAT/SMT solvers, DPLL/CDCL, Stålmarck), and LDT provides a practical neuro-symbolic realization suitable for supervised machine learning objectives.

The architecture further unifies perspectives from masked token diffusion, by casting deduction as a systematic mapping from partial states to more informative ones. By making the lattice operations first-class citizens within the transformer pipeline, the architecture maintains interpretability in the abstract state evolution and decouples model expressivity from parametric scaling—enabling small models to achieve high reasoning fidelity.

Practically, LDT highlights the limitations of LLM-based CoT for hard combinatorial reasoning: their inability to represent sound iterative deduction and manage search space branching is a critical limitation for many real-world tasks involving constraint satisfaction or proof search.

Limitations and Future Directions

While LDT excels on logically well-structured problems with fixed rulesets, tasks requiring meta-induction over arbitrary rule inference (e.g., ARC-AGI) remain challenging. The naïve extension to such environments produces unreliable abstention/conflict detection. Further research might focus on combining LDT's abstract domain methodology with adaptive program synthesis domains or integrating stronger latent variable propagation across deduction steps, e.g., using deeper recursive supervision and explicit cross-step state.

Another compelling direction is to generalize the lattice construction or the abstract domain to admit richer expressivity, for instance, over programs or non-grid-structured domains, broadening the theoretical applicability to more complex forms of generalized reasoning.

Conclusion

Lattice Deduction Transformers provide a sound, neuro-symbolic deduction paradigm, occupying an operational niche between symbolic constraint solvers and generalized neuro-reasoners. The approach defines a rigorous interface between neural computation and logical soundness, achieving high accuracy, sound inference, and sample efficiency on canonical hard reasoning tasks where black-box LLMs and classic neural architectures fail. The work sets a clear agenda for future models—where interpretability, soundness, and symbolic structure interface directly with scalable, learnable architectures.