Lattice Deduction Transformer (LDT)

• Lattice Deduction Transformer (LDT) is a recurrent transformer that integrates explicit lattice projection and abstract interpretation for sound logical deduction.
• It uses a looped transformer with iterative lattice updates and on-policy training that mirrors search-based constraint solving to ensure monotonic candidate elimination.
• Empirical benchmarks on Sudoku and Maze problems show LDT achieves 100% accuracy and soundness faster than previous models, even at smaller parameter counts.

The Lattice Deduction Transformer (LDT) is a recurrent transformer architecture designed to perform logically sound deduction by explicitly projecting its latent state through a symbolic lattice between recurrent passes. LDT couples a purpose-built, looped transformer with explicit lattice projection, on-policy training that mirrors constraint-solver search, and domain-agnostic supervision based on abstract interpretation. This approach enables LDT—at small parameter counts and with fast training—to empirically guarantee soundness: it either produces a correct answer or abstains, never returning an invalid solution, across hard logical benchmarks where both prior recurrent reasoners and LLMs fail (Davis et al., 9 May 2026).

1. Architecture: Recurrent Transformer Backbone and Lattice Interface

LDT is structured around a looped, small-scale transformer reminiscent of Sotaku and Universal Transformers. Each forward pass in LDT comprises 16 internal iterations of a 4-layer, 4-head transformer with embedding dimension 128 (for Sudoku/Snowflake) or 192 (for Maze), utilizing a small feed-forward multiplier (×4) and dropout. The initial input to each iteration $\ell$ is the current abstract state $x_\ell$, encoded as per-cell multi-hot raw logits for all candidate symbols, supplemented by a dedicated "conflict" [CLS] logit.

The positional information is provided by learned 2D embeddings (and 2D rotary embeddings in the case of large mazes). The model maintains stability in the latent through the reinjection of the input lattice encoding as a residual at each iteration. After each transformer's update, the system outputs updated per-cell candidate logits $b_\ell$ and a conflict logit $c_\ell$. Training supervision is applied to all 16 outputs, but only the final iteration’s results are needed in inference.

2. Lattice Projection, Abstract Domains, and Logical Deduction

The core innovation lies in maintaining the model's hidden state as a pointwise powerset lattice $A = \{1\ldots k\} \to \mathcal{P}(V)$, where $k$ is the number of positions (cells or grid points) and $V$ is the vocabulary of possible symbols (digits, path-markers, etc.). This abstract state $x$ is a $k \times |V|$ tensor of "alive-candidate" sigmoid scores, plus an additional sigmoid for the [CLS] conflict bit.

Between transformer passes, the model projects its continuous logits back to the lattice using threshold elimination:

$$x'_i(v) \gets x_i(v) \quad \text{if} \; \sigma(b_i(v)) \geq \theta_{\text{elim}}, \; \text{else} \; 0,$$

thereby implementing the meet operation ($x_\ell$0): candidate sets per cell can only shrink, never expand, ensuring monotonic descent through the lattice. The only operations permitted are meet ($x_\ell$1) as cellwise intersection and join ($x_\ell$2) as cellwise union, guaranteeing that, under elimination, no valid solution candidates can be prematurely discarded.

3. On-Policy Training: Mirroring Search-Based Constraint Solving

Training is performed on-policy through a "Solve" loop that emulates deduction in a search-based constraint solver. Rather than static trajectory supervision, LDT alternates deduction and stochastic branching:

• Initialization with the "top" lattice element ($x_\ell$3, all candidates alive).
• At each step, the model thresholds out low-confidence candidates via $x_\ell$4, checks for conflict or complete solution, and, if unresolved, randomly selects one of the remaining multi-alive cells, samples an alive candidate (with softmax at temperature $x_\ell$5), and "pins" it in $x_\ell$6. Conflicts (empty candidate cell or high [CLS] logit) cause the chain to backtrack.
• Each step computes a loss $x_\ell$7 based on the overlap of $x_\ell$8 with the ground-truth solution set $x_\ell$9.

This mechanism ensures training encounters the actual partially-deduced lattice states seen at inference.

4. Supervision via Abstract Interpretation

Supervision leverages an abstract interpretation scheme: at each step, LDT is trained against an abstraction $b_\ell$0 of all ground-truth solutions consistent with $b_\ell$1. With $b_\ell$2 being a pool of sample valid solutions, abstraction yields, per position $b_\ell$3:

$b_\ell$4

The training target becomes $b_\ell$5, thus LDT learns to eliminate only candidates not present in any valid solution. The per-iteration loss function combines three terms, averaged over $b_\ell$6:

• $b_\ell$7: weighted binary cross-entropy on $b_\ell$8 vs. $b_\ell$9, with false eliminations heavily penalized ($c_\ell$0),
• $c_\ell$1: symmetric BCE on the conflict bit against $c_\ell$2,
• $c_\ell$3: cross-entropy at cells with singleton $c_\ell$4, indicating the true value to keep.

The overall loss is:

$c_\ell$5

with $c_\ell$6, $c_\ell$7. This domain-agnostic abstraction supports both unique-solution (Sudoku, $c_\ell$8) and multi-solution (Maze, $c_\ell$9) problems.

5. Soundness Guarantees and Theoretical Underpinnings

LDT’s deduction process is empirically and formally sound. Since each operation only eliminates candidates guaranteed not to occur in any surviving solution (by construction of $A = \{1\ldots k\} \to \mathcal{P}(V)$0 and thresholding), the deduction head cannot exclude viable values. The conflict [CLS] mechanism is trained to detect unsatisfiable states (i.e., those failing to match any $A = \{1\ldots k\} \to \mathcal{P}(V)$1). During inference, elevating the conflict threshold ($A = \{1\ldots k\} \to \mathcal{P}(V)$2) reduces false-positive conflicts while still identifying genuine unsatisfiability. Consequently, LDT, after sufficient training and proper threshold tuning, either produces a correct solution or abstains (timeout or conflict), with no observed errors in benchmark experiments.

Formally, if $A = \{1\ldots k\} \to \mathcal{P}(V)$3 never under-approximates the set of viable candidates, any fully pinned, conflict-free state aligns with some $A = \{1\ldots k\} \to \mathcal{P}(V)$4, where $A = \{1\ldots k\} \to \mathcal{P}(V)$5 represents the full solution set. This ensures all returned answers are valid.

6. Benchmark Performance and Comparative Results

Extensive evaluation on three logical deduction benchmarks demonstrates LDT’s empirical efficiency and soundness:

Benchmark	LDT (800K/1.8M params)	Sotaku (800K)	TRM (5M)	HRM (27M)	Frontier LLMs (Claude, GPT-5.4, DeepSeek V4-Pro)
Sudoku-Extreme	100%	98.9%	87.4%	55%	0%
Snowflake Sudoku	100%	N/A	N/A	N/A	0%
Maze-Hard	99.9% (K=512)	N/A	85.3%	74.5%	0%

• On Sudoku-Extreme (1000 hard instances, unique solution), an 800K-parameter LDT achieves 100% accuracy and empirical soundness after 4000 training steps (~15 minutes on a B200 GPU), with a median inference time of ~0.03s per puzzle. Sotaku achieves only 98.9% after 2h 40m; TRM and HRM perform worse and require much larger models and longer training.
• On Snowflake Sudoku (1000 variable-topology puzzles), the same 800K LDT (adapted only in grid masking) achieves perfect accuracy after only 4 minutes of training; all chain-of-thought LLMs score 0%.
• On Maze-Hard (1000 instances, 30×30 grid, multi-solution), a 1.8M-parameter LDT achieves 99.3% accuracy (K=1) and 99.9% (K=512) after 20,000 steps (~9.8h training), outperforming TRM (85.3%) and HRM (74.5%) at higher parameter counts and training cost.

In all cases, frontier LLMs (Claude Opus 4.6, GPT-5.4, DeepSeek V4-Pro) achieve 0% accuracy.

7. Distinctions from Previous Recurrent Reasoners

Prior small-model recurrent reasoners such as Sotaku, TRM, and HRM rely on static data, hand-engineered heuristics, heavy data augmentation, and require prolonged training—hours to days—while achieving only 55–99% solution rates on challenging instances and lacking any soundness guarantee. LDT distinguishes itself through:

• Explicit lattice projection, ensuring monotonic and sound candidate elimination.
• On-policy “Solve” loop, training the model on actual inference trajectories and decision branches.
• Abstract-interpretation–based supervision, enabling domain-agnostic, sound deduction with minimal inductive bias.

LDT achieves empirically sound, efficient logical deduction unmatched by previous small recurrent models or large-scale chain-of-thought LLMs, delivering 100% on two distinct Sudoku benchmarks and 99.9% on a challenging maze suite—all within minutes or several hours of wall-clock training, at substantially lower parameter counts and computational cost (Davis et al., 9 May 2026). A plausible implication is that such architectures, which couple symbolic abstraction with small recurrent reasoning, define a strong lower bound for efficient and reliable deductive reasoning in constraint-based problem domains.