Differentiable Logic Gate Networks

• DLGNs are neural architectures that replace traditional affine transforms with discrete Boolean logic gates, using a train-soft, infer-hard scheme to form explicit logic circuits.
• Architectural variants such as trainable wiring, convolutional logic trees, and threshold layers extend their applicability to image tasks and hardware synthesis, achieving high accuracies like 98.68% on MNIST and 86.29% on CIFAR-10.
• Research addresses key challenges—including vanishing gradients and discretization errors—with innovations like input-wise parameterization and Gumbel noise injection, drastically improving training efficiency and hardware deployment.

Differentiable Logic Gate Networks (DLGNs), also called LGNs or DDLGNs in parts of the literature, are neural architectures whose elementary units are discrete logic gates rather than affine transforms followed by nonlinear activations. During training, each two-input neuron is relaxed into a differentiable mixture over the 16 Boolean functions on $\{0,1\}^2$; at inference, the relaxation is hardened by selecting a single gate, producing a sparse Boolean circuit that can be executed as bit-level logic (Petersen et al., 2022). This train-soft, infer-hard paradigm has since been extended to convolutional vision models, trainable wiring, silicon-aware synthesis, ternary and recurrent variants, and task-specific systems for image restoration, video copy detection, and hardware-triggering workloads (Petersen et al., 2024, Mommen et al., 8 Jul 2025, Fieldhouse et al., 21 Apr 2026, Damera et al., 27 Feb 2026).

1. Formal model and computational primitive

In the standard binary formulation, a DLGN neuron has exactly two inputs and learns which of the 16 possible Boolean operators to apply. The original relaxation assigns each neuron a 16-dimensional logit vector $\mathbf z$, converts it to a categorical distribution with a softmax, and computes the expected output of the gate under real-valued probabilistic logic. A canonical form is

$$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$

where $f_i$ denotes the $i$-th relaxed Boolean function (Petersen et al., 2022). The real-valued gate surrogates are interpreted as expectations under independent Bernoulli inputs; representative examples are $A\land B \mapsto A\cdot B$, $A\lor B \mapsto A+B-AB$, and $A\oplus B \mapsto A+B-2AB$ (Petersen et al., 2022).

This formulation differs from conventional feed-forward networks in two respects. First, the learnable object is gate identity rather than a dense weight vector. Second, the deployed model is not a quantized approximation of a multiply-accumulate network, but a literal Boolean circuit. The standard architecture is layered and sparse: each neuron consumes two signals from earlier layers, and classification is typically performed by a Group-Sum head that partitions the final activations into class bins and sums them, optionally scaled by a temperature $\tau$ (Petersen et al., 2022, Brändle et al., 30 Sep 2025).

The Group-Sum head is central to the DLGN literature because it connects Boolean hidden computation to multiclass logits without reintroducing dense arithmetic. In the large-class study, the class-probability vector is written as a softmax over per-class sums divided by $\tau$, and the paper emphasizes that $\mathbf z$0 is not a minor tuning constant but a decisive control parameter for neuron utilization and output-layer behavior (Brändle et al., 30 Sep 2025). This suggests that, even in a logic-circuit paradigm, the output interface remains an important source of inductive bias.

2. Architectural organization and training variants

The baseline DLGN uses fixed pseudo-random wiring: the topology is frozen after initialization, and training only selects gate types (Petersen et al., 2022). Several later works modify this assumption. One line makes the interconnections themselves differentiable by assigning each gate input a softmax distribution over candidate source nodes from the previous layer and annealing the connection temperature toward an argmax distribution. In this formulation, wiring and gate function are both optimized, and the strongest reported MNIST result in the fully connection-trainable setting is 98.68% with 2×8000 gates, while 8000 simple logic gates are sufficient to achieve over 98% on MNIST (Mommen et al., 8 Jul 2025).

A separate line broadens DLGNs into more interpretable differentiable logic networks for mixed tabular inputs. The architecture there combines a ThresholdLayer, a LogicLayer of two-input gates, and a SumLayer, with alternating phases that separately optimize neuron functions and connections (Yue et al., 2024). Thresholds are trained with sigmoidal relaxations and later hardened to Heaviside-style comparisons, so the final model can be read as explicit comparisons and Boolean rules rather than only as a gate netlist (Yue et al., 2024). On 20 tabular classification datasets, that work reports the best average balanced-class accuracy rank of 2.3 and notes that inference may require up to a thousand times fewer logic-gate operations than NNs (Yue et al., 2024).

For image tasks, the most important architectural extension is convolution. "Convolutional Differentiable Logic Gate Networks" replaces randomly connected layers on flattened inputs with convolutional logic-gate trees, logical OR pooling, and residual initialization (Petersen et al., 2024). A kernel is implemented as a complete binary tree of gates whose parameters are shared across spatial positions, preserving a convolutional inductive bias without introducing multiply-accumulate kernels. The paper states that this allows scaling logic gate networks up by over one order of magnitude and reports 86.29% accuracy on CIFAR-10 using 61 million logic gates, which it describes as 29x smaller than the previous best accuracy-matched baseline (Petersen et al., 2024).

These variants are methodologically distinct but structurally compatible. Fixed random wiring favors minimal training overhead, trainable wiring moves DLGNs toward trainable Boolean circuits, threshold layers make them suitable for non-binary inputs, and convolutional logic trees supply spatial weight sharing. The literature therefore does not treat "DLGN" as a single frozen blueprint, but as a family of logic-gate-based models linked by the same soft gate-selection mechanism.

3. Optimization pathologies and parameterization remedies

A persistent theme in the literature is that DLGNs are easy to deploy but hard to train. Three recurrent failure modes are vanishing gradients, discretization error, and high training cost (Rüttgers et al., 26 Sep 2025). The standard "softmax over all Boolean functions" parameterization is diagnosed as redundant because functions and their negations receive independent weights, creating symmetry-driven gradient cancellation. To remove that redundancy, "Light Differentiable Logic Gate Networks" introduces Input-Wise Parametrization (IWP), which assigns weights to input patterns rather than to full Boolean-function identities. For binary gates, this reduces the number of parameters per gate from 16 to 4, yielding a 4x smaller model, up to 1.86x faster backward pass, 1.11x faster forward pass in the reported 80-layer setting, 4x less VRAM, and 8.5x fewer training steps to converge (Rüttgers et al., 26 Sep 2025).

A second remedy targets the train-test mismatch directly. "Mind the Gap: Removing the Discretization Gap in Differentiable Logic Gate Networks" introduces Gumbel LGNs, which inject Gumbel noise and use a straight-through estimator so that the forward pass becomes discrete while gradients are borrowed from a soft Gumbel-Softmax surrogate (Yousefi et al., 9 Jun 2025). The paper reports 4.5× faster in wall-clock time, a 98% reduction in discretization gap, and a 100% reduction in unused gates; the baseline DLGN is said to leave about 49.81% unused gates, whereas Gumbel LGNs report 0.00% unused gates (Yousefi et al., 9 Jun 2025). The same work gives a curvature-based interpretation: the noise-averaged objective adds an implicit Hessian-trace regularizer, favoring flatter minima and hence more robust hardening (Yousefi et al., 9 Jun 2025).

The binary gate vocabulary itself also creates a scaling bottleneck. Extending the gate-selection paradigm to ternary Kleene logic $\mathbf z$1 would require a softmax over $\mathbf z$2 two-input gates. "Polynomial Surrogate Training for Differentiable Ternary Logic Gate Networks" replaces categorical gate distributions with a degree-$\mathbf z$3 polynomial having 9 coefficients,

$\mathbf z$4

which the paper states is a 2,187× parameter reduction relative to a 19,683-logit softmax (Damera et al., 27 Feb 2026). At CIFAR-10 scale, these ternary networks train 2–3× faster than binary DLGNs, and the paper reports that the hardening gap contracts with overparameterization, falling from about 14.1 percentage points at medium scale to 3.7 points at 512K neurons (Damera et al., 27 Feb 2026).

Collectively, these works show that the computational primitive of DLGNs is simple, but its parameterization is nontrivial. A large part of later DLGN research is therefore not about new downstream tasks, but about making the original logic-gate relaxation numerically viable at greater depth, larger gate vocabularies, and smaller hardening gaps.

4. Silicon, FPGA, and hardware realization

Because a trained DLGN already resembles a logic circuit, it can be mapped more directly to digital hardware than conventional neural networks. The most explicit ASIC-oriented demonstration is "Silicon Aware Neural Networks", which converts a discretized DLGN one-to-one into a gate-level netlist for a Cadence-provided SkyWater 130nm standard-cell library (Fieldhouse et al., 21 Apr 2026). The paper maps individual logic functions to cells such as AND2X1, XOR2X1, NOR2X1, NAND2X1, INVX1, BUFX2, TIELO, and TIEHI, and implements the GroupSum layer with ADDHX1 and ADDFX1 in a popcount / binary adder tree (Fieldhouse et al., 21 Apr 2026).

The same work introduces an area-aware loss based on the expected cell area per neuron,

$\mathbf z$5

with the reported sweep identifying $\mathbf z$6 as a useful trade-off coefficient (Fieldhouse et al., 21 Apr 2026). For MNIST, the baseline DLGN reports 98.04% accuracy with average area per neuron 9.380 $\mathbf z$7 and total area 3,001,592 $\mathbf z$8, whereas area-aware training reports 97.66% accuracy with 6.107 $\mathbf z$9 average area and 1,954,215 $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$0 total area (Fieldhouse et al., 21 Apr 2026). In post-layout analysis of the final SkyWater macro, the paper reports 97.49% MNIST accuracy, 41.8 million inferences/s, 83.88 mW, and a worst-case critical path delay of 23.9 ns; it also notes that the comparison table lists 15 ns latency and 352 pJ per inference, while the post-layout section gives 2.0 nJ under an input switching activity factor of 0.7 (Fieldhouse et al., 21 Apr 2026).

FPGA studies sharpen a different point: synthesis cost is driven less by gate semantics than by output-layer structure. "Resource Utilization of Differentiable Logic Gate Networks Deployed on FPGAs" decomposes LUT consumption into input logic, combinational decision logic, and summation/classification logic, and reports that the final layer width $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$1 is the dominant driver of timing and resource use (Wormald et al., 4 May 2026). End caps with $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$2 reduce LUT usage by about 28%, FF usage by about 10%, and may improve latency by up to 4 cycles, while synthesis success across the explored space is only 31.2% (Wormald et al., 4 May 2026). This indicates that DLGN deployment is not only a question of logic-gate count, but also of how the GroupSum stage is physically realized.

In an application-specific FPGA setting, "Rapid Inference of Logic Gate Neural Networks for Anomaly Detection in High Energy Physics" reports an LGN variant with 3 cycles latency, 0 DSPs, 856 FFs, and 19,977 LUTs, compared with a QKeras baseline at 16 cycles, 697 DSPs, 50,368 FFs, and 159,447 LUTs (Gerlach et al., 31 Oct 2025). The paper emphasizes zero DSP usage as a distinctive advantage of logic-gate inference in Level-1 triggering (Gerlach et al., 31 Oct 2025). Across these hardware papers, the main message is consistent: DLGNs are not merely "hardware inspired"; they are concrete gate-level artifacts that can be synthesized, routed, and analyzed with standard digital-design flows.

5. Empirical range and application domains

The earliest DLGN results were strongest on small or binarizable classification tasks. The original paper reports a discretized MNIST model at 98.47% accuracy with $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$3 per image on CPU and 50 ns on GPU, and states that DLGNs can exceed one million MNIST images per second on a single CPU core (Petersen et al., 2022). On CIFAR-10, the same paper reaches 62.14% with the largest reported model, establishing the efficiency argument but also exposing the difficulty of natural-image learning with purely random-connectivity logic layers (Petersen et al., 2022).

Later vision work extends that empirical envelope. Convolutional DLGNs improve CIFAR-10 substantially to 86.29% with 61 million logic gates (Petersen et al., 2024). A separate scalability study shows that DLGNs can distinguish thousands of classes effectively on a synthetic task, with the large DLGN outperforming an MLP up to 300 classes and still exceeding the MLP at 2000 classes, while on a combined 67-class MNIST-like task a tuned DLGN with $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$4 reaches 83.42% against an MLP at 83.53% (Brändle et al., 30 Sep 2025). The same paper also reports the main large-scale limitation: on ImageNet-32, no tested DLGN configuration matches the MLP beyond around 100 classes, and with 6 layers each output neuron depends on only $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$5 inputs, about 0.7% of the 9216-dimensional threshold-expanded input (Brändle et al., 30 Sep 2025).

The task range is no longer limited to feed-forward classification. "Recurrent Deep Differentiable Logic Gate Networks" reports 5.00 BLEU and 30.9% accuracy on WMT'14 English-German, with a collapsed inference model at 4.39 BLEU and 27.7% (Bührer et al., 8 Aug 2025). "Differentiable Logic Cellular Automata" reports perfect learning of Conway’s Game of Life and recurrent pattern-generation experiments, including checkerboard generation that continues to converge when grid size and rollout length are scaled up by 4× (Miotti et al., 5 Jun 2025). The two papers make different "first" claims: the cellular-automata work says it is, to the authors’ knowledge, the first successful application of DLGNs in a recurrent architecture, while RDDLGN presents the first implementation of Recurrent Deep Differentiable Logic Gate Networks for sequence-to-sequence learning (Miotti et al., 5 Jun 2025, Bührer et al., 8 Aug 2025). A plausible implication is that "recurrent DLGN" has emerged independently in domain-specific and sequence-modeling forms rather than through a single canonical design.

Low-level vision and retrieval have also become active application areas. "LogicIR" adapts logic-gate networks to image restoration with a UNet-inspired architecture, differentiable bit decoding, and index shuffling; on BSD68 it reports 27.71 dB at 169.3 G BOPs with rotational ensemble, and an ablation improves from 17.19 dB for a stacked CLGN baseline to 27.83 dB for full LogicIR-S (Lee et al., 25 Jun 2026). "Efficient Logic Gate Networks for Video Copy Detection" reports best configurations around 0.985–0.986 accuracy/F1, $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$6AP near 0.997–0.998, descriptor sizes of 0.25–0.5 kB, and throughput exceeding 11k samples/s (Fojcik, 23 Apr 2026). In high-energy physics anomaly detection, CLGNs are reported to perform on par with or better than a quantized neural-network baseline while retaining highly favorable FPGA characteristics (Gerlach et al., 31 Oct 2025). These results indicate that DLGNs are no longer confined to proof-of-concept Boolean classifiers; they now span symbolic tabular models, image classification, restoration, recurrent dynamics, retrieval, and hardware-triggering systems.

6. Interpretability, related formalisms, and unresolved issues

One attraction of DLGNs is that the final model is intrinsically discrete. In the interpretable logic-network literature, threshold units become explicit comparisons, gate nodes become named Boolean operations, and class heads become sums of activated rules (Yue et al., 2024). That work further notes that trained expressions can be simplified with symbolic simplification tools and that thresholds outside $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$7 may act as implicit feature pruning (Yue et al., 2024). In this sense, DLGNs differ from post-hoc rule extraction: the circuit is the model.

Yet several unresolved issues remain. Training cost is still high in the baseline formulation, deeper networks remain vulnerable to vanishing gradients and discretization mismatch, and scaling to harder real-world image data remains limited even when synthetic many-class performance is strong (Rüttgers et al., 26 Sep 2025, Yousefi et al., 9 Jun 2025, Brändle et al., 30 Sep 2025). Hardware deployment also introduces non-ML constraints: the SkyWater 130nm work reports that an earlier architecture with 6 layers of 64,000 neurons each was difficult to route in the 1P5M process, prompting a narrower 18 layers of 4,000 neurons each design (Fieldhouse et al., 21 Apr 2026). This suggests that "logic-friendly" does not mean physically unconstrained.

The boundaries of the DLGN concept are themselves evolving. TTnet is a related differentiable truth-table CNN whose filters are constrained to tractable truth tables and later rewritten as DNF/CNF rules, Boolean decision trees, or compact Boolean circuits (Benamira et al., 2022). WARP-LUTs are presented as a Walsh–Hadamard-based extension and improvement over DLGNs, replacing double-exponential gate-parameter growth with a $$a'=\sum_{i=0}^{15}\frac{e^{z_i}}{\sum_j e^{z_j}}\, f_i(a_1,a_2),$$8-parameter LUT relaxation; for 4-input logic blocks, the paper contrasts about 65,000 DLGN parameters per node with 16 for WARP-LUTs, and reports roughly one third of the wall-clock step time on an A100 GPU in a large CIFAR-10 model (Gerlach et al., 17 Oct 2025). Ternary PST further generalizes the framework to many-valued logic with explicit UNKNOWN states and selective prediction (Damera et al., 27 Feb 2026). The literature therefore contains both a core DLGN lineage and a surrounding ecosystem of truth-table, Walsh-basis, and polynomial-surrogate approaches that address the same deployment-oriented logic-learning problem from different representational angles.

Taken together, these developments place DLGNs at the intersection of differentiable program induction, sparse symbolic computation, and digital hardware synthesis. Their distinguishing premise is stable across the literature: a model can be trained in a continuous space yet deployed as an exact logic circuit. What remains unsettled is not the viability of that premise, but the most effective parameterization, output design, scaling law, and hardware substrate for making logic-gate learning competitive beyond the task regimes where it already excels.