--- title: 'HGQ-LUT: Efficient LUT-Centric Inference' url: https://www.emergentmind.com/topics/hgq-lut type: topic --- # HGQ-LUT: Efficient LUT-Centric Inference HGQ-LUT denotes a class of lookup-table-centric inference methods in which quantization is made sufficiently fine-grained, hierarchical, or hardware-guided that deployment reduces to table lookups plus reductions instead of conventional multiply-accumulate pipelines. In its most explicit usage, HGQ-LUT is the LUT-aware training framework introduced in "HGQ-LUT: Fast LUT-Aware Training and Efficient Architectures for DNN Inference," which combines LUT-Dense and LUT-Conv layers with element-wise heterogeneous quantization, zero-bit pruning, and an end-to-end FPGA compilation flow [2604.22293]. In related literature, the label is also used descriptively for broader constructions that share the same principle, including hierarchical nested-lattice quantization with small LUTs, FPGA tabulation of Kolmogorov-Arnold edge functions, group-wise hierarchical linear quantization for LLM LUT kernels, and in-DRAM per-group LUT generation for W4A4 inference [2505.13164] [2512.12850] [2510.19482] [2606.08891]. ## 1. Terminology and scope The label is not used uniformly across the literature. In the FPGA DNN setting, HGQ-LUT is a named framework built around LUT-aware training, HGQ quantizers, and automatic compilation to logic LUTs [2604.22293]. In several adjacent works, however, the papers do not use the term explicitly; instead, later technical summaries use it to denote the broader idea of choosing quantization, grouping, and table organization so inference can be executed through LUT-native hardware paths [2505.13164] [2512.12850] [2510.19482] [2606.08891]. | Usage | Representative formulation | Primary substrate | |---|---|---| | Explicit HGQ-LUT | LUT-Dense and LUT-Conv with HGQ quantizers | FPGA logic LUTs | | Hierarchical nested-lattice quantization with LUT decoding | \(M\)-layer inner-product reconstruction from a small LUT | Cache-resident LUTs | | Hardware-guided KAN tabulation | 1D spline edge functions discretized into L-LUTs | FPGA logic LUTs | | HLQ with LUT-based GEMM | \(\hat W = \sum_{j=0}^{q-1} s_j b_j + z\) | Edge CPU LUT kernels | | PALUTE HGQ-LUT interpretation | Per-group W4A4 LUTs generated and queried in DRAM | M3D DRAM PIM | This breadth matters because a common misconception is to treat HGQ-LUT as a single, fixed algorithm. The literature instead contains a specific framework and a family resemblance: fine-grained or hierarchical quantization is coupled to LUT execution, and hardware costs are exposed during training or code construction rather than deferred to post hoc synthesis. ## 2. Core mathematical formulations In the explicit FPGA framework, the central abstraction is the 1-input logical LUT, where “one input” refers to the logical signal and that signal is multi-bit in hardware. A LUT-Dense layer is defined as $$ a_i^{(l)} = \sum_{j=1}^{N} \mathrm{L\textrm{-}LUT}_{i,j}\!\big(a_j^{(l-1)}\big). $$ This is presented as a relaxation of a dense layer, and by reparameterizing intermediate activations, $$ a_i'^{(l)} = \sum_{j=1}^{N} w_{ij}^{(l)} \,\phi^{(l-1)}\!\big(a_j'^{(l-1)}\big) + b_i^{(l)}, $$ with $$ \mathrm{L\textrm{-}LUT}_{i,j}(x)=w_{ij}^{(l)}\phi^{(l-1)}(x)+\frac{b_i^{(l)}}{N}, $$ the LUT formulation becomes equivalent to the reparameterized dense layer. LUT-Conv is obtained by preceding LUT-Dense with an im2col transform, so convolutional receptive fields are unfolded into vectors and processed with the same LUT machinery [2604.22293]. KANELÉ applies the same LUT-native idea to Kolmogorov-Arnold Networks. A KAN layer with input dimension \(d_{\mathrm{in}}\) and output dimension \(d_{\mathrm{out}}\) is a matrix of univariate edge functions, $$ (x_{l+1})_j = \sum_{i=1}^{d_l} \phi_{j,i}(x_{l,i}), $$ and edge activations are parameterized with B-spline bases on a fixed domain \([a,b]\). Because each \(\phi_{q,p}\) is a learned one-dimensional function, it can be tabulated directly, $$ \mathrm{LUT}[i] = \phi(a+i\Delta), $$ with optional interpolation at inference. This edge-based decomposition makes pruning structurally simple and makes FPGA inference a matter of L-LUT evaluation followed by pipelined summation [2512.12850]. In hierarchical nested-lattice quantization for matrix multiplication, the LUT object is not a neural activation table but a codebook of inner products between low-rate lattice-code components. A rate-\(R\) code is split into \(M\) layers of rate \(R/M\), replacing a single LUT of size \(2^{2dR}\) with a smaller LUT of size $$ 2^{2dR/M}, $$ while reconstructing inner products through $$ \hat x^\top \hat y = \sum_{i=0}^{M-1}\sum_{j=0}^{M-1} q^{i+j} L\!\big(b_i(x), b_j(y)\big). $$ The formal purpose is to keep LUT-based inner-product decoding feasible at high quantization rates without materially increasing distortion [2505.13164]. For edge LLM inference on CPUs, ELUTQ introduces Hierarchical Linear Quantization, again with an explicitly linear decomposition: $$ \hat W = \sum_{j=0}^{q-1}s_j\cdot b_j + z. $$ Because the representation is linear in the binary planes \(b_j\), GEMM can be written as $$ A\hat W = \sum_{j=0}^{q-1} s_j\,(A b_j) + (A\mathbf{1})\,z, $$ so the runtime path becomes LUT lookups for activation–binary-pattern dot products plus scalar accumulations, eliminating dequantization overhead on the critical path [2510.19482]. ## 3. Quantization, pruning, and resource-aware optimization The defining training idea in HGQ-LUT is that quantization is not an afterthought but a differentiable, element-wise optimization variable coupled to a LUT-aware resource surrogate. Inputs use WRAP-mode quantization, outputs use SAT-mode quantization, and zero-bit support is native: if either the input or output element’s bit-width of a given L-LUT is set to zero, that L-LUT is pruned automatically. The training objective augments task loss with a LUT resource term, $$ L_\mathrm{total} = L_\mathrm{task} + \beta \sum_{\ell} R_{\mathrm{LUT}}^{(\ell)}(\theta,\mathbf{b}), $$ where the per-L-LUT surrogate depends on input bit-width \(m\), output bit-width \(n\), and FPGA LUT primitive parameters. For \(m \ge Y\), $$ \mathrm{EBOPs}_{\mathrm{L\textrm{-}LUT}} = 2^{m-X}n, $$ and empirically $$ \#\mathrm{LUTs} \approx \mathrm{EBOPs}^{0.985}. $$ A single scalar \(\beta\) schedule is then used to trace an accuracy-resource Pareto frontier in one run [2604.22293]. RQP, or resource-oriented quantiser pruning, extends the HGQ line by making bit-widths themselves the optimization target at weight granularity and by replacing monotonic resource reduction with a direct jump to the target resource region. Its EBOPs proxy is $$ \mathrm{EBOPs} = \sum_{(i,j)\in \mathcal{M}} b_i b_j + \sum_{(k,l)\in \mathcal{A}} \max(b_k,b_l), $$ with LUT usage modeled as \(\mathrm{LUT} \approx \mathrm{EBOPs}^{0.985}\). RQP introduces one-shot target-budget bit-width reallocation, spectral-constrained topology compensation, and bidirectional \(\beta\) scheduling, so the search can move around the target budget instead of compressing irreversibly toward it. On jet substructure classification, this reduces search cost by up to \(20.58\times\) relative to monotonic HGQ workflows while keeping a competitive Pareto frontier [2606.30382]. KANELÉ adopts quantization-aware training via Brevitas and inserts quantizers at the network input and after each layer so the training graph matches inference precision. Layer outputs are quantized as $$ x_{l+1,q}= s_l \cdot \mathrm{Quantize}[n_l]\!\left(\frac{\mathrm{clip}(x_{l+1},a,b)}{s_l}\right), $$ with STE for gradients. Bit-widths are explored layer-wise from 1 to 8 bits, and the reported ablations show LUT use drops exponentially with bit-width while diminishing returns appear below 6 bits. Pruning exploits the additive independence of KAN edges through an \(L_2\)-norm importance metric and an exponential warm-up threshold \(\tau(t)\), so resource pressure is aligned with direct hardware savings in logic LUTs and FFs [2512.12850]. ELUTQ places the optimization focus on HLQ parameter search and low-bit LLM deployment. In HLQ-GPTQ, group-wise scales and zero-points are optimized by a nearest-neighbor codebook selection followed by continuous updates of \(s\) and \(z\); an alternating least-squares variant is also described. Efficient finetuning then proceeds in two stages: a block-wise reconstruction phase minimizing $$ \big\|\mathcal{F}(\mathbf{W},\mathbf{X})-\mathcal{F}(\mathrm{HLQ}(\mathbf{W};s,z),\mathbf{X})\big\|, $$ followed by end-to-end tuning of scales only. The result is a quantization format that remains orthogonal to existing PTQ or QAT pipelines while staying compatible with LUT execution [2510.19482]. ## 4. Compilation flows and hardware realizations The explicit HGQ-LUT framework is tightly integrated with HGQ2 and da4ml. During training, each L-LUT is represented by an MLP surrogate implemented with regular tensor operations, typically monolithic GEMMs or einsums. During compilation, the truth table for each L-LUT is generated by enumerating all possible input bit patterns, evaluating the trained surrogate, and applying output quantization. The compiled model is lowered to da4ml’s Distributed Arithmetic Instruction Set with a custom L-LUT instruction, then emitted as Verilog or VHDL. Bit-exact verification is performed with a DAIS interpreter and RTL-level checks through GHDL or Verilator, and hybrid models mixing L-LUT instructions with arithmetic instructions are supported natively [2604.22293]. KANELÉ provides a separate FPGA-oriented design flow for KANs. Training is followed by conversion of surviving edges to L-LUTs, enumeration of input state spaces, storage of integer truth tables in JSON, and generation of VHDL RTL for per-layer packages, LUT entities, adder trees, pipeline registers, and memory initialization files. The hardware mapping uses distributed ROM built from logic LUTs rather than BRAM or DSPs, with aggressive pipelining to sustain reported clocks of \(800\) to \(1736\) MHz and initiation interval \(II=1\) [2512.12850]. PALUTE relocates the LUT substrate from FPGA logic to Monolithic 3D DRAM. Its memory fabric uses a Vertical Bitline topology in which a bank comprises \(1{,}024\) MATs, each channel is a \(4\times4\) array of banks, and the chip integrates a \(4\times4\) array of channels. A near-memory LUT generator on a hybrid-bonded logic die materializes GEMM and unary-operator LUTs in three cycles at \(200\) MHz and writes them into a dedicated middle LUT region. Under the stated geometry and row-buffer bandwidth assumptions, the design reaches up to \(8{,}388{,}608\) simultaneous LUT reads, making group-wise HGQ-LUT-style inference a memory-system problem rather than a logic-synthesis problem [2606.08891]. Dedicated accelerator and CPU realizations show that the same abstraction also survives outside FPGA workflows. FIGLUT replaces FP-INT arithmetic with RAC units reading conflict-free FFLUT or half-size hFFLUT tables; with \(\mu=4\) and \(k=32\), the design is tuned to reduce arithmetic energy while keeping LUT fan-out manageable. ELUTQ, by contrast, targets ARM NEON-class CPUs, rearranging one-bit weight matrices offline, building activation-group LUTs at runtime, and using vectorized table-lookup instructions so inference proceeds through bit-serial LUT-based GEMM rather than dequantize-then-multiply kernels [2503.06862] [2510.19482]. ## 5. Representative systems and reported performance | System | Representative reported result | Domain | |---|---|---| | HGQ-LUT | \(197\times\) training speedup over NLA | FPGA LUT-aware training | | KANELÉ | Up to \(\sim2700\times\) latency speedup and \(>4000\times\) LUT reduction vs prior KAN-on-FPGA | LUT-native KAN inference | | PALUTE | \(1{,}264\) TPS at \(0.16\) W | Edge LLM PIM | | ELUTQ | Over \(25\) tokens/s on Apple M2 for 2-bit LLaMA2-7B | CPU edge LLM | | RQP | Up to \(20.58\times\) reduction in search cost | FPGA quantiser pruning | Within the explicit FPGA DNN workflow, HGQ-LUT reports per-batch training time comparable to plain HGQ and a \(197\times\) speedup over NLA. On HLF JSC, using two HGQ-LUT layers of dimensions 20 and 5, it reports the best accuracy-LUT Pareto frontier on both OpenML and CERNBox datasets, with a pronounced advantage in the low-LUT region. On TGC muon tracking, a hybrid architecture achieves the same accuracy as a plain HGQ baseline with \(\sim 1/3\) lower latency and \(\sim 1/4\) lower LUT utilization. On CEPC drift chamber cluster counting, under an embedded budget below \(10\)k LUTs, the reported implementation uses \(6813\) LUTs, \(1\) DSP, \(0\) BRAM, \(903\) FFs, \(122.1\) MHz \(F_{\max}\), latency \(154\) cycles, and \(II=151\) cycles [2604.22293]. KANELÉ emphasizes the case where the learned function itself is already decomposed into LUT-friendly univariate components. On Moons, Wine, and Dry Bean, the reported KANELÉ designs have latencies of \(2.9\) to \(7.1\) ns, \(F_{\max}\) of \(842\) to \(1736\) MHz, logic LUTs of \(67\) to \(534\) to \(402\), FFs of \(57\) to \(686\) to \(471\), and \(0\) BRAM and \(0\) DSP, against prior KAN-on-FPGA baselines with latencies \(1280\) to \(6880\) to \(18960\) ns and much larger resource footprints. On JSC CERNBox it reports \(75.1\%\) accuracy with \(5034\) LUTs, \(1917\) FFs, \(870\) MHz \(F_{\max}\), \(8.1\) ns latency, and Area\(\times\)Delay \(4.1\times10^4\); on JSC OpenML it reports \(76.0\%\) with \(1232\) LUTs and \(987\) MHz \(F_{\max}\); on MNIST it reaches \(96.3\%\) with \(3809\) LUTs and \(9.3\) ns latency. On MLPerf Tiny ToyADMOS it reports AUC \(0.83\), \(29{,}981\) LUTs, \(17{,}643\) FFs, \(II=1\), throughput \(228\)M inferences/s, latency \(0.07\,\mu\)s, and energy \(0.01\,\mu\)J/inference [2512.12850]. RQP focuses on search efficiency rather than a new inference primitive, but its results are directly relevant to HGQ-LUT deployments because the resource proxy is the same LUT-facing EBOP model. Around a \(400\)-EBOP target, standard HGQ requires up to \(200\)k epochs and \(372.96\) minutes to reach \(73.64\%\) accuracy at \(398\) EBOPs, whereas RQP uses \(6\)k total epochs and \(18.12\) minutes to reach \(73.87\%\) at \(399\) EBOPs. Post-implementation on a Xilinx Virtex Ultrascale+ xcvu13p-flga2577-2-e, representative RQP points include \(368\) LUTs at \(671.217\) MHz and \(8.94\) ns, \(2189\) LUTs at \(657.462\) MHz and \(21.29\) ns, and \(11{,}219\) LUTs at \(510.204\) MHz and \(41.16\) ns [2606.30382]. In LLM deployment, ELUTQ and PALUTE illustrate two different endpoints of the same design philosophy. ELUTQ reports that for LLaMA3-8B, HLQ reduces perplexity by about \(8\%\) at 3-bit and \(85\%\) at 2-bit precision under post-training quantization, and that efficient finetuning further improves 2-bit performance within two hours. For 2-bit LLaMA2-7B on an Apple M2 chip with 4 threads and batch size \(1\), the system achieves over \(25\) tokens/s, and measured kernel latencies show essentially no added runtime overhead relative to uniform quantization inside LUT kernels. PALUTE reports \(1{,}264\) TPS end-to-end throughput at \(0.16\) W, corresponding to \(7{,}738\) TPS/W, with \(12.8\times\) energy efficiency over CHIME, \(1.6\times\) over FIGLUT, and \(2.0\times\) area efficiency over PIMPAL under W4A4 across Qwen3-4B models [2510.19482] [2606.08891]. FIGLUT provides the accelerator-side comparison point for weight-only FP-INT LUT GEMM. For the same 3-bit weight precision, it reports \(59\%\) higher TOPS/W and \(20\%\) lower perplexity than state-of-the-art accelerator designs; at matched perplexity, it reports \(98\%\) higher TOPS/W by performing \(2.4\)-bit operations. The design also reports up to \(2.4\times\) energy-efficiency gains over FIGNA at Q2 and about \(1.6\times\) at Q3, showing that LUT execution can outperform integer-multiplier pipelines when the quantization format remains compatible with bit-serial table indexing [2503.06862]. ## 6. Limitations, misconceptions, and open directions A first limitation is terminological. The acronym is explicit in the FPGA LUT-aware training paper, but several related papers state that they do not use the term explicitly and only match it “in spirit” or through a closest construct. A precise reading therefore distinguishes named HGQ-LUT from broader hardware-guided or hierarchical LUT-native quantization schemes rather than conflating them into one canonical algorithm [2505.13164] [2512.12850] [2606.08891] [2510.19482]. A second limitation is structural. In the explicit FPGA framework, 1-input L-LUTs are less expressive than high-fan-in LUTs, truth-table enumeration still grows as \(2^m\), backend LUT packing is vendor-specific when \(m