LUT-Conv: Lookup-Table Convolution Methods

• LUT-Conv is a family of methods that replace runtime multiply-accumulate operations with indexed retrieval from precomputed tables.
• These techniques include im2col-based LUT-Dense layers, tabulated image operators, and FPGA-native Boolean implementations for efficient convolution.
• Key challenges include exponential table size growth with dimensionality, necessitating sparse kernel designs and optimized connectivity for practical deployment.

LUT-Conv denotes a family of lookup-table-based formulations of convolutional or convolution-adjacent computation in which runtime multiply-accumulate evaluation is replaced, wholly or partly, by indexed retrieval from precomputed tables. The literature does not use the term in a single uniform sense. In one explicit formulation, HGQ-LUT defines LUT-Conv as an im2col-based convolution built on a LUT-Dense layer and compiled into logic LUTs for FPGA inference (Sun et al., 24 Apr 2026). In image restoration and video filtering, closely related systems tabulate local patch-to-output mappings and execute them by lookup and interpolation rather than by conventional convolution (Huang et al., 2023, Yang et al., 20 Mar 2025, Li et al., 2024, Li et al., 11 Sep 2025, Zhang et al., 12 Oct 2025). In FPGA-native neural inference, related work also uses logic LUTs as learned Boolean operators inside convolutional windows or as low-bit multiplication primitives inside convolution engines (Wang et al., 2019, Xie et al., 2024, Sen et al., 2024).

1. Terminological scope and variants

The most precise contemporary use of the name appears in HGQ-LUT, where LUT-Conv is a convolutional layer implemented by applying im2col before a LUT-Dense computation (Sun et al., 24 Apr 2026). Outside that paper, the same design space is populated by methods that do not always use the identical name but realize the same underlying substitution: local convolution-like mappings are precomputed and stored in tables rather than evaluated by arithmetic at inference time.

Interpretation	Representative mechanism	Representative papers
Layer-native LUT-Conv	im2col followed by LUT-Dense with 1-input logical LUTs	(Sun et al., 24 Apr 2026)
Local patch LUT operator	quantized neighborhood to output lookup, often with interpolation	(Huang et al., 2023, Yang et al., 20 Mar 2025, Zhang et al., 12 Oct 2025, Li et al., 2024, Li et al., 11 Sep 2025)
Logic-LUT convolution	learned $K$-input Boolean operator inside a conv window	(Wang et al., 2019)
LUT-based conv arithmetic	lookup-based multipliers or MAC decomposition inside conv datapaths	(Xie et al., 2024, Sen et al., 2024)

A restricted but important variant is LUT conversion of pointwise convolution. ICELUT uses only $1\times1$ convolutions during training, then replaces the pointwise backbone by channel LUTs and the coefficient head by weight LUTs, yielding what it describes as purely LUT inference without any convolutional neural network at deployment (Yang et al., 2024). This establishes that, in practice, LUT-Conv may denote anything from explicit convolutional layers to LUT-compiled channel-mixing operators.

2. Core mathematical formulations

The cleanest formal definition is the HGQ-LUT LUT-Dense layer,

$$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$

with LUT-Conv realized by placing an im2col operation before this computation so that each flattened receptive field becomes the input vector to the same operator (Sun et al., 24 Apr 2026). In this formulation, convolution is not represented as a single high-fan-in truth table; it is factorized into many 1-input logical LUTs followed by summation.

LUTNet adopts a different FPGA-native formulation. It replaces the binarized inner-product term by learned $K$-input Boolean operators,

$$y = f\left(\sum_{n=1}^{\tilde{N}} g_n\left(\tilde{\boldsymbol{x}}^{(n)}\right)\right),$$

where each $\tilde{\boldsymbol{x}}^{(n)}$ is selected from the same convolutional window, preserving convolutional locality while replacing XNOR-like products by learned LUT functions (Wang et al., 2019). Here LUT-Conv is closer to logic synthesis of a local Boolean operator than to table lookup over intensities.

A third recurrent formulation is the tabulated local image operator. In such systems, a quantized neighborhood is used as a direct table index, and table size grows exponentially with the number of indexed dimensions. RFE-LUT formalizes this with

$$S(\mathbf{B}) = \left(\prod_{j=1}^{d}\left(\frac{2^8}{b_j}+1\right)\right)mB,$$

under a diagonal lattice basis $\mathbf{B}=\operatorname{diag}(b_1,\dots,b_d)$, making explicit that memory scales with the product of per-dimension quantization counts (Zhang et al., 12 Oct 2025). Video in-loop filtering work writes the same constraint as

$$MS = \left(2^{8-q}+1\right)^n \times V \times 8\ \text{bit},$$

with $n$ the cached dimension and $1\times1$0 the sampling interval (Li et al., 11 Sep 2025). DnLUT states the analogous growth law for restoration as

$1\times1$1

which is precisely why naive color-spatial LUTs become intractable (Yang et al., 20 Mar 2025).

These formulations all encode the same structural fact: LUT-Conv exchanges arithmetic complexity for memory complexity, and the dominant variable is not depth alone but the dimensionality of the indexed local state.

3. Architectural realizations in vision models

In image super-resolution, HKLUT shows a storage-driven design in which local convolution-like operators are decomposed into extremely small staged LUTs. Its specialized kernel patterns reduce the number of sampled pixels to two or three, while asymmetric MSB/LSB branches allocate the larger receptive field only to the MSB branch. The resulting models occupy 100 KB for HKLUT-S and 112.5 KB for HKLUT-L, in contrast to multi-megabyte earlier LUT-SR systems (Huang et al., 2023). This is a direct response to the exponential $1\times1$2 storage law: receptive field is preserved by sparse patterns, rotation ensemble, and multistage progressive upsampling rather than by increasing the per-LUT input dimension.

DnLUT makes the same trade-off for color denoising. Its Pairwise Channel Mixer processes the pairs $1\times1$3, $1\times1$4, and $1\times1$5 with $1\times1$6, depth-2 local operators that remain representable as 4D LUTs, while the L-shaped convolution enlarges spatial support with 3D LUTs rather than 4D spatial LUTs. The paper reports 500KB storage, 20X faster inference, and 0.1% energy consumption compared to DnCNN, while also emphasizing that a naive $1\times1$7 LUT would require 582.6 TB under its sampling regime (Yang et al., 20 Mar 2025). The architectural lesson is that channel interaction and receptive-field growth must be co-designed under LUT dimensionality constraints.

RFE-LUT generalizes this line of work from low-level restoration to high-level vision. Its lattice vector quantizer learns per-dimension quantization resolution, its Irregular Dilated Convolution enlarges spatial span without increasing the number of active LUT inputs, and its U-shaped cascaded LUT structure compounds context hierarchically (Zhang et al., 12 Oct 2025). This suggests that the principal barrier to broader LUT-Conv deployment is not lookup itself but inefficient use of table capacity.

ICELUT occupies a narrower but conceptually important corner of the field. It restricts the train-time network to pointwise $1\times1$8 convolutions plus a split fully connected head, then converts the pointwise backbone into branch LUTs of shape $1\times1$9 and the coefficient head into weight LUTs, achieving a purely LUT-based image enhancer at inference (Yang et al., 2024). This is not general spatial convolution, but it demonstrates that LUT conversion is particularly natural for channel-mixing operators.

4. FPGA-native interpretations

On FPGAs, LUT-Conv often means direct use of logic LUT resources rather than software-side table lookup. HGQ-LUT is the most explicit recent example: it trains with regular tensor operations, then enumerates the truth tables of 1-input logical LUTs and lowers them to an IR with L-LUT instructions for RTL generation and bit-exact verification (Sun et al., 24 Apr 2026). Its central practical claim is that LUT-aware training can be made GPU-efficient by avoiding high-fan-in trainable logic during training.

LUTNet represents an earlier logic-centric route. It inserts learned $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$0-input Boolean functions into convolutional layers and fully connected layers, with additional inputs chosen from the same convolutional window so that the receptive-field shape remains unchanged (Wang et al., 2019). In this sense, convolution is preserved structurally while the local arithmetic primitive is replaced by a logic LUT.

A different hardware interpretation treats LUTs as multipliers rather than as full local operators. LUTMUL embeds fixed low-bit weights into LUT6/LUT6_2 contents, uses activation bits as addresses, and reports an average of 2 LUTs per 4-bit multiplication. Its accelerator supports pointwise convolution, depthwise convolution, and standard convolution through a streaming convolution generator (Xie et al., 2024). The related LUT-NA framework instead decomposes high-precision MACs into lower-precision LUT-based sub-MACs through a divide-and-conquer strategy and reports up to $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$1 lower area than traditional LUT-based techniques and up to $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$2 lower area than conventional digital MAC-based techniques (Sen et al., 2024). These systems do not tabulate entire patch responses, but they are still LUT-Conv in the broader sense that convolution is realized through LUT-native arithmetic.

5. Training, quantization, and compilation workflows

Despite their diversity, LUT-Conv systems share a common deployment pattern. A trainable surrogate is optimized first; the deployed table is derived afterward. HKLUT trains a small CNN surrogate for each LUT, enumerates all possible quantized inputs, and stores INT8 outputs obtained by multiplying FP32 outputs by 127 and rounding down (Huang et al., 2023). ICELUT similarly trains a pointwise-CNN-plus-SFC predictor and then transfers both backbone and head to channel and weight LUTs for deployment (Yang et al., 2024).

Some systems explicitly adapt training to the deployed table. DnLUT converts trained local operators into optimized lookup tables post-training and then performs 2000 iterations of LUT-aware fine-tuning (Yang et al., 20 Mar 2025). The video in-loop filtering frameworks train a DNN over a restricted reference range, cache its outputs into clipped LUTs, and then finetune the clipped LUT to compensate for interpolation and clipping error (Li et al., 2024, Li et al., 11 Sep 2025). RFE-LUT moves this coupling earlier by jointly optimizing network weights and quantizer steps under a memory-regularized objective (Zhang et al., 12 Oct 2025).

HGQ-LUT differs in that the deployed entity is not a numeric table over image intensities but a bank of logical truth tables. Its truth-table extraction enumerates all possible input combinations of each L-LUT, passes them through the MLP realization used during training, and quantizes the outputs, after which the arithmetic training graph is discarded and FPGA logic is generated (Sun et al., 24 Apr 2026). Across these variants, quantization is not merely a compression step; it defines the address space of the deployed convolutional operator.

6. Constraints, misconceptions, and research directions

A common misconception is that LUT-Conv denotes one standardized operator. The literature shows otherwise. Some methods replace a full convolution-like local mapping by a table over quantized neighborhoods; some only replace $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$3 channel mixing; some synthesize Boolean operators inside convolution windows; and some only LUT-ize the multiplication stage of a conventional convolution pipeline (Yang et al., 2024, Wang et al., 2019, Xie et al., 2024). The term is therefore best treated as a family of lookup-driven realizations of local neural computation.

A second misconception is that LUT-Conv removes the scaling problem of convolution. It does not. It relocates the bottleneck. The dominant limitation becomes exponential growth in table size with indexed dimensionality. DnLUT’s 582.6 TB estimate for a naive $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$4 LUT, HKLUT’s emphasis on $$a^{(l)}_i = \sum_{j=1}^{N} \text{L-LUT}_{i,j}(a^{(l-1)}_j),$$5 storage, and RFE-LUT’s explicit memory regularizer all describe the same structural obstacle (Yang et al., 20 Mar 2025, Huang et al., 2023, Zhang et al., 12 Oct 2025). Most successful systems therefore cap the active input dimension, use sparse kernel patterns, split channels or bit planes, or compose multiple small LUTs across stages.

A third misconception is that lookup-based layers are always more expressive than conventional layers. HGQ-LUT explicitly restricts itself to 1-input logical LUTs for training efficiency, which improves practicality but also means that multi-input interactions must emerge through summation and depth rather than inside a single truth table (Sun et al., 24 Apr 2026). Conversely, broader LUT-based DNN work on SparseLUT shows that when fan-in is fixed, connectivity choice becomes critical, and random sparse connectivity can materially limit accuracy (Lou et al., 17 Mar 2025). This suggests that future LUT-Conv systems may need not only better quantization and receptive-field design but also structured connectivity optimization adapted to convolutional weight sharing.

The present trajectory of the field is therefore hybrid rather than monolithic. The papers collectively indicate that LUT-Conv is most effective when applied to operators with small effective dimensionality, strong locality, or fixed low-bit arithmetic, and when integrated with architectural devices that preserve context without enlarging a single table beyond tractable size (Zhang et al., 12 Oct 2025, Huang et al., 2023, Yang et al., 20 Mar 2025).