LOCALUT: DRAM-PIM LUT-Based Low-Bit DNN Inference

• LOCALUT is a DRAM-PIM architecture using operation-packed LUTs to perform multiple low-bit MAC operations per lookup, reducing arithmetic complexity.
• It employs LUT canonicalization and a reordering LUT to eliminate redundancy and mitigate the overhead of dynamic weight permutation on PIM cores.
• The design uses LUT slice streaming to efficiently leverage DRAM capacity and SRAM speed, achieving up to 1.82× speedup in end-to-end DNN inference.

Searching arXiv for the specified paper and closely related LUT/PIM context. LOCALUT is a DRAM-PIM architecture and execution method for low-bit DNN inference that replaces much of arithmetic computation with lookup-table (LUT) accesses, then pushes that idea further by exploiting a specific capacity–computation tradeoff: use abundant DRAM capacity to store larger, more expressive LUTs so that each lookup performs not just one multiplication, but multiple packed multiply-accumulate (MAC) operations at once. In the cited work, the manuscript names the design SlimWalk, while the artifact and repository link are LoCaLUT; in this sense, “LOCALUT” denotes that same proposed system. The design is evaluated on a real UPMEM-based platform and is presented as a method for efficient low-bit quantized DNN inference in DRAM-based processing-in-memory (Hong et al., 6 Apr 2026).

1. Architectural setting and design objective

LOCALUT is motivated by a mismatch at the center of DRAM-based PIM. DRAM chips have plenty of storage and high internal bandwidth, but are a poor substrate for dense arithmetic logic. Their process technology is optimized for capacity rather than logic, and practical PIM products therefore expose only a small amount of fixed-function compute, often tied to a narrow set of precisions. The paper identifies this as a problem for modern quantized DNN inference, which increasingly spans formats such as W1A3, W1A4, W2A2, W4A4, and also discusses low-bit floating-point variants in a bank-level PIM study.

Within that setting, LUT-based computation is used as an alternative compute mechanism that maps input operands to precomputed results, thereby eliminating the need for arithmetic logic. The method is particularly attractive for low-bit quantized inference because LUTs reduce logic complexity and are naturally precision-flexible: changing bitwidths or number formats does not require new arithmetic hardware, only different table contents. LOCALUT therefore frames DRAM capacity as a substitute for logic density, using memory as a compute substrate rather than treating memory merely as storage.

The targeted workload is low-bit transformer inference on real PIM hardware. The evaluation includes BERT-base, OPT-125M, and ViT-Base under quantization settings including W1A3, W1A4, W2A2, and W4A4. In end-to-end execution, the PIM banks handle GEMMs such as QKV projections, output projections, and feed-forward GEMMs, while the host handles non-GEMM operations such as softmax, normalization, GELU, and quantization/dequantization.

2. Operation-packed LUTs and the capacity–computation tradeoff

The core idea is operation-packed LUTs. Instead of the common one-LUT-lookup-per-operation style, LOCALUT uses LUTs that combine several low-bit MAC terms into one table entry. If a weight value has bitwidth $b_w$, an activation has bitwidth $b_a$, and each LUT output uses $b_o$ bytes, then a single-operation LUT needs

$b_o \cdot 2^{b_w+b_a}$

bytes. LOCALUT packs $p$ operations together, so the LUT size becomes

$b_o \cdot 2^{(b_w+b_a)\cdot p}.$

This grows exponentially with $p$, but one lookup computes $p$ multiplications plus their local sum. With packing degree $p=3$, the LUT entry can directly store

$w_1 a_1 + w_2 a_2 + w_3 a_3.$

In the paper’s example, weight vector $b_a$0 and activation vector $b_a$1 map to the result $b_a$2. A single lookup therefore returns an entire partial dot product.

This design is explicitly a trade of memory capacity for throughput. Increasing $b_a$3 reduces the number of lookups, but the LUT size grows rapidly. The method is therefore viable primarily in low-bit regimes, where the operand space remains small enough that the packed tables remain feasible with DRAM-scale storage. The paper’s design-space discussion indicates that, with half the UPMEM bank used for LUTs, $b_a$4 can reach around 8 for some low-bit settings such as $b_a$5, while $b_a$6 is around 5. Without canonicalization, those packing degrees fall to 6 and 3, respectively.

A plausible implication is that LOCALUT is not merely replacing multipliers with table accesses; it is rebalancing the compute substrate itself. Arithmetic intensity is shifted into table construction and indexing, while DRAM capacity becomes the primary resource controlling effective MAC density.

3. LUT canonicalization and reordering

A central obstacle is that operation-packed LUTs become very large. LOCALUT’s first key optimization is therefore LUT canonicalization, which identifies and removes redundancy inside packed LUTs. The redundancy arises because the packed MAC result is invariant to a joint permutation of the operands. For example,

$b_a$7

More generally, the same partial inner product appears under many permutations of the packed activation vector. LOCALUT eliminates this by forcing activations into a canonical sorted order and storing only one representative entry per equivalence class.

Before canonicalization, the number of activation-side columns in the LUT is

$b_a$8

because there are $b_a$9 possible values per activation and $b_o$0 packed positions. After canonicalization, the number of distinct columns becomes the number of multisets of size $b_o$1 drawn from $b_o$2 activation symbols:

$b_o$3

This changes growth from exponential in $b_o$4 to a high-order polynomial for fixed $b_o$5. The reported reductions are large: for $b_o$6, canonicalization reduces LUT size by 12.4× at $b_o$7 and 611.1× at $b_o$8. For W1A3 across the considered packing degrees, combining canonicalization and the auxiliary remapping structure yields total LUT reduction of 1.68× to 358× relative to the original operation-packed LUT (Hong et al., 6 Apr 2026).

Canonicalization introduces a second problem: if activations are sorted to reach the canonical column index, the corresponding packed weight vector must be reordered in exactly the same permutation. Dynamic bit unpacking, permutation, and repacking on a weak PIM core is expensive. LOCALUT therefore introduces the reordering LUT, a small auxiliary LUT whose row index is the packed weight vector and whose column index is the permutation induced by sorting the activations. Its output is the already-reordered packed weight vector in canonical form.

The paper’s example uses activations $b_o$9, which sort to $b_o \cdot 2^{b_w+b_a}$0; the corresponding weights $b_o \cdot 2^{b_w+b_a}$1 must be permuted to $b_o \cdot 2^{b_w+b_a}$2. The reordering LUT directly maps from the original packed weight plus the permutation to that canonicalized weight key. Since the reordering LUT only needs to encode all possible packed weights and all possible permutations, its activation-side width is just $b_o \cdot 2^{b_w+b_a}$3 columns. The paper emphasizes that straightforward canonicalization alone can hurt performance because weight index reordering overhead dominates on the in-memory CPU; the reordering LUT removes most of that bottleneck.

4. LUT slice streaming and execution model

Even after canonicalization, the most effective LUT for a high packing degree may still be too large to fit entirely in the PIM unit’s fast local SRAM buffer. LOCALUT’s third major idea is LUT slice streaming, which exploits the DRAM-buffer hierarchy by streaming only relevant LUT columns into the buffer and reusing them across many weight vectors.

The paper first studies two simple placements. A full LUT in the DRAM bank can support larger packing degree because a UPMEM bank has 64 MB, but DRAM accesses are slow. A smaller LUT in the local buffer benefits from fast SRAM access, but the 64 KB local buffer only allows a smaller packing degree. In experiments, the local-buffer LUT consistently outperforms the DRAM-resident LUT despite its smaller packing degree, because SRAM lookup is much faster than repeated DRAM accesses.

LOCALUT extends this buffer-resident execution style with streaming. For a fixed group of canonicalized activations, GEMM repeatedly touches only a small subset of LUT columns while scanning across many weight rows. LOCALUT therefore stores the large canonical LUT and reordering LUT in DRAM, but streams only the needed columns, or slices, into the local buffer. It processes $b_o \cdot 2^{b_w+b_a}$4 activation vectors at a time, loading $b_o \cdot 2^{b_w+b_a}$5 columns from the canonical LUT and $b_o \cdot 2^{b_w+b_a}$6 columns from the reordering LUT into SRAM, then reusing those buffered slices across many weight vectors before moving to the next activation group. The paper describes this as effectively LUT-slice-stationary rather than activation-stationary.

The work also provides a performance model for choosing the packing degree and whether slice streaming is beneficial. For matrix multiplication of $b_o \cdot 2^{b_w+b_a}$7 and $b_o \cdot 2^{b_w+b_a}$8, and assuming a LUT slice contains $b_o \cdot 2^{b_w+b_a}$9 entries, the total execution time for slice streaming is modeled as

$p$0

where $p$1 is the latency to load one canonical-LUT entry plus one reordering-LUT entry from DRAM into the local buffer, and $p$2 is the local lookup cost, including reordering LUT access, canonical LUT access, and accumulation. The first term is the LUT-slice loading cost; the second is the cost of all packed lookups and reductions.

The optimal packing degree is chosen as

$p$3

If the entire LUT can fit in the local buffer at packing degree $p$4, the streaming term disappears and the cost becomes

$p$5

The printed equations in the provided text are partially corrupted, but the intended logic is explicit: increasing $p$6 reduces the number of lookups by a factor of $p$7, while slice size grows exponentially through $p$8; streaming becomes beneficial when reuse across many weight rows is high enough to amortize DRAM-to-buffer slice transfer cost.

5. Hardware realization and empirical results

LOCALUT is implemented on a real UPMEM system. The evaluation platform has 32 ranks of UPMEM DIMMs, totaling up to 2048 PIM processing elements, one per bank piece. Each bank has a 64 MB DRAM array, a 64 KB SRAM local buffer, and an in-order processor. LOCALUT uses about half of both DRAM-bank capacity and local-buffer capacity for LUT storage, with the rest reserved for weights, activations, outputs, and metadata. The host CPU computes the performance model, chooses $p$9, decides whether to use slice streaming, builds the LUTs, and broadcasts them to the PIM banks.

During inference, activations are quantized on the host, partitioned into groups of $b_o \cdot 2^{(b_w+b_a)\cdot p}.$0, sorted into canonical form, and sent together with the permutation metadata. The PIM banks then execute GEMM: with canonicalization plus reordering, each packed weight is transformed into its canonical key via the reordering LUT, then indexed into the canonical LUT to produce partial dot products, which are accumulated locally and written back.

Quantitatively, on standalone GEMM, LOCALUT achieves a geometric mean speedup of 2.87× over Naive PIM and 1.77× over LUT Tensor Core (LTC), with peaks up to 4.73× over Naive PIM and 1.93× over LTC depending on bitwidth and matrix size. On end-to-end model inference across BERT, ViT, and OPT, it reports a geometric mean speedup of 1.77× over Naive PIM and 1.82× over LTC; the 1.82× figure is the headline number emphasized in the abstract. Compared to the basic operation-packed LUT baseline, the full design adds roughly 22% additional speedup through the combined effect of canonicalization, reordering LUT, and slice streaming (Hong et al., 6 Apr 2026).

The ablations isolate the role of each mechanism. The base OP design already benefits from one lookup doing more work. Adding LUT canonicalization (OP+LC) reduces capacity pressure and allows larger packing degrees, but performance can drop because runtime weight reordering on the in-memory processor becomes expensive. Adding reordering LUT (OP+LC+RC) largely recovers that loss. Adding slice streaming (OP+LC+RC+SS) yields the full system and enables larger effective packing degrees by exploiting DRAM bank capacity while retaining SRAM-speed access for streamed slices.

Energy results are also favorable. Across BERT, ViT, and OPT, LOCALUT is more energy efficient than all major baselines. For W1Ax settings it achieves 3.37× lower energy than Naive PIM and 1.88× lower energy than LTC. For W4A4, LOCALUT still improves energy by 1.16× over Naive PIM. On accuracy, the method is intended for exact low-bit arithmetic under a chosen quantization scheme, in contrast to product-quantization approaches that approximate dot products via centroids.

6. Applicability, limitations, and broader significance

LOCALUT is explicitly best suited to very low-bit regimes. As bitwidth increases, the canonical LUT and especially the reordering-related structures grow quickly, reducing feasible packing degree and eroding the advantage over direct arithmetic. The paper notes that beyond about 4-bit weights/activations, performance becomes less favorable, which is why the design aligns most naturally with 1-bit, 2-bit, and 4-bit inference.

The scheme also consumes substantial memory capacity. LOCALUT deliberately trades capacity for throughput and replicates LUTs across banks, which can become problematic for very large models or memory-constrained settings. In the current UPMEM implementation, the design still depends on a weak in-memory processor for residual unpacking and control logic; the paper’s breakdown shows that index-reordering-related computation still dominates a significant fraction of kernel time even after introducing the reordering LUT, while the reordering LUT access itself accounts for only 6.9% of total GEMM kernel time. End-to-end performance also remains affected by the usual PIM-system issue that non-GEMM operations and inter-bank communication route through the host.

These constraints delimit the scope of the method, but they do not alter its central architectural contribution. LOCALUT reframes LUT-based inference for PIM as a systematic exploitation of memory capacity as compute substrate. Its three techniques—canonicalization, reordering LUT, and slice streaming—convert an otherwise impractical exponential-LUT design into a working system on real hardware. A plausible implication is that the work contributes not only a specific inference engine for low-bit DNNs, but also a broader design principle for memory-rich, low-logic accelerators: data structures that appear capacity-expensive in conventional accelerators may become attractive when they align with the memory hierarchy and reduce logic pressure.

In that sense, LOCALUT denotes more than a particular LUT organization. It is a DRAM-PIM inference system in which multiple MACs are packed into each lookup, redundant LUT entries are removed through canonicalization, weights are remapped by an auxiliary reordering LUT, and reusable LUT slices are streamed through the DRAM/SRAM hierarchy. On the reported UPMEM platform, this yields up to a 1.82× geometric-mean end-to-end speedup over prior LUT-based baselines while preserving exact low-bit arithmetic semantics under the chosen quantization scheme (Hong et al., 6 Apr 2026).