---
title: 'TLMM: Table-Lookup Ternary MatMul'
url: https://www.emergentmind.com/topics/table-lookup-based-ternary-matrix-multiplication-tlmm
type: topic
---

# TLMM: Table-Lookup Ternary MatMul

Searching arXiv for recent papers on table-lookup-based ternary matrix multiplication and related fast matrix multiplication schemes.
Table-Lookup-Based Ternary Matrix Multiplication (TLMM) denotes a family of matrix-multiplication methods in which ternary structure—typically weights or coefficients in $\{-1,0,+1\}$—is exploited so that direct multiplications are replaced by table lookups, additions, subtractions, and sign handling. In the systems literature, TLMM is used for mixed-precision GEMM and GEMV with ternary weights and higher-precision activations, especially for low-bit neural inference on CPUs, NPUs, ASICs, and FPGAs [2407.00088; 2510.15926; 2511.21910]. In the algebraic literature, ternary matrix-multiplication schemes over $\mathbb{Z}_T=\{-1,0,1\}$ are valued because they “correspond to simple additions and subtractions without multiplication by constants,” making them directly suitable for efficient implementation and especially valuable for TLMM-style realization [2603.02398; 2606.02480].

## 1. Computational model

A canonical TLMM workflow is organized as **Phase 1: LUT Build** and **Phase 2: Fetch and Accumulate**. For matrix-vector or matrix-matrix multiplication with ternary weights, the activation vector is partitioned into groups of size $\mu$ (or $G$), all $3^\mu$ possible partial sums are computed for each group, and the resulting values are stored in a lookup table. During execution, grouped ternary weights serve as LUT indices, and the matrix product is reconstructed by accumulating the selected entries [2604.25183; 2510.15926].

For the hardware design-space formalization, if $\mathbf{w}\in\{-1,0,+1\}^n$ is partitioned into $L$ groups of size $\mu$, the LUT-build stage computes all partial dot products for each group, and the fetch stage evaluates
\[
y = \sum_{\text{groups } i=1}^{L} \text{LUT}_i[\text{index}(w_{i,1},...,w_{i,\mu})].
\]
The corresponding compute tile is parameterized as
\[
n \times m = (L\cdot\mu)\times K,
\]
where $\mu$ is the LUT group size, $L$ the number of parallel LUT blocks, and $K$ the number of parallel fetch units per LUT [2604.25183].

TeLLMe v2 gives the same principle in grouped-activation form. With ternary weights partitioned into groups of size $G$, each group generates $3^G$ possible sums, packed offline as indices and realized online through a table:
\[
\mathrm{TL}[idx] = \sum_{g=1}^G w_g^{(idx)} \cdot a_g,
\qquad
o_k = \sum_{l=1}^{L} \mathrm{TL}^{(l)}[idx_{l,k}].
\]
Here, the runtime kernel becomes a repeated sequence of table construction from activations and indexed accumulation from preprocessed ternary weight groups [2510.15926].

A broader formulation appears in T-MAC for mixed-precision GEMM. Low-bit weights are decomposed into bitplanes,
\[
A \times W = \sum_{i=0}^{n-1} 2^i (A \times W_i),
\]
and each bit-slice computation is replaced by group-wise table lookups. TLMM is the ternary specialization of this lookup-based matrix-multiplication paradigm [2407.00088].

## 2. Encodings, symmetry, and local arithmetic

TLMM depends critically on how ternary values are encoded and how lookup-table redundancy is removed. One direct software realization uses a **2-bit encoding for ternary values**:
- \(1 \rightarrow (x^+, x^-) = (1, 0)\)
- \(0 \rightarrow (0, 0)\)
- \(-1 \rightarrow (0, 1)\)

Each matrix is represented by separate “plus” and “minus” bitplanes. For ternary–ternary multiplication,
\[
(z^+, z^-) = ((x^+ \land y^+) \lor (x^- \land y^-),\; (x^+ \land y^-) \lor (x^- \land y^+)),
\]
and the scalar dot-product contribution is accumulated as
\[
c = \sum_{t=1}^{k} (z^+ - z^-).
\]
This implementation uses Boolean logic, SIMD, and popcount rather than explicit precomputed lookup tables, but it is described as conceptually similar to lookup-based processing and is relevant because it shows that ternary matrix multiplication can be reduced to sign-separated bit manipulations [2205.09120].

In lookup-centric accelerators, symmetry reduction is central. Platinum chooses chunk size $c=5$ for ternary weights, so a raw LUT has $3^5=243$ entries, then applies **mirror consolidation** so that only one of each symmetric entry pair is stored, reducing storage to $\lceil 3^c/2 \rceil$ entries per LUT. Platinum also packs $c$ ternary weights into a base-3 integer, described as **1.6 bits/weight**, to minimize memory footprint and align LUT indices for sequential access [2511.21910].

The hardware generator for 1.58-bit LLM inference formalizes three recurring optimizations: **Symmetry**, **Redundancy**, and **Sparsity**. Symmetry stores only half the LUT entries and reconstructs negative results by sign flipping; redundancy reuses common sub-expressions in the adder network; sparsity exploits zeros in ternary weights to skip unnecessary additions. Its adder count per LUT is bounded by
\[
\frac{\#\text{adders}}{\text{LUT}} \leq (\mu - 1)\cdot \frac{3^{\mu} - 1}{2} - R(\mu) - \mu \cdot S(\mu),
\]
with $R(\mu)$ and $S(\mu)$ denoting savings from redundancy and sparsity [2604.25183].

Related software–hardware co-design work uses **table symmetrization** to halve table size and **table quantization** to reduce LUT storage. In the bit-serial LUT Tensor Core, table symmetrization exploits signed low-bit symmetry so that only half the entries are stored and the remainder are obtained by sign handling [2408.06003].

## 3. Architectural realizations

The published systems span general-purpose CPUs, mobile NPUs, FPGA soft logic, edge-FPGA LLM accelerators, and dedicated ASICs. Their common objective is to eliminate dequantization or direct multiply-add execution for ternary or ultra-low-bit matrix multiplication.

| Work | Platform | Representative result |
|---|---|---|
| T-MAC [2407.00088] | CPU | up to 4x throughput, 70% energy reduction |
| Platinum [2511.21910] | ASIC | up to 73.6x speedup, 32.4x energy reduction, 0.96mm2 |
| T-MAN [2511.11248] | NPU | 1.4x prefill, 3.1x decoding, 84% energy savings |
| TeLLMe v2 [2510.15926] | Edge FPGA | up to 25 tokens/s, 0.45--0.96 s TTFT, under 5 W |

On CPUs, T-MAC introduces LUT-based kernels for mixed-precision GEMM without dequantization and reports **up to 4x increase in throughput and 70% reduction in energy consumption compared to llama.cpp**. For **BitNet-b1.58-3B**, it reports **30 tokens/s with a single core and 71 tokens/s with eight cores on M2-Ultra, and 11 tokens/s on Raspberry Pi 5** [2407.00088].

On ARM CPUs, a different fast ternary matrix-multiplication path uses NEON SIMD and bitplane logic. The reported outcome is that **TNN** and **TBN** matrix multiplications are **3.6 times faster than full-precision, 2.5 times faster than 8-bit, and 1.4 times faster than 4-bit matrix multiplication** on ARM Cortex-A73, while remaining about **2.9 times slower than the authors' binary matrix multiplication** [2205.09120].

On NPUs, T-MAN uses table lookup to subsume unsupported low-bit operations through **fused two-level table-based dequantization** and **concurrency-hierarchy-guided tiling**. It reports **1.4x and 3.1x speedup for prefill and decoding respectively, and 84% energy savings compared to the baseline NPU methods** [2511.11248].

On edge FPGAs, TeLLMe is described as **the first table-lookup-based ternary LLM accelerator for low-power edge FPGAs that fully supports both prefill and autoregressive decoding using 1.58-bit weights and 8-bit activations**. Under a **5 W power budget**, it reports **up to 25 tokens/s decoding throughput** and **0.45--0.96 s time-to-first-token for 64--128 token prompts** [2510.15926].

On ASICs, Platinum replaces dynamic shortcut detection with offline-generated construction paths and path-adaptable execution. On **BitNet b1.58-3B**, it reports **up to 73.6x, 4.09x, and 2.15x speedups** over **SpikingEyeriss, Prosperity, and 16-thread T-MAC (CPU)**, and **32.4x, 3.23x, and 20.9x** energy reductions, **all within a 0.96mm2 chip area** [2511.21910].

FPGA soft-logic work generalizes the lookup idea beyond LLM kernels. TLMAC maps unique groups of quantized weights to lookup-based processing elements and reports that entire **ImageNet-scale quantised models with full-precision accuracy** can be implemented using lookup-based computing on one commercially available FPGA [2403.11414]. A separate all-digital MADDNESS-based accelerator reports **2.5 times higher energy efficiency (174 TOPS/W) and 5 times higher area efficiency (2.01 TOPS/mm2)** than the conventional accelerator in post-layout simulation [2506.16800].

## 4. Design-space trade-offs

A central issue in TLMM is the choice of LUT group size. The generator study formalizes the trade-off: **Increasing $\mu$ increases LUT build cost (exponentially), decreases accumulation cost (linearly)**. It further states that the **optimal architecture is fundamentally governed by the activation data type**: LUT-based reuse offers significant gains for **FP16**, while it yields **diminishing returns for small integer types** such as **INT8** [2604.25183].

The same study provides a parameterized area model,
\[
\text{Build+}_\text{cost} \sim \frac{3.069^\mu}{1.938} \cdot \frac{n}{\mu},\quad
\text{Accumulate+}_\text{cost} \sim \frac{n \cdot m}{\mu},\quad
\text{MUX}_\text{cost} \sim \frac{n \cdot m}{\mu} \cdot \frac{3^\mu-1}{2},
\]
and concludes that **maximizing core size consistently improves area density compared to highly tiled approaches**. Its optimized designs achieve a **2.2x area reduction compared to multiplier-based baselines**, and benchmarking against prior accelerators yields **up to a 1.2x area improvement** after correcting suboptimal parameters [2604.25183].

Platinum addresses a different bottleneck: LUT construction overhead. Traditional construction requires **$(k-1)2^k$ (or $3^k$) additions per chunk**, which becomes prohibitive as $k$ grows. Platinum computes the entire construction path **offline using minimum spanning tree (MST)** algorithms and stores the resulting **build_path** in SRAM, replacing runtime scheduling with a lightweight pipeline [2511.21910].

T-MAN highlights that prefill and decoding favor conflicting table layouts and tilings. Its solution is a unified layout that supports both the matrix core and vector units without maintaining separate weight copies. The design uses asynchronous DMA, on-chip TCM buffering, and vector-unit VLUT instructions for decoding [2511.11248].

The FPGA literature emphasizes routing and memory organization as first-order concerns. TLMAC uses spectral clustering and simulated annealing to reduce LUT utilization and routing congestion in soft logic [2403.11414]. TeLLMe v2 uses **fine-grained analytic URAM-based weight buffer management**, a **streaming dataflow architecture**, and specialized attention implementations to make the TLMM engine sustainable within an edge-FPGA resource budget [2510.15926].

## 5. Ternary fast matrix multiplication schemes and their relevance to TLMM

A separate but closely related line of work studies matrix-multiplication schemes whose bilinear coefficients are restricted to $\mathbb{Z}_T=\{-1,0,1\}$. In this setting, ternary coefficients are described as “highly desirable” because they require only addition and subtraction and avoid multiplication by constants. This makes them directly suitable for efficient implementation and especially relevant to TLMM [2603.02398].

The open-source flip graph framework supports **binary ($\mathbb{Z}_2$), modular ternary ($\mathbb{Z}_3$), and integer ternary ($\mathbb{Z}_T$)** coefficients and covers **680 schemes ranging from $(2 \times 2 \times 2)$ to $(16 \times 16 \times 16)$**. It reports **79 matrix multiplication schemes** with improved rank, **276 schemes now in $\mathbb{Z}_T$ coefficients**, **117 in integer coefficients**, and a new **$4 \times 4 \times 10$ scheme requiring only 115 multiplications**, with **$\omega\approx 2.80478$** [2603.02398].

The later meta flip graph and serendipitous product results extend the search to **all 680 rectangular formats with dimensions up to $16 \times 16 \times 16$** and improve ranks for **206 formats**. They report **84 new ternary schemes** where previously only integer or rational coefficients were known, and the distribution across all investigated formats is **375 ternary, 18 integer, and 287 rational**. The same work reports **23 new schemes with $\omega < \log_2 7$**, bringing the total to **52**, with **4** of the new schemes over $\mathbb{Z}_T$ [2606.02480].

At the level of individual constructions, an exact **56-addition, rank-23 scheme for general $3\times3$ matrix multiplication** uses **23 multiplications** and **56 additions/subtractions**, with all tensor coefficients in $\{-1,0,1\}$. Correctness is certified by the **729 Brent equations over $\mathbb{Z}$**, and the algorithm works over **arbitrary associative, possibly noncommutative, coefficient rings** [2604.27645].

A GPU-accelerated ternary meta flip graph study further maps present feasibility boundaries. It reports new best ranks for **$4 \times 5 \times 12$**, **$5 \times 6 \times 10$**, and **$6 \times 7 \times 9$**, and concludes from the analysis of **164 known schemes** that **92 can be implemented in $Z_T$**, while **72 could not be found in the ternary field with current methods** [2511.20317].

These results do not describe lookup-table hardware directly, but they enlarge the library of ternary coefficient schemes that can be realized using additions, subtractions, and sign flips. This suggests a tighter connection between algebraic matrix-multiplication optimization and practical TLMM deployment than is present for schemes with large integers or rationals [2606.02480].

## 6. Theoretical perspective, misconceptions, and open questions

TLMM is not synonymous with explicit precomputed tables in every implementation. A common misconception is that ternary acceleration always requires an actual lookup array. The ARM CPU work explicitly states that its ternary kernels **do not use explicit table lookup for ternary multiplication**; instead, ternary products are realized directly through Boolean logic, NEON vectorization, and popcount, while remaining conceptually similar to table-lookup acceleration [2205.09120].

Another misconception is that LUT-based ternary execution is uniformly optimal across data types and platforms. The architectural exploration for 1.58-bit LLM inference shows that the benefit is strongly activation-dependent: **while LUT-based reuse offers significant gains for high-cost arithmetic (e.g., FP16), it yields diminishing returns for small integer types** [2604.25183].

At the theoretical end, lookup tables also affect asymptotic models. Over a constant-sized finite field, any algebraic matrix-multiplication algorithm using $O(N^\omega)$ operations can be converted into one using
\[
O\!\left(\frac{N^\omega}{(\log N)^{\omega/2 - 1}}\right)
\]
bit operations, and **Strassen’s algorithm can be converted into an algorithm using $O(N^{2.81} / (\log N)^{0.4})$ bit operations** [2211.04643]. This result concerns bit complexity rather than accelerator microarchitecture, but it establishes that lookup operations can change the quantitative meaning of “fast matrix multiplication” even in classical algebraic settings [2211.04643].

Several practical bottlenecks recur across the literature. The all-digital MADDNESS accelerator attributes **over 94% energy and 50–80% area consumption** to the decoder/SRAM array as parameters scale [2506.16800]. TLMAC identifies routing congestion as a limiting factor for larger multi-bit FPGA deployments [2403.11414]. TeLLMe v2, T-MAN, and Platinum all devote substantial design effort to weight layout, precomputation scheduling, and on-chip buffering rather than only to the lookup operator itself [2510.15926; 2511.11248; 2511.21910].

The field is unusually open-source. Public artifacts include **T-MAC**, **T-MAN**, the **ternary_flip_graph** framework, **FastMatrixMultiplication**, and the **ternary-lut-dse** hardware generator [2407.00088; 2511.11248; 2603.02398; 2606.02480; 2604.25183]. A plausible implication is that TLMM research is converging toward a shared experimental base in which algorithmic ternary schemes, LUT-based execution strategies, and architecture-level cost models can be evaluated together rather than as isolated subfields.

Source: https://www.emergentmind.com/topics/table-lookup-based-ternary-matrix-multiplication-tlmm