---
title: 'Bit-Column-Serial: Concepts & Techniques'
url: https://www.emergentmind.com/topics/bit-column-serial
type: topic
---

# Bit-Column-Serial: Concepts & Techniques

Searching arXiv for recent and foundational papers on Bit-Column-Serial and closely related bit-serial/bit-column methods.
Bit-column-serial denotes a class of bit-serial representations, dataflows, and hardware organizations in which computation or storage is structured around **bit-columns**: sets of bits of equal significance drawn across multiple operands, weights, or memory elements. In matrix multiplication and DNN acceleration, bit-column-serial execution decomposes multi-bit operands into binary bit-planes or grouped weight columns and processes one bit-column per cycle, accumulating significance-weighted partial sums; in processing-in-memory, each physical memory column can act as an independent SIMD lane under bit-serial control; and in MRAM, serially connected devices can be interpreted as a “memory column” whose aggregate resistance encodes multiple levels [2603.14988; 2507.12444; 1806.08862; 2012.11890; 1903.08949].

## 1. Concept and scope

The core abstraction is the **bit-column**. In the matrix-multiplication formulation used by BISMO, a multi-bit matrix is decomposed into binary matrices $A^{[i]}$ and $B^{[j]}$, and full-precision multiplication is expressed as a weighted sum of binary matrix products:
$$
C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).
$$
This is the canonical bit-serial decomposition underlying several later systems [1806.08862].

A more specific use of the term appears in bitSMM, where a $B$-bit matrix multiply is decomposed into a **time sequence of bit-column multiplications**. Individual bit-columns of $A$ and $B$ are fed into a 2D grid of bit-serial MACs, and after $B$ bit-planes the array accumulates all $2B$ partial products into the full-precision result [2603.14988].

In DNN accelerators such as BitWave and SparseCol, the term refers not only to serial execution but also to **structured bit-level sparsity**. There, weights are grouped, examined column-wise in sign-magnitude form, and entire all-zero bit-columns are skipped. This preserves regularity in memory access and scheduling while reducing both memory footprint and redundant MAC work [2507.12444; 2606.16016].

A broader interpretation appears in in-memory and in-cache computing. SIMDRAM stores data in a vertical layout so that each DRAM column becomes an independent SIMD lane for bit-serial execution, while Neural Cache stores transposed bit-slices so that each SRAM column holds one entire element and can participate in in-situ bit-serial arithmetic [2012.11890; 1805.03718]. A distinct storage-oriented usage occurs in multi-bit MRAM, where serially connected pMTJs form one memory column and the total resistance encodes multiple states [1903.08949].

This suggests that “bit-column-serial” is best understood as a unifying organizational principle rather than a single algorithm: significance-aligned bits are exposed as first-class scheduling, compression, or storage units.

## 2. Algebraic and dataflow foundations

In bit-serial matrix multiplication, an operand is expanded into binary bit-planes. BISMO writes
$$
A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad
B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},
$$
so that multiplication reduces to $p \cdot q$ binary matrix multiplies plus a weighted sum [1806.08862]. For signed integers or two’s-complement fixed-point, the same pattern applies, with an extra sign weight of $\pm 1$ on the most significant bit-planes [1806.08862].

bitSMM uses a more constrained bit-column schedule. For matrices $A$ and $B$ quantized to $B$ bits, each element is written as
$$
a_i = \sum_{b=0}^{B-1} 2^b a_i^{(b)}, \qquad
b_j = \sum_{b=0}^{B-1} 2^b b_j^{(b)},
$$
with $a_i^{(b)}, b_j^{(b)} \in \{0,1\}$ [2603.14988]. At MAC position $(r,c)$ the local bit-serial MAC computes
$$
p_{ij}^{(b)} = a_i^{(b)} \cdot b_j^{(b)},
$$
and the final dot product is accumulated as
$$
C_{rc} = \sum_{b=0}^{B-1} 2^b \left(\sum_{i=0}^{N-1} a_i^{(b)} b_j^{(b)}\right).
$$
The defining operational feature is that the array accepts streams of bit-columns over time: $A$’s bits flow down columns MSb-first, while $B$’s bits flow across rows LSb-first [2603.14988].

BitWave formulates the same general idea at grouped weight level. For a group of $G$ weights, the $p$-th bit-column is
$$
B_p = [w_0[p], w_1[p], \dots, w_{G-1}[p]].
$$
If $B_p$ is all-zero, the column is skipped. Only non-zero columns are stored and streamed, with a $P$-bit index vector indicating which significance positions are present [2507.12444]. For each non-zero $p$,
$$
\mathrm{psum}[p] = \sum_{c=0}^{G-1} (w_c[p] \cdot a_c), \qquad
y_{\mathrm{group}} += (\mathrm{psum}[p] \ll p).
$$
Because the shift amount $p$ is shared by the whole group, one shifter and one adder suffice per column rather than per weight bit [2507.12444].

BBS adopts a related column abstraction for sparsity analysis. For fixed significance $b$, the set $\{W_i^b\}_{i=0..N-1}$ is the $b$th bit-column of a weight group, and the dot product is written as
$$
y=\sum_{b=0}^{p-1}2^b\Bigl(\sum_{i=0}^{N-1}W_i^bA_i\Bigr).
$$
This formulation supports column-wise zero/one pruning and compression [2409.05227].

Across these formulations, the common property is that significance is explicit in the execution schedule. The latency cost scales with precision, but the hardware cost of each primitive operation is reduced, and column-level regularity can be exploited for compression or skipping.

## 3. Microarchitectural realizations

A recurrent implementation strategy is the **systolic or spatial array of simple bit-serial units**. bitSMM uses an $R \times C$ grid of identical `bitSerialMAC` cells. Above each column is a parallel-to-serial loader for weights, and along each row is a similar loader for activations. Single-bit pipeline registers between adjacent MACs cause bit-columns to move “rightward across rows and downward across columns” [2603.14988]. After all bit-planes pass, an output-read enable traverses the array in a “snake” pattern, one MAC per cycle [2603.14988].

bitSMM evaluates two MAC variants. The **Booth-inspired multiplier** maintains a small Booth accumulator, inspects the current and previous multiplier bits, and issues “add $M$,” “subtract $M$,” or “no op,” followed by arithmetic right shift. The **standard binary multiplication with correction (SBMwC)** follows the textbook two’s-complement fix: ordinary shifted accumulation for all but the final multiplier bit, and subtraction of the properly shifted multiplicand at the MSb when needed [2603.14988]. These variants illustrate that bit-column-serial scheduling is compatible with multiple signed-arithmetic implementations.

BISMO realizes bit-serial matrix multiplication as a three-stage streaming pipeline: Fetch, Execute, and Result. The Execute stage is a $D_m \times D_n$ array of DPUs, each taking $D_k$ bits from row and column buffers, performing AND followed by popcount, applying a software-supplied weight $2^{i+j}$, and accumulating locally [1806.08862]. The optimized BISMO version replaces a costly barrel shifter by **wavefront scheduling** over decreasing $i+j$, so the accumulator needs only a 1-bit shift between successive wavefronts [1901.00370].

BitWave’s architecture is organized around 512 Bit-Column-Serial Compute Engines (BCEs). Each BCE contains a sign-magnitude multiplier array, an adder tree, and one barrel shifter that applies the significance-based shift once to the accumulated column sum [2507.12444]. A **Zero-Column Index Parser (ZCIP)** streams the index, extracts sign information, and emits $(p,\mathrm{sync\_count})$ pairs to drive non-zero-column fetch and execution [2507.12444]. The architecture further supports seven spatial-unrolling configurations, selected offline by ZigZag, so that the 512 BCEs can be reshaped to match wide, shallow, narrow, deep, or depthwise layers [2507.12444].

SparseCol similarly maps grouped non-zero bit-columns to a PE array, but names its units **Sparse-Column Computation Units (SCUs)**. Each SCU contains an SMM (Sign-Magnitude Multiplier), a First Adder Tree, a Unified Shifter, and a Second Adder. The **Sparse Column Index Decoder (SCID)** is implemented as 128 parallel 8-bit “find-ones” units and generates shift amounts and sign-bit control for each SCU [2606.16016]. The paper attributes its efficiency in part to processing a group-level shift once rather than shifting individual bit-cells [2606.16016].

BitMoD extends the bit-column-serial idea to mixed numerical datatypes for LLMs. Its PE consumes one term per cycle from each of four adjacent weight columns and performs a four-stage pipeline: exponent alignment and sign generation, bit-serial mantissa multiply, accumulation with bit significance, and on-the-fly dequantization [2411.11745]. A unified term representation allows the same datapath to process INT8, INT6, FP4, FP3, and extended variants by changing the number of terms and the per-term fields [2411.11745].

A plausible implication is that bit-column-serial hardware is less defined by a single primitive than by the repeated use of three structural motifs: compact 1-bit or column-wise multipliers, shared significance handling, and explicit serialization over bit significance.

## 4. Structured sparsity, compression, and dynamic precision

A major development beyond early bit-serial matrix multiplication is the exploitation of **column-structured bit-level sparsity**. BitWave argues that existing bit-serial accelerators can skip zero bits but suffer from inefficient memory accesses because non-zero bits occur at irregular indices. Its alternative is to compress out all-zero columns within groups of sign-magnitude weights, store only non-zero columns, and drive a highly regular datapath with a small index vector [2507.12444].

The choice of **sign-magnitude** rather than two’s-complement is central in BitWave. The paper states that in two’s-complement many small negative weights consume multiple leading 1 bits, reducing co-occurring zero columns, whereas sign-magnitude raises bit-column sparsity in ResNet-18 conv2 from approximately $17\%$ to approximately $59\%$ [2507.12444]. Group size $G$ governs the trade-off between index overhead and the probability that a whole column is zero; typical $G=8,16,32$ were found best [2507.12444].

BitWave also introduces a post-training **Bit-Flip** optimization. For a candidate layer and a larger target zero-column count, it adjusts individual weights by flipping up to one bit in sign-magnitude representation to satisfy the new zero-column count while minimizing $\ell_2$ distance to the original weight. If accuracy on a calibration set does not degrade, the move is accepted [2507.12444]. The reported effect is an increase of $+0.5$ to $1$ zero columns per group and an additional compression-ratio gain of approximately $1.1\times$ to $1.6\times$ with less than $0.5\%$ accuracy or PESQ/F1 drop [2507.12444].

SparseCol formalizes **bit-column sparsity** as a ratio over grouped weights:
$$
S_{bc} = \frac{N_{zero}}{N_g \cdot W_p},
$$
where $N_{zero}$ counts zero columns among groups and bit positions [2606.16016]. The paper contrasts this with unstructured bit-level sparsity, emphasizing that zero-column skipping creates a small set of structured zero patterns per group and simplifies index/vector decode [2606.16016]. It likewise uses sign-magnitude plus optional post-training Bit-Flip, but stresses that the method requires no retraining [2606.16016].

BBS generalizes the sparsity view by introducing **bi-directional bit-level sparsity**. For a column $W^b$, zero-bit sparsity $S_0^b$ and one-bit sparsity $S_1^b$ satisfy $S_1^b = 1 - S_0^b$, so one can either skip zeros or invert the column and skip ones. The BBS sparsity is
$$
S_{\mathrm{BBS}}^b = \max(S_0^b, S_1^b) \ge \tfrac{1}{2}.
$$
This guarantees that every bit-column has at least $50\%$ sparsity [2409.05227]. BBS adds two post-training binary-pruning methods, Rounded Column Averaging and Zero-Point Shifting, and packs the resulting metadata into an 8-bit word for hardware simplicity [2409.05227].

Dynamic precision is another recurring property. bitSMM supports runtime-configurable operand precision from 1 to 16 bits [2603.14988]. BISMO similarly supports any integer or fixed-point widths at runtime with one hardware instance [1806.08862]. SparseCol and BitWave both exploit the fact that every bit-column costs one cycle or one column step, so reducing effective precision directly reduces work [2606.16016; 2507.12444]. BitMoD extends that principle from precision to datatype, using a unified bit-serial representation for multiple low-bit integer and floating-point formats [2411.11745].

## 5. Processing-in-memory, in-cache, and storage interpretations

Bit-column-serial organization is not confined to standalone accelerators. In SIMDRAM, the defining transformation is a **vertical layout** in which each bit-slice of every word is stored in a single DRAM column. Each DRAM column then behaves as an independent SIMD lane, and multi-bit operations are computed bit-serially using majority-based logic composed from triple-row activation and inversion [2012.11890]. The framework compiles arbitrary operations into sequences of MAJ and NOT, maps inputs to rows, and executes them inside DRAM under controller-managed timing constraints [2012.11890].

Neural Cache applies an analogous idea in SRAM arrays. Data are stored in a transposed bit-slice layout so that each column holds one entire element and each bit-slice occupies a distinct word-line. By asserting selected word-lines, the array performs in-situ AND and bit-serial addition; the full dot product is decomposed into $B^2$ bit phases over significance pairs $(k,\ell)$ [1805.03718]. The architecture uses per-column accumulators, carry latches, and a small slice-level FSM so that thousands of subarrays execute the same bit-serial instruction [1805.03718].

PIMSAB places a 1-bit processing element on each SRAM bit-line in a CRAM array and streams operands bit-plane by bit-plane. The architecture emphasizes that bit-serial computation is “divisible,” supporting adaptive precision, bit slicing for large widths, efficient constant operations, shuffle logic, and hierarchical communication via an H-tree and mesh NoC [2311.11384]. Although the paper does not foreground “bit-column” in the same sparsity-centric sense as BitWave or SparseCol, it clearly adopts a column-oriented bit-serial compute model [2311.11384].

The MRAM example is different in purpose but related in organization. In serially connected pMTJ columns, each pMTJ occupies one bit-position in a series string, and the total resistance
$$
R_{\mathrm{total}} = \sum_{i=1}^{N} R_i, \qquad R_i \in \{R_P, R_{AP}\}
$$
encodes one of $N+1$ resistance levels [1903.08949]. Thus $N$ junctions form one memory column storing up to $\log_2(N+1)$ bits. The authors describe this as a density-improving serial-column mapping, not a compute dataflow [1903.08949].

These cases show that the term can denote either a computational schedule over significance-aligned columns or a physical storage organization in which columns themselves become the locus of state encoding or SIMD execution.

## 6. Performance characteristics, trade-offs, and limitations

A central performance property of bit-column-serial computation is **inverse scaling with precision**. bitSMM gives, for one $R \times C$ portion of an $M \times P$ multiply with $N=R$ at $B$ bits,
$$
T_{\mathrm{compute}} = (N+1)B,\qquad
T_{\mathrm{readout}} = RC,\qquad
T_{\mathrm{total}} = (N+1)B + RC.
$$
Its peak throughput is
$$
\mathrm{OPS}_{\mathrm{peak}} = \frac{R \cdot C \cdot f_{\mathrm{clk}}}{B},
$$
or equivalently
$$
\mathrm{GOPS} = (R \times C / B)\times (f_{\mathrm{clk}}/10^9).
$$
Hence halving $B$ approximately doubles throughput [2603.14988].

On an AMD ZCU104 FPGA at $300$ MHz with a $64 \times 16$ array and Booth MAC, bitSMM reports **19.2 GOPS peak** and **2.973 GOPS/W**; in ASAP7 at $1$ GHz and $64 \times 16$, it reports **73.22 GOPS**, **40.8 GOPS/W**, and **552 GOPS/mm$^2$** [2603.14988]. These figures quantify the efficiency of a compact bit-column systolic array under runtime-configurable precision.

The original BISMO reports a peak binary throughput of **6.554 TOPS** and **1.4 binary TOPS/W** on a Xilinx PYNQ-Z1 configuration with an $8 \times 8$ DPU array, $D_k=256$, and $200$ MHz clock [1806.08862]. The improved BISMO reaches **15.36 T binary OPS/s** on Ultra96 at $300$ MHz and **2.13 TOPS/W**, while larger out-of-context synthesis on VU9P scales to **783 binary TOPS** [1901.00370]. Both papers emphasize the benefit of pipeline overlap and wide binary datapaths, but also note the cost that a $w \times a$ multiply takes roughly $w \cdot a$ times as many cycles as a binary multiply [1806.08862; 1901.00370].

For structured-sparsity accelerators, performance is driven not only by precision but also by the number of non-zero columns. BitWave evaluates ResNet-18, MobileNetV2, a CNN-LSTM speech denoiser, and BERT-Base, reporting up to **13.25$\times$ speedup** and **7.71$\times$ efficiency** relative to state-of-the-art sparsity-aware accelerators, with **1.138 mm$^2$** area and **17.56 mW** power in 16 nm [2507.12444]. SparseCol reports **1320 BTOPS/W** peak efficiency, **745.02 BTOPS/W** system-level efficiency on CNN classification tasks, and **850.5 BTOPS/W** on transformer architectures, outperforming state-of-the-art sparse processors in efficiency by **6.8$\times$** [2606.16016].

BBS reports an average **1.66$\times$ reduction in model size**, up to **3.03$\times$ speedup**, and **2.44$\times$ energy saving** compared to prior DNN accelerators [2409.05227]. BitMoD reports average speedups of **1.69$\times$** and **1.48$\times$** versus ANT and OliVe, respectively, while using a bit-serial PE to support multiple numerical datatypes with a small encoder overhead [2411.11745].

The limitations are equally recurrent. BISMO notes higher latency and memory-bandwidth pressure because bit-serial execution streams many small bit-planes [1806.08862]. SparseCol identifies a trade-off in prior bit-serial sparsity schemes between irregular memory/computation patterns and complex online scheduling logic [2606.16016]. BitWave similarly argues that existing bit-level sparsity accelerators suffer from inefficient memory access when non-zero bits have irregular indices [2507.12444]. In the MRAM case, scaling to higher bit counts narrows write windows because switching-voltage overlap and process variations become more severe [1903.08949].

This suggests that the primary research trajectory in bit-column-serial systems is not merely faster serialization, but the reconciliation of three constraints: precision scalability, regular memory access, and high utilization under sparsity.

Source: https://www.emergentmind.com/topics/bit-column-serial