Bit-Column-Serial: Concepts & Techniques

• Bit-Column-Serial is a technique that decomposes multi-bit operands into binary bit-columns, aligning significance for precise, serial processing.
• It leverages structured sparsity and grouped operations to reduce memory access overhead while improving throughput and energy efficiency.
• The approach underpins diverse architectures—from matrix multiplication in DNN accelerators to in-memory computation—with dynamic precision and scalable designs.

Searching arXiv for recent and foundational papers on Bit-Column-Serial and closely related bit-serial/bit-column methods. Bit-column-serial denotes a class of bit-serial representations, dataflows, and hardware organizations in which computation or storage is structured around bit-columns: sets of bits of equal significance drawn across multiple operands, weights, or memory elements. In matrix multiplication and DNN acceleration, bit-column-serial execution decomposes multi-bit operands into binary bit-planes or grouped weight columns and processes one bit-column per cycle, accumulating significance-weighted partial sums; in processing-in-memory, each physical memory column can act as an independent SIMD lane under bit-serial control; and in MRAM, serially connected devices can be interpreted as a “memory column” whose aggregate resistance encodes multiple levels (Antunes et al., 16 Mar 2026, Shi et al., 16 Jul 2025, Umuroglu et al., 2018, Hajinazar et al., 2020, Rzeszut et al., 2019).

1. Concept and scope

The core abstraction is the bit-column. In the matrix-multiplication formulation used by BISMO, a multi-bit matrix is decomposed into binary matrices $A^{[i]}$ and $B^{[j]}$, and full-precision multiplication is expressed as a weighted sum of binary matrix products:

$$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$

This is the canonical bit-serial decomposition underlying several later systems (Umuroglu et al., 2018).

A more specific use of the term appears in bitSMM, where a $B$-bit matrix multiply is decomposed into a time sequence of bit-column multiplications. Individual bit-columns of $A$ and $B$ are fed into a 2D grid of bit-serial MACs, and after $B$ bit-planes the array accumulates all $2B$ partial products into the full-precision result (Antunes et al., 16 Mar 2026).

In DNN accelerators such as BitWave and SparseCol, the term refers not only to serial execution but also to structured bit-level sparsity. There, weights are grouped, examined column-wise in sign-magnitude form, and entire all-zero bit-columns are skipped. This preserves regularity in memory access and scheduling while reducing both memory footprint and redundant MAC work (Shi et al., 16 Jul 2025, Shi et al., 14 Jun 2026).

A broader interpretation appears in in-memory and in-cache computing. SIMDRAM stores data in a vertical layout so that each DRAM column becomes an independent SIMD lane for bit-serial execution, while Neural Cache stores transposed bit-slices so that each SRAM column holds one entire element and can participate in in-situ bit-serial arithmetic (Hajinazar et al., 2020, Eckert et al., 2018). A distinct storage-oriented usage occurs in multi-bit MRAM, where serially connected pMTJs form one memory column and the total resistance encodes multiple states (Rzeszut et al., 2019).

This suggests that “bit-column-serial” is best understood as a unifying organizational principle rather than a single algorithm: significance-aligned bits are exposed as first-class scheduling, compression, or storage units.

2. Algebraic and dataflow foundations

In bit-serial matrix multiplication, an operand is expanded into binary bit-planes. BISMO writes

$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$

so that multiplication reduces to $p \cdot q$ binary matrix multiplies plus a weighted sum (Umuroglu et al., 2018). For signed integers or two’s-complement fixed-point, the same pattern applies, with an extra sign weight of $B^{[j]}$0 on the most significant bit-planes (Umuroglu et al., 2018).

bitSMM uses a more constrained bit-column schedule. For matrices $B^{[j]}$1 and $B^{[j]}$2 quantized to $B^{[j]}$3 bits, each element is written as

$B^{[j]}$4

with $B^{[j]}$5 (Antunes et al., 16 Mar 2026). At MAC position $B^{[j]}$6 the local bit-serial MAC computes

$B^{[j]}$7

and the final dot product is accumulated as

$B^{[j]}$8

The defining operational feature is that the array accepts streams of bit-columns over time: $B^{[j]}$9’s bits flow down columns MSb-first, while $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$0’s bits flow across rows LSb-first (Antunes et al., 16 Mar 2026).

BitWave formulates the same general idea at grouped weight level. For a group of $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$1 weights, the $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$2-th bit-column is

$$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$3

If $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$4 is all-zero, the column is skipped. Only non-zero columns are stored and streamed, with a $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$5-bit index vector indicating which significance positions are present (Shi et al., 16 Jul 2025). For each non-zero $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$6,

$$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$7

Because the shift amount $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$8 is shared by the whole group, one shifter and one adder suffice per column rather than per weight bit (Shi et al., 16 Jul 2025).

BBS adopts a related column abstraction for sparsity analysis. For fixed significance $$C = \sum_{i=0}^{p-1}\sum_{j=0}^{q-1} 2^{i+j} \cdot (A^{[i]} \cdot B^{[j]}).$$9, the set $B$0 is the $B$1th bit-column of a weight group, and the dot product is written as

$B$2

This formulation supports column-wise zero/one pruning and compression (Chen et al., 2024).

Across these formulations, the common property is that significance is explicit in the execution schedule. The latency cost scales with precision, but the hardware cost of each primitive operation is reduced, and column-level regularity can be exploited for compression or skipping.

3. Microarchitectural realizations

A recurrent implementation strategy is the systolic or spatial array of simple bit-serial units. bitSMM uses an $B$3 grid of identical bitSerialMAC cells. Above each column is a parallel-to-serial loader for weights, and along each row is a similar loader for activations. Single-bit pipeline registers between adjacent MACs cause bit-columns to move “rightward across rows and downward across columns” (Antunes et al., 16 Mar 2026). After all bit-planes pass, an output-read enable traverses the array in a “snake” pattern, one MAC per cycle (Antunes et al., 16 Mar 2026).

bitSMM evaluates two MAC variants. The Booth-inspired multiplier maintains a small Booth accumulator, inspects the current and previous multiplier bits, and issues “add $B$4,” “subtract $B$5,” or “no op,” followed by arithmetic right shift. The standard binary multiplication with correction (SBMwC) follows the textbook two’s-complement fix: ordinary shifted accumulation for all but the final multiplier bit, and subtraction of the properly shifted multiplicand at the MSb when needed (Antunes et al., 16 Mar 2026). These variants illustrate that bit-column-serial scheduling is compatible with multiple signed-arithmetic implementations.

BISMO realizes bit-serial matrix multiplication as a three-stage streaming pipeline: Fetch, Execute, and Result. The Execute stage is a $B$6 array of DPUs, each taking $B$7 bits from row and column buffers, performing AND followed by popcount, applying a software-supplied weight $B$8, and accumulating locally (Umuroglu et al., 2018). The optimized BISMO version replaces a costly barrel shifter by wavefront scheduling over decreasing $B$9, so the accumulator needs only a 1-bit shift between successive wavefronts (Umuroglu et al., 2019).

BitWave’s architecture is organized around 512 Bit-Column-Serial Compute Engines (BCEs). Each BCE contains a sign-magnitude multiplier array, an adder tree, and one barrel shifter that applies the significance-based shift once to the accumulated column sum (Shi et al., 16 Jul 2025). A Zero-Column Index Parser (ZCIP) streams the index, extracts sign information, and emits $A$0 pairs to drive non-zero-column fetch and execution (Shi et al., 16 Jul 2025). The architecture further supports seven spatial-unrolling configurations, selected offline by ZigZag, so that the 512 BCEs can be reshaped to match wide, shallow, narrow, deep, or depthwise layers (Shi et al., 16 Jul 2025).

SparseCol similarly maps grouped non-zero bit-columns to a PE array, but names its units Sparse-Column Computation Units (SCUs). Each SCU contains an SMM (Sign-Magnitude Multiplier), a First Adder Tree, a Unified Shifter, and a Second Adder. The Sparse Column Index Decoder (SCID) is implemented as 128 parallel 8-bit “find-ones” units and generates shift amounts and sign-bit control for each SCU (Shi et al., 14 Jun 2026). The paper attributes its efficiency in part to processing a group-level shift once rather than shifting individual bit-cells (Shi et al., 14 Jun 2026).

BitMoD extends the bit-column-serial idea to mixed numerical datatypes for LLMs. Its PE consumes one term per cycle from each of four adjacent weight columns and performs a four-stage pipeline: exponent alignment and sign generation, bit-serial mantissa multiply, accumulation with bit significance, and on-the-fly dequantization (Chen et al., 2024). A unified term representation allows the same datapath to process INT8, INT6, FP4, FP3, and extended variants by changing the number of terms and the per-term fields (Chen et al., 2024).

A plausible implication is that bit-column-serial hardware is less defined by a single primitive than by the repeated use of three structural motifs: compact 1-bit or column-wise multipliers, shared significance handling, and explicit serialization over bit significance.

4. Structured sparsity, compression, and dynamic precision

A major development beyond early bit-serial matrix multiplication is the exploitation of column-structured bit-level sparsity. BitWave argues that existing bit-serial accelerators can skip zero bits but suffer from inefficient memory accesses because non-zero bits occur at irregular indices. Its alternative is to compress out all-zero columns within groups of sign-magnitude weights, store only non-zero columns, and drive a highly regular datapath with a small index vector (Shi et al., 16 Jul 2025).

The choice of sign-magnitude rather than two’s-complement is central in BitWave. The paper states that in two’s-complement many small negative weights consume multiple leading 1 bits, reducing co-occurring zero columns, whereas sign-magnitude raises bit-column sparsity in ResNet-18 conv2 from approximately $A$1 to approximately $A$2 (Shi et al., 16 Jul 2025). Group size $A$3 governs the trade-off between index overhead and the probability that a whole column is zero; typical $A$4 were found best (Shi et al., 16 Jul 2025).

BitWave also introduces a post-training Bit-Flip optimization. For a candidate layer and a larger target zero-column count, it adjusts individual weights by flipping up to one bit in sign-magnitude representation to satisfy the new zero-column count while minimizing $A$5 distance to the original weight. If accuracy on a calibration set does not degrade, the move is accepted (Shi et al., 16 Jul 2025). The reported effect is an increase of $A$6 to $A$7 zero columns per group and an additional compression-ratio gain of approximately $A$8 to $A$9 with less than $B$0 accuracy or PESQ/F1 drop (Shi et al., 16 Jul 2025).

SparseCol formalizes bit-column sparsity as a ratio over grouped weights:

$B$1

where $B$2 counts zero columns among groups and bit positions (Shi et al., 14 Jun 2026). The paper contrasts this with unstructured bit-level sparsity, emphasizing that zero-column skipping creates a small set of structured zero patterns per group and simplifies index/vector decode (Shi et al., 14 Jun 2026). It likewise uses sign-magnitude plus optional post-training Bit-Flip, but stresses that the method requires no retraining (Shi et al., 14 Jun 2026).

BBS generalizes the sparsity view by introducing bi-directional bit-level sparsity. For a column $B$3, zero-bit sparsity $B$4 and one-bit sparsity $B$5 satisfy $B$6, so one can either skip zeros or invert the column and skip ones. The BBS sparsity is

$B$7

This guarantees that every bit-column has at least $B$8 sparsity (Chen et al., 2024). BBS adds two post-training binary-pruning methods, Rounded Column Averaging and Zero-Point Shifting, and packs the resulting metadata into an 8-bit word for hardware simplicity (Chen et al., 2024).

Dynamic precision is another recurring property. bitSMM supports runtime-configurable operand precision from 1 to 16 bits (Antunes et al., 16 Mar 2026). BISMO similarly supports any integer or fixed-point widths at runtime with one hardware instance (Umuroglu et al., 2018). SparseCol and BitWave both exploit the fact that every bit-column costs one cycle or one column step, so reducing effective precision directly reduces work (Shi et al., 14 Jun 2026, Shi et al., 16 Jul 2025). BitMoD extends that principle from precision to datatype, using a unified bit-serial representation for multiple low-bit integer and floating-point formats (Chen et al., 2024).

5. Processing-in-memory, in-cache, and storage interpretations

Bit-column-serial organization is not confined to standalone accelerators. In SIMDRAM, the defining transformation is a vertical layout in which each bit-slice of every word is stored in a single DRAM column. Each DRAM column then behaves as an independent SIMD lane, and multi-bit operations are computed bit-serially using majority-based logic composed from triple-row activation and inversion (Hajinazar et al., 2020). The framework compiles arbitrary operations into sequences of MAJ and NOT, maps inputs to rows, and executes them inside DRAM under controller-managed timing constraints (Hajinazar et al., 2020).

Neural Cache applies an analogous idea in SRAM arrays. Data are stored in a transposed bit-slice layout so that each column holds one entire element and each bit-slice occupies a distinct word-line. By asserting selected word-lines, the array performs in-situ AND and bit-serial addition; the full dot product is decomposed into $B$9 bit phases over significance pairs $B$0 (Eckert et al., 2018). The architecture uses per-column accumulators, carry latches, and a small slice-level FSM so that thousands of subarrays execute the same bit-serial instruction (Eckert et al., 2018).

PIMSAB places a 1-bit processing element on each SRAM bit-line in a CRAM array and streams operands bit-plane by bit-plane. The architecture emphasizes that bit-serial computation is “divisible,” supporting adaptive precision, bit slicing for large widths, efficient constant operations, shuffle logic, and hierarchical communication via an H-tree and mesh NoC (Arora et al., 2023). Although the paper does not foreground “bit-column” in the same sparsity-centric sense as BitWave or SparseCol, it clearly adopts a column-oriented bit-serial compute model (Arora et al., 2023).

The MRAM example is different in purpose but related in organization. In serially connected pMTJ columns, each pMTJ occupies one bit-position in a series string, and the total resistance

$B$1

encodes one of $B$2 resistance levels (Rzeszut et al., 2019). Thus $B$3 junctions form one memory column storing up to $B$4 bits. The authors describe this as a density-improving serial-column mapping, not a compute dataflow (Rzeszut et al., 2019).

These cases show that the term can denote either a computational schedule over significance-aligned columns or a physical storage organization in which columns themselves become the locus of state encoding or SIMD execution.

6. Performance characteristics, trade-offs, and limitations

A central performance property of bit-column-serial computation is inverse scaling with precision. bitSMM gives, for one $B$5 portion of an $B$6 multiply with $B$7 at $B$8 bits,

$B$9

Its peak throughput is

$2B$0

or equivalently

$2B$1

Hence halving $2B$2 approximately doubles throughput (Antunes et al., 16 Mar 2026).

On an AMD ZCU104 FPGA at $2B$3 MHz with a $2B$4 array and Booth MAC, bitSMM reports 19.2 GOPS peak and 2.973 GOPS/W; in ASAP7 at $2B$5 GHz and $2B$6, it reports 73.22 GOPS, 40.8 GOPS/W, and 552 GOPS/mm$2B$7 (Antunes et al., 16 Mar 2026). These figures quantify the efficiency of a compact bit-column systolic array under runtime-configurable precision.

The original BISMO reports a peak binary throughput of 6.554 TOPS and 1.4 binary TOPS/W on a Xilinx PYNQ-Z1 configuration with an $2B$8 DPU array, $2B$9, and $$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$0 MHz clock (Umuroglu et al., 2018). The improved BISMO reaches 15.36 T binary OPS/s on Ultra96 at $$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$1 MHz and 2.13 TOPS/W, while larger out-of-context synthesis on VU9P scales to 783 binary TOPS (Umuroglu et al., 2019). Both papers emphasize the benefit of pipeline overlap and wide binary datapaths, but also note the cost that a $$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$2 multiply takes roughly $$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$3 times as many cycles as a binary multiply (Umuroglu et al., 2018, Umuroglu et al., 2019).

For structured-sparsity accelerators, performance is driven not only by precision but also by the number of non-zero columns. BitWave evaluates ResNet-18, MobileNetV2, a CNN-LSTM speech denoiser, and BERT-Base, reporting up to 13.25$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$4 speedup and 7.71$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$5 efficiency relative to state-of-the-art sparsity-aware accelerators, with 1.138 mm$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$6 area and 17.56 mW power in 16 nm (Shi et al., 16 Jul 2025). SparseCol reports 1320 BTOPS/W peak efficiency, 745.02 BTOPS/W system-level efficiency on CNN classification tasks, and 850.5 BTOPS/W on transformer architectures, outperforming state-of-the-art sparse processors in efficiency by 6.8$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$7 (Shi et al., 14 Jun 2026).

BBS reports an average 1.66$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$8 reduction in model size, up to 3.03$$A = \sum_{i=0}^{p-1} 2^i \cdot A^{[i]}, \qquad B = \sum_{j=0}^{q-1} 2^j \cdot B^{[j]},$$9 speedup, and 2.44$p \cdot q$0 energy saving compared to prior DNN accelerators (Chen et al., 2024). BitMoD reports average speedups of 1.69$p \cdot q$1 and 1.48$p \cdot q$2 versus ANT and OliVe, respectively, while using a bit-serial PE to support multiple numerical datatypes with a small encoder overhead (Chen et al., 2024).

The limitations are equally recurrent. BISMO notes higher latency and memory-bandwidth pressure because bit-serial execution streams many small bit-planes (Umuroglu et al., 2018). SparseCol identifies a trade-off in prior bit-serial sparsity schemes between irregular memory/computation patterns and complex online scheduling logic (Shi et al., 14 Jun 2026). BitWave similarly argues that existing bit-level sparsity accelerators suffer from inefficient memory access when non-zero bits have irregular indices (Shi et al., 16 Jul 2025). In the MRAM case, scaling to higher bit counts narrows write windows because switching-voltage overlap and process variations become more severe (Rzeszut et al., 2019).

This suggests that the primary research trajectory in bit-column-serial systems is not merely faster serialization, but the reconciliation of three constraints: precision scalability, regular memory access, and high utilization under sparsity.