BS-Sparsity-Enabled Two-State Coding (BSTC)

• BSTC is a bit-slice–native weight-compression technique that exploits high-order bit-slice sparsity to reduce weight-access latency in large language models.
• It groups bit-slices into fixed-size blocks, emitting a 0 header for all-zero groups and a 1 plus literal payload for non-zero groups, enabling lockstep de/compression with minimal overhead.
• Integrated within the MCBP accelerator, BSTC achieves roughly 2.7× DRAM bandwidth reduction and cuts decoding time by 35–40%, significantly accelerating inference.

Searching arXiv for the specified paper and closely related accelerator work to ground the article in current literature. BS-Sparsity-Enabled Two-State Coding (BSTC) is a bit-slice–native weight-compression method introduced as one of the three key innovations in MCBP, a bit-grained compute-memory efficient algorithm-hardware co-design for accelerating LLM inference (Wang et al., 12 Sep 2025). Within MCBP, BSTC targets the weight-access bottleneck in the decoding stage by exploiting significant sparsity in high-order bit-slice weight, rather than operating at the conventional value level. The method compresses each bit-slice independently, uses a two-state code that distinguishes all-zero groups from non-zero groups, and is co-designed with the bit-serial BRCR engine so that compression, decompression, and computation proceed in lockstep with zero realignment overhead (Wang et al., 12 Sep 2025).

1. Position within MCBP and the decoding bottleneck

MCBP is presented as a memory-compute efficient LLM inference accelerator that addresses three sources of inference latency: inefficiencies in GEMM operations, weight access, and KV cache access (Wang et al., 12 Sep 2025). Its three key innovations are BS-repetitiveness-enabled computation reduction (BRCR), BS-sparsity-enabled two-state coding (BSTC), and Bit-grained progressive prediction (BGPP). BSTC specifically addresses weight access by exploiting significant sparsity in high-order bit-slice weight (Wang et al., 12 Sep 2025).

The motivation for BSTC is tied to a limitation of existing Transformer accelerators and prior compression schemes. In the decoding stage, the dominating cost of weight-fetch has traditionally been attacked at the “value level,” including run-length coding, Huffman coding, and sparse-matrix formats, but these schemes leave unexploited sparsity that appears only after decomposing each 8-bit weight into individual bit-planes (Wang et al., 12 Sep 2025). When an LLM is quantized to 8 bits, value-level sparsity is typically only 5–10%. Moreover, value-centric layouts store all eight bits of every weight contiguously, so if a bit-serial GEMM engine needs only the MSB-slice, it must still read all eight bits and then perform bit-reorder in SRAM or logic to construct the 1-bit stream (Wang et al., 12 Sep 2025).

Section 2.2 of the source paper profiles Llama-7B and Qwen-7B and reports that MSB slices alone are 70–80% zeros, with empirical sparsity satisfying $SR_i \ge 65\%$ for $i \ge 3$ (Wang et al., 12 Sep 2025). This establishes the central premise of BSTC: a compression scheme designed at the bit-slice level can expose sparsity that value-level coding cannot. The paper further states that BSTC lines up perfectly with the bit-serial compute in the CAM-based BRCR engine, exploits the 60–80% zero-rates found in the high-order bit-slices, and costs almost zero extra reorder logic (Wang et al., 12 Sep 2025).

2. Bit-slice formulation and two-state mapping

BSTC begins from an 8-bit quantized weight matrix $W \in \mathbb{Z}^{H \times H}$, decomposed into $b = 8$ bit-slices according to

$$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$

For each bit-slice $W_i$, sparsity is measured as $SR_i = \#\text{zeros} / \#\text{entries}$ (Wang et al., 12 Sep 2025). Empirically, the paper reports $SR_i \ge 65\%$ for $i \ge 3$, and therefore only slices 3 through 7 are encoded, while slices 0, 1, and 2 remain uncompressed (Wang et al., 12 Sep 2025).

Within a given bit-slice $W_i$, each column of length $i \ge 3$0 is divided into fixed groups of $i \ge 3$1 rows. Each group is thus an $i \ge 3$2-bit vector $i \ge 3$3, with at most $i \ge 3$4 possible patterns per column (Wang et al., 12 Sep 2025). BSTC defines a two-state mapping on each such group:

• if $i \ge 3$5 then emit code “0”
• else emit code “1” $i \ge 3$6 $i \ge 3$7

where $i \ge 3$8 denotes bit-concatenation (Wang et al., 12 Sep 2025).

This definition gives the code its “two-state” character: the header bit indicates either an all-zero group or a non-zero group followed by a literal payload. The decoder therefore consumes a serial stream of 1-bit headers, where a ‘0’ means that the entire group is all zeros and a ‘1’ means that the next $i \ge 3$9 cycles contain literal bits (Wang et al., 12 Sep 2025). No further pointer or index information is needed. The paper states that choosing the same group size $W \in \mathbb{Z}^{H \times H}$0 for BSTC as for the BRCR merge engine, with $W \in \mathbb{Z}^{H \times H}$1 in the design, allows BSTC de/compression and compute to proceed in lockstep, with zero realignment overhead (Wang et al., 12 Sep 2025).

A concise summary of the coding rule is as follows.

Element	Definition
Weight decomposition	$W \in \mathbb{Z}^{H \times H}$2
Encoded slices	$W \in \mathbb{Z}^{H \times H}$3
Group structure	Column-wise groups of $W \in \mathbb{Z}^{H \times H}$4 rows
Zero group	Emit “0”
Non-zero group	Emit “1” $W \in \mathbb{Z}^{H \times H}$5 $W \in \mathbb{Z}^{H \times H}$6
Design group size	$W \in \mathbb{Z}^{H \times H}$7

This design directly couples the coding format to the bit-serial datapath. A plausible implication is that the format is not merely a compression layer but a representational choice intended to eliminate the conventional boundary between memory layout and compute layout.

3. Encoding and decoding procedure

The source presents high-level pseudocode for BSTC encoding and decoding on one bit-slice (Wang et al., 12 Sep 2025). Encoding processes the bit-slice in row-major chunks of $W \in \mathbb{Z}^{H \times H}$8. For each row block and each column, it extracts the corresponding $W \in \mathbb{Z}^{H \times H}$9-bit column group $b = 8$0. If $b = 8$1 is equal to $b = 8$2, it appends a single header bit 0; otherwise it appends a header bit 1 followed by the literal bits of $b = 8$3 (Wang et al., 12 Sep 2025).

Decoding reconstructs a bit-slice $b = 8$4 by scanning the input bitstream in the same row-block and column order. For each group, the decoder reads one header bit. If the header is 0, the corresponding group in $b = 8$5 remains $b = 8$6; if the header is 1, the decoder reads the next $b = 8$7 bits and writes them into the group (Wang et al., 12 Sep 2025).

Two implementation properties are emphasized. First, there are no pointers or variable-length integer codes: every group yields exactly 1 or $b = 8$8 bits (Wang et al., 12 Sep 2025). Second, the decoder can therefore be simple, clocked at one header per cycle with an $b = 8$9-bit shift-register to collect the next $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$0 bits (Wang et al., 12 Sep 2025). This distinguishes BSTC from value-level schemes that require explicit indexing or arbitrary-byte pattern handling.

Because the method encodes slices independently and groups are fixed-size, the representation remains structurally regular despite being variable-length at the group level. This regularity is central to the claim that the compressed stream is itself aligned to the single-bit serial datapath (Wang et al., 12 Sep 2025).

4. Hardware realization and parallel decompression

The hardware realization described for MCBP places BSTC within an SRAM-banked architecture. Figure 1(c) is described as showing an on-chip SRAM layout in which the $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$1 bit-slice is partitioned along the hidden dimension into $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$2 sub-matrices, one per SRAM bank (Wang et al., 12 Sep 2025). During offline weight-load, the controller writes each sub-matrix’s compressed stream into its bank at a known row address, and the start-addresses are stored in a tiny “address area” consisting of 3 SRAM rows (Wang et al., 12 Sep 2025).

At decode time, the controller performs four steps (Wang et al., 12 Sep 2025):

1. Reads start-address for bank $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$3.
2. Reads one SRAM row, 64 bits wide, per cycle.
3. Feeds each bit into a lightweight “two-state” decoder:
   • A 1-bit comparator checks for header == 0 versus header == 1.
   • On a ‘1’, a small serial-in parallel-out (SIPO) of width $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$4 collects the next $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$5 bits.
   • On a ‘0’, it simply outputs $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$6 zeros.
4. Steers the decoder’s $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$7-bit output into the bit-slice register file row_block.

The paper states that because every bank and every row block is aligned, all 20 decoders can run fully in parallel; for $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$8 this corresponds to 20 banks $$W = \sum_{i=0}^{b-1} 2^i \cdot W_i, \qquad W_i \in \{0,1\}^{H \times H}.$$9 4 rows/bank = 80 rows total (Wang et al., 12 Sep 2025). Under this organization, the entire $W_i$0 bit-slice can be decompressed in $W_i$1 cycles (Wang et al., 12 Sep 2025).

The architectural significance of this realization is that decompression is not treated as an off-critical-path preprocessing stage. Instead, it is integrated into the operating rhythm of the bit-serial engine. The paper explicitly attributes to BSTC nearly zero extra reorder logic and zero realignment overhead, both of which are consequences of the shared group granularity between coding and compute (Wang et al., 12 Sep 2025).

5. Quantitative effects on weight access and decoding latency

The paper reports several quantitative effects for BSTC in Section 6.2 and Figure 2 (Wang et al., 12 Sep 2025). Applying BSTC to the MSB slices, specifically $W_i$2, achieves on average a 2.7× reduction in DRAM bandwidth for weights in the decoding stage, compared to an uncompressed bit-serial design (Wang et al., 12 Sep 2025).

For the 3rd–7th slices, the reported sparsity is approximately 70–75%, and the average size per group under the two-state code is given as

$W_i$3

bits per $W_i$4-bit group, yielding $W_i$5–$W_i$6 for $W_i$7 (Wang et al., 12 Sep 2025). End-to-end across all compressed slices, this yields a 2.5–3× reduction in weight bits fetched (Wang et al., 12 Sep 2025). In the full-model end-to-end decoding latency study, BSTC alone cuts weight traffic by 44% and total decoding time by 35–40% (Wang et al., 12 Sep 2025). The paper also states, with reference to Table 9, that BSTC’s overall weight-access reduction across 26 benchmarks is 44% on average, translating to 2.7× lower DRAM bandwidth from weight fetch alone (Wang et al., 12 Sep 2025).

The following table organizes the principal reported metrics.

Quantity	Reported value
MSB-slice zero-rates	70–80% zeros
Empirical sparsity for encoded slices	$W_i$8 for $W_i$9
Encoded slices	3rd–7th slices
Average DRAM bandwidth reduction	2.7×
Reduction in weight bits fetched	2.5–3×
Weight traffic reduction in full-model study	44%
Total decoding time reduction from BSTC alone	35–40%

These measurements are reported specifically for the role of BSTC within the MCBP design rather than as isolated software-only compression ratios. This suggests that the relevant performance criterion is not only compressed size but also whether the compressed representation can be consumed at the pace required by the bit-serial accelerator.

6. Relation to prior weight-compression schemes

The source contrasts BSTC with value-level compression methods such as RLE, Huffman, and CSC/CSR (Wang et al., 12 Sep 2025). These methods see only 5–10% zeros in full bytes and must either leave the lower 6 bits uncompressed or carry $SR_i = \#\text{zeros} / \#\text{entries}$0 plus pointer overhead when compressing arbitrary byte-patterns (Wang et al., 12 Sep 2025). As described in the paper, they therefore cannot exploit the 70–80% zero-rates of high-order bit-planes without substantial index traffic (Wang et al., 12 Sep 2025).

By contrast, BSTC is characterized in the source as having four specific advantages (Wang et al., 12 Sep 2025):

• It works at the bit-plane level, so it captures every zero in the MSB.
• It uses only a 1-bit “non-zero” marker, with no full pointers, amortized across $SR_i = \#\text{zeros} / \#\text{entries}$1 bits.
• It needs no data reorder hardware, because the same bit-serial datapath processes headers and literals.
• It decompresses at DRAM bandwidth, not at compute-side reorder buffers.

Taken together, the paper states that BSTC delivers an order-of-magnitude better sparsity exploitation, described as 70% versus 7%, with less than 10% area overhead for en/decoders, and integrates seamlessly into the BRCR group compute (Wang et al., 12 Sep 2025). The formulation “70% vs. 7%” reflects the comparison between bit-plane-level sparsity and value-level sparsity reported in the source, rather than a universal sparsity law.

The comparison also clarifies a common misconception: the relevant sparsity for bit-serial inference is not necessarily the sparsity of quantized byte values as stored in conventional layouts. BSTC is based on the observation that sparsity emerges asymmetrically across bit-slices, particularly in high-order slices, and that exploiting this structure requires both a different coding unit and a different storage layout (Wang et al., 12 Sep 2025).

7. Significance, constraints, and interpretation

BSTC is described in the source as the first weight-compression that is natively bit-slice aware, aligns perfectly with bit-serial compute, and mines the high MSB zero-rates of quantized LLMs for nearly 3× weight-access reduction with tiny decoder logic and zero reorder overhead (Wang et al., 12 Sep 2025). Within MCBP, this role is specifically complementary to BRCR, which reduces redundant GEMM computations, and BGPP, which reduces KV cache access (Wang et al., 12 Sep 2025). The result is a design in which weight access is not optimized independently of compute, but through an algorithm-hardware co-design that couples representation, decoder structure, banking, and execution granularity.

At the system level, the source reports that extensive experiments on 26 benchmarks show MCBP achieves 9.43× speed up and 31.1× higher energy efficiency than Nvidia A100 GPU, and 35×, 5.2×, and 3.2× energy saving than Spatten, FACT, and SOFA, respectively (Wang et al., 12 Sep 2025). These are properties of MCBP as a whole, not BSTC in isolation. A careful interpretation is therefore that BSTC should be understood as the weight-access component of a larger accelerator architecture rather than as a standalone compression codec.

The design also has explicit scope conditions. Only slices 3 through 7 are encoded, while slices 0 through 2 remain uncompressed (Wang et al., 12 Sep 2025). The paper’s implementation uses the same group size as the BRCR merge engine, with $SR_i = \#\text{zeros} / \#\text{entries}$2 (Wang et al., 12 Sep 2025). This suggests that BSTC’s benefits depend on the presence of significant high-order bit-slice sparsity and on architectural compatibility with bit-serial processing. The paper’s central claim is not merely that compressed representation saves bandwidth, but that the representation can be decoded in parallel, at the required throughput, with negligible alignment cost (Wang et al., 12 Sep 2025).