Bit-Level Sparsity BSQ in Neural Networks

• Bit-Level Sparsity BSQ is defined as the proportion of zero bits in quantized neural network weights, revealing over 50% zero bits even with low value-level sparsity.
• Reordering and bidirectional techniques align similar bit-columns to compress redundant data and achieve significant runtime speedups and energy savings.
• Efficient algorithms leverage statistical bit similarity to prune non-informative columns, optimizing crossbar mapping in RRAM-based compute-in-memory architectures.

Bit-level sparsity (often abbreviated BSQ) refers to the substantial presence of zero bits within the quantized weights (and, in some contexts, activations) of deep neural networks. Unlike value-level sparsity—where entire weights are zero—bit-level sparsity exploits the fact that most bits in a fixed-width quantization representation are zero, creating significant opportunities for computation and memory reduction in both digital and mixed-signal acceleration contexts. Contemporary research leverages not only raw zero bits but also the structural patterns of bit similarity, bidirectionally dominant bits, and strategically pruned or compressed bit-planes to maximize architectural efficiency while maintaining model fidelity.

1. Bit-Level Sparsity: Definition and Statistical Structure

Bit-level sparsity is formally defined as the proportion of zero bits in the quantized representation of neural network weights. For B-bit two's-complement quantization, each weight $x$ is expressed as

$$x = -x_{B-1} \cdot 2^{B-1} + \sum_{i=0}^{B-2} x_i \cdot 2^i,$$

where $x_i$ denotes the corresponding bit. For a network with $p$ value-level sparsity (fraction of exactly-zero weights), nonzero weights—assuming i.i.d. Bernoulli(0.5) per bit after randomization—have average $0.5$ zero bits per position, yielding aggregate bit-level zero fraction:

$$P_{\rm 0bit} = p \cdot 1 + (1-p) \cdot 0.5 = 0.5p + 0.5.$$

Thus, even with low value sparsity, over half of all bit-storage may be zeros, and this rate increases with pruning. The statistical correlation of bits between columns enables structures such as column similarity and bidirectional bit-sparsity, which underpin recent compression and compute-reduction strategies (Yang et al., 18 Nov 2025).

2. Algorithmic Exploitation: Reordering, Pruning, and Bi-directional Techniques

Standard compute-in-memory (CIM) accelerators cannot directly benefit from unstructured bit-level sparsity as only entire zero rows/columns enable skipping. The introduction of bit-level similarity and reordering strategies addresses this (Yang et al., 18 Nov 2025).

The bit-level weight reordering algorithm operates over a matrix $M_{m\times n}$ (flattened, bit-expanded weights):

• Compute the pairwise similarity metric $S(i, j) = m - sHD(i, j)$, where $sHD$ is the sum of XORs over $m$ rows.
• Identify pairs with maximal similarity (least Hamming distance).
• Reorder rows and columns so that highly similar columns align within operation units (OUs), and identical (or all-zero) columns are retained once (or dropped).
• Form OU tiles where redundant or similar bits can be compacted, effectively halving crossbar columns under moderate sparsity. This matches bit-level sparsity and similarity into hardware-aligned groups, compressing computation and storage.

Bidirectional bit-level sparsity (BBS) schemes extend classical single-side sparsity by allowing each bit-column to be processed as its sparsest direction (zeros or ones). By inverting columns heavily dominated by ones, BBS guarantees effective skip ratios of at least 50%, ensuring balanced PE utilization and simplifying hardware scheduling (Chen et al., 2024).

Binary pruning methods—such as rounded column averaging or zero-point shifting—enable retraining-free compression by making entire bit-columns constant and thus removable, compressing tensor storage while providing on-the-fly hardware-decoding (Chen et al., 2024).

3. Impact on RRAM-Based Accelerators and Crossbar Mapping

In resistive RAM (RRAM)-based CIM architectures, weights are mapped to crossbar arrays with each bit occupying a physical cell. The requirement to maintain structured compute patterns traditionally inhibits joint exploitation of weight sparsity and CIM (Yang et al., 18 Nov 2025).

The reordering methodology addresses this:

• Bit-level sparsity is mapped such that all-zero or identical columns are grouped, enabling whole-column elimination.
• Bit similarity extends this by treating column pairs with high matching as a combinable resource (only one physical column needed per pair).
• Mapping after reordering ensures that the majority of stored crossbar columns are densely packed with effective compute data, and inactive columns are removed. Simulation on CNNs (LeNet5, AlexNet, VGG16, GoogleNet, ResNet18) demonstrates average performance improvements of 61.24% and energy savings between 1.51× and 2.52× compared to prior RePIM baselines (Yang et al., 18 Nov 2025).

4. Quantitative Gains and Performance Modeling

The key metrics for architectural evaluation are:

• Crossbar invocation count (CCQ): Number of active OUs required per inference.
• Energy per invocation (EC): RRAM and controller energy for each OU.
• Performance (P): $P = 1/(CCQ \cdot EC)$; speedup and energy saving are measured relative to best baselines.

The speedup and energy efficiency closely track the reduction in CCQ, as EC remains nearly constant. Under moderate sparsity (30–60%), bit-level similarity yields significant gains (compression factors of up to 2×), while at high sparsity ($p > 80\%$), all-zero strategies dominate and both approaches converge (Yang et al., 18 Nov 2025). Control logic overhead for reordering is negligible compared to total CIM energy (~0.48 mW vs. base levels).

5. Relation to Broader Bit-Level Sparsity and Mixed-Precision Approaches

Bit-level sparsity is a unifying theme across digital and analog accelerators:

• Bit-slice and bit-plane pruning: As in Panacea (Kam et al., 2024), bit-level sparsity in quantized weights/activations enables outer-product skipping by compressing or skipping entire bit-slices or vectors, supported in hardware via RLE and tile-based dataflow.
• Regularization and mixed-precision quantization: Modern BSQ approaches use $\ell_1$ or group-Lasso penalties, or fully differentiable continuous masks (see MSQ (Han et al., 30 Jul 2025), CSQ (Xiao et al., 2022), and classical BSQ (Yang et al., 2021)), automatically driving unneeded bits to zero and dynamically reducing model precision.
• Bidirectional and structured patterns: Methods like BBS (Chen et al., 2024) and column-similarity grouping (Yang et al., 18 Nov 2025) go beyond random sparsity, producing structured, hardware-friendly zeros or ones, minimizing imbalance and index inefficiency.
• Hardware implementations: BitWave (Shi et al., 16 Jul 2025), BitVert (Chen et al., 2024), and RRAM crossbar designs natively support varied compressed bit-tensor encodings and fast on-the-fly decoding.

The column-similarity reordering and bit-level compression thus represent state-of-the-art techniques for exploiting the inherent fine-grained redundancy and sparsity present in quantized neural network weights, particularly in nonvolatile and in-memory compute substrates.

6. Limitations, Practical Constraints, and Outlook

Current schemes, while highly effective, impose statistical and hardware constraints:

• Group size and block configuration (OU height, pairing strategies) dictate achievable sparsity compression; larger groupings introduce combinatorial explosion and decreased match probability (Yang et al., 18 Nov 2025).
• For practical RRAM devices, two's-complement quantization and column grouping must be balanced against cell endurance and process variation.
• Algorithmic limits arise as unstructured sparsity (random zeros) gives way to structured, optimally-compressed patterns only under careful quantizer or regularizer design.

Future research aims to further co-design quantizer algorithms, bit-level encoding, and low-variance activation/weight reordering routines, targeting even more aggressive bit-level sparsification without accuracy penalty, and seamless integration for both analog CIM and fully-digital accelerators. Extending these techniques to activations, recurrent or transformer-based models, and hybrid value/bit-level sparsity pipelines remains an active field of exploration (Yang et al., 18 Nov 2025, Kam et al., 2024, Han et al., 30 Jul 2025, Chen et al., 2024).