BUI-GF: Bit-Wise Uncertainty Guard Filtering

• The paper demonstrates that BUI-GF fuses sparsity prediction with full-precision execution in a bit-serial workflow, reducing latency by 30% with minimal accuracy loss.
• It computes dynamic uncertainty intervals at each bit-round to safely prune uninformative Key–Query pairs, achieving up to 70% reduction in computation.
• BUI-GF’s integration in PADE shows a robust hardware design with a 7.4× latency reduction and 31.1× energy efficiency improvement over traditional GPU-based methods.

Bit-Wise Uncertainty Interval-Enabled Guard Filtering (BUI-GF) is a core algorithmic innovation in the hardware-accelerated PADE architecture for dynamic sparse attention. BUI-GF enables precise, predictor-free early pruning of uninformative Key–Query pairs (termed iEQKs) in Transformer self-attention, fusing the roles of sparsity prediction and full-precision execution into a single, bit-serial workflow. This approach eliminates the overhead and redundancy of traditional multi-stage predictor-based sparsity methods while maintaining accuracy and hardware efficiency (Wang et al., 16 Dec 2025).

1. Motivation and Context

Transformer models compute an $S \times S$ matrix of dot-products between Query and Key tokens ($Q \times K^T$), incurring prohibitive quadratic computational and memory costs. Dynamic sparsity (DS) approaches mitigate this overhead by pruning dot-products associated with low-relevance Key–Query pairs. Conventional practices deploy a low-precision sparsity predictor stage—often an MSB-multiplied or log-domain estimator—to decide which pairs to evaluate in full. However, such divisions present multiple inherent bottlenecks: at 8-bit quantization, the predictor can consume over 60% of total power and area due to its specialized logic and data movement redundancy [(Wang et al., 16 Dec 2025), Fig. 2(a)]. Furthermore, operations in the predictor stage are not reused, leading to inefficiency.

The Stage-Fusion paradigm in PADE replaces the predictor–executor dichotomy with a bit-serial execution stream. In this regime, pruning decisions are made incrementally with successive bit-plane revelations. Naive strategies that use only the most significant bits (MSB) to estimate importance are error-prone due to the high variance in two’s complement representations, risking irrevocable elimination of relevant pairs from a single inaccurate partial computation.

BUI-GF addresses these limitations by replacing static, estimator-based gating with dynamic, uncertainty-interval–aware guard filtering at each bit round, allowing safe early pruning decisions that preserve accuracy while maintaining minimal hardware and energy costs.

2. Theoretical and Mathematical Basis

Let $x$ denote a $p$-bit two’s-complement integer Key element represented as: $$x = -b_{p-1} \cdot 2^{p-1} + \sum_{i=0}^{p-2} b_i \cdot 2^i$$ After processing $r$ MSB bit-planes, the partial dot-product between a fixed Query row $Q_i$ and a Key $K_j$ is $S_{i,j}^r$. The residual contribution from the unprocessed $p-r$ LSB planes is uncertain but bounded.

The Bit-wise Uncertainty Interval (BUI) defines: $S_{i,j}^{r,\min} = S_{i,j}^r + I_i^{r,\min}$

$S_{i,j}^{r,\max} = S_{i,j}^r + I_i^{r,\max}$

where $I_i^{r,\min}, I_i^{r,\max}$ are bounds on the possible contributions from the lower bit-planes, dependent only on %%%%10%%%% and the current bit position.

Explicitly, for bit-plane $b$ (with $0$ as LSB): $L_b = \sum_{i=b+1}^{p-1} x_i 2^i$

$U_b = L_b + 2^b - 1$

These intervals are precomputed and stored in a compact look-up table (LUT), indexed by $r$, providing efficient bounds at each stage [(Wang et al., 16 Dec 2025), Fig. 10(c)].

To derive the dynamic pruning threshold at bit round $r$, BUI-GF computes a softmax-inspired, max-based gating value: $$\mathcal{T}_i = \max_j S_{i,j}^{:,\min} - \alpha \cdot \mathrm{radius}$$ Here, $S_{i,:}^{:,\min}$ aggregates all candidate lower bounds for row $i$, $\alpha$ is a tunable coefficient, and the radius is empirically set (typically $\approx 5$ for 8-bit systems).

Pruning logic at round $r$ is:

• If $S_{i,j}^{r,\max} \leq \mathcal{T}_i$, Key $j$ is permanently pruned (iEQK).
• If $S_{i,j}^{r,\min} > \mathcal{T}_i$, Key $j$ is accepted.
• Otherwise, the pair remains "uncertain" for evaluation in the next bit round.

3. Algorithmic Workflow

The end-to-end BUI-GF procedure for each Query row $i$ operates as follows:

Initialize partial scores S[j]=0 for all j Lookup (I_min[r], I_max[r]) for r=0…p−1 from BUI table for r in 0…p−1: for each surviving j: L[j] = S[j] + I_min[r] T = max_j L[j] − α·radius for each surviving j: Δ = dot(Q_i, K_j^r) // 1-bit Key plane S[j] += Δ if S[j] + I_max[r] ≤ T: prune j // iEQK // else if S[j] + I_min[r] > T: // early accept (optional) // mark j as final survivor // else, continue to next bit // After r=p-1, remaining j are iQKs return list of survivors

The guard filter (BUI-GF) dynamically categorizes each candidate Key–Query pair into pruned, accepted, or uncertain classes at every bit round, leveraging the certainty provided by interval bounds and maximizing reuse of partial computation.

4. Hardware Integration and Architectural Mapping

BUI-GF is architecturally embedded within PADE’s QK Processing Unit (QK-PU), which comprises 128 bit-serial processing lanes. Each lane maintains:

• A scoreboard to accumulate 32 partial sums ($S[j]$).
• A grouped sparsity-aware ANDer tree (GSAT) for rapid $\Delta$ computation.
• A Decision Unit interfacing with BUI interval data and broadcast thresholds to determine progression or eviction for each Key.

A central BUI-GF Module:

• Collects per-lane lower bounds and computes $\mathcal{T}_i$ via a max-reduction tree.
• Broadcasts $\mathcal{T}_i$ and $I_{\min/\max}$ to all decision units for synchronized filtering.
• Relies on a compact LUT (8 × 2 for 8-bit entries) in the BUI Generator for all relevant interval data [(Wang et al., 16 Dec 2025), Fig. 10].

This design is complemented by overlapping bit-plane DRAM fetch and partial updates in an out-of-order (BS-OOE) manner, increasing utilization and throughput.

5. Empirical Results and Measured Impact

Adoption of BUI-GF produces a 30% reduction in end-to-end QK latency relative to a bit-serial baseline without BUI-based filtering (ablation study, [(Wang et al., 16 Dec 2025), Fig. 16(a)]). Across 22 diverse benchmarks, BUI-GF retains $\leq 0.1\%$ accuracy loss compared to 8-bit dense attention, while enabling $\sim 70\%$ reduction in QK computation. In conjunction with BS-OOE and ISTA, the broader PADE system achieves a $7.4\times$ reduction in latency and $31.1\times$ energy efficiency improvement over Nvidia H100 GPU. The principal contribution to these gains is from BUI-GF’s predictor-free early termination mechanism, which dominates the observed efficiency gap.

6. Comparative Analysis with Predictor-Based DS Approaches

Predictor-based DS hardware, exemplified by Sanger, DOTA, and SOFA accelerators, leverages an auxiliary low-bit estimator (MSB multiply, log-shift, top-$k$) with explicit thresholding followed by full-precision recomputation for selected pairs. These stages are not synergistic: at low bit-widths, the predictor logic/traffic dominates, cumulatively exceeding $60\%$ of power at 8-bit execution.

BUI-GF provides an alternative with the following characteristics:

Feature	Predictor-Based DS	BUI-GF (in PADE)
Predictor Power	High (>60% at 8b)	Low (12.1% total overhead)
Area Overhead	Significant	Small LUT (4.9%)
Data Reuse	None (redundant)	Complete (bit-serial reuse)
Sorting/Top-$k$	Yes	None

BUI-GF thus delivers predictor-level sparsity accuracy, superior hardware resource reuse, and eliminates redundant computation. The resulting system adheres to the "ideal DS" prescription: minimal logic, practical integration, preserved accuracy, and maximal sparsity realization (Wang et al., 16 Dec 2025).

7. Practical and Theoretical Implications

BUI-GF enables a generalized architectural motif for dynamic neural sparsity: uncertainty-aware, predictor-free, bit-serial execution. It eliminates the structural bottlenecks of split-stage designs in sparse attention accelerators without any significant loss in precision. A plausible implication is that similar interval-enabled speculative filtering may be broadly applicable in other domains where fine-grained, bit-serial computation coalesces with stringent energy or area constraints.