---
title: 'BLC: Best Low-Rank Approximation under Clipping'
url: https://www.emergentmind.com/topics/best-low-rank-approximation-under-clipping-blc
type: topic
---

# BLC: Best Low-Rank Approximation under Clipping

Best Low-rank Approximation under Clipping (BLC) is a method central to the FLRQ (Flexible Low-Rank Quantization) framework, designed for efficient and accurate post-training quantization (PTQ) of large language models. BLC focuses on minimizing quantization error by alternating scalable low-rank extraction with outlier clipping and quantization of the residual, thereby providing robust, near-monotone error decrease and high quantization quality at minimal computational overhead [2601.05684].

## 1. Formal Definition and Objective

BLC addresses the quantized low-rank approximation of a neural network weight matrix $W \in \mathbb{R}^{m \times n}$, targeting efficient storage and inference without costly fine-tuning. Given $W$, a desired bit-width $d$ for uniform quantization, and a calibration activation matrix $X \in \mathbb{R}^{n \times B}$ (with $B$ activation samples), the goal is to decompose $W$ as:
- A rank-$r$ matrix $W_r = UV^T$ with $U \in \mathbb{R}^{m \times r}$ and $V \in \mathbb{R}^{n \times r}$,
- A clipping threshold $\tau \geq 0$,
- A quantized residual $W_q = \textrm{Quant}\left( \textrm{Clip}(W - W_r, \tau), d \right)$,

such that the end-to-end error on the calibration batch,
$$
E(U, V, \tau) := \| WX - [W_r + W_q] X \|_2,
$$
is minimized. In matrix norm notation, the objective is:
$$
\min_{U, V, \tau \geq 0} \| W - [UV^T + \textrm{Quant}( \textrm{Clip}(W - UV^T, \tau), d ) ] \|_F^2,
$$
where the clipping operator $\textrm{Clip}(A, \tau)_{ij} = \textrm{sign}(A_{ij}) \cdot \min(|A_{ij}|, \tau)$, and quantization uses uniform $d$-bit codebooks (with possible group-wise scaling or zero-point schemes).

## 2. Optimization Motivation

Low-rank preconditioning with $W_r$ captures the majority of the variance in $W$, isolating heavy-tail outliers in the residual $R = W - W_r$. Directly quantizing $R$ can produce excessive rounding errors, especially from outlier values. By clipping $R$ to $\pm \tau$, the dynamic range for quantization is reduced, thus improving quantization fidelity for the bulk of residual entries at the expense of zeroing a small fraction of large outliers.

A joint, closed-form optimization over all variables $(U, V, \tau)$ is computationally intractable. BLC therefore uses an alternating iterative approach:
- Recompute the best rank-$r$ low-rank component of the current residual (via R1-Sketch flexible rank selection).
- Search for the optimal clipping threshold $\tau$ that, after quantizing the residual, minimizes calibration error.

The working assumption is that reproducing $WX$ with small $\ell_2$ error on the calibration set is a sufficient proxy for maintaining accuracy post-quantization.

## 3. Iterative BLC Procedure

The central loop of BLC alternates between low-rank extraction and clipping/quantization threshold search. The following pseudocode outlines the key steps within the context of FLRQ:

```python
# Inputs:
#   W ∈ ℝ^(m×n):      weight matrix
#   X ∈ ℝ^(n×B):      calibration data
#   d:                quantization bits (e.g. 2, 3, 4)
#   p_clp_search:     clipping fractions to search
#   epochs:           number of BLC epochs

# (1) Initialize
W_r = initial_low_rank(W)       # via SVD or R1‐FLR
W_q = Quant(Clip(W - W_r, τ0), d)
bestE = +∞
best_{W_r,W_q} = {W_r, W_q}

# (2) Main BLC Loop
for epoch in range(epochs):
    E = || W X - (W_r + W_q) X ||_2
    if E < bestE:
        bestE = E
        best_{W_r, W_q} = {W_r, W_q}
    R = W - W_q
    U, V = R1-FLR(R)             # Flexible-rank low-rank extraction
    W_r = U V^T
    bestτ, bestWq = argmin_{τ ∈ p_clp_search} || W X - (W_r + Quant(Clip(W - W_r, τ), d)) X ||_2
    W_q = bestWq

return best_{W_r, W_q}
```

R1-FLR leverages the fast Rank-1 Sketch (with Gaussian projection) to extract layer- and data-dependent rank efficiently, allowing for outlier-aware low-rank representations. The clipping threshold search is performed over $10$ logarithmically spaced fractions of the residual's maximum absolute value.

## 4. Theoretical Properties

The theoretical properties of BLC arise from the error guarantees of R1-Sketch and the monotone error reduction obtained via alternating minimization:
- With $r = 1$ power iteration and expectation, the R1-Sketch error bound is
  $$
  \mathbb{E} \|A - A_1\| \leq \sigma_2 + [1 + 4\sqrt{2n/(1-1)}]^{1/(it+1)} \sigma_2,
  $$
  guaranteeing top-singular-vector approximation within a small constant factor per power iteration (Halko et al., 2011).
- Each BLC epoch does not increase the calibration error $E$; storing the best solution ensures non-increasing objective. Although the problem is non-convex, convergence within $O(1)$ epochs is typical for 3–4 bit quantization, and $O(10–20)$ for 2 bits.
- Per-epoch complexity is dominated by the cost of R1-Sketch (i.e., $O(\text{it} \cdot n^2)$, implemented as GEMV BLAS-2 operations) and the threshold search (which is a small loop over preselected $\tau$ values). Empirically, using $it = 2$ for R1-Sketch costs only $6$ GEMV operations and is 2–5$\times$ faster than truncated SVD [2601.05684].

## 5. Practical Parameter Choices

Practical deployment of BLC involves several empirically validated settings:
- R1-Sketch with $it = 2$ is sufficient to match truncated SVD accuracy at a fraction of the computational cost (see Table 13 in [2601.05684]).
- Quantization is performed using group size $128$, consistent with AWQ default.
- Calibration data consists of $128$ sequences of $2048$ tokens from WikiText2.
- The initial low-rank component $W_r$ can be derived via a full SVD or a single R1-FLR pass.
- Clipping thresholds $\tau$ are searched over $10$ logarithmically spaced fractions of the absolute max of the current residual.
- The recommended number of BLC epochs is 1–2 for 4-bit quantization, 2–5 for 3 bits, and 10–30 for 2 bits (see Figure 11 and Table 15 in [2601.05684]).

## 6. Empirical Validation and Comparisons

BLC achieves state-of-the-art accuracy and efficiency in quantized large language models. Key experimental results on benchmark models and tasks include:
- On OPT-1.3B with W3A16 quantization, BLC improves perplexity (PPL) from 15.80 (without BLC) to 15.53 (with BLC), a delta of $-0.27$.
- For W2A16 on OPT-1.3B, PPL without BLC increases to above $10^4$ due to overflow; BLC reduces this to 22.99.
- On six zero-shot tasks for LLaMA2-7B at 3 bits, FLRQ+BLC improves average accuracy from 53.7% to 54.4%.
- Ablation shows that removing BLC degrades 2-bit OPT-1.3B PPL from 22.99 to 29.32.
- Throughput and latency overhead of FLRQ+LoRA on W4A16 is only 4–6% relative to the baseline (Figure 7).
- Compared with fixed-rank LQER at rank 256, adaptive FLRQ achieves similar or better PPL with average rank ≈ 39 and extra storage ≈ 0.24 bits per parameter (Table 9).

BLC thus provides an alternating low-rank extraction and outlier quantization procedure with robust calibration loss reduction, fast convergence, and minimal overhead, underpinning accurate and efficient 2–4 bit quantization for large-scale models [2601.05684].

Source: https://www.emergentmind.com/topics/best-low-rank-approximation-under-clipping-blc