BLC: Best Low-Rank Approximation under Clipping

• The paper introduces BLC as a novel method that integrates low-rank extraction with outlier clipping to minimize quantization error.
• It employs an alternating iterative approach that refines rank approximations and clipping thresholds, ensuring rapid convergence and minimal overhead.
• Empirical validations show significant improvements in perplexity and quantization fidelity for 2–4 bit post-training quantization of large language models.

Best Low-rank Approximation under Clipping (BLC) is a method central to the FLRQ (Flexible Low-Rank Quantization) framework, designed for efficient and accurate post-training quantization (PTQ) of LLMs. BLC focuses on minimizing quantization error by alternating scalable low-rank extraction with outlier clipping and quantization of the residual, thereby providing robust, near-monotone error decrease and high quantization quality at minimal computational overhead (Gul et al., 9 Jan 2026).

1. Formal Definition and Objective

BLC addresses the quantized low-rank approximation of a neural network weight matrix $W \in \mathbb{R}^{m \times n}$, targeting efficient storage and inference without costly fine-tuning. Given $W$, a desired bit-width $d$ for uniform quantization, and a calibration activation matrix $X \in \mathbb{R}^{n \times B}$ (with $B$ activation samples), the goal is to decompose $W$ as:

• A rank-$r$ matrix $W_r = UV^T$ with $U \in \mathbb{R}^{m \times r}$ and $V \in \mathbb{R}^{n \times r}$,
• A clipping threshold $W$0,
• A quantized residual $W$1,

such that the end-to-end error on the calibration batch,

$W$2

is minimized. In matrix norm notation, the objective is:

$W$3

where the clipping operator $W$4, and quantization uses uniform $W$5-bit codebooks (with possible group-wise scaling or zero-point schemes).

2. Optimization Motivation

Low-rank preconditioning with $W$6 captures the majority of the variance in $W$7, isolating heavy-tail outliers in the residual $W$8. Directly quantizing $W$9 can produce excessive rounding errors, especially from outlier values. By clipping $d$0 to $d$1, the dynamic range for quantization is reduced, thus improving quantization fidelity for the bulk of residual entries at the expense of zeroing a small fraction of large outliers.

A joint, closed-form optimization over all variables $d$2 is computationally intractable. BLC therefore uses an alternating iterative approach:

• Recompute the best rank-$d$3 low-rank component of the current residual (via R1-Sketch flexible rank selection).
• Search for the optimal clipping threshold $d$4 that, after quantizing the residual, minimizes calibration error.

The working assumption is that reproducing $d$5 with small $d$6 error on the calibration set is a sufficient proxy for maintaining accuracy post-quantization.

3. Iterative BLC Procedure

The central loop of BLC alternates between low-rank extraction and clipping/quantization threshold search. The following pseudocode outlines the key steps within the context of FLRQ:

$B$7

R1-FLR leverages the fast Rank-1 Sketch (with Gaussian projection) to extract layer- and data-dependent rank efficiently, allowing for outlier-aware low-rank representations. The clipping threshold search is performed over $d$7 logarithmically spaced fractions of the residual's maximum absolute value.

4. Theoretical Properties

The theoretical properties of BLC arise from the error guarantees of R1-Sketch and the monotone error reduction obtained via alternating minimization:

• With $d$8 power iteration and expectation, the R1-Sketch error bound is

$d$9

guaranteeing top-singular-vector approximation within a small constant factor per power iteration (Halko et al., 2011).

• Each BLC epoch does not increase the calibration error $X \in \mathbb{R}^{n \times B}$0; storing the best solution ensures non-increasing objective. Although the problem is non-convex, convergence within $X \in \mathbb{R}^{n \times B}$1 epochs is typical for 3–4 bit quantization, and $X \in \mathbb{R}^{n \times B}$2 for 2 bits.
• Per-epoch complexity is dominated by the cost of R1-Sketch (i.e., $X \in \mathbb{R}^{n \times B}$3, implemented as GEMV BLAS-2 operations) and the threshold search (which is a small loop over preselected $X \in \mathbb{R}^{n \times B}$4 values). Empirically, using $X \in \mathbb{R}^{n \times B}$5 for R1-Sketch costs only $X \in \mathbb{R}^{n \times B}$6 GEMV operations and is 2–5$X \in \mathbb{R}^{n \times B}$7 faster than truncated SVD (Gul et al., 9 Jan 2026).

5. Practical Parameter Choices

Practical deployment of BLC involves several empirically validated settings:

• R1-Sketch with $X \in \mathbb{R}^{n \times B}$8 is sufficient to match truncated SVD accuracy at a fraction of the computational cost (see Table 13 in (Gul et al., 9 Jan 2026)).
• Quantization is performed using group size $X \in \mathbb{R}^{n \times B}$9, consistent with AWQ default.
• Calibration data consists of $B$0 sequences of $B$1 tokens from WikiText2.
• The initial low-rank component $B$2 can be derived via a full SVD or a single R1-FLR pass.
• Clipping thresholds $B$3 are searched over $B$4 logarithmically spaced fractions of the absolute max of the current residual.
• The recommended number of BLC epochs is 1–2 for 4-bit quantization, 2–5 for 3 bits, and 10–30 for 2 bits (see Figure 1 and Table 15 in (Gul et al., 9 Jan 2026)).

6. Empirical Validation and Comparisons

BLC achieves state-of-the-art accuracy and efficiency in quantized LLMs. Key experimental results on benchmark models and tasks include:

• On OPT-1.3B with W3A16 quantization, BLC improves perplexity (PPL) from 15.80 (without BLC) to 15.53 (with BLC), a delta of $B$5.
• For W2A16 on OPT-1.3B, PPL without BLC increases to above $B$6 due to overflow; BLC reduces this to 22.99.
• On six zero-shot tasks for LLaMA2-7B at 3 bits, FLRQ+BLC improves average accuracy from 53.7% to 54.4%.
• Ablation shows that removing BLC degrades 2-bit OPT-1.3B PPL from 22.99 to 29.32.
• Throughput and latency overhead of FLRQ+LoRA on W4A16 is only 4–6% relative to the baseline (Figure 2).
• Compared with fixed-rank LQER at rank 256, adaptive FLRQ achieves similar or better PPL with average rank ≈ 39 and extra storage ≈ 0.24 bits per parameter (Table 9).

BLC thus provides an alternating low-rank extraction and outlier quantization procedure with robust calibration loss reduction, fast convergence, and minimal overhead, underpinning accurate and efficient 2–4 bit quantization for large-scale models (Gul et al., 9 Jan 2026).