Squeeze10-LLM: Ultra-Low-Bit Quantization Framework

• Squeeze10-LLM is a PTQ framework that achieves 10× compression by quantizing 80% of weights to 1 bit and 20% to 4 bits, averaging 1.6-bit precision.
• It introduces PBAR to assess activation sensitivity for weight salience and FIAS to control cumulative activation drift during quantization.
• Empirical results on LLaMA and LLaMA2 models demonstrate significant zero-shot accuracy improvements over previous sub-2-bit quantization methods.

Searching arXiv for Squeeze10-LLM and closely related low-bit LLM quantization work to ground the article. Squeeze10-LLM is a post-training quantization (PTQ) framework for LLMs that targets extreme weight compression while retaining usable task performance. It is designed to “squeeze” 16-bit model weights by 10 times through a staged mixed-precision scheme in which 80% of the weights are quantized to 1 bit and 20% to 4 bits, yielding an average precision of 1.6 bits per weight. The method addresses the usual failure mode of sub-2-bit weight-only quantization—severe accuracy degradation—by combining two mechanisms: Post-Binarization Activation Robustness (PBAR), which ranks weights by how harmful binarization would be to activations, and Full Information Activation Supervision (FIAS), which prevents cumulative layerwise drift during PTQ (Zhu et al., 24 Jul 2025).

1. Scope and deployment objective

Squeeze10-LLM is situated in the problem of deploying LLMs under strict memory, storage, and bandwidth constraints. The motivating premise is that LLMs have massive parameter counts and high computational costs, so lowering weight precision can reduce storage and accelerate inference. The paper specifically focuses on ultra-low-bit weight-only PTQ, with the goal of pushing beyond conventional 2-bit PTQ into the sub-2-bit regime without retraining the original model (Zhu et al., 24 Jul 2025).

The central target is a 10× compression of 16-bit weights. The paper attains this with a mixed-precision allocation rather than a uniform low-bit format. Its average weight precision is

$0.8 \cdot 1 + 0.2 \cdot 4 = 1.6.$

This arithmetic is fundamental to the method’s identity: Squeeze10-LLM is not a uniform 1-bit model, and it is not a generic “low-bit” scheme in which every parameter receives the same budget. A common misconception is to read the method as pure binarization. In fact, its design depends on preserving a minority of weights at 4 bits, with the 1.6-bit figure arising from the 80/20 split rather than from a novel scalar datatype (Zhu et al., 24 Jul 2025).

The deployment significance of this design is that it treats ultra-low-bit PTQ as a selective resource-allocation problem. The paper argues that sub-2-bit quantization fails not only because information is lost, but also because salient weights are difficult to identify and quantization errors compound across layers. Squeeze10-LLM is therefore framed as a weight selection and supervision strategy as much as a quantization format.

2. Staged mixed-precision quantization pipeline

The framework proceeds in three stages. First, it performs 4-bit uniform quantization as an intermediate buffer. Second, it selectively binarizes weights using PBAR to decide which parameters should remain at 4 bits and which can be reduced to 1 bit. Third, it applies mixed-bit supervision with FIAS so that later quantization steps are guided by original pretrained activations rather than already corrupted intermediate outputs (Zhu et al., 24 Jul 2025).

The initial 4-bit stage is not merely an implementation convenience. The paper reports that 4-bit works best as a “buffering” precision because it most closely matches the activation distribution of the original model among tested intermediate precisions from 2 to 8 bits. This gives the method a staged character: it does not jump directly from 16-bit to 1-bit, but uses 4-bit as a bridge between full precision and binarization (Zhu et al., 24 Jul 2025).

For standard $k$-bit asymmetric uniform quantization, the paper uses

$$w_q = \mathrm{clamp}\left(\left\lfloor\frac{w}{s}\right\rceil + z,0,2^{k}-1\right), \qquad \hat{w} = s \times (w_q - z)$$

with

$$s=\frac{\max(w)-\min(w)}{2^k-1}, \qquad z=\left\lfloor\frac{-\min(w)}{s}\right\rceil.$$

For binarization, it uses

$$\mathrm{sign}(x)= \begin{cases} -1, & x \leq 0 \ 1, & x > 0 \end{cases}.$$

These definitions clarify that the framework combines ordinary uniform quantization with a sign-based binary representation rather than introducing a new quantizer. The novelty lies in how the model transitions between these formats and in how the weight subsets are chosen (Zhu et al., 24 Jul 2025).

The final mixed-precision schedule is fixed and explicit: the top 20% most salient weights are quantized to 4 bits, while the remaining 80% are quantized to 1 bit. The paper also studies other high-bit proportions and finds that larger high-bit ratios improve accuracy, but that 20% at 4 bits is a strong tradeoff for the target 1.6-bit regime. This suggests that the method is optimized around a specific operating point rather than presented as a universally optimal bit allocation.

3. PBAR and activation-aware weight salience

PBAR, or Post-Binarization Activation Robustness, is the paper’s refined weight significance metric. Its purpose is to decide which parameters are too important to binarize. The paper characterizes PBAR as combining three signals: Hessian-based salience, weight-outlier sensitivity, and post-binarization activation range change. The key claim is that weight importance should be judged not only by parameter-space criteria such as magnitude or Hessian score, but also by how much activations change after binarization (Zhu et al., 24 Jul 2025).

The activation-aware term is introduced because existing mixed-precision methods can miss weights whose binarization causes disproportionate disruption to layer outputs. PBAR explicitly measures the activation-range change induced by quantizing an individual weight element from high precision to 1 bit. Conceptually, if that change is large, the weight is deemed important and is preferentially kept at 4 bits; if the activation range barely changes, the weight can be safely binarized (Zhu et al., 24 Jul 2025).

The final PBAR salience score is expressed as

$\mathbf{M} = \mathbf{V} + \lambda \mathbf{B},$

where $\mathbf{V}$ denotes Hessian-based salience, $\mathbf{B}$ denotes post-binarization activation sensitivity, and $\lambda$ is a balancing coefficient. The appendix reports that for LLaMA2-7B the best value is

$\lambda = 3 \times 10^{-4}.$

The significance of PBAR is methodological. It moves salience estimation from a purely parameter-centric view toward an activation-centric one. In the ultra-low-bit regime, this is especially consequential because the distinction between “replaceable” and “non-replaceable” weights is sharper than in 4-bit or 8-bit PTQ. A plausible implication is that PBAR treats mixed-precision quantization less as approximate numeric compression and more as structured preservation of activation-critical subspaces.

4. FIAS and the control of cumulative error propagation

FIAS, or Full Information Activation Supervision, addresses a separate failure mode: cumulative activation drift. In standard PTQ, once a layer is quantized, its quantized output becomes the next layer’s input. Small distortions introduced early can therefore propagate and amplify through later layers. The paper argues that this is especially damaging in aggressive mixed-precision settings, where even modest early perturbations can destabilize downstream salience estimation and final accuracy (Zhu et al., 24 Jul 2025).

FIAS changes the supervision signal used during quantization. Rather than repeatedly updating activations using already-quantized preceding layers, it keeps using the original pretrained activations to supervise later quantization steps. The paper describes this as preserving “full-information activations” throughout PTQ. In effect, the supervisory target is held fixed to the unquantized model, so later layers are not quantized against progressively corrupted inputs (Zhu et al., 24 Jul 2025).

This design has several consequences stated in the paper: it avoids activation drift, prevents quantization noise from compounding, stabilizes salience estimation across layers, and improves the reliability of Hessian- and activation-based scoring. Relative to PBAR, which decides what to preserve at higher precision, FIAS determines how the entire staged compression process is stabilized. The ablation results indicate that FIAS contributes more strongly than PBAR to final performance, which is consistent with the paper’s emphasis on layerwise error accumulation as a dominant difficulty in sub-2-bit PTQ (Zhu et al., 24 Jul 2025).

A common misunderstanding is to treat Squeeze10-LLM as simply a better importance metric. That is incomplete. The framework combines a selection mechanism and a supervision mechanism, and the paper’s ablations indicate that staged quantization, PBAR, and FIAS are all contributing factors rather than interchangeable variants.

5. Experimental evaluation and empirical performance

The evaluation covers LLaMA and LLaMA2 models, including LLaMA-7B, LLaMA-30B, LLaMA-65B, LLaMA2-7B, LLaMA2-13B, and LLaMA2-70B. The paper measures perplexity on WikiText2, Penn Treebank (PTB), and C4, and zero-shot accuracy on BoolQ, HellaSwag, PIQA, WinoGrande, ARC-c, and ARC-e. Compared baselines include GPTQ, AWQ, QuIP, OmniQuant, PB-LLM, and full-precision 16-bit models (Zhu et al., 24 Jul 2025).

The headline result is that Squeeze10-LLM achieves state-of-the-art performance for sub-2-bit weight-only quantization and improves average zero-shot accuracy from about 43% to 56% on six classification tasks. The paper’s most explicit example is LLaMA2-7B, where the average accuracy rises from 43.95% for PB-LLM to 56.04% for Squeeze10-LLM (Zhu et al., 24 Jul 2025).

Model	PB-LLM avg. accuracy	Squeeze10-LLM avg. accuracy
LLaMA2-7B	43.95%	56.04%
LLaMA2-13B	44.92%	64.93%
LLaMA2-70B	67.72%	70.83%
LLaMA-7B	47.16%	57.31%
LLaMA-65B	68.79%	70.30%

For LLaMA2-7B, the paper reports the following task-level accuracies for Squeeze10-LLM: BoolQ 67.43, HellaSwag 46.03, PIQA 72.20, WinoGrande 64.56, ARC-c 32.94, and ARC-e 64.48. These gains are described as substantial across nearly all tasks when compared with PB-LLM (Zhu et al., 24 Jul 2025).

The paper also reports scaling behavior. On LLaMA2-70B, the jump from 67.72% to 70.83% indicates that the method remains effective at larger model sizes and comes close to full precision. This suggests that the staged mixed-precision strategy does not only rescue small models from collapse, but remains competitive as parameter count increases.

Ablation results reinforce the architectural interpretation of the method. Removing PBAR lowers performance slightly. Removing FIAS lowers performance more noticeably. Removing both lowers it further, and removing both together with staged quantization lowers it still more. On WinoGrande, for example, the reported accuracies are 64.56% for the full method, 64.33% without PBAR, 62.19% without FIAS, 61.25% without both, and 56.43% without both and without staged quantization (Zhu et al., 24 Jul 2025). These figures support the claim that PBAR improves salience selection, FIAS mitigates propagation error, and staged quantization is itself structurally important.

6. Interpretation, limitations, and place within ultra-low-bit PTQ

Squeeze10-LLM’s main conceptual contribution is to show that sub-2-bit weight-only quantization need not be treated as uniform numeric degradation. The framework instead treats extreme compression as a coordinated procedure: use 4-bit as a transition point, keep only the most activation-sensitive weights at 4 bits, and avoid supervising later layers with already distorted activations. The paper argues that this combination makes aggressive PTQ substantially more robust than naive binarization (Zhu et al., 24 Jul 2025).

Several limitations are stated or directly implied. The method still requires a special mixed-precision assignment rather than simple uniform quantization. There is extra overhead for storing salient weight indices. Performance depends on calibration and salience estimation. The reported evaluation is mainly on LLaMA and LLaMA2 families and on standard zero-shot benchmarks. The appendix notes that the storage cost is manageable, with an additional bit for index storage and possible compression via sparse formats such as CSR. The paper also indicates that, despite strong results, some accuracy gap remains at extremely low bits, especially relative to full precision on smaller models (Zhu et al., 24 Jul 2025).

These caveats help delimit what “10× squeezing” means in practice. It does not mean lossless compression, and it does not mean that 1-bit quantization has been made universally reliable. Rather, the paper shows that the specific 1.6-bit operating point can be made viable through a staged and activation-aware PTQ pipeline. This suggests a broader lesson for low-bit LLM deployment: once bit-width is pushed below 2, the dominant issue is no longer just quantizer design, but the interaction among salience ranking, activation stability, and cross-layer supervision.

In that sense, Squeeze10-LLM occupies a specific place in the literature on LLM compression. It is a weight-only PTQ method targeted at an operating regime where conventional low-bit assumptions cease to be adequate. Its reported contribution is not merely that it compresses aggressively, but that it identifies a concrete recipe—4-bit buffering, PBAR-based selective binarization, and FIAS-based supervision—that preserves substantially more zero-shot capability than earlier sub-2-bit PTQ baselines (Zhu et al., 24 Jul 2025).