EntQuant: Data-Free Extreme Compression

• EntQuant is a data-free post-training compression framework that decouples inference precision from storage cost using entropy coding.
• It employs an ℓ1-based rate–distortion optimization to drive 8-bit weight representations toward extremely low entropy, achieving rates near 2 bits per parameter.
• The framework unifies fast, model-agnostic techniques with high-fidelity compression, maintaining competitive performance on large models without calibration data.

EntQuant is a data-free post-training compression framework introduced in “Float8@2bits: Entropy Coding Enables Data-Free Model Compression” (Putzky et al., 30 Jan 2026). It decouples numerical precision from storage cost by fixing a moderately high-precision inference format—Float8 or Int8—and then entropy-coding the resulting low-entropy weight stream down to a target rate such as approximately $2$ bits per parameter, without changing kernel precision. Within the formulation reported by the paper, EntQuant is positioned between fast, data-free, model-agnostic methods that can collapse below $4$ bits and data-dependent or recovery-based methods that offer higher fidelity at higher computational cost; its central claim is that these regimes can be unified in a single framework for extreme compression (Putzky et al., 30 Jan 2026).

1. Conceptual basis and place in post-training quantization

Traditional post-training quantization ties storage cost directly to bit-width: Int8 implies $8$ bpp, Int4 implies $4$ bpp, and Int2 implies $2$ bpp. EntQuant changes this coupling. Rather than lowering the numerical format used by inference kernels, it keeps an $8$-bit representation for execution and reduces storage cost through entropy coding of the quantized symbol stream. The paper states the key insight as follows: by optimizing the empirical distribution of quantized weights for low entropy, one can achieve arbitrarily low storage cost while retaining the expressivity of an $8$-bit weight format at inference time (Putzky et al., 30 Jan 2026).

This design yields a specific separation of concerns. Numerical precision is associated with the inference kernel, whereas storage cost is associated with the entropy of the encoded weight stream. In the terminology of the paper, EntQuant is the first framework to unite the advantages of fast, data-free, model-agnostic compression and data-dependent high-fidelity compression, and it is explicitly aimed at the extreme compression regime. The framework is also described as the first data-free PTQ method that breaks the rigid link between bit-width and storage via entropy coding.

2. Mathematical formulation

The paper formulates EntQuant for a weight matrix $W \in \mathbb{R}^{M\times N}$, a “mild” quantizer $Q_\gamma(\cdot)$ into an $8$-bit format $4$0 (Float8 or Int8), and per-channel scales $4$1. The quantized matrix is written as

$4$2

The empirical entropy, interpreted as expected bits per parameter, is

$4$3

where $4$4 is the frequency histogram of code symbols. The optimization objective is a rate–distortion Lagrangian: $4$5 with distortion

$4$6

surrogate rate term

$4$7

and $4$8 controlling the bits-per-parameter budget (Putzky et al., 30 Jan 2026).

The paper further states the entropy bound

$4$9

This bound motivates the use of the $8$0 proxy for entropy control. By tuning $8$1, the method trades off reconstruction error against empirical entropy; the paper reports that entropy versus $8$2 is log-linear and model-independent. The result is a single optimization mechanism that steers an $8$3-bit quantized representation toward very low-entropy symbol distributions.

3. Algorithmic pipeline

EntQuant operates layer-wise, or block-wise, and does not use data. The workflow given in the paper is explicit (Putzky et al., 30 Jan 2026):

• Step 0: load full-precision $8$4 in Float16 or BFloat16.
• Step 1: initialize scales $8$5 via AbsMax,

$8$6

• Step 2: optimize $8$7 by minimizing

$8$8

with L-BFGS and straight-through gradients through $8$9.

• Step 3: quantize,

$4$0

• Step 4: flatten $4$1 into a symbol stream $4$2 and feed it to an ANS encoder, producing bitstream $4$3, while also storing the frequency table and $4$4.

Inference is likewise specified. On device, ANS$4$5 decodes $4$6, the quantized weights are rescaled as $4$7, and a fused GEMM such as Marlin Float8 is called for

$4$8

The paper emphasizes two implementation granularities. First, scales are channel-wise. Second, entropy coding is block-wise, with one block stated as approximately $4$9 MiB of weight symbols. This architecture is central to the framework’s claim that inference can still run in a standard $2$0-bit format even when stored weights occupy approximately $2$1 bits per parameter.

4. Empirical behavior and reported benchmarks

The reported evaluation spans data-free baselines, calibration or fine-tuning methods, and instruction-tuned or reasoning-oriented models (Putzky et al., 30 Jan 2026).

Setting	EntQuant	Baseline context
LLaMA-2 7B @ $2$2 bpp	PPL(C4) $2$3; LM-Eval $2$4	HQQ@3 bpp $2$5; NF4@4 bpp $2$6; LM-Eval $2$7
LLaMA-2 7B @ $2$8 bpp	PPL $2$9; LM-Eval $8$0	HQQ collapses with PPL $8$1
LLaMA-2 70B @ $8$2 bpp	$8$3	GPTQ $8$4; OmniQuant $8$5; QuIP# $8$6; EfficientQAT $8$7
LLaMA-2 70B @ $8$8 bpp	$8$9	GPTQ — / $8$0; OmniQuant $8$1; QuIP# $8$2; EfficientQAT $8$3

For the LLaMA-2 family, the paper states that at approximately $8$4 bpp EntQuant outperforms NF4 and HQQ by a wide margin and matches $8$5-bit quality, while at approximately $8$6 bpp it remains usable. For LLaMA-2 70B, the reported comparison is that at $8$7 bpp EntQuant matches fine-tuned methods to within $8$8 accuracy loss, and at $8$9 bpp it remains competitive (Putzky et al., 30 Jan 2026).

The instruction-tuned evaluations are used to support robustness claims beyond standard perplexity and short benchmark suites. The paper reports the following examples:

• LLaMA-3.1 8B Instruct @ $W \in \mathbb{R}^{M\times N}$0 bpp: GSM8K $W \in \mathbb{R}^{M\times N}$1 versus base $W \in \mathbb{R}^{M\times N}$2; GPQA $W \in \mathbb{R}^{M\times N}$3 versus $W \in \mathbb{R}^{M\times N}$4; IFEval $W \in \mathbb{R}^{M\times N}$5 versus $W \in \mathbb{R}^{M\times N}$6.
• LLaMA-3.1 70B Instruct @ $W \in \mathbb{R}^{M\times N}$7 bpp: performance is within $W \in \mathbb{R}^{M\times N}$8 of the full-precision base across all four benchmarks.
• LLaMA-3.1 70B Instruct @ $W \in \mathbb{R}^{M\times N}$9 bpp and $Q_\gamma(\cdot)$0 bpp: performance degrades gracefully, and at $Q_\gamma(\cdot)$1 bpp the model remains functional, with GSM8K $Q_\gamma(\cdot)$2 versus $Q_\gamma(\cdot)$3.

These results are presented as evidence that EntQuant retains functional performance on more complex benchmarks with instruction-tuned models, not only on standard evaluation sets.

5. Systems design, implementation, and runtime characteristics

The implementation stack reported by the paper consists of PyTorch, Optimum-Quanto, and HuggingFace Transformers. The quantization backend uses Float8, specifically torch.float8e4m3, together with Marlin fused kernels for inference. The entropy coder is NVIDIA nvCOMP ANS with chunk size $Q_\gamma(\cdot)$4 KiB and block-wise joint encoding of all weights in a transformer block (Putzky et al., 30 Jan 2026).

The runtime profile is a significant part of the framework’s practical identity. The paper reports compression time below $Q_\gamma(\cdot)$5 minutes for a $Q_\gamma(\cdot)$6B-parameter model on one H100, with typical runs much smaller than $Q_\gamma(\cdot)$7 minutes on smaller models. At inference, the reported overhead is $Q_\gamma(\cdot)$8–$Q_\gamma(\cdot)$9 slower than BFloat16, on par with NF4, and faster than HQQ. The GPU bottleneck is identified as ANS decoding, which the paper describes as compute-bound and efficiently interleaved with GEMM kernels.

These implementation choices are not incidental. They reflect the framework’s defining claim that very low storage rates can be combined with a conventional high-throughput inference path. The paper’s systems contribution is therefore not only the compression objective but also the integration of ANS decoding into the runtime path of transformer inference.

6. Robustness claims, scope, and significance

EntQuant is described as requiring no calibration or fine-tuning data. The paper further states that the method remains stable under distribution shifts inherent in new benchmarks (Putzky et al., 30 Jan 2026). Within the paper’s framing, this is important because the contrast class includes data-dependent methods whose robustness can be uncertain under distribution shift.

The practical use cases stated or implied by the reported evaluations center on extreme-compression deployment of large models. Because EntQuant compresses a $8$0B-parameter model in less than $8$1 minutes, retains Float8 or Int8 inference precision, and operates without calibration data, it is presented as a route to accessible post-training compression in settings where recovery training is undesirable or infeasible. A plausible implication is that storage budgets near $8$2 bpp need not force a move to $8$3-bit execution kernels.

In the paper’s own summary, EntQuant is the first data-free PTQ framework that: breaks the rigid link between bit-width and storage via entropy coding; applies a simple $8$4-based rate–distortion optimization to produce extremely low-entropy $8$5-bit weights; and integrates high-throughput ANS decoding into the inference pipeline. The reported outcome is state-of-the-art extreme compression down to approximately $8$6 bpp, matching or exceeding both data-free and data-dependent baselines while requiring no calibration data or recovery training (Putzky et al., 30 Jan 2026).