4-bit Group Scaling Quantization

Updated 8 July 2026

4-bit GSQ is a quantization method that partitions tensor weights into groups with a shared scale factor and represents each weight with a 4-bit code.
It balances trade-offs by using smaller groups for improved local accuracy while larger groups reduce metadata overhead, impacting efficiency.
Empirical analyses show variable impacts on accuracy and throughput across different models and tasks, underscoring its role as a versatile deployment baseline.

Searching arXiv for papers on 4-bit group scaling quantization and closely related methods. 4-bit Group Scaling Quantization (GSQ) is a family of low-precision quantization schemes in which a tensor is partitioned into small groups, each group shares a scale factor, and each value in the group is represented with a 4-bit code. In recent large-language-model deployment work, GSQ is described as an advanced weight quantization method for post-training compression that builds on Activation-aware Weight Quantization (AWQ) and applies group-wise scaling to preserve model fidelity under 4-bit weights (Sk et al., 15 Aug 2025). The same grouped shared-scale principle also underlies microscaled FP4 formats such as NVFP4 and MXFP4, which quantize values into small groups sharing a scale (Egiazarian et al., 12 May 2026). As a result, 4-bit GSQ functions both as a practical deployment baseline and as the reference point against which non-uniform codebooks, asymmetric microscaling, salient-bit compression, and multi-grid quantizers are evaluated (Zhao et al., 22 Jan 2025, Lee et al., 2024, Lee et al., 23 Jan 2025).

1. Quantization model and mathematical form

In the group-scaling formulation documented for LLM benchmarking, a weight vector $\mathbf{w} = [w_1, w_2, \ldots, w_n]$ is partitioned into $G$ groups, each of size $g=n/G$ . For each group $i$ , the scaling factor is

$s_i = \max |\mathbf{w}^{(i)}| / (2^{b-1} - 1),$

with $b=4$ . Each weight in the group is quantized as

$q_j = \mathrm{clip}\left(\mathrm{round}\left(\frac{w_j}{s_i}\right), -2^{b-1}, 2^{b-1}-1\right),$

and reconstructed during inference as

$\hat{w}_j = q_j \cdot s_i.$

This is the canonical shared-scale scalar quantization pattern: the group-specific scale captures local dynamic range, while the per-value code remains a 4-bit integer (Sk et al., 15 Aug 2025).

The same benchmark emphasizes two implementation constraints. First, model weight matrices are partitioned into groups along the last dimension. Second, the last tensor dimension must be divisible by the group size, and the group size must remain fixed across all layers (Sk et al., 15 Aug 2025). These restrictions are not incidental; they define the tensor layout, scale metadata layout, and kernel compatibility of a GSQ deployment.

A broader implication of this formulation is that 4-bit GSQ is not merely “INT4.” Its characteristic feature is the granularity at which scale is shared. The quality of the approximation therefore depends not only on the 4-bit codebook, but also on how much statistical heterogeneity is forced into each shared-scale group.

2. Granularity, group size, and scale overhead

Group size is the central design variable in 4-bit GSQ. In one empirical study, group sizes 16 and 64 were evaluated for LLM weight quantization (Sk et al., 15 Aug 2025). In another, GSQ/AWQ appears as a group-scaling baseline with group size 128 (Zhao et al., 22 Jan 2025). QRazor explicitly contrasts its own 16- or 32-element groups with “typical groupwise quantization,” noting that GSQ uses 128 (Lee et al., 23 Jan 2025). In microscaled FP4 formats, practical small-group formats such as MXFP or NVFP are described as using 16–32 values per block (Egiazarian et al., 12 May 2026).

These papers converge on the same technical tension. Smaller groups reduce quantization error caused by diverse value distributions within a group, but they also increase the frequency with which scales or related metadata must be stored and applied. QRazor makes this overhead explicit by contrasting its bit-selection mechanism with GSQ’s need to store and multiply per-group FP16/32 scale factors (Lee et al., 23 Jan 2025). The benchmark study correspondingly reports that reducing GSQ group size to 16 improved accuracy but slightly increased latency and reduced throughput and memory efficiency, whereas using group size 64 increased throughput at the cost of a small drop in accuracy (Sk et al., 15 Aug 2025).

This trade-off explains why group scaling is often treated as a continuum rather than a single method. Coarser GSQ is more deployment-friendly but more exposed to within-group outliers; finer GSQ improves local fidelity but taxes metadata handling and implementation efficiency.

3. Empirical behavior in LLM deployment

A direct evaluation of 4-bit GSQ on LLaMA 1B, Qwen 0.5B, and Phi 1.5B used MS MARCO, BoolQ, and GSM8K, reporting accuracy, memory usage, inference latency, and throughput on an NVIDIA A100 SXM4 40GB GPU (Sk et al., 15 Aug 2025). The results show that GSQ behavior is task- and model-dependent rather than uniformly beneficial.

Several concrete outcomes illustrate this variability. On LLaMA 1B with MS MARCO, accuracy improved from 81.12% to 84.04% under GSQ. On Phi 1.5B with MS MARCO, accuracy changed from 99.80 to 99.82. On Qwen 0.5B with BoolQ, accuracy changed from 56.21 to 55.81 (Sk et al., 15 Aug 2025). For LLaMA 1B on BoolQ, GSQ increased throughput from 69.57 to 364.91 tokens/sec (Sk et al., 15 Aug 2025). At the same time, the study reports that quantization does not guarantee memory reduction and that quantized models sometimes used more memory because of implementation overhead (Sk et al., 15 Aug 2025).

The same benchmark identifies a stable task pattern. Retrieval on MS MARCO could be minimally affected or even improved, QA on BoolQ was relatively robust, and GSM8K was particularly sensitive to precision loss (Sk et al., 15 Aug 2025). This suggests that 4-bit GSQ is often adequate for retrieval- and classification-like behaviors, but brittle for mathematical reasoning workloads where small perturbations may alter multistep token trajectories.

A separate scaling-law study places these observations in a broader bit-width context. It characterizes 4-bit quantization as a safe, near-lossless default that is mature and well-supported in existing hardware, but reports that 2-bit, 3-bit, and ternary quantization can surpass 4-bit in the accuracy-model size trade-off when training schemes and quantization functions are optimized across bit-widths (Liu et al., 4 Feb 2025). Thus, 4-bit GSQ remains practically important even when it is not Pareto-optimal in a size-normalized sense.

4. Critiques from alternative 4-bit quantization schemes

Several recent methods use 4-bit GSQ as the baseline they seek to exceed, and their critiques clarify the boundaries of the shared-scale paradigm.

GANQ treats GSQ/AWQ as a uniform, group-scaled baseline that still requires dequantization on current GPUs. In its 4-bit LLaMA-2-7B perplexity comparison, GSQ/AWQ with group size 128 reports 5.60 on WikiText2, while GANQ reports 5.57, using non-uniform per-channel codebooks and LUT-based mixed-precision GEMM rather than standard dequantization-based execution (Zhao et al., 22 Jan 2025). The criticism here is not directed at 4-bit precision alone, but at the combination of uniform quantization and dequantization-heavy kernels.

QRazor critiques GSQ from a different angle. It argues that per-group scale factors impose both memory and compute overhead, especially when compared with a bitwise “significant data razoring” scheme that retains only salient bits and stores a small flag for each group. QRazor further claims an advantage from using smaller groups, such as 16 or 32, instead of the 128-element grouping associated with GSQ (Lee et al., 23 Jan 2025). In this comparison, GSQ is framed as a scale-centric method whose simplicity becomes a liability when group heterogeneity is high.

AMXFP4 focuses on microscaled FP4, where groups share a scale but values are represented in FP4 rather than integer grids. It identifies a tradeoff: microscaling suppresses activation outliers but increases group-wise asymmetry, which symmetric shared scales do not capture well. AMXFP4 therefore introduces separate positive and negative scales per group and reports that it outperforms MXFP4 by 3% on VQA and exceeds rotation-based methods by 1.6% on CSQA (Lee et al., 2024). This is a critique of symmetric GSQ-style sharing rather than of grouping itself.

“Grid Games” challenges another implicit assumption of GSQ-like formats: that each group should use a single fixed 4-bit grid. The paper formalizes adaptive two-grid selection per group and reports that dual-grid methods recover more than 97.5% of full-precision accuracy on an 8-model mean, compared with less than 97% for standard NVFP4; its hardware-friendly SFP4 variant yields approximately 23% lower KL divergence than standard NVFP4 (Egiazarian et al., 12 May 2026). The central claim is that shared-scale grouping remains useful, but the one-grid-per-group design is unnecessarily restrictive.

The acronym “GSQ” is not unique in current literature, and disambiguation is necessary because several distinct methods use similar groupwise metadata ideas.

Name	Domain	Distinguishing feature
4-bit Group Scaling Quantization	LLM PTQ/inference	Shared per-group scale with 4-bit values (Sk et al., 15 Aug 2025)
GSQ-Tuning	On-device LLM fine-tuning	Group-Shared Exponents Integer format; group size 32; $32 \times 4 + 5 = 133$ bits per group, about 4.16 bits/value (Zhou et al., 18 Feb 2025)
GSQ (“Gumbel-Softmax Quantization”)	LLM PTQ	Jointly learns per-coordinate assignments and per-group scales via Gumbel-Softmax (Dadgarnia et al., 20 Apr 2026)
Grouped Spherical Quantization	Vision tokenizers	Grouped vector quantization with spherical codebooks; 4-bit is only a codebook-size special case (Wang et al., 2024)

GSQ-Tuning generalizes the group-shared idea from inference compression to fully quantized training. Its Group-Shared Exponents Integer representation stores a shared exponent for a group and integer mantissas for group members, enabling fully integer-based inference and training. The paper reports accuracy comparable to BF16-based fine-tuning while significantly reducing 1.85x memory usage, and compared with FP8 it reports 5x lower power consumption and 11x smaller chip area with the same performance (Zhou et al., 18 Feb 2025). Although this is not standard AWQ-style GSQ, it preserves the core principle that local magnitude redundancy can be exploited through group-shared metadata.

By contrast, the 2026 “GSQ” paper redefines the acronym as “Gumbel-Softmax Quantization.” Its method still uses symmetric scalar group-wise quantization, but it replaces heuristic assignment with a Gumbel-Softmax relaxation that jointly optimizes per-coordinate grid assignments and per-group scales while remaining compatible with existing scalar inference kernels (Dadgarnia et al., 20 Apr 2026). In that sense it is both an acronym collision and a continuation of the group-scaled scalar quantization line.

6. Limitations, misconceptions, and open directions

A persistent misconception is that 4-bit GSQ is automatically the best compression regime for LLMs. The scaling-law evidence does not support that conclusion. Under a unified training and quantization framework, ternary, 2-bit, and 3-bit models are reported to match or exceed 4-bit in the size-accuracy trade-off, even though 4-bit remains the most mature setting from the standpoint of kernels and hardware support (Liu et al., 4 Feb 2025). The practical dominance of 4-bit GSQ therefore reflects ecosystem readiness as much as intrinsic optimality.

A second misconception is that group scaling automatically yields lower memory use and latency. The benchmark on LLaMA 1B, Qwen 0.5B, and Phi 1.5B reports that quantized models sometimes used more memory because of implementation overhead, and latency improvements were not universal (Sk et al., 15 Aug 2025). The same result complicates a simplistic “4-bit equals faster” narrative: deployment outcomes depend on tensor shapes, kernel design, and how scales are handled at runtime.

Current research directions refine rather than abandon the group-scaling abstraction. One line adds expressivity inside each group, as in adaptive multi-grid formats (Egiazarian et al., 12 May 2026). Another relaxes symmetry by separating positive and negative shared scales, as in AMXFP4 (Lee et al., 2024). A third attempts to remove scale-induced runtime cost altogether, either through salient-bit extraction without per-group floating scales (Lee et al., 23 Jan 2025) or through LUT-based execution that avoids dequantization (Zhao et al., 22 Jan 2025). A plausible implication is that the long-term evolution of 4-bit GSQ will be driven less by the question of whether grouping is useful, and more by how much structure can be embedded into each group without sacrificing kernel compatibility.

In that sense, 4-bit Group Scaling Quantization occupies a transitional position. It is the mature, hardware-conscious baseline of low-bit LLM deployment, yet it is also the object of sustained revision: finer grouping, richer grids, asymmetric scales, learned assignments, and dequantization-free arithmetic all emerge as attempts to retain the deployment simplicity of shared-scale quantization while reducing its approximation error.