AlphaQ: Calibration-Free MoE Quantization

• AlphaQ is a calibration-free mixed-precision quantization method for MoE architectures that uses heavy-tailed weight spectra from HT-SR theory as an importance signal.
• It formulates bit allocation as a global optimization problem that integrates structural importance with quantization noise under a strict bit budget to maintain near BF16 performance.
• AlphaQ avoids calibration biases by basing allocations solely on pretrained weights, demonstrating superior results over calibration-based methods on various MoE models.

AlphaQ is a calibration-free mixed-precision bit-allocation method for Mixture-of-Experts (MoE) quantization. It addresses the deployment bottleneck of frontier MoE LLMs, where sparse expert activation reduces compute per token but does not remove the requirement that all expert weights remain resident in memory at inference time. AlphaQ assigns different bit-widths to different experts or layers without using training data or calibration data for the allocation decision, replacing activation-based importance estimation with a weight-spectrum-based signal derived from Heavy-Tailed Self-Regularization (HT-SR) theory. In the formulation presented in "AlphaQ: Calibration-Free Bit Allocation for Mixture-of-Experts Quantization" (Yang et al., 3 Jun 2026), modules with more heavy-tailed spectra are treated as better trained and therefore assigned higher precision under a global bit-budget constraint.

1. Problem formulation and motivation

MoE architectures scale model capacity through sparse expert activation, but their deployment remains memory-bound because all expert weights must reside in memory. This makes quantization, and especially mixed-precision quantization, a natural systems-level intervention: different experts and sublayers need not be stored at identical precision if their contribution to model quality is heterogeneous (Yang et al., 3 Jun 2026).

The immediate target of AlphaQ is the bit-allocation stage of post-training quantization. Prior MoE quantization schemes typically determine per-expert or per-layer precision from calibration activations, reconstruction error, expert usage frequency, or sensitivity measured on a calibration set. The paper argues that this strategy is structurally problematic for frontier MoE LLMs because the original training data is proprietary and inaccessible, calibration sets are biased and incomplete, and expert importance estimated from surrogate data can become tied to a specific domain. A central failure mode is that some experts are under-triggered or overemphasized by the calibration distribution, leading to domain bias and poor cross-domain generalization.

The motivating empirical example is Mixtral-8×7B, for which PMQ bit allocations vary substantially depending on whether calibration comes from C4, MATH, or GitHub-Code, with corresponding performance shifts. This is presented not as an incidental implementation detail but as evidence that calibration-based bit allocation can inherit distributional artifacts from the chosen surrogate dataset. AlphaQ is designed to eliminate this dependency by using only pretrained weights at the allocation stage.

2. Spectral heavy-tailedness as an importance signal

AlphaQ replaces activation-based importance estimation with a structural criterion derived from HT-SR theory. The underlying intuition is that well-trained neural networks often exhibit heavy-tailed empirical spectral density (ESD) in their weight matrices, and that heavier-tailed spectra indicate richer learned structure, stronger correlations, and better training sufficiency (Yang et al., 3 Jun 2026). In AlphaQ, this is operationalized as a ranking principle:

More heavy-tailed weight spectra → better trained / more important module → higher bit-width.

For a weight matrix $\mathbf{W}_i \in \mathbb{R}^{m \times n}$, the correlation matrix is defined as

$\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$

Its empirical spectral density is

$$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$

where $\lambda_j(\mathbf{X}_i)$ are the eigenvalues. AlphaQ quantifies heavy-tailedness through PL_Alpha_Hill, a robust power-law tail exponent estimate:

$$p(\lambda) \propto \lambda^{-\alpha}, \qquad \lambda_{\min} < \lambda < \lambda_{\max}.$$

The Hill estimator used is

$$\alpha = 1 + \left(\frac{1}{k}\sum_{i=1}^{k}\ln\frac{\lambda_{n-i+1}}{\lambda_{n-k}}\right)^{-1}.$$

Smaller $\alpha$ corresponds to a heavier tail and therefore higher estimated importance.

The paper also uses FARMS (Fixed-Aspect-Ratio Matrix Subsampling) to reduce aspect-ratio bias in spectral estimation across differently shaped matrices. This is significant because MoE architectures contain modules with heterogeneous dimensions, including attention layers and expert sublayers such as up, gate, and down projections. A plausible implication is that AlphaQ is not merely ranking experts at a coarse block level, but constructing a finer-grained structural prior over heterogeneous modules using a common spectral statistic.

3. Allocation objective and optimization procedure

AlphaQ does not allocate bits from $\alpha$ alone. It combines structural importance from heavy-tailedness with a quantization-noise model. Under a uniform-noise approximation, the quantization error variance for layer $l$ at bit-width $b$ scales like $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$0, yielding

$\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$1

This is then importance-scaled by the alpha score:

$\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$2

where $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$3 and $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$4 is a data-free curvature parameter. Because smaller $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$5 increases the weighting, heavily structured modules become more expensive to quantize aggressively.

Bit allocation is formulated as a budget-constrained multiple-choice knapsack / ILP problem. Let $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$6 denote the candidate bit-width set, $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$7 indicate whether layer $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$8 is assigned bit-width $\mathbf{X}_i = \mathbf{W}_i^\top \mathbf{W}_i.$9, $$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$0 be the number of parameters in layer $$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$1, and $$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$2 be the global bit budget. The optimization is

$$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$3

The method is global rather than local: it solves one budgeted optimization across all target modules instead of assigning bits block by block. This directly targets the “block-wise local optimum” issue identified in prior methods (Yang et al., 3 Jun 2026).

The pipeline is explicitly weight-only at the allocation stage:

1. Take the pretrained MoE model weights.
2. Partition modules or layers of interest.
3. Compute PL_Alpha_Hill for each target module using Hill + FARMS.
4. Interpret smaller $$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$4 as higher importance.
5. Compute layer-wise quantization-noise cost for each candidate bit-width.
6. Combine importance and noise into $$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$5.
7. Solve the global ILP / knapsack under the total bit budget.
8. Quantize the model using the chosen bit-widths with any PTQ backend.

The paper is explicit that “calibration-free” applies to bit allocation rather than to the entire downstream quantization workflow: the actual quantization backend can still be a standard PTQ method such as GPTQ.

4. Relationship to calibration-based MoE quantization

AlphaQ is positioned against calibration-based MoE quantization methods such as PMQ, MxMoE, and MoEQuant. The distinction is methodological rather than merely practical. Calibration-based approaches infer importance from sampled activations or reconstruction behavior on input data, whereas AlphaQ infers importance from model weights alone (Yang et al., 3 Jun 2026).

This difference yields several consequences stated in the paper. First, AlphaQ is data-independent and therefore less sensitive to domain shift. Second, it avoids dependence on imperfect surrogates for inaccessible training distributions. Third, because the optimization is global, it can exploit heterogeneous importance across blocks more effectively than methods that allocate budget locally.

The paper’s ablation logic further sharpens this distinction. It argues that alpha-only and noise-only objectives are each weaker than the combined objective, and that global allocation is preferable to block-wise allocation because block importance is heterogeneous. This suggests that AlphaQ is not simply a calibration-free substitute for an existing heuristic, but a structured redefinition of the allocation problem in which spectral importance and quantization cost are jointly optimized.

The default choice of curvature parameter is also data-free:

$$\mu_{\mathbf{X}_i} := \frac{1}{n} \sum_{j=1}^{n} \delta_{\lambda_j(\mathbf{X}_i)},$$6

This keeps the entire allocation procedure independent of external data, consistent with the paper’s central claim that allocation bias can be removed without sacrificing performance.

5. Empirical evaluation and reported performance

AlphaQ is evaluated on DeepSeekV2-Lite, Qwen1.5-MoE, Mixtral-8×7B, and Qwen3-30B-A3B, with comparisons against Uniform, PMQ, and additional baselines in selected settings. Across models and budgets, the paper reports that AlphaQ consistently outperforms calibration-based baselines under matched bit budgets (Yang et al., 3 Jun 2026).

The most emphasized result is on Qwen1.5-MoE. At 3.5-bit average expert precision, AlphaQ achieves near BF16 accuracy while using only 7.6 GB of weights versus 28.94 GB in BF16, corresponding to roughly 4× compression. In the 5-task setting, BF16 average accuracy is 67.93 and AlphaQ at 3.5-bit reaches 68.04. In the 6-task appendix setting for the A2.7B variant, BF16 is 69.87 and AlphaQ at 3.5-bit reaches 69.86, effectively matching BF16-level accuracy at that budget.

At lower precision, the method remains competitive. For Qwen1.5-MoE at 2.5-bit, the paper states that average accuracy remains much better than Uniform and PMQ and that perplexity is significantly reduced relative to baselines.

A compact summary of the reported comparisons is given below.

Model / setting	Reported AlphaQ result	Baseline comparison
Qwen1.5-MoE, 3.5-bit	Avg. 68.04; 7.6 GB	BF16 Avg. 67.93; 28.94 GB
Mixtral-8×7B, 2.5-bit	Avg. 70.76; PPL 5.17	PMQ Avg. 70.38; PPL 5.32
Mixtral-8×7B, 3.5-bit	Avg. 74.29	PMQ Avg. 73.32; Uniform 73.81
DeepSeekV2-Lite, 3.5-bit	Avg. 70.07	PMQ 69.75; Uniform 69.09
DeepSeekV2-Lite, 2.5-bit	Avg. 66.74	PMQ 65.75; Uniform 65.15
Qwen3-30B-A3B, 2.5-bit	Avg. 69.56	PMQ 65.88; Uniform 68.76
OLMoE-1B-7B, 3-bit	Avg. 28.86	Best PMQ variant 27.69; BF16 35.29

For Mixtral-8×7B, AlphaQ slightly outperforms PMQ at both 2.5-bit and 3.5-bit. At 2.5-bit, average accuracy is 70.76 for AlphaQ versus 70.38 for PMQ and 67.86 for Uniform, with perplexity improving from 5.32 to 5.17 relative to PMQ. At 3.5-bit, AlphaQ again leads with average accuracy 74.29, compared with 73.32 for PMQ and 73.81 for Uniform. The efficiency numbers reported for Mixtral include 13.41 GB at 2-bit with speedup about 1.69×, and 22.03 GB at 3.5-bit with compression about 4.4×.

On DeepSeekV2-Lite, AlphaQ improves over PMQ and Uniform at all budgets reported. On Qwen3-30B-A3B, the 2.5-bit result shows a wider margin over PMQ, indicating that the method generalizes to larger-scale MoE models. The paper also states that AlphaQ outperforms the recent DynaMo method on overlapping benchmarks.

6. Ablations, limitations, and significance

The ablation study isolates the components of AlphaQ’s objective. On OLMoE-1B-7B, the reported results are:

• Noise-only: PPL 11.22, Avg Acc 63.74
• Alpha-only: PPL 10.03, Avg Acc 66.23
• Alpha + Noise (Direct): PPL 9.56, Avg Acc 66.81
• Alpha + Noise (Ours): PPL 9.19, Avg Acc 67.11

These numbers support three specific conclusions: alpha is stronger than noise alone, combining alpha and noise is better than either component in isolation, and the particular integration used by AlphaQ is stronger than a direct combination (Yang et al., 3 Jun 2026).

A second ablation compares global allocation to block-wise allocation. Global allocation yields better perplexity on both Mixtral 2.5-bit (5.17 versus 5.81) and DeepSeekV2-Lite 2.5-bit (6.64 versus 6.99). A third ablation compares layer-wise and expert-wise allocation; layer-wise allocation performs better on Mixtral 2-bit (PPL 6.11 versus 6.28) and DeepSeekV2-Lite 3-bit (PPL 6.69 versus 6.81). This is presented as evidence that finer-grained, layer-aware allocation is preferable to coarser expert-level assignment.

The paper is explicit about several limitations. AlphaQ currently covers weight-only quantization, leaving activation bit allocation for future work. It assumes that heavy-tailedness is a valid proxy for importance, a hypothesis supported on the tested MoE LLMs but not proven universally across architectures. It addresses allocation bias, but other calibration-dependent stages of quantization, including GPTQ error compensation, may still introduce bias. The benchmark set is also not exhaustive with respect to all existing MoE quantization or compression methods.

Within those limits, the significance of AlphaQ is methodological and practical. It shows that mixed-precision allocation for MoE models can be driven by structural information encoded in weight spectra rather than by sampled activations. This suggests a broader reorientation of post-training compression for models whose training distributions are inaccessible: importance estimation need not be calibration-dependent if sufficiently informative weight-space statistics are available. In the specific setting studied, AlphaQ uses HT-SR-inspired spectral heavy-tailedness to allocate precision globally, producing strong empirical results and, most notably, near BF16 performance on Qwen1.5-MoE at 3.5-bit average precision with roughly 4× memory compression (Yang et al., 3 Jun 2026).