Quantization with Expert-Selection Calibration (QESC)

• QESC is a quantization method for MoE models that calibrates router outputs to maintain reliable expert selection despite low-bit perturbations.
• It employs a layer-wise calibration procedure using a TopK-MSE loss to align routing behavior between full-precision and quantized models.
• Empirical results demonstrate improvements in perplexity and accuracy across multiple MoE LLMs with minimal additional calibration overhead.

Searching arXiv for the cited QESC-related papers to ground the article in current records. arXiv search: (Chen et al., 3 Aug 2025) “EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts LLMs”; (Park et al., 4 Jun 2026) “Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Models”; (Fu et al., 16 Jun 2025) “EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization”; (Hu et al., 2 May 2025) “MoEQuant: Enhancing Quantization for Mixture-of-Experts LLMs via Expert-Balanced Sampling and Affinity Guidance”; (Chu et al., 19 Nov 2025) “Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference”. Quantization with Expert-Selection Calibration (QESC) is the quantization component of the EAC-MoE framework for mixture-of-experts (MoE) LLMs. It was introduced as a response to a specifically MoE failure mode: low-bit post-training quantization can preserve ordinary layer reconstruction only imperfectly, yet even small perturbations in the hidden states that feed the router can change the top-$K$ expert assignment, thereby altering the conditional computation path itself (Chen et al., 3 Aug 2025). QESC therefore augments weight-only post-training quantization with a router calibration stage whose purpose is to preserve expert-selection behavior between the full-precision and quantized model. In the broader literature, it belongs to a family of router-aware or expert-selection-aware MoE quantization methods, but it remains distinct in its explicit focus on calibrating routers layer by layer under low-bit perturbations (Chen et al., 3 Aug 2025).

1. Origin and conceptual basis

QESC appears in "EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts LLMs" (Chen et al., 3 Aug 2025). In that framework, QESC addresses quantization, while a separate module, Pruning based on Expert-Selection Frequency (PESF), addresses inference speed by pruning less frequently used experts for a current task (Chen et al., 3 Aug 2025). The motivating claim is that MoE compression should not be treated as a dense-model quantization problem with a sparse layer substitute; rather, the routing mechanism is itself part of the quantity that must be preserved.

A decoder-only MoE replaces the dense feed-forward network with a router and $N$ experts. For token representation $\bm{x}$, the router produces routing logits

$\bm{r} = \{r_0,\cdots,r_{N-1}\},$

then expert-selection scores

$\bm{s} = \text{Softmax}(\bm{r}).$

The top-$K$ experts are selected, and the MoE output is

$$\bm{z} = \sum_{j=0}^{K-1} \frac{\bm{s}_{e_j}}{\sum_{i=0}^{K-1} \bm{s}_{e_i}} \cdot E_{e_j}(\bm{x}).$$

This sparse routing is central to MoE efficiency and specialization, but it also creates discontinuity: a small quantization perturbation can change expert ranking and therefore change which experts are executed (Chen et al., 3 Aug 2025).

The QESC paper names this failure mode the expert-shift problem. In ordinary post-training quantization for dense models, the objective is typically to find quantized weights $\bm{W}_q$ minimizing output reconstruction error,

$$\arg\min_{\bm{W}_q} \|\bm{W}\bm{X} - \bm{W}_q\bm{X}\|_2^2,$$

where $\bm{X}$ is calibration data. QESC argues that this objective is insufficient for MoE models because the router boundary is sensitive to small perturbations. If $N$0 only by a small margin, quantization-induced perturbations $N$1 can invert the ordering whenever

$N$2

The resulting error is not merely numerical drift in a fixed path; it is a change in the chosen subnetwork (Chen et al., 3 Aug 2025).

2. MoE routing sensitivity and the expert-shift problem

The empirical motivation for QESC is unusually direct. On WikiText2 validation, the QESC paper compares four conditions: full precision without expert shift, full precision forced to use quantized expert scores, quantized model forced to use full-precision expert scores, and ordinary quantized model with shifted routing. For Mixtral-8x7B, perplexity changes from $N$3 to $N$4 to $N$5 to $N$6; for Deepseek-moe-16b-base, it changes from $N$7 to $N$8 to $N$9 to $\bm{x}$0 (Chen et al., 3 Aug 2025). These comparisons isolate routing drift as a major contributor to degradation.

The paper therefore treats router preservation as a first-class calibration objective. Its implementation choices are correspondingly asymmetric across model components: MHSA components are quantized to 4-bit; experts in MoE layers are quantized to 2-bit, 2.5-bit, or 3-bit depending on the experiment; the router is kept at original precision and calibrated (Chen et al., 3 Aug 2025). This asymmetry is justified by model structure: the router occupies less than $\bm{x}$1 of non-embedding parameters, yet its effect on expert selection is disproportionate (Chen et al., 3 Aug 2025).

QESC is explicitly post-training, training-free, calibration-set-based, and is layered on top of GPTQ-style weight-only quantization. GPTQ is used as the underlying quantizer because it reduces group-wise quantization error using Hessian-based estimation

$\bm{x}$2

and error compensation. The quantization setup is weight-only, group-wise asymmetric, with group size 128. The average model bit-widths reported are approximately 2.06, 2.54, and 3.03 bits, resulting from MHSA at 4-bit, experts at 2 / mixed 2&3 / 3-bit, and router at original precision (Chen et al., 3 Aug 2025).

This architecture-specific decomposition also distinguishes QESC from later MoE PTQ methods that quantize the router itself. For example, EAQuant calibrates router quantization parameters with a combined logit-MSE and routing-KL objective, whereas QESC preserves the router by keeping it in full precision and calibrating its weights against perturbed hidden states (Fu et al., 16 Jun 2025).

3. Calibration objective and layer-wise procedure

QESC calibrates the router, not by adding an explicit bias or temperature term to router outputs, but by optimizing the router weight matrix so that router outputs on quantized inputs better match the full-precision routing behavior (Chen et al., 3 Aug 2025). The method is explicitly layer-by-layer, which is intended to prevent routing drift from accumulating across depth.

The paper’s pipeline is as follows. Using the WikiText2 calibration dataset, the model is processed sequentially, layer by layer. In each transformer layer, the MHSA part is quantized, the router in the MoE block is calibrated, and all experts in that MoE block are quantized. The next layer is then processed using the partially quantized-and-calibrated model (Chen et al., 3 Aug 2025). This means each router is calibrated under the actual perturbed input distribution produced by already-quantized upstream layers, rather than under a purely full-precision trajectory.

A naive router-calibration objective would align all router outputs with ordinary MSE. QESC rejects this because many experts are irrelevant for top-$\bm{x}$3 selection on a given token. It instead introduces TopK-MSE loss: $\bm{x}$4 where $\bm{x}$5 is the router weight matrix and $\bm{x}$6 is the router input obtained from the quantized model (Chen et al., 3 Aug 2025). The indices are taken from the full-precision router output, so the calibration objective explicitly privileges the experts that mattered to the original routing decision.

The practical interpretation is straightforward. Let $\bm{x}$7 denote the router input hidden state from the full-precision pathway and $\bm{x}$8 the corresponding router input after upstream quantization. Then $\bm{x}$9 and $\bm{r} = \{r_0,\cdots,r_{N-1}\},$0 are compared only on the full-precision top-$\bm{r} = \{r_0,\cdots,r_{N-1}\},$1 entries. This is a teacher-student alignment over the routing-relevant subset rather than over all experts (Chen et al., 3 Aug 2025).

The paper also reports model-specific values for the TopK-MSE hyperparameter $\bm{r} = \{r_0,\cdots,r_{N-1}\},$2: $\bm{r} = \{r_0,\cdots,r_{N-1}\},$3 for Phi3.5-moe, $\bm{r} = \{r_0,\cdots,r_{N-1}\},$4 for Deepseek-moe-16b-base, and $\bm{r} = \{r_0,\cdots,r_{N-1}\},$5 for Qwen1.5-MoE-A2.7B (Chen et al., 3 Aug 2025). These values are larger than the number of actually selected experts, which the paper interprets as a way to cover incorrectly unselected experts while avoiding the long tail of irrelevant logits.

4. Empirical results and ablations

The QESC evaluation uses four MoE LLMs: Mixtral-8x7B, Phi3.5-moe, Deepseek-moe-16b-base, and Qwen1.5-MoE-A2.7B. Metrics include WikiText2 perplexity, average accuracy on eight zero-shot tasks from LM Harness, and additional results on GSM8K and HumanEval (Chen et al., 3 Aug 2025).

At 2.06-bit average precision, QESC consistently improves over GPTQ and usually over PMQ. For Mixtral-8x7B, GPTQ scores 62.56 average accuracy while QESC reaches 66.31; for Phi3.5-moe, GPTQ scores 64.45 and QESC 65.03; for Deepseek-moe-16b-base, GPTQ scores 54.88 and QESC 57.05; for Qwen1.5-MoE-A2.7B, GPTQ scores 57.76 and QESC 59.52 (Chen et al., 3 Aug 2025). Corresponding perplexities also improve: Mixtral from 5.51 to 5.09, Deepseek from 8.27 to 7.99, and Qwen from 9.92 to 8.30 (Chen et al., 3 Aug 2025).

At 3.03-bit average precision, the main practical significance is closeness to full precision. QESC reaches 72.21 on Mixtral-8x7B versus baseline 72.64, 68.49 on Phi3.5-moe versus 69.62, 61.22 on Deepseek-moe-16b-base versus 61.38, and 62.89 on Qwen1.5-MoE-A2.7B versus 64.72 (Chen et al., 3 Aug 2025). The paper highlights that for Mixtral-8x7B and Deepseek-moe-16b-base, QESC keeps zero-shot average accuracy within roughly 0.5 percentage points of full precision at 3.03-bit (Chen et al., 3 Aug 2025).

On more difficult tasks, the pattern persists. For Mixtral-8x7B at 3.03-bit, GSM8K rises from 52.29 with GPTQ to 55.34 with QESC, while HumanEval pass@10 rises from 40.83 to 46.34; the full-precision references are 58.30 and 59.15, respectively (Chen et al., 3 Aug 2025). The paper interprets these improvements as especially relevant for math and code tasks, where expert specialization appears to matter strongly.

Ablation results isolate the calibration objective. Under 2.06-bit, replacing TopK-MSE with plain MSE degrades both perplexity and average accuracy. For Phi3.5-moe, MSE gives PPL 5.33 and 0-shot 64.52, while TopK-MSE gives PPL 5.22 and 0-shot 65.03; for Deepseek-moe-16b-base, MSE gives PPL 8.16 and 0-shot 55.91, while TopK-MSE gives PPL 7.99 and 0-shot 57.05; for Qwen1.5-MoE-A2.7B, MSE gives PPL 9.02 and 0-shot 58.44, while TopK-MSE gives PPL 8.30 and 0-shot 59.52 (Chen et al., 3 Aug 2025). This confirms the paper’s claim that generic output matching is weaker than routing-aware matching.

The calibration overhead is small. Relative to GPTQ, router calibration adds approximately 1.3%–2.8% extra offline quantization time: 0.02 h on Mixtral-8x7B, 0.03 h on Phi3.5-moe, 0.05 h on Deepseek-moe-16b-base, and 0.02 h on Qwen1.5-MoE-A2.7B (Chen et al., 3 Aug 2025).

5. Relation to adjacent methods and terminological clarifications

QESC is best understood as one member of a broader MoE quantization line that uses routing information to define what should be preserved, but nearby methods differ substantially in what they calibrate.

MoEQuant makes calibration expert-selection aware at two earlier stages: Expert-Balanced Self-Sampling constructs calibration sets that better balance expert usage, and Affinity-Guided Quantization weights quantization losses and Hessian statistics by router coefficients $\bm{r} = \{r_0,\cdots,r_{N-1}\},$6 (Hu et al., 2 May 2025). EAQuant, by contrast, combines expert-aware smoothing aggregation, router logits distribution alignment, and expert-level calibration data balance; its router-alignment objective matches both pre-softmax logits and post-softmax routing distributions, but it does not introduce an explicit top-$\bm{r} = \{r_0,\cdots,r_{N-1}\},$7 assignment loss (Fu et al., 16 Jun 2025). VSRAQ goes further toward explicit routing consistency by decomposing its calibration objective into value alignment and structure alignment, preserving not only routing-relevant logits but also expert ordering and top-$\bm{r} = \{r_0,\cdots,r_{N-1}\},$8 boundary margins (Park et al., 4 Jun 2026).

Later work also diversified the meaning of “expert-selection-aware” quantization. DynaExq does not recalibrate quantizer parameters; instead, it treats expert precision as a runtime-managed resource, using router probabilities and activation events to maintain an EMA hotness score and to promote or demote experts between HIGH and LOW precision under an HBM budget (Chu et al., 19 Nov 2025). GEMQ takes a different position: after quantization, it fine-tunes router weights with quantized experts frozen so as to minimize cross-entropy on calibration data, arguing that rigid preservation of full-precision routing is not always optimal once expert weights have changed (Deng et al., 21 May 2026). AlphaQ, conversely, is a critique of calibration-dependent expert-importance estimation: it proposes a calibration-free bit-allocation method based on expert-wise spectral heavy-tailedness rather than routing statistics (Yang et al., 3 Jun 2026).

The term QESC itself should also not be conflated with the unrelated ESC of "Evolution Strategy-Based Calibration for Low-Bit Quantization of Speech Models," where ESC means evolution strategy and there is no expert-selection or MoE routing mechanism (Rakotoarivony, 9 Mar 2026). Similarly, FAQ is a family-aware calibration-data regeneration method for LLM PTQ; its “expert guidance” refers to an external judge LLM rather than MoE expert selection (Xiao et al., 16 Jan 2026).

6. Scope, limitations, and subsequent significance

QESC has several clear limitations as presented in its source paper. The method keeps the router at original precision, so it is not a fully low-bit solution for every component (Chen et al., 3 Aug 2025). It is argued to be orthogonal to other PTQ methods focused on reducing quantization error itself, such as GPTQ, OmniQuant, and QuaRot, but it is experimentally combined only with GPTQ in the paper (Chen et al., 3 Aug 2025). The strongest evidence is on four MoE models up to roughly 50B total parameters, and the paper does not provide detailed optimizer settings for router calibration (Chen et al., 3 Aug 2025).

Conceptually, QESC is also narrower than some later routing-aware objectives. It preserves the original top-$\bm{r} = \{r_0,\cdots,r_{N-1}\},$9 routing pattern by calibrating routers against full-precision routing-relevant logits, but it does not formalize ranking preservation, boundary-margin preservation, or task-loss-based router adaptation as separate objectives (Chen et al., 3 Aug 2025). This suggests two distinct later interpretations of the same underlying problem. One interpretation, exemplified by VSRAQ, treats routing consistency itself as the calibration target and formalizes value and structure losses around the full-precision router (Park et al., 4 Jun 2026). Another interpretation, exemplified by GEMQ, treats post-quantization routing as something that may need to change and therefore calibrates the router to the quantized experts rather than to the original assignments (Deng et al., 21 May 2026). A plausible implication is that “expert-selection calibration” can mean either preserving original selection or adapting selection after expert perturbation, depending on which failure mode is considered primary.

Even with these limitations, QESC established a durable formulation: in MoE LLMs, quantization error is not adequately characterized by ordinary reconstruction metrics because expert choice is itself part of the computation. Its lasting contribution is therefore methodological rather than merely empirical. It defined router calibration as a lightweight, training-free PTQ stage; it demonstrated that top-$\bm{s} = \text{Softmax}(\bm{r}).$0-focused router matching is stronger than full-logit MSE; and it helped set the terms for later work on routing-consistent quantization, expert-aware calibration data, runtime precision scheduling, and calibration-free alternatives (Chen et al., 3 Aug 2025).