QuantLRM: Fine-Tuning Driven LRM Quantization

• QuantLRM is a weight-only quantization method that uses fine-tuning weight updates to determine channel importance in large reasoning models.
• It introduces the 'protecting both ends' hypothesis, preserving both substantially updated and minimally changed weights to enhance reasoning performance.
• The method integrates with AWQ-style scaling and demonstrates improvements up to 6.55% in low-bit quantization across various fine-tuning regimes.

QuantLRM is a post-training, weight-only quantization method for Large Reasoning Models (LRMs) that uses fine-tuning traces—specifically, weight updates between a pre-fine-tuned model and its fine-tuned version—to estimate which channels should be protected during quantization (Zhang et al., 31 Jan 2026). It targets the specific problem of accurate low-bit quantization of LRMs after reasoning-incentivized post-training, rather than generic LLM compression, and is motivated by the claim that standard PTQ signals such as activation statistics and second-order or Hessian approximations do not fully capture what supervised fine-tuning, direct preference optimization, or reinforcement-learning-based reasoning post-training changed or preserved in the model (Zhang et al., 31 Jan 2026). Its central conceptual contribution is the hypothesis of “protecting both ends”: for reasoning-tuned models, both the smallest and the largest weight updates during fine-tuning are informative, whereas updates of intermediate magnitude are less diagnostic (Zhang et al., 31 Jan 2026).

1. Problem setting and conceptual basis

QuantLRM is situated in the deployment regime where modern LRMs are produced by substantial post-training pipelines, including supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning variants such as RLVR, but conventional PTQ treats the final checkpoint as a static artifact and discards the training history (Zhang et al., 31 Jan 2026). The method treats this as wasteful, because the difference between pre- and post-fine-tuning weights is assumed to encode which parameters were critical to adaptation.

The paper’s intuition is explicitly inspired by classical magnitude pruning. If absolute weight magnitude can serve as a proxy for pruning importance, QuantLRM asks whether absolute weight update magnitude during reasoning-oriented fine-tuning can serve as a proxy for quantization importance (Zhang et al., 31 Jan 2026). The naïve version of that idea would preserve channels with the largest updates, but the paper reports that this is not sufficient: simply ranking channels by large update magnitude performs worse than activation-based importance. This leads to the stronger claim that fine-tuning dynamics in LRMs are structurally U-shaped rather than monotonic.

Under this view, very large updates likely correspond to parameters heavily used to encode new reasoning behavior during fine-tuning, whereas very small or zero updates may indicate parameters that optimization intentionally left nearly untouched because they are important for pre-existing general capabilities such as language modeling, instruction following, or stable backbone functionality (Zhang et al., 31 Jan 2026). A plausible implication is that QuantLRM reinterprets reasoning post-training not only as selective modification, but also as selective preservation.

2. Fine-tuning traces and the “protecting both ends” hypothesis

QuantLRM computes importance from weight deltas between checkpoints. The paper defines channel importance for a channel $c$ in module $m$ at layer $l$ as the average of individual weight-importance scores: $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$ where $N$ is the number of weights in the channel (Zhang et al., 31 Jan 2026). For comparison, the paper also states the activation-based channel importance used in validation experiments: $I_{l,c}^m = \sum_{i=1}^{M} (a_i^m)^2,$ where $M$ is sequence length and $a_i^m$ is the activation on that channel (Zhang et al., 31 Jan 2026).

The distinctive step is the definition of $i_{l,c}^{m,n}$ from update magnitudes. QuantLRM fits restricted quadratic functions over weight updates, split at the global median update, with defaults $y_{\min}=1$ and $m$0, so that the median maps to $m$1 and both the minimum and maximum map to $m$2 (Zhang et al., 31 Jan 2026). This is the formal implementation of “protecting both ends”: middle updates receive low scores, while both extremes receive high scores.

The method then addresses the case $m$3, which the paper identifies as empirically common because zero updates can account for more than $m$4 of all weights (Zhang et al., 31 Jan 2026). To avoid distortion of the left quadratic fit, QuantLRM excludes zero updates when fitting the quadratics, recomputes the median over positive updates only, assigns zero updates the baseline value $m$5, and then explicitly amplifies channels with many zero updates. If $m$6 is the number of zero updates in a channel, the final importance is

$m$7

The $m$8 avoids nullifying a channel when $m$9 (Zhang et al., 31 Jan 2026).

This zero-handling step is not incidental. The paper presents it as a crucial part of the final design, because QuantLRM is not merely a nonlinear remapping of update magnitudes; it is a channel-importance measure that explicitly rewards both large adaptation and concentrated preservation (Zhang et al., 31 Jan 2026).

3. Quantization mechanism and integration with AWQ-style scaling

Once channel importance is computed, QuantLRM uses it inside an AWQ-style scaling-based quantization objective rather than introducing a new quantizer (Zhang et al., 31 Jan 2026). For input features $l$0, fine-tuned weights $l$1, quantizer $l$2, and scaling vector $l$3, the loss is

$l$4

To inject fine-tuning signals, QuantLRM sets

$l$5

and searches over 20 candidates for $l$6 in the interval $l$7, module by module (Zhang et al., 31 Jan 2026).

The resulting workflow is explicit in the paper: compute checkpoint deltas, fit the restricted quadratics, count zero updates, form channel importance, grid-search $l$8, and then quantize (Zhang et al., 31 Jan 2026). This makes QuantLRM a better importance-driven scaling policy for weight-only PTQ, not a replacement for the surrounding quantization infrastructure.

A common misconception is that QuantLRM is a generic low-rank correction scheme or a quantization-aware training procedure. The paper does not describe it that way. It requires checkpoint differences, but it does not require training-time gradients or optimizer states, and it remains much cheaper than QAT (Zhang et al., 31 Jan 2026). Another misconception is that it only protects heavily changed weights; the method’s defining claim is precisely that carefully preserved weights are also informative.

4. Validation and benchmark results

The paper first validates the “protecting both ends” hypothesis through a mixed-precision probe. In these experiments, R1-Distill-Qwen-32B is quantized to 3-bit while keeping $l$9 of weights in 16-bit according to different protection signals. Direct SFT update magnitude underperforms activation-based protection, and “SFT mid” is also poor, but “SFT both ends” reaches the best average score, 70.28, compared with 68.45 for activation on the AWQ calibration set and 66.80 for activation on reasoning calibration (Zhang et al., 31 Jan 2026). The paper presents this as evidence that the useful signal is specifically the U-shaped one, not simply the magnitude of large updates.

The main evaluation spans SFT, DPO, and RL fine-tuning regimes over four reasoning benchmarks: AIME-120, FOLIO, temporal sequences from BIG-Bench Hard, and GPQA-Diamond (Zhang et al., 31 Jan 2026). Validation uses AIME 2021 plus 100 randomly sampled FOLIO training instances. For calibration, GPTQ, GPTAQ, and ANY3 use 128 C4 samples of length 2048, whereas AWQ and QuantLRM use 128 samples of length 512; the paper also notes that activation-based methods did not benefit from a larger reasoning calibration set, so default calibration is retained for fairness (Zhang et al., 31 Jan 2026).

The primary focus is weight-only W3A16, with W4A16 reported separately. On SFT models at 3-bit, QuantLRM is reported as best across all three tested models. For R1-Llama-70B, average score rises to 71.30, versus 69.18 for AWQ and 68.65 for GPTQ. For R1-Qwen-32B, QuantLRM reaches 73.58, versus 70.55 for ANY3 and 70.50 for AWQ. For R1-Qwen3-8B, QuantLRM reaches 66.13, versus 64.48 for AWQ (Zhang et al., 31 Jan 2026). The paper emphasizes that gains are especially noticeable on AIME-120 and summarizes the SFT improvements as up to 3.03% average over the best PTQ baseline and at least 1.65% on several SFT LRMs.

The gains persist under preference and reinforcement-learning fine-tuning. On Olmo-3-7B-Think-DPO, QuantLRM achieves 58.40 average, slightly above AWQ’s 58.18 and well above GPTQ’s 53.80. On Olmo-3-7B-Think trained with RLVR, QuantLRM reaches 63.23, versus 56.68 for GPTQ and 56.15 for AWQ (Zhang et al., 31 Jan 2026). The paper reports this as its headline result: an average improvement of 6.55% on a reinforcement-learning fine-tuned model relative to the strongest baseline.

The ablations directly support the method’s central design. On R1-Qwen-32B, “SFT both ends (no zero handling)” yields 62.88 average, assigning zero updates directly to $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$0 gives 64.23, assigning them to $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$1 gives 63.68, but excluding zeros from quadratic fitting and then counting zero updates reaches 70.28, a 5.95% gain over the no-zero-handling version (Zhang et al., 31 Jan 2026). On R1-Llama-8B, “SFT mid” is unhelpful, and on R1-Qwen3-8B, activation-based protection on AWQ calibration gives 18.83, while “SFT both ends” gives 65.75 in the mixed-precision test (Zhang et al., 31 Jan 2026).

5. Pseudo-fine-tuning, practical deployment, and computational cost

QuantLRM requires some notion of checkpoint differences, which creates a practical difficulty for non-fine-tuned models or cases where the pre-fine-tuned checkpoint is unavailable. The paper addresses this with pseudo-fine-tuning: a short, simplified fine-tuning run on the target model performed solely to collect update traces (Zhang et al., 31 Jan 2026). The demonstration uses Qwen3-1.7B with SFT on the OpenR1-Math-94k dataset from LlamaFactory. Weight updates are collected at multiple training steps, and QuantLRM performance improves as training proceeds; the paper reports that it eventually matches or surpasses AWQ after about 1956 training steps, and recommends pseudo-fine-tuning for a few thousand steps or a few epochs when needed (Zhang et al., 31 Jan 2026).

This extension broadens applicability, but the paper does not claim pseudo-fine-tuning is theoretically identical to real fine-tuning. A plausible implication is that QuantLRM needs informative update trajectories more than it needs the exact official training history.

The deployment overhead is reported as low. On 4-bit Olmo-3-7B-Think, QuantLRM adds 2 min 27 s of one-time preparation time versus none for AWQ, and its scaling-factor search takes 6 min 35 s versus 6 min 28 s for AWQ (Zhang et al., 31 Jan 2026). Inference speed is essentially identical—30.33 vs 30.24 tokens/s—because both use the same AWQ kernel, and GPU memory use is also reported as similar (Zhang et al., 31 Jan 2026). Inference evaluation uses vLLM, and latency experiments use the AWQ kernel for 4-bit models because that kernel supports only 4-bit inference.

Several engineering details qualify the method’s practicality. Because the pipeline uses only 16-bit arithmetic, large channel-importance values can overflow on very large models, especially when zero-update counts are large. To mitigate this, the authors slice $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$2 of MLP modules into pieces when counting zero updates on large models such as R1-Llama-70B and R1-Qwen-32B, then average the zero counts before multiplying by quadratic scores (Zhang et al., 31 Jan 2026). The paper also reports a model-specific adaptation—multiplying channel importance by absolute activation values—that helped on R1-Llama-70B but did not generalize, so such adjustments were selected by validation performance.

6. Position within LLM quantization research and documented limitations

QuantLRM occupies a distinct position among recent quantization methods. It is not a low-rank error-reconstruction method such as SRR, which decomposes quantized weights as $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$3 and treats rank allocation as a preservation-versus-reconstruction problem (Cho et al., 2 Feb 2026). It is also not a flexible low-rank PTQ method such as FLRQ, which incrementally selects layerwise ranks and alternates low-rank extraction with clipped residual quantization (Gul et al., 9 Jan 2026). Instead, QuantLRM leaves the weight-only PTQ formulation intact and changes the importance signal that drives AWQ-style channel scaling (Zhang et al., 31 Jan 2026).

This also distinguishes it from QAT-style approaches such as LR-QAT, which place low-rank auxiliary weights inside the quantizer and train them so they can be merged into the final low-bit tensor (Bondarenko et al., 2024), and from LoRDS, which factorizes the quantization scaling manifold as $$I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},$$4 and uses that low-rank scale parameterization across PTQ, QAT, and PEFT (Tang et al., 30 Jan 2026). QuantLRM is closer to PTQ saliency methods such as AWQ and GPTQ in deployment style, but it replaces activation or curvature proxies with fine-tuning dynamics (Zhang et al., 31 Jan 2026).

A second misconception is that QuantLRM is the strongest choice across all bit-widths. The paper explicitly states that its strongest contribution is to push W3A16 quantization of LRMs. In W4A16, PTQ is described as already near-saturated: on R1-Llama-70B, QuantLRM is competitive at 75.35 average, but not best, with AWQ at 75.60 (Zhang et al., 31 Jan 2026). The method therefore derives most of its significance from the low-bit reasoning regime where prior methods are known to degrade reasoning performance.

The central limitation is access to meaningful fine-tuning signals. Real checkpoint pairs are ideal; pseudo-fine-tuning is an effective approximation, but it remains an approximation (Zhang et al., 31 Jan 2026). The paper’s empirical case against activation-based and second-order methods is also primarily comparative rather than mechanistic: it shows that QuantLRM’s channel-importance measure is more effective than activation or second-order information on the reported tasks, especially at 3 bits, but it does not provide a deeper Hessian-versus-update theory (Zhang et al., 31 Jan 2026).

QuantLRM’s broader significance lies in the view it advances: post-training quantization for reasoning models can benefit from fine-tuning dynamics, not only from activations or curvature. This suggests a shift in what is treated as useful compression metadata. A plausible implication is that future PTQ methods for LRMs may increasingly treat training traces as first-class structure, especially when reasoning performance, rather than generic perplexity, is the deployment target.