---
title: 'QuantLRM: Fine-Tuning Driven LRM Quantization'
url: https://www.emergentmind.com/topics/quantlrm
type: topic
---

# QuantLRM: Fine-Tuning Driven LRM Quantization

QuantLRM is a post-training, weight-only quantization method for Large Reasoning Models (LRMs) that uses fine-tuning traces—specifically, weight updates between a pre-fine-tuned model and its fine-tuned version—to estimate which channels should be protected during quantization [2602.02581]. It targets the specific problem of accurate low-bit quantization of LRMs after reasoning-incentivized post-training, rather than generic LLM compression, and is motivated by the claim that standard PTQ signals such as activation statistics and second-order or Hessian approximations do not fully capture what supervised fine-tuning, direct preference optimization, or reinforcement-learning-based reasoning post-training changed or preserved in the model [2602.02581]. Its central conceptual contribution is the hypothesis of “protecting both ends”: for reasoning-tuned models, both the smallest and the largest weight updates during fine-tuning are informative, whereas updates of intermediate magnitude are less diagnostic [2602.02581].

## 1. Problem setting and conceptual basis

QuantLRM is situated in the deployment regime where modern LRMs are produced by substantial post-training pipelines, including supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning variants such as RLVR, but conventional PTQ treats the final checkpoint as a static artifact and discards the training history [2602.02581]. The method treats this as wasteful, because the difference between pre- and post-fine-tuning weights is assumed to encode which parameters were critical to adaptation.

The paper’s intuition is explicitly inspired by classical magnitude pruning. If absolute weight magnitude can serve as a proxy for pruning importance, QuantLRM asks whether absolute weight update magnitude during reasoning-oriented fine-tuning can serve as a proxy for quantization importance [2602.02581]. The naïve version of that idea would preserve channels with the largest updates, but the paper reports that this is not sufficient: simply ranking channels by large update magnitude performs worse than activation-based importance. This leads to the stronger claim that fine-tuning dynamics in LRMs are structurally U-shaped rather than monotonic.

Under this view, very large updates likely correspond to parameters heavily used to encode new reasoning behavior during fine-tuning, whereas very small or zero updates may indicate parameters that optimization intentionally left nearly untouched because they are important for pre-existing general capabilities such as language modeling, instruction following, or stable backbone functionality [2602.02581]. A plausible implication is that QuantLRM reinterprets reasoning post-training not only as selective modification, but also as selective preservation.

## 2. Fine-tuning traces and the “protecting both ends” hypothesis

QuantLRM computes importance from weight deltas between checkpoints. The paper defines channel importance for a channel \(c\) in module \(m\) at layer \(l\) as the average of individual weight-importance scores:
\[
I_{l,c}^m = \frac{1}{N}\sum_{n=1}^{N} i_{l,c}^{m,n},
\]
where \(N\) is the number of weights in the channel [2602.02581]. For comparison, the paper also states the activation-based channel importance used in validation experiments:
\[
I_{l,c}^m = \sum_{i=1}^{M} (a_i^m)^2,
\]
where \(M\) is sequence length and \(a_i^m\) is the activation on that channel [2602.02581].

The distinctive step is the definition of \(i_{l,c}^{m,n}\) from update magnitudes. QuantLRM fits restricted quadratic functions over weight updates, split at the global median update, with defaults \(y_{\min}=1\) and \(y_{\max}=10\), so that the median maps to \(y_{\min}\) and both the minimum and maximum map to \(y_{\max}\) [2602.02581]. This is the formal implementation of “protecting both ends”: middle updates receive low scores, while both extremes receive high scores.

The method then addresses the case \(\Delta W_{\min}=0\), which the paper identifies as empirically common because zero updates can account for more than \(1\%\) of all weights [2602.02581]. To avoid distortion of the left quadratic fit, QuantLRM excludes zero updates when fitting the quadratics, recomputes the median over positive updates only, assigns zero updates the baseline value \(y_{\min}\), and then explicitly amplifies channels with many zero updates. If \(Z_{l,c}^m\) is the number of zero updates in a channel, the final importance is
\[
I_{l,c}^m = \left( \frac{1}{N}\sum_{n=1}^{N} f\!\left(\Delta W_{l,c}^{m,n}\right) \right)\cdot (Z_{l,c}^m + 1).
\]
The \(+1\) avoids nullifying a channel when \(Z_{l,c}^m=0\) [2602.02581].

This zero-handling step is not incidental. The paper presents it as a crucial part of the final design, because QuantLRM is not merely a nonlinear remapping of update magnitudes; it is a channel-importance measure that explicitly rewards both large adaptation and concentrated preservation [2602.02581].

## 3. Quantization mechanism and integration with AWQ-style scaling

Once channel importance is computed, QuantLRM uses it inside an AWQ-style scaling-based quantization objective rather than introducing a new quantizer [2602.02581]. For input features \(X\), fine-tuned weights \(W'\), quantizer \(Q(\cdot)\), and scaling vector \(s\), the loss is
\[
\mathcal{L}(s) = \left| Q\!\left(W' \cdot \operatorname{diag}(s)\right) \left(\operatorname{diag}(s)^{-1}\cdot X\right) - W'X \right|.
\]
To inject fine-tuning signals, QuantLRM sets
\[
s = (I_l^m)^\alpha, \qquad \alpha^* = \arg\min_{\alpha} \mathcal{L}\big((I_l^m)^\alpha\big),
\]
and searches over 20 candidates for \(\alpha\) in the interval \([0,1]\), module by module [2602.02581].

The resulting workflow is explicit in the paper: compute checkpoint deltas, fit the restricted quadratics, count zero updates, form channel importance, grid-search \(\alpha\), and then quantize [2602.02581]. This makes QuantLRM a better importance-driven scaling policy for weight-only PTQ, not a replacement for the surrounding quantization infrastructure.

A common misconception is that QuantLRM is a generic low-rank correction scheme or a quantization-aware training procedure. The paper does not describe it that way. It requires checkpoint differences, but it does not require training-time gradients or optimizer states, and it remains much cheaper than QAT [2602.02581]. Another misconception is that it only protects heavily changed weights; the method’s defining claim is precisely that carefully preserved weights are also informative.

## 4. Validation and benchmark results

The paper first validates the “protecting both ends” hypothesis through a mixed-precision probe. In these experiments, R1-Distill-Qwen-32B is quantized to 3-bit while keeping \(5\%\) of weights in 16-bit according to different protection signals. Direct SFT update magnitude underperforms activation-based protection, and “SFT mid” is also poor, but “SFT both ends” reaches the best average score, **70.28**, compared with **68.45** for activation on the AWQ calibration set and **66.80** for activation on reasoning calibration [2602.02581]. The paper presents this as evidence that the useful signal is specifically the U-shaped one, not simply the magnitude of large updates.

The main evaluation spans SFT, DPO, and RL fine-tuning regimes over four reasoning benchmarks: AIME-120, FOLIO, temporal sequences from BIG-Bench Hard, and GPQA-Diamond [2602.02581]. Validation uses AIME 2021 plus 100 randomly sampled FOLIO training instances. For calibration, GPTQ, GPTAQ, and ANY3 use 128 C4 samples of length 2048, whereas AWQ and QuantLRM use 128 samples of length 512; the paper also notes that activation-based methods did not benefit from a larger reasoning calibration set, so default calibration is retained for fairness [2602.02581].

The primary focus is weight-only W3A16, with W4A16 reported separately. On SFT models at 3-bit, QuantLRM is reported as best across all three tested models. For R1-Llama-70B, average score rises to **71.30**, versus **69.18** for AWQ and **68.65** for GPTQ. For R1-Qwen-32B, QuantLRM reaches **73.58**, versus **70.55** for ANY3 and **70.50** for AWQ. For R1-Qwen3-8B, QuantLRM reaches **66.13**, versus **64.48** for AWQ [2602.02581]. The paper emphasizes that gains are especially noticeable on AIME-120 and summarizes the SFT improvements as up to **3.03%** average over the best PTQ baseline and at least **1.65%** on several SFT LRMs.

The gains persist under preference and reinforcement-learning fine-tuning. On Olmo-3-7B-Think-DPO, QuantLRM achieves **58.40** average, slightly above AWQ’s **58.18** and well above GPTQ’s **53.80**. On Olmo-3-7B-Think trained with RLVR, QuantLRM reaches **63.23**, versus **56.68** for GPTQ and **56.15** for AWQ [2602.02581]. The paper reports this as its headline result: an average improvement of **6.55%** on a reinforcement-learning fine-tuned model relative to the strongest baseline.

The ablations directly support the method’s central design. On R1-Qwen-32B, “SFT both ends (no zero handling)” yields **62.88** average, assigning zero updates directly to \(y_{\max}\) gives **64.23**, assigning them to \(y_{\min}\) gives **63.68**, but excluding zeros from quadratic fitting and then counting zero updates reaches **70.28**, a **5.95%** gain over the no-zero-handling version [2602.02581]. On R1-Llama-8B, “SFT mid” is unhelpful, and on R1-Qwen3-8B, activation-based protection on AWQ calibration gives **18.83**, while “SFT both ends” gives **65.75** in the mixed-precision test [2602.02581].

## 5. Pseudo-fine-tuning, practical deployment, and computational cost

QuantLRM requires some notion of checkpoint differences, which creates a practical difficulty for non-fine-tuned models or cases where the pre-fine-tuned checkpoint is unavailable. The paper addresses this with pseudo-fine-tuning: a short, simplified fine-tuning run on the target model performed solely to collect update traces [2602.02581]. The demonstration uses Qwen3-1.7B with SFT on the OpenR1-Math-94k dataset from LlamaFactory. Weight updates are collected at multiple training steps, and QuantLRM performance improves as training proceeds; the paper reports that it eventually matches or surpasses AWQ after about **1956 training steps**, and recommends pseudo-fine-tuning for **a few thousand steps or a few epochs** when needed [2602.02581].

This extension broadens applicability, but the paper does not claim pseudo-fine-tuning is theoretically identical to real fine-tuning. A plausible implication is that QuantLRM needs informative update trajectories more than it needs the exact official training history.

The deployment overhead is reported as low. On 4-bit Olmo-3-7B-Think, QuantLRM adds **2 min 27 s** of one-time preparation time versus none for AWQ, and its scaling-factor search takes **6 min 35 s** versus **6 min 28 s** for AWQ [2602.02581]. Inference speed is essentially identical—**30.33 vs 30.24 tokens/s**—because both use the same AWQ kernel, and GPU memory use is also reported as similar [2602.02581]. Inference evaluation uses vLLM, and latency experiments use the AWQ kernel for 4-bit models because that kernel supports only 4-bit inference.

Several engineering details qualify the method’s practicality. Because the pipeline uses only 16-bit arithmetic, large channel-importance values can overflow on very large models, especially when zero-update counts are large. To mitigate this, the authors slice \(\Delta W\) of MLP modules into pieces when counting zero updates on large models such as R1-Llama-70B and R1-Qwen-32B, then average the zero counts before multiplying by quadratic scores [2602.02581]. The paper also reports a model-specific adaptation—multiplying channel importance by absolute activation values—that helped on R1-Llama-70B but did not generalize, so such adjustments were selected by validation performance.

## 6. Position within LLM quantization research and documented limitations

QuantLRM occupies a distinct position among recent quantization methods. It is not a low-rank error-reconstruction method such as SRR, which decomposes quantized weights as \(Q + LR\) and treats rank allocation as a preservation-versus-reconstruction problem [2602.02001]. It is also not a flexible low-rank PTQ method such as FLRQ, which incrementally selects layerwise ranks and alternates low-rank extraction with clipped residual quantization [2601.05684]. Instead, QuantLRM leaves the weight-only PTQ formulation intact and changes the importance signal that drives AWQ-style channel scaling [2602.02581].

This also distinguishes it from QAT-style approaches such as LR-QAT, which place low-rank auxiliary weights inside the quantizer and train them so they can be merged into the final low-bit tensor [2406.06385], and from LoRDS, which factorizes the quantization scaling manifold as \(S=BA\) and uses that low-rank scale parameterization across PTQ, QAT, and PEFT [2601.22716]. QuantLRM is closer to PTQ saliency methods such as AWQ and GPTQ in deployment style, but it replaces activation or curvature proxies with fine-tuning dynamics [2602.02581].

A second misconception is that QuantLRM is the strongest choice across all bit-widths. The paper explicitly states that its strongest contribution is to push W3A16 quantization of LRMs. In W4A16, PTQ is described as already near-saturated: on R1-Llama-70B, QuantLRM is competitive at **75.35** average, but not best, with AWQ at **75.60** [2602.02581]. The method therefore derives most of its significance from the low-bit reasoning regime where prior methods are known to degrade reasoning performance.

The central limitation is access to meaningful fine-tuning signals. Real checkpoint pairs are ideal; pseudo-fine-tuning is an effective approximation, but it remains an approximation [2602.02581]. The paper’s empirical case against activation-based and second-order methods is also primarily comparative rather than mechanistic: it shows that QuantLRM’s channel-importance measure is more effective than activation or second-order information on the reported tasks, especially at 3 bits, but it does not provide a deeper Hessian-versus-update theory [2602.02581].

QuantLRM’s broader significance lies in the view it advances: post-training quantization for reasoning models can benefit from fine-tuning dynamics, not only from activations or curvature. This suggests a shift in what is treated as useful compression metadata. A plausible implication is that future PTQ methods for LRMs may increasingly treat training traces as first-class structure, especially when reasoning performance, rather than generic perplexity, is the deployment target.

Source: https://www.emergentmind.com/topics/quantlrm