MB-CLoRA: Cascaded Low-Rank Adapters for Multi-Bit PTQ

• The paper introduces a cascaded low-rank compensation mechanism that hierarchically reuses a high-bit adapter backbone to extend calibration across multiple bit-widths.
• MB-CLoRA enables a one-pass post-training quantization, reducing redundancy and training cost while supporting real-time switching across 4-8 bits.
• Empirical results demonstrate improved low-bit accuracy with zero inference overhead, confirming the effectiveness of integrating MB-CLoRA within the QuEPT framework.

Searching arXiv for the specified paper and closely related low-rank/quantization context. Multi-Bit Cascaded Low-Rank Adapters (MB-CLoRA) are a low-rank compensation mechanism introduced within QuEPT, a post-training quantization scheme for Transformers that targets elastic multi-bit deployment through a single calibration pass. MB-CLoRA is designed for the setting in which one model must operate across several predefined bit-widths, specifically $b \in \{4,5,6,7,8\}$, without repeated optimization. Its central idea is to organize low-rank adapters hierarchically so that higher-bit adapters provide a shared backbone and lower-bit adapters inherit and extend that backbone, thereby reducing redundancy across bit-widths while improving low-bit calibration quality and preserving real-time switching capability (Xu et al., 13 Feb 2026).

1. Motivation and problem setting

Transformer layers are described as extremely sensitive to quantization noise, and that sensitivity varies substantially across bit-widths, for example between 4 bit and 8 bit. In a multi-bit post-training quantization setting, calibration must therefore accommodate several operating points simultaneously. The underlying difficulty is that naïve sharing strategies and naïve separation strategies each incur a distinct cost: training a separate adapter for each bit-width has high training and storage cost, while sharing a single low-rank compensation across all bit-widths leads to poor low-bit performance (Xu et al., 13 Feb 2026).

MB-CLoRA addresses this tension by introducing a hierarchical low-rank compensation mechanism. In the formulation reported for QuEPT, high-bit adapters form the backbone, and mid- and low-bit adapters inherit and extend that backbone. This construction is explicitly intended to reduce redundancy across bit-widths and to reuse higher-precision knowledge during lower-bit calibration. The reported goal is not merely parameter sharing in the generic sense, but a structured transfer of calibration capacity across precision tiers.

The immediate significance of this design is operational: after a one-shot PTQ calibration procedure, the same model can be deployed at 4, 5, 6, 7, or 8 bits by selecting the corresponding adapter slice, with no re-optimization required. This suggests that MB-CLoRA is best understood as a mechanism for elastic precision within PTQ rather than as a conventional task-adaptation LoRA variant.

2. Architecture and cascaded rank design

MB-CLoRA inserts a low-rank adapter inside the weight clipping operator of each linear or attention weight matrix $W \in \mathbb{R}^{p \times t}$. Two learnable matrices are introduced,

$$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$

so that the compensation applied to the weight is

$R = BA,$

with total rank

$r = r_h + r_m + r_l.$

The target bit-width set is partitioned into three tiers:

• high-bit: $\mathcal{B}_H = \{7,8\}$ with rank $r_h$,
• mid-bit: $\mathcal{B}_M = \{5,6\}$ with extra rank $r_m$,
• low-bit: $\mathcal{B}_L = \{4\}$ with extra rank $W \in \mathbb{R}^{p \times t}$0 (Xu et al., 13 Feb 2026).

The defining feature of the method is the cascaded structure. For any $W \in \mathbb{R}^{p \times t}$1, the effective rank $W \in \mathbb{R}^{p \times t}$2 is

$W \in \mathbb{R}^{p \times t}$3

The compensation matrix for bit-width $W \in \mathbb{R}^{p \times t}$4 is then the leading submatrix

$W \in \mathbb{R}^{p \times t}$5

At inference for bit-width $W \in \mathbb{R}^{p \times t}$6, the system quantizes $W \in \mathbb{R}^{p \times t}$7 through clipping, rounding, and dequantization, with no further parameter changes. The architectural consequence is that low-bit operation does not replace higher-bit calibration structure; it extends it. A plausible implication is that the method imposes an ordering on representational corrections, where more severe quantization regimes receive strictly additional low-rank capacity rather than an unrelated compensation.

3. Quantization formulation and calibration objective

For each transformer block $W \in \mathbb{R}^{p \times t}$8 and each bit-width $W \in \mathbb{R}^{p \times t}$9, the quantized weight is defined as

$$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$0

where $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$1 and $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$2 are the learned scale and zero-point, and $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$3 (Xu et al., 13 Feb 2026).

Calibration is performed in a one-shot, block-wise reconstruction regime. In each block, three bit-widths are sampled, one from each tier: $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$4, $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$5, and $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$6. The inputs $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$7 are fused via Multi-Bit Token Merging (MB-ToMe), and the quantized outputs $$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$8 are compared with the full-precision output using MAE. The joint objective is

$$A \in \mathbb{R}^{r \times t}, \qquad B \in \mathbb{R}^{p \times r},$$9

where $R = BA,$0 are the clipping thresholds that determine $R = BA,$1 and $R = BA,$2, and $R = BA,$3 depends on $R = BA,$4 through cascaded rank selection.

Several implementation details are specified. The pipeline per block is: freeze $R = BA,$5 and activation quantization scales; initialize $R = BA,$6; sample $R = BA,$7; fuse incoming features $R = BA,$8 via MB-ToMe; forward-quantize for each sampled bit-width; compute $R = BA,$9 loss with respect to the full-precision block output; backpropagate only into $r = r_h + r_m + r_l.$0; then move to the next block. Typical hyper-parameters include Adam with learning rate approximately $r = r_h + r_m + r_l.$1, no weight decay, batch size 32 per block, and one pass per block. The reported data slices are 1,024 unlabeled ImageNet images for ViTs, 128 C4 text samples for LLaMA LLMs, and 128 image-caption pairs from ShareGPT4V’s COCO subset for LLaVA-OV multimodal models. The paper further states that MAE showed approximately $r = r_h + r_m + r_l.$2 better accuracy than MSE, and that the MB-ToMe selection ratio is approximately $r = r_h + r_m + r_l.$3 of “stable” tokens (Xu et al., 13 Feb 2026).

4. Cross-bit synergy and switching behavior

The method’s rationale is articulated in terms of cross-bit synergy. Because the higher-bit tiers are calibrated first, their ranks form the base $r = r_h + r_m + r_l.$4. All lower-bit settings inherit those parameters and learn only the extra correction capacity $r = r_h + r_m + r_l.$5 or $r = r_h + r_m + r_l.$6 on top. The paper states that gradients from $r = r_h + r_m + r_l.$7 gently diffuse into $r = r_h + r_m + r_l.$8 and $r = r_h + r_m + r_l.$9, so that higher-precision structure guides and regularizes low-bit compensation rather than letting the different bit-width solutions diverge (Xu et al., 13 Feb 2026).

This organization distinguishes MB-CLoRA from two boundary cases that are explicitly considered in the QuEPT analysis. It is not equivalent to a fully shared adapter, because additional rank is allocated to lower tiers. It is also not equivalent to independent per-tier adapters, because the lower tiers inherit the high-bit backbone. A common misconception would therefore be to treat MB-CLoRA as only a parameter-saving device. The reported formulation assigns it a dual role: reducing redundancy across bit-widths and converting cross-bit competition into synergy.

The switching mechanism is correspondingly simple. After one calibration pass, deployment at bit-width $\mathcal{B}_H = \{7,8\}$0 consists of loading $\mathcal{B}_H = \{7,8\}$1. No re-training or per-bit optimization is required. In the terminology of QuEPT, this supports real-time switching between uniform quantization and mixed precision quantization without repeated optimization. This suggests that MB-CLoRA is coupled not only to low-rank reconstruction, but also to a runtime model-management objective in which several quantized operating points are exposed from a single calibrated parameterization.

5. Computational and storage characteristics

The reported inference overhead is zero. The reason given is that the LoRA modules are inside the clipping operator; once training is complete, one simply adds $\mathcal{B}_H = \{7,8\}$2 to $\mathcal{B}_H = \{7,8\}$3 and discards $\mathcal{B}_H = \{7,8\}$4. Accordingly, MB-CLoRA does not impose an additional online module execution cost during inference (Xu et al., 13 Feb 2026).

The storage overhead is described as a small fraction of the base model. For each weight matrix $\mathcal{B}_H = \{7,8\}$5, the additional parameters are $\mathcal{B}_H = \{7,8\}$6 and $\mathcal{B}_H = \{7,8\}$7, with $\mathcal{B}_H = \{7,8\}$8. The paper summarizes this as approximately $\mathcal{B}_H = \{7,8\}$9 parameters versus $r_h$0 full-precision parameters. Example ranks are $r_h$1 for ViT-S and $r_h$2 for LLaMA2-7B, corresponding to $r_h$3 for ViT small and $r_h$4 for LLaMA2-7B.

Training cost is reported as comparable to a single-bit PTQ pass. On ViT-S, QuEPT requires approximately 17 minutes, compared with PTMQ’s 430 minutes, while single-bit ERQ on ViT-S is summarized as approximately $r_h$5 layers; QuEPT instead performs a single shot through all bit-widths. The total training time is stated as on the order of 10–40 minutes on a single 3090 GPU. These figures are reported for QuEPT as a whole, but MB-CLoRA is one of the specific components that make the one-shot multi-bit calibration regime feasible.

6. Empirical behavior and ablation evidence

The empirical results reported for QuEPT provide direct evidence on the contribution of MB-CLoRA. On ViT-S for ImageNet top-1 accuracy, QuEPT achieves 75.1 at W4A4, 79.7 at W5A5, 80.6 at W6A6, 81.0 at W7A7, and 81.2 at W8A8, with a training time of 17 minutes and FP32 accuracy of 81.3. The comparative rows listed are ERQ, with 68.9, 78.8, 80.5, 80.9, and 81.2 across W4A4 through W8A8 and time $r_h$6, and PTMQ, with 76.1 at W6A6, 77.1 at W7A7, 78.2 at W8A8, and 430 minutes (Xu et al., 13 Feb 2026).

For LLaMA2-7B with weight and activation quantization, the reported values are: at W4A4, DuQuant obtains Wiki 6.28, C4 7.90, and 0-shot average 59.11, while QuEPT obtains Wiki 6.33, C4 7.86, and 0-shot average 61.62; at W6A6, SmoothQ obtains Wiki 6.20, C4 7.76, and 0-shot average 61.40, while QuEPT obtains Wiki 5.53, C4 7.03, and 0-shot average 65.48; at W8A8, QuEPT reports Wiki 5.48, C4 6.98, and 0-shot average 66.23. These are system-level QuEPT results, but they contextualize the role of MB-CLoRA inside a broader elastic-precision pipeline.

The MB-CLoRA ablation on ViT-S top-1 isolates the sharing strategy. A fully shared configuration with $r_h$7 gives 73.7 at W4A4, 80.6 at W6A6, and 81.1 at W8A8. An independent configuration with $r_h$8 gives 73.5, 80.7, and 81.2. MB-CLoRA with $r_h$9 gives 74.7, 80.6, and 81.1. This directly supports the claim that the cascaded design improves low-bit accuracy relative to both fully shared and independent baselines, while leaving higher-bit performance essentially unchanged.

Additional ablations clarify interactions with other QuEPT components. For MB-ToMe on ViT-S, Case 1 (random) yields 73.7 at W4A4, 78.6 at W5A5, and 80.6 at W6A6; Case 2 (uniform fuse) yields 73.5, 78.9, and 80.7; Case 3 (selective) yields 74.7, 79.4, and 80.6. The module-wise ablation shows the following ViT-S results: without clipping, MB-CLoRA, MB-ToMe, and MAE, performance is 69.2 at W4A4, 80.1 at W6A6, and 81.1 at W8A8; adding clipping alone gives 70.1, 80.2, and 81.1; clipping plus MB-CLoRA gives 72.2, 79.2, and 80.7; clipping plus MB-ToMe gives 73.0, 80.1, and 80.9; clipping plus MB-CLoRA and MB-ToMe gives 73.6, 80.1, and 81.0; clipping plus MB-CLoRA, MB-ToMe, and MAE gives 74.7, 80.7, and 81.1. Taken together, these results indicate that MB-CLoRA contributes most clearly at the low-bit end, and that its effect is strengthened when combined with MB-ToMe and MAE-based reconstruction.

7. Interpretation, scope, and limitations

Within QuEPT, MB-CLoRA is presented as a mechanism for multi-bit PTQ rather than as a generic low-rank fine-tuning method. Its role is to reconstruct block-wise multi-bit errors under one-shot calibration on a small data slice and to support deployment across predefined bit-widths through rank slicing. In that sense, the method belongs to the intersection of post-training quantization, low-rank compensation, and elastic precision Transformers (Xu et al., 13 Feb 2026).

The reported evidence supports three bounded conclusions. First, MB-CLoRA improves low-bit accuracy by sharing and extending high-precision adaptation. Second, it incurs no extra inference cost because the compensation can be folded into the quantized weight before deployment. Third, it enables a single unified quantized model that can be switched across 4–8 bit with minimal storage and training overhead. These claims are directly stated in the QuEPT account.

At the same time, the available evidence is tied to the specific QuEPT design. MB-CLoRA is trained jointly with clipping parameters and used together with MB-ToMe in the one-shot block-wise reconstruction procedure. A plausible implication is that its empirical behavior should be interpreted as part of an integrated calibration framework rather than as a standalone module whose effect is independent of data slice selection, block-wise reconstruction, or token-fusion strategy. The ablations do show a direct low-bit benefit from the cascaded sharing strategy itself, but they do not separate MB-CLoRA from the broader assumptions of QuEPT’s calibration regime.