QuantVSR: Low-Bit PTQ for Video Super-Resolution
- The paper introduces a low-bit post-training quantization method integrating a low-bit branch, auxiliary skip connection, and learnable bias correction for video super-resolution.
- It leverages spatio-temporal complexity aware rank allocation to dynamically assign full-precision capacity, preserving temporal consistency and reconstruction fidelity.
- Empirical results demonstrate that QuantVSR nearly matches full-precision performance at 4-bit precision while significantly reducing parameters and computational costs.
Searching arXiv for the specified paper and directly related context. {"query":"arXiv (Chai et al., 6 Aug 2025) QuantVSR Low-Bit Post-Training Quantization for Real-World Video Super-Resolution"} arxiv_search({"query":"(Chai et al., 6 Aug 2025)"}) QuantVSR is a low-bit post-training quantization method for real-world video super-resolution (VSR) developed for diffusion-based models whose slow processing speeds and heavy resource consumption hinder practical deployment. It is formulated on top of MGLD-VSR, a diffusion-based UNet for real-world VSR, and combines a low-bit branch, a low-rank full-precision auxiliary branch, and a learnable bias correction mechanism to preserve fidelity under aggressive quantization. The method targets the specific difficulty of quantizing VSR models, where temporal characteristics and high fidelity requirements make direct transfer of image-oriented quantization strategies inadequate. In the reported experiments, QuantVSR is evaluated at W6A6 and W4A4 precision and is described as obtaining comparable performance with the full-precision model while significantly outperforming recent leading low-bit quantization methods (Chai et al., 6 Aug 2025).
1. Problem Setting and Model Scope
QuantVSR addresses post-training quantization for real-world VSR in the diffusion-model regime. The underlying motivation is explicit: diffusion models have shown superior performance in real-world VSR, but their computational cost and memory footprint impede deployment. Quantization is therefore used as a compression strategy, yet the paper identifies two obstacles specific to VSR: temporal dynamics and stringent reconstruction fidelity (Chai et al., 6 Aug 2025).
The base model is MGLD-VSR, described as a diffusion-based UNet for real-world video super-resolution. Within this setting, QuantVSR replaces each full-precision Linear, Conv2d, and Conv3d layer with a quantized layer comprising three components: a low-bit branch, a skip-connection full-precision auxiliary branch, and a learnable bias alignment term. This decomposition is central to the method’s design because it preserves an explicit high-precision correction path while allowing most arithmetic to be performed in low precision.
A plausible implication is that QuantVSR is not merely a generic PTQ wrapper but a VSR-specific quantization framework structured around the error modes induced by spatio-temporal data. That interpretation is supported by the explicit use of both spatial and temporal statistics in rank allocation and by the addition of a bias-correction module aimed at systematic quantization error.
2. Quantized Layer Formulation
In QuantVSR, each target layer is transformed into a dual-branch structure. The first branch is the low-bit computation path. After Hadamard smoothing, activations and weights are quantized to the designated bit-widths $W_b/A_b$ and multiplied in integer arithmetic. The second branch is a full-precision auxiliary skip connection implemented with two low-rank matrices,
$L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$
which approximate the original weight $W$. A learnable bias alignment vector $A_\text{bias}$, with the same shape as the layer bias, is added to absorb average quantization bias (Chai et al., 6 Aug 2025).
The workflow is divided into two optimization stages. In the STCA stage, the method computes a layer-specific rank $r$ from calibration data, initializes $L_1$ and $L_2$ by truncated SVD, and then jointly refines the low-rank branch and the quantized residual branch. In the LBA stage, the low-rank and low-bit parameters are frozen, and only $A_\text{bias}$ is trained. Both stages minimize the same reconstruction objective, namely mean squared error between the full-precision layer output and the quantized layer output.
This architecture differs from a single-path low-bit approximation because the full-precision auxiliary path remains active during compensation. The ablation data suggest that this auxiliary structure is not incidental: removing the skip connection or replacing adaptive rank allocation with a fixed-rank alternative produces large degradations in PSNR.
3. Spatio-Temporal Complexity Aware Rank Allocation
The spatio-temporal complexity aware (STCA) mechanism is the method’s principal device for assigning layer-specific full-precision capacity. For a layer input
$\mathbf{X}\in\mathbb{R}^{T\times C\times H\times W},$
QuantVSR defines temporal complexity as
$C_t = \frac{1}{T-1}\sum_{t=1}^{T-1}\frac{1}{C\,H\,W}\|\mathbf{X}_{t+1}-\mathbf{X}_{t}\|_{2}^{2},$
and spatial complexity as
$L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$0
where $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$1 denotes standard deviation over spatial dimensions (Chai et al., 6 Aug 2025).
The rank-allocation procedure uses a small calibration set consisting of 1,800 UNet feature pairs from REDS30. From the empirical distributions of $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$2, the lower and upper thresholds $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$3 and $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$4 are defined as the 25th and 75th percentiles. Each layer is initialized with $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$5. For each calibration sample, the rank is incremented by one if $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$6 and $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$7, decremented by one if $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$8 and $L_1\in\mathbb{R}^{m\times r}, \qquad L_2\in\mathbb{R}^{r\times n},$9, and otherwise left unchanged. The final rank is clamped to $W$0 and rounded to the nearest multiple of 8 for efficient GPU use.
After rank assignment, $W$1 and $W$2 are initialized by the top-$W$3 SVD of $W$4, and the low-rank branch is jointly fine-tuned with the low-bit residual branch on calibration MSE. The reported ablation makes the effect of STCA concrete. On SPMCS at W4A4, the no-skip-connection configuration yields PSNR $W$5 dB, a fixed-rank skip connection (SVDSC, $W$6) yields PSNR $W$7 dB, and STCA yields PSNR $W$8 dB. The average rank learned by STCA is about 24, compared with fixed rank 32 in SVDQuant. This suggests that layerwise heterogeneity in spatio-temporal complexity is materially important and that adaptive allocation uses full-precision capacity more efficiently than a uniform-rank policy.
4. Learnable Bias Alignment and Quantization Error Modeling
QuantVSR introduces learnable bias alignment (LBA) to address systematic bias induced by quantization. The paper expresses the expected discrepancy between quantized and full-precision multiplication as
$W$9
with $A_\text{bias}$0 and $A_\text{bias}$1 (Chai et al., 6 Aug 2025).
LBA addresses this by inserting a learned bias vector $A_\text{bias}$2 after the low-bit multiplication. During the final PTQ stage, all weights and low-rank branches are frozen and only $A_\text{bias}$3 is trained using the same MSE objective used elsewhere in the framework. At inference time, $A_\text{bias}$4 is fused into the layer bias, so the module introduces no extra inference cost.
The ablation shows that LBA produces a measurable but small improvement beyond STCA alone. On SPMCS at W4A4, STCA alone attains PSNR $A_\text{bias}$5 dB and SSIM $A_\text{bias}$6, while STCA + LBA attains PSNR $A_\text{bias}$7 dB and SSIM $A_\text{bias}$8, with slight gains in LPIPS, DISTS, and DOVER. A common misconception would be that the dominant problem in low-bit VSR is solely low-rank approximation error. The reported results suggest instead that residual systematic bias remains relevant even after adaptive low-rank compensation, albeit at a smaller magnitude than the gains attributable to STCA.
5. Quantization Scheme, Calibration, and Optimization Objective
QuantVSR evaluates W6A6 and W4A4 settings, with W8A8 used in ablations. The quantizer is uniform integer quantization: $A_\text{bias}$9 where $r$0 is the scale, $r$1 is the zero-point, $r$2 is the integer range, and for signed $r$3-bit quantization,
$r$4
The operator $r$5 denotes round-to-nearest (Chai et al., 6 Aug 2025).
To mitigate outliers, inputs and weights are multiplied by a random Hadamard matrix $r$6 before quantization, following the same principle referenced as QuaRot in the source description. Calibration uses 1,800 pairs of noisy-latent and cleaner-latent features sampled from the diffusion UNet on REDS30.
The optimization target in both the STCA refinement stage and the LBA stage is the reconstruction loss
$r$7
During the STCA stage, gradients flow through $r$8 and the quantized residual branch, with STE used for the rounding operations. During the LBA stage, gradients update only $r$9. No additional regularizers are reported beyond this intermediate-feature reconstruction MSE.
These design choices indicate a strongly reconstruction-oriented PTQ pipeline. Rather than introducing auxiliary distributional or perceptual objectives at calibration time, the method relies on accurate layer-output matching and uses architectural compensation—adaptive low-rank correction plus bias correction—to recover downstream fidelity.
6. Empirical Results, Compression, and Temporal Behavior
The reported compression figures are given for the UNet only. The full-precision W32A32 model has 935 M parameters and 1,881 G operations. QuantVSR at W6A6 reduces this to 204 M parameters and 446 G operations, corresponding to $L_1$0 and $L_1$1, respectively. QuantVSR at W4A4 reduces it further to 146 M parameters and 328 G operations, corresponding to $L_1$2 parameters and $L_1$3 operations (Chai et al., 6 Aug 2025).
| Setting | Parameters / Operations | Reported change |
|---|---|---|
| FP (W32A32) | 935 M / 1,881 G | baseline |
| QuantVSR W6A6 | 204 M / 446 G | $L_1$4 / $L_1$5 |
| QuantVSR W4A4 | 146 M / 328 G | $L_1$6 / $L_1$7 |
Selected quantitative results are reported on the MGLD-VSR backbone. On REDS4 synthetic at W4A4, the full-precision model attains PSNR $L_1$8 dB and SSIM $L_1$9, while QuantVSR W4A4 attains PSNR $L_2$0 dB and SSIM $L_2$1. The best competing 4-bit method listed, SVDQuant, attains PSNR $L_2$2 dB and SSIM $L_2$3. On MVSR4x, characterized as real-world, the full-precision model attains PSNR $L_2$4 dB and SSIM $L_2$5, QuantVSR attains PSNR $L_2$6 dB and SSIM $L_2$7, and SVDQuant attains PSNR $L_2$8 dB and SSIM $L_2$9.
The evaluation also includes LPIPS and DISTS, no-reference IQA metrics—CLIP-IQA, MUSIQ, NIQE, and MANIQA—and VQA metrics DOVER and $A_\text{bias}$0. The summary given is that QuantVSR nearly matches the full-precision model on reference metrics and outperforms all 4-bit and 6-bit baselines across the board. Qualitative and temporal-consistency results are similarly specific: even at 4 bits, QuantVSR is reported to preserve sharp textures and stable motion without flicker, whereas prior PTQ methods produce artifacts or temporal jitter.
These results are notable because the strongest reductions are achieved at W4A4, where low-bit degradation would typically be expected to be pronounced. The reported numbers indicate that the method’s compensation mechanisms are sufficient, at least on the tested benchmarks, to keep reconstruction quality close to full precision while maintaining markedly lower parameter count and operation count.
7. Interpretation Within Low-Bit VSR Research
Within the framing provided by the paper, QuantVSR is defined by three interacting components: a complexity-aware per-layer low-rank full-precision branch via STCA, low-bit quantization with random rotations and STE, and a learned bias corrector via LBA (Chai et al., 6 Aug 2025). The empirical narrative of the ablation study indicates that these components are not interchangeable. The largest quality recovery comes from the adaptive full-precision skip path, while LBA contributes a smaller but consistent improvement.
This decomposition clarifies the paper’s research contribution relative to a simpler interpretation of post-training quantization. QuantVSR is not presented as a purely numerical quantizer; it is a structured approximation strategy tailored to diffusion-based VSR layers under spatio-temporal variation. The use of calibration statistics from REDS30, the per-layer rank adaptation, and the explicit treatment of systematic bias all point toward a PTQ regime in which preserving temporal consistency is as important as preserving per-frame distortion metrics.
The paper’s summary claim is correspondingly narrow and technical: QuantVSR achieves 4-bit video super-resolution with almost no loss in PSNR and SSIM, approximately $A_\text{bias}$1 reduction in compute, approximately $A_\text{bias}$2 reduction in parameters, and superior temporal consistency. Given the reported results, a plausible implication is that low-bit PTQ for diffusion-based VSR need not be restricted to mild compression regimes such as 8-bit inference, provided that full-precision compensation is allocated adaptively and systematic bias is explicitly corrected.