QuantDemoire: Efficient PTQ for Demoiréing

• QuantDemoire is a framework for image demoiréing quantization that compresses deep models for edge deployment while preserving critical image quality.
• It combines an outlier-aware quantizer with frequency-aware calibration to mitigate distribution outliers and spectral distortions common in low-bit quantization.
• Experimental results indicate over 4 dB PSNR improvement versus standard methods, achieving substantial reductions in model parameters and computation.

QuantDemoire is a post-training quantization framework for image demoiréing, introduced to compress deep demoiréing models for edge deployment without incurring the severe quality collapse observed under standard low-bit quantization. The framework is tailored to the specific failure modes of demoiréing: distribution outliers in activations and weights, and weakened representations in smooth regions that manifest as visible banding artifacts. Its design therefore combines an outlier-aware quantizer with a frequency-aware calibration strategy, and it is evaluated on the ESDNet backbone across UHDM, FHDMi, and LCDMoiré under W8A8, W6A6, W4A4, and W3A3 settings (Chen et al., 5 Oct 2025).

1. Problem setting and technical motivation

Image demoiréing is a low-level restoration task that removes screen-capture moiré stripes. QuantDemoire starts from the observation that directly applying existing quantization methods to demoiréing models causes unusually severe degradation, even when comparable approaches are effective for classification or some restoration tasks. The paper attributes this to two coupled causes: outliers that inflate quantization ranges and compress usable precision for the bulk of values, and disproportionate damage to smooth low- and mid-frequency image regions, where low-bit quantization introduces banding artifacts (Chen et al., 5 Oct 2025).

This framing is significant because it recasts quantization failure in demoiréing as more than a generic numeric approximation problem. The degradation is described as both distributional and spectral. In particular, the task is sensitive not only to pointwise quantization error, but also to frequency-domain distortion in relatively flat regions where the model must preserve subtle tonal transitions. A plausible implication is that PTQ for demoiréing cannot be treated as a direct transplant of methods validated on higher-level vision tasks.

2. Outlier-aware quantization

The first component of QuantDemoire is an outlier-aware quantizer. For activations, it replaces global min/max calibration with random sampling so that rare extreme values do not dominate the quantization range. If $\text{Sample}_\gamma(\cdot)$ denotes random selection of a fraction $\gamma$ of elements, the sampled channelwise activation maximum is estimated as

$$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$

and the smoothing factor is

$$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$

After smoothing, the activation bounds are likewise derived from sampled activations,

$$u_a = \max(\text{Sample}_{\gamma_2}(\mathbf{X}\oslash)), \qquad l_a = \min(\text{Sample}_{\gamma_2}(\mathbf{X}\oslash)),$$

where $\oslash$ denotes division by the smoothing factor (Chen et al., 5 Oct 2025).

For weights, QuantDemoire uses a different mechanism because weights are much fewer in number and discarding their extremes is reported to be too damaging. It adopts a mixed-precision weight quantizer in which values outside percentile thresholds are stored in FP16, while the remainder are quantized: $$T_{\mathrm{low}} = P_\beta(\mathbf{W}), \qquad T_{\mathrm{high}} = P_{1-\beta}(\mathbf{W}),$$

$$\mathbf{W}_{\text{outlier}}= \{w_i \in \mathbf{W} \mid w_i < T_{\mathrm{low}} \ \text{or}\ w_i > T_{\mathrm{high}}\},$$

$$\mathbf{W}_{\text{normal}}=\mathbf{W}\setminus \mathbf{W}_{\text{outlier}}, \qquad \hat{\mathbf{W}}=\mathbf{W}_{\text{outlier}} \cup Q(\mathbf{W}_{\text{normal}}).$$

The reported choice is $\beta=0.005$, so 0.5% of weights at both tails are preserved in FP16 with negligible overhead. The paper’s interpretation is that activation outliers can be sampled away from calibration statistics, whereas weight outliers are structurally important but sparse enough to preserve separately (Chen et al., 5 Oct 2025).

The underlying fake quantization remains standard affine integer quantization: $\gamma$0

$\gamma$1

with quantized convolution

$\gamma$2

and straight-through estimation through rounding,

$\gamma$3

The novelty lies not in the affine quantizer itself, but in how calibration statistics and exceptional weights are handled.

3. Frequency-aware calibration

The second component is a frequency-aware calibration strategy used to optimize quantization parameters, especially activation boundaries, during PTQ calibration. The motivation is that ordinary pixel-space reconstruction losses do not sufficiently penalize banding artifacts in smooth regions. QuantDemoire therefore extracts low- and mid-frequency components from both the quantized output and the ground truth, then minimizes losses on those representations rather than relying only on full-spectrum image supervision (Chen et al., 5 Oct 2025).

The extraction operator is defined recursively from the $\gamma$4 kernel

$\gamma$5

with increasing dilation: $\gamma$6

$\gamma$7

For extraction depth $\gamma$8, the low/mid-frequency representation is $\gamma$9. Calibration then minimizes

$$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$0

where $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$1 is the low/mid-frequency component of the quantized output, $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$2 is the corresponding ground-truth component, $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$3 is pixel-wise L1 loss, and $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$4 is a VGG16-based perceptual loss. Only activation quantization boundaries are optimized; all model weights remain fixed. The reported choice is $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$5, described as the best balance between retaining mid-frequency structure and emphasizing the smooth regions where banding emerges (Chen et al., 5 Oct 2025).

This component is central to the framework’s task specificity. The calibration target is explicitly frequency selective, which differs from PTQ schemes that treat all reconstruction discrepancies as equally important in pixel space.

4. Experimental protocol and evaluation configuration

QuantDemoire is evaluated on UHDM, FHDMi, and LCDMoiré using ESDNet as the full-precision backbone. Baselines are MinMax, Percentile, 2DQuant, and SVDQuant, all applied as post-training quantization methods under the same static-quantization setup. Quantization is per-channel for weights and per-tensor for activations (Chen et al., 5 Oct 2025).

The calibration procedure uses 200 randomly selected training image pairs per dataset. These are cropped, then optimized for 4 epochs with Adam, learning rate $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$6, batch size 1, and cyclic cosine annealing. The sampling rates are $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$7, and the smoothing exponent is $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$8. The reported bit-width settings are W8A8, W6A6, W4A4, and W3A3, where $$M_j = \max\left(\left|\text{Sample}_{\gamma_1}(\mathbf{X}_j)\right|\right),$$9 denotes weight precision and $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$0 denotes activation precision (Chen et al., 5 Oct 2025).

A practical feature of the evaluation is that it spans both moderate and aggressive compression. This matters because the paper’s strongest claims concern low-bit regimes, especially W4A4 and W3A3, where generic PTQ baselines deteriorate sharply.

5. Quantitative performance and ablation evidence

The principal quantitative claim is that QuantDemoire achieves large reductions in parameters and computation while maintaining image quality, and that it outperforms existing quantization methods by over 4 dB on W4A4. On UHDM, the full-precision ESDNet reaches 22.12 dB PSNR; under W4A4, QuantDemoire achieves 21.08 dB, whereas MinMax, Percentile, 2DQuant, and SVDQuant obtain 16.51 dB, 16.85 dB, 17.07 dB, and 14.68 dB, respectively. On FHDMi at W4A4, QuantDemoire reports 20.28 dB against 16.01 dB, 16.92 dB, 14.80 dB, and 12.43 dB for the same baselines. On LCDMoiré at W4A4, it reaches 31.28 dB, again substantially exceeding the quantized baselines (Chen et al., 5 Oct 2025).

Dataset	Baselines at W4A4 PSNR (dB)	QuantDemoire PSNR (dB)
UHDM	16.51 / 16.85 / 17.07 / 14.68	21.08
FHDMi	16.01 / 16.92 / 14.80 / 12.43	20.28
LCDMoiré	lower than 31.28	31.28

Compression is also explicit. At 4-bit, the model shrinks from 5.93M parameters and 13.52G ops to 0.79M parameters and 1.80G ops, corresponding to 86.61% parameter reduction and 86.68% operation reduction. The paper further states that even at 3-bit, QuantDemoire remains much stronger than alternative PTQ approaches (Chen et al., 5 Oct 2025).

The ablations are tightly aligned with the framework’s two design hypotheses. For activation range estimation on UHDM W4A4, sample-based estimation reaches 17.61 dB, compared with 16.51 dB for MinMax and 16.85 dB for Percentile. Sample-based smoothing improves over “Smooth (Raw),” raising PSNR from 20.43 to 20.52 dB and SSIM from 0.7408 to 0.7538. For weights, “Discard Outliers” drops to 16.29 dB, whereas “Store Outlier” reaches 20.92 dB. Randomly storing the same number of weights does not match explicit outlier selection. The outlier ratio study identifies $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$1 as a good trade-off, while $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$2 gives slightly better PSNR at higher parameter and operation cost. For calibration, switching from spatial-domain optimization to low- and mid-frequency calibration improves UHDM W4A4 from 20.92 dB to 21.08 dB and reduces LPIPS; $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$3 outperforms $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$4 and $$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$5 (Chen et al., 5 Oct 2025).

These results directly support the paper’s main thesis: demoiréing degradation under quantization is dominated by both tail statistics and frequency-specific fidelity loss, and both must be addressed simultaneously.

6. Position within demoiréing research

QuantDemoire belongs to a different line of work from demoiréing methods that primarily redesign restoration pipelines. For example, FDNet formulates demoiréing as self-adaptive learning from a focused moiré image and a defocused moiré-free blur image, using a U-Net-like demoiré network and a blur-kernel generator jointly optimized at test time under the reconstruction objective

$$s_j = M_j^\alpha \big/ \max\!\left(|\mathbf{W}_j|\right)^{1-\alpha}.$$6

with no separate pretraining phase (Liu et al., 2020). QuantDemoire, by contrast, assumes a strong full-precision demoiréing backbone, namely ESDNet, and addresses the deployment problem of compressing that model while preserving demoiréing quality (Chen et al., 5 Oct 2025).

This distinction clarifies the scope of QuantDemoire. It is not a new restoration backbone, a new supervised training protocol, or a new two-image inference setup. Instead, it is a model-compression framework specialized to demoiréing. A common misconception would be to view it as a generic quantization recipe; the paper argues the opposite, namely that demoiréing is unusually sensitive to outliers and to low-/mid-frequency distortions in smooth regions, so task-specific PTQ design is required (Chen et al., 5 Oct 2025).

In that sense, QuantDemoire extends demoiréing research from restoration accuracy toward efficient deployment. This suggests a broader methodological point: once demoiréing is treated as a restoration problem with distinctive spectral failure modes, quantization policy itself becomes part of the restoration design space rather than a purely downstream compression step.