Compression with Dead-Zone Quantizer (CoDeQ)

• Compression with Dead-zone Quantizer (CoDeQ) is a unified methodology that integrates dead-zone scalar quantization to perform both magnitude pruning and uniform quantization in a single, differentiable operator.
• It enables explicit rate-distortion and sparsity control by tuning quantizer parameters such as step size, offset, and dead-zone width, thereby optimizing performance across image and model compression tasks.
• Empirical results show that CoDeQ matches the rate-distortion performance of state-of-the-art codecs and achieves competitive neural network accuracies with reduced bit operations and enhanced sparsity.

Compression with Dead-zone Quantizer (CoDeQ) represents a unified methodology for signal and model compression that leverages the properties of dead-zone scalar quantization. Dead-zone quantizers introduce adjustable regions around zero where all values are mapped to zero, effecting both quantization and magnitude pruning in a single operator. This principle enables efficient image compression and supports highly sparse, low-precision neural networks in a fully differentiable, end-to-end optimization procedure. CoDeQ, as formalized in (Zhou et al., 2020) and further generalized for model compression in (Wenshøj et al., 15 Dec 2025), provides explicit control over rate-distortion tradeoffs and network sparsity by varying quantizer parameters, with provable near-optimal performance across a spectrum of applications.

1. Dead-Zone Quantizer: Formulation and Properties

The dead-zone quantizer is a variant of the uniform scalar quantizer that incorporates a widened zero bin. For a $b$-bit quantizer with step size $\Delta$, the classical form maps an input $x$ as

$$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$

assigning values within $|x| \leq \Delta/2$ to zero, thereby pruning small coefficients.

CoDeQ generalizes this with an explicit dead-zone width $d \geq \Delta$:

$$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$

This scheme is equivalent to magnitude pruning with threshold $\tau = d/2$, combined with uniform quantization above the threshold. For deep learning compression, this mapping is fully differentiable via straight-through estimators, enabling its use in joint pruning–quantization settings (Wenshøj et al., 15 Dec 2025).

The dead-zone quantizer for image latent coefficients is similarly parameterized by step size $Q$ and offset $\delta$:

$\Delta$0

where $\Delta$1 tunes the symmetry and width of the dead zone. In practice, $\Delta$2 provides optimal rate-distortion performance for learned codecs (Zhou et al., 2020).

2. Network Architectures and Training Protocols

CoDeQ-based image compression methods employ autoencoder architectures in the tradition of Ballé et al. (2017). The encoder $\Delta$3 consists of three strided convolutions with GDN nonlinearities, mapping $\Delta$4 patches to $\Delta$5 latent representations. The decoder $\Delta$6 features three transposed convolutions with inverse GDN. Bottleneck dimensionality is fixed at 128 coefficients per patch; models are trained over 1 million steps with batch size 8 and learning rate $\Delta$7 (Zhou et al., 2020).

Training is performed using the RaDOGAGA (Rate-Distortion Optimization Guided Autoencoder for Isometric Embedding in Euclidean Latent Space) framework. RaDOGAGA adds distortion-driven orthogonality regularization in latent space:

• Primary distortion: $\Delta$8 (MSE or MS-SSIM),
• Jacobian orthogonality: $\Delta$9 with $x$0 and $x$1,
• Log-rate penalty via a learned CDF prior $x$2.

The total loss is

$x$3

with metric-specific ($x$4, $x$5, $x$6) settings for MSE and MS-SSIM. The induced latent space becomes isometric to the selected fidelity metric, ensuring rate-distortion optimality over subsequent quantization (Zhou et al., 2020).

In the context of model compression, the dead-zone widths $x$7 and optional bit-widths $x$8 are parameterized and learned end-to-end via unconstrained scalars $x$9 and $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$0, using formulas:

$$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$1

$$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$2

and quantization step

$$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$3

The optimization objective is

$$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$4

with $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$5 typically the cross-entropy over quantized weights (Wenshøj et al., 15 Dec 2025).

3. Rate-Distortion Optimization and Sparsity Control

CoDeQ enables explicit, tunable rate-distortion control by modulating the quantizer step size $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$6 and offset $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$7 (for image compression) or the dead-zone widths $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$8 (for model compression). In the classical image codec setting, rate $$\bar{x} = \mathrm{clip}\Bigl(\lfloor x/\Delta \rceil,\,-Q_b,\,Q_b\Bigr), \quad Q_b = 2^{b-1}-1,$$9 and distortion $|x| \leq \Delta/2$0 are jointly optimized with a Lagrange multiplier $|x| \leq \Delta/2$1, tracing the rate-distortion curve via selection of operating points.

Under RaDOGAGA’s loss, the latent space becomes orthonormal and metric-isometric, enabling the dead-zone quantizer to function equivalently to a learned KLT/DCT transform followed by scalar quantization. Experimentally, sweeping $|x| \leq \Delta/2$2 achieves bitrates from $|x| \leq \Delta/2$3 to $|x| \leq \Delta/2$4, matching RD performance of individually tuned models (Zhou et al., 2020).

For model compression, varying $|x| \leq \Delta/2$5 at runtime directly adjusts layer sparsity, decoupled from quantization bit-width. Regularization via $|x| \leq \Delta/2$6 controls global sparsity, and $|x| \leq \Delta/2$7 encourages low precision. The optimization is differentiable and performed end-to-end; all pruning and quantization decisions are learned jointly (Wenshøj et al., 15 Dec 2025).

4. Algorithmic Pipeline and Inference Procedure

Image compression with CoDeQ is realized by the following inference steps:

• Input: $|x| \leq \Delta/2$8,
• Analysis transform: $|x| \leq \Delta/2$9,
• Dead-zone quantization: $d \geq \Delta$0,
• Entropy coding: Bits via arithmetic encoding with $d \geq \Delta$1 from $d \geq \Delta$2,
• Dequantization: $d \geq \Delta$3,
• Synthesis transform: $d \geq \Delta$4.

Model compression follows a fully differentiable QAT loop:

• Initialize weights and parameters,
• For each layer, compute $d \geq \Delta$5, $d \geq \Delta$6, $d \geq \Delta$7, $d \geq \Delta$8,
• Quantize/prune: $d \geq \Delta$9,
• Compute task loss and $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$0 regularizers,
• Backpropagate using STE for non-differentiable components,
• Update parameters via gradient descent (Wenshøj et al., 15 Dec 2025).

5. Empirical Results and Rate-Sparsity Benchmarks

On Kodak images, CoDeQ’s variable-rate method matches the rate-distortion envelope of independently trained Ballé et al. (2017) models for both PSNR and MS-SSIM criteria:

• PSNR: CoDeQ deviates by $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$1 dB from multi-model baseline,
• MS-SSIM: deviation within $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$2 dB,
• Best results for dead-zone offset $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$3 (Zhou et al., 2020).

Visual inspection indicates superior retention of fine textures at low bitrates compared to non–CoDeQ baselines, with minor edge artifacts attributed to scalar uniform quantization. At high bitrate, reconstructions are nearly identical.

For ImageNet/ResNet-18 model compression:

• Full precision baseline: $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$4 Top-1, $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$5 BOPs,
• SQL (joint ADMM): $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$6, $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$7 BOPs,
• QST-B: $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$8, $$\tilde{q}(x;\Delta,d) = \begin{cases} 0, & |x| \leq d/2, \ \mathrm{sign}(x)\,\left(d/2 + \Delta\, \mathrm{clip}\left(\left\lfloor \frac{|x| - d/2}{\Delta} \rceil, -Q_b, Q_b\right)\right) \right), & |x| > d/2. \end{cases}$$9 BOPs,
• CoDeQ (fixed 4-bit + learned sparsity): $\tau = d/2$0, $\tau = d/2$1 BOPs,
• CoDeQ (mixed precision + learned sparsity): $\tau = d/2$2, $\tau = d/2$3 BOPs.

Layer-wise analysis shows CoDeQ assigns higher bit-width/lower sparsity to first/last layers and lower bit-width/higher sparsity to middle layers, matching canonical heuristic allocations but discovered automatically in the end-to-end process (Wenshøj et al., 15 Dec 2025).

6. Discussion: Implementation, Limitations, and Extensions

CoDeQ offers several operational advantages:

• Eliminates the need for auxiliary search phases or parameter selection procedures outside the training loop,
• Pruning and quantization are unified, architecture-agnostic, and controlled by global regularization,
• Fixed 4-bit variant matches mixed-precision performance, favoring hardware compatibility.

Limitations include the use of layer-wise granularity for quantization and pruning; finer granularity (channel-, group-, or block-wise) and structured sparsity remain open for future development. The current scale factors utilize absmax statistics; direct learning of scale could further decrease quantization error.

Potential extensions of the CoDeQ framework are:

• Per-channel/group dead-zone control for sparsity granularity,
• Joint scale learning with regularization,
• Structured dead-zone quantization for hardware exploitation,
• Integration with post-training coding (e.g., Huffman, distillation) for maximal compression.

CoDeQ bridges magnitude pruning and uniform quantization, leveraging dead-zone properties for efficient compression in both data and model domains, with strong empirical performance and minimal operational complexity (Zhou et al., 2020, Wenshøj et al., 15 Dec 2025).