Pack-PTQ: Optimized Low-Bit Neural Quantization

• Pack-PTQ is an advanced post-training quantization method that groups neural network blocks to preserve cross-layer dependencies and maintain accuracy in ultra-low-bit regimes.
• It leverages Hessian-guided adaptive grouping and pack-wise reconstruction to optimize quantizer parameters efficiently using minimal calibration data.
• Mixed-precision assignment targets sensitive packs, yielding substantial performance improvements across CNNs, ViTs, and point-cloud networks.

Pack-PTQ is an advanced methodology for post-training quantization (PTQ) of neural networks, designed to preserve model accuracy in ultra-low-bit regimes by explicitly reconstructing groups of blocks—termed "packs"—rather than treating each layer independently. The core innovation involves adaptive grouping based on Hessian-derived metrics, mixed-precision assignment tailored to quantization sensitivity, and loss minimization on a small calibration set. Pack-PTQ has demonstrated substantial improvements over prior PTQ methods for both convolutional (CNN), transformer (ViT), and point-cloud (PointNet) architectures, especially in high-compression settings (Li et al., 1 May 2025).

1. Hessian-Guided Adaptive Packing

Pack-PTQ introduces a principled strategy to partition a neural network into non-overlapping "packs," each comprising consecutive blocks. This grouping aims to capture cross-block dependencies during quantization, mitigating the severe performance drop commonly seen in block-wise PTQ when operating at low bit-widths.

A block's importance is estimated via a second-order Taylor expansion of the loss, with the expected loss change for quantized block $b$ approximated as:

$$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$

where $\Delta z = z_q - z$, $g^{(z)}$ is the loss gradient with respect to the output $z$ of the block, and $H^{(z)}$ is the corresponding Hessian. Instead of the full Hessian, Pack-PTQ uses the mean entry:

$$S \approx \frac{2\,\mathbb{E}[\mathcal{L}_q^b - \Delta z^T g^{(z)}]}{\mathbb{E}[\Delta z^T \Delta z]}$$

Lower $S$ indicates the block is less sensitive to quantization noise. The packing algorithm proceeds backward from the network's end: at each step, the contiguous set from the current end to the block with minimum score forms a new pack; the process repeats until all blocks are assigned. This yields packs whose internal dependencies are preserved, allowing joint optimization within each group.

2. Pack-Wise Reconstruction Objective

For each pack $P^{(\ell)}$, the quantization parameters—scales and zero-points for both weights and activations—are optimized by minimizing the reconstruction error across a small calibration set:

$$\mathcal{L}_{\rm rec} = \mathbb{E}_{x \sim \text{calib}} \bigl\|P_q^{(\ell)}(x) - P^{(\ell)}(x)\bigr\|_F$$

This objective ensures alignment between the quantized and original pack outputs, accounting for intra-pack dependencies. The optimization is performed using Adam (learning rate $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$0), batch size $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$1, and $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$2 iterations per pack, providing both efficiency and strong empirical results without the risk of overfitting to the calibration data.

3. Mixed-Precision Bit Assignment

Different packs exhibit heterogeneous sensitivity to quantization. Pack-PTQ formulates mixed-precision assignment as a discrete constrained optimization problem: for $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$3 packs, assign bit-widths $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$4 such that

$$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$5

Here, $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$6 denotes the number of parameters in pack $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$7, and

$$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$8

captures the mean sensitivity based on block importance and quantization loss. The problem is solved greedily (by sorting packs by $$\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z$$9) or with dynamic programming, maximizing model performance under a given memory budget $\Delta z = z_q - z$0. Packs that are more sensitive to quantization are preferentially assigned higher bit-widths.

4. End-to-End Algorithmic Workflow

Pack-PTQ's process consists of four principal components:

1. Block Importance Scoring: Compute the importance score $\Delta z = z_q - z$1 for each block using calibration data via a single forward and backward pass.
2. Packing: Iteratively partition blocks into packs using the score-minimizing scheme described above.
3. Bit-Width Assignment: For each pack, calculate sensitivity $\Delta z = z_q - z$2 and solve the discrete optimization for bit allocation under resource constraints.
4. Pack-Wise Reconstruction: For each pack, jointly optimize the quantizer parameters to minimize output mismatch between the quantized and reference networks.

This is formalized in a high-level pseudocode provided in the original work.

5. Implementation Considerations

• Calibration Data: The method requires only a small calibration set (typically $\Delta z = z_q - z$3–$\Delta z = z_q - z$4 samples, e.g., a subset of ImageNet).
• Optimization Settings: Adam optimizer, learning rate $\Delta z = z_q - z$5, $\Delta z = z_q - z$6 iterations per pack, with cosine decay and no weight decay.
• Quantization Schemes: Uniform quantization is used; activations use symmetric quantizers, weights employ affine quantization, both clamped to $\Delta z = z_q - z$7.
• Runtime Complexity: Importance scoring in $\Delta z = z_q - z$8 (per block), packing in $\Delta z = z_q - z$9 (negligible for $g^{(z)}$0), per-pack reconstruction involves local gradient steps. Overall wall-time ranges from $g^{(z)}$1 to $g^{(z)}$2 hours on a single high-end GPU for modern vision networks.

6. Empirical Results and Evaluation

Pack-PTQ demonstrates significant improvements over leading PTQ methods in diverse settings:

• ImageNet Benchmarks: On ResNet18 (W3/A3), accuracy improved from 56.89% (BRECQ) to 66.73% with Pack-PTQ using mixed-precision; MobileNetV2 from 12.27% to 63.54%.
• Vision Transformers: On ViT-S, the score rose from 0.42% (BRECQ) to 55.56% (Pack-PTQ+MP); DeiT-S similarly showcased large gains.
• 3D Point Clouds: For PointNet on ModelNet40, Pack-PTQ maintained sub-0.5% drops from full-precision at 3/3 bit-width, and with mixed-precision even surpassed baseline performance.
• Ablation Studies: Both random and Hessian-guided packing improve over block-wise reconstruction. Mixed-precision confers further gains, with the combined approach yielding the best results.
• Efficiency: Quantization wall-times ranged from 0.79 hours (ResNet18) to 6.82 hours (Swin-S).

A tabulation of select ImageNet results:

Model	Baseline (BRECQ, 3/3)	Pack-PTQ (w/o MP)	Pack-PTQ (+MP)
ResNet18	56.89%	64.46%	66.73%
MobileNetV2	12.27%	58.83%	63.54%
ViT-S	0.42%	47.68%	55.56%
DeiT-S	14.63%	53.04%	58.38%

7. Practical Guidelines and Applicability

Several practical recommendations are indicated for effective use of Pack-PTQ:

• Packing granularity is crucial: grouping low-sensitivity blocks with subsequent layers yields better reconstructions.
• Small calibration sets suffice; pack-wise (not network-wise) reconstruction avoids overfitting.
• Mixed-precision assignment should be employed where possible, especially for lightweight or transformer models.
• Default hyperparameters are robust; no extensive search required for standard scenarios.
• The entire process is feasible on a single GPU within a few hours.
• The method generalizes across CNNs, ViTs, and 3D networks, with empirical gains commonly 5–10 percentage points above prior PTQ baselines in ultra-low-bit (e.g., W3/A3) contexts.

This suggests that Pack-PTQ provides a flexible, high-performance PTQ toolkit for a wide range of architectures and deployment settings, particularly when memory or computational budgets are severely constrained (Li et al., 1 May 2025).