---
title: 'Pack-PTQ: Optimized Low-Bit Neural Quantization'
url: https://www.emergentmind.com/topics/pack-ptq
type: topic
---

# Pack-PTQ: Optimized Low-Bit Neural Quantization

Pack-PTQ is an advanced methodology for post-training quantization (PTQ) of neural networks, designed to preserve model accuracy in ultra-low-bit regimes by explicitly reconstructing groups of blocks—termed "packs"—rather than treating each layer independently. The core innovation involves adaptive grouping based on Hessian-derived metrics, mixed-precision assignment tailored to quantization sensitivity, and loss minimization on a small calibration set. Pack-PTQ has demonstrated substantial improvements over prior PTQ methods for both convolutional (CNN), transformer (ViT), and point-cloud (PointNet) architectures, especially in high-compression settings [2505.00259].

## 1. Hessian-Guided Adaptive Packing

Pack-PTQ introduces a principled strategy to partition a neural network into non-overlapping "packs," each comprising consecutive blocks. This grouping aims to capture cross-block dependencies during quantization, mitigating the severe performance drop commonly seen in block-wise PTQ when operating at low bit-widths. 

A block's importance is estimated via a second-order Taylor expansion of the loss, with the expected loss change for quantized block $b$ approximated as:

\[
\mathcal{L}_q^b \approx \Delta z^T g^{(z)} + \tfrac12\,\Delta z^T H^{(z)}\,\Delta z
\]

where $\Delta z = z_q - z$, $g^{(z)}$ is the loss gradient with respect to the output $z$ of the block, and $H^{(z)}$ is the corresponding Hessian. Instead of the full Hessian, Pack-PTQ uses the mean entry:

\[
S \approx \frac{2\,\mathbb{E}[\mathcal{L}_q^b - \Delta z^T g^{(z)}]}{\mathbb{E}[\Delta z^T \Delta z]}
\]

Lower $S$ indicates the block is less sensitive to quantization noise. The packing algorithm proceeds backward from the network's end: at each step, the contiguous set from the current end to the block with minimum score forms a new pack; the process repeats until all blocks are assigned. This yields packs whose internal dependencies are preserved, allowing joint optimization within each group.

## 2. Pack-Wise Reconstruction Objective

For each pack $P^{(\ell)}$, the quantization parameters—scales and zero-points for both weights and activations—are optimized by minimizing the reconstruction error across a small calibration set:

\[
\mathcal{L}_{\rm rec} = \mathbb{E}_{x \sim \text{calib}} \bigl\|P_q^{(\ell)}(x) - P^{(\ell)}(x)\bigr\|_F
\]

This objective ensures alignment between the quantized and original pack outputs, accounting for intra-pack dependencies. The optimization is performed using Adam (learning rate $4 \times 10^{-5}$), batch size $32$, and $2{,}000$ iterations per pack, providing both efficiency and strong empirical results without the risk of overfitting to the calibration data.

## 3. Mixed-Precision Bit Assignment

Different packs exhibit heterogeneous sensitivity to quantization. Pack-PTQ formulates mixed-precision assignment as a discrete constrained optimization problem: for $M$ packs, assign bit-widths $b_j \in \{k_1, \dots, k_K\}$ such that

\[
\max_{(b_1, \dots, b_M)} \sum_{j=1}^M b_j\,\Omega_j  \quad \text{s.t.} \quad \sum_{j=1}^M b_j p_j \leq C
\]

Here, $p_j$ denotes the number of parameters in pack $j$, and 

\[
\Omega_j = \frac{1}{n_j} \sum_{i = 1}^{n_j} S_{j}[i] \cdot \mathcal{L}^q_{j}[i]
\]

captures the mean sensitivity based on block importance and quantization loss. The problem is solved greedily (by sorting packs by $\Omega_j/p_j$) or with dynamic programming, maximizing model performance under a given memory budget $C$. Packs that are more sensitive to quantization are preferentially assigned higher bit-widths.

## 4. End-to-End Algorithmic Workflow

Pack-PTQ's process consists of four principal components:

1. **Block Importance Scoring**: Compute the importance score $S_b$ for each block using calibration data via a single forward and backward pass.
2. **Packing**: Iteratively partition blocks into packs using the score-minimizing scheme described above.
3. **Bit-Width Assignment**: For each pack, calculate sensitivity $\Omega_j$ and solve the discrete optimization for bit allocation under resource constraints.
4. **Pack-Wise Reconstruction**: For each pack, jointly optimize the quantizer parameters to minimize output mismatch between the quantized and reference networks.

This is formalized in a high-level pseudocode provided in the original work.

## 5. Implementation Considerations

- **Calibration Data**: The method requires only a small calibration set (typically $512$–$2{,}000$ samples, e.g., a subset of ImageNet).
- **Optimization Settings**: Adam optimizer, learning rate $4 \times 10^{-5}$, $2{,}000$ iterations per pack, with cosine decay and no weight decay.
- **Quantization Schemes**: Uniform quantization is used; activations use symmetric quantizers, weights employ affine quantization, both clamped to $[0, 2^k - 1]$.
- **Runtime Complexity**: Importance scoring in $O(n)$ (per block), packing in $O(n^2)$ (negligible for $n \sim 100$), per-pack reconstruction involves local gradient steps. Overall wall-time ranges from $0.8$ to $7$ hours on a single high-end GPU for modern vision networks.

## 6. Empirical Results and Evaluation

Pack-PTQ demonstrates significant improvements over leading PTQ methods in diverse settings:

- **ImageNet Benchmarks**: On ResNet18 (W3/A3), accuracy improved from 56.89% (BRECQ) to 66.73% with Pack-PTQ using mixed-precision; MobileNetV2 from 12.27% to 63.54%.
- **Vision Transformers**: On ViT-S, the score rose from 0.42% (BRECQ) to 55.56% (Pack-PTQ+MP); DeiT-S similarly showcased large gains.
- **3D Point Clouds**: For PointNet on ModelNet40, Pack-PTQ maintained sub-0.5% drops from full-precision at 3/3 bit-width, and with mixed-precision even surpassed baseline performance.
- **Ablation Studies**: Both random and Hessian-guided packing improve over block-wise reconstruction. Mixed-precision confers further gains, with the combined approach yielding the best results.
- **Efficiency**: Quantization wall-times ranged from 0.79 hours (ResNet18) to 6.82 hours (Swin-S).

A tabulation of select ImageNet results:

| Model        | Baseline (BRECQ, 3/3) | Pack-PTQ (w/o MP) | Pack-PTQ (+MP) |
|--------------|-----------------------|-------------------|----------------|
| ResNet18     | 56.89%                | 64.46%            | 66.73%         |
| MobileNetV2  | 12.27%                | 58.83%            | 63.54%         |
| ViT-S        | 0.42%                 | 47.68%            | 55.56%         |
| DeiT-S       | 14.63%                | 53.04%            | 58.38%         |

## 7. Practical Guidelines and Applicability

Several practical recommendations are indicated for effective use of Pack-PTQ:

- Packing granularity is crucial: grouping low-sensitivity blocks with subsequent layers yields better reconstructions.
- Small calibration sets suffice; pack-wise (not network-wise) reconstruction avoids overfitting.
- Mixed-precision assignment should be employed where possible, especially for lightweight or transformer models.
- Default hyperparameters are robust; no extensive search required for standard scenarios.
- The entire process is feasible on a single GPU within a few hours.
- The method generalizes across CNNs, ViTs, and 3D networks, with empirical gains commonly 5–10 percentage points above prior PTQ baselines in ultra-low-bit (e.g., W3/A3) contexts.

This suggests that Pack-PTQ provides a flexible, high-performance PTQ toolkit for a wide range of architectures and deployment settings, particularly when memory or computational budgets are severely constrained [2505.00259].

Source: https://www.emergentmind.com/topics/pack-ptq