InfoQ: Mixed-Precision Quantization

• InfoQ is a mixed-precision quantization framework that measures global sensitivity using mutual information diagnostics to improve per-layer bit allocation.
• The method formulates bit allocation as an integer linear programming problem, optimizing for constraints like model size and BitOps without heavy retraining.
• It leverages Sliced Mutual Information and observer layers to rapidly assess sensitivity, enabling competitive performance in high-compression regimes.

Searching arXiv for the specified paper and closely related mixed-precision quantization work. InfoQ is a mixed-precision quantization framework for deep neural networks in which per-layer bit-width allocation is driven by measured changes in global information flow rather than by local sensitivity proxies or expensive search procedures. It addresses mixed-precision quantization as a constrained optimization problem under budgets such as model size or BitOps, and its central claim is that the sensitivity of a layer should be assessed by how quantizing that layer alters the information retained by downstream representations about the input and the labels. The framework is training-free in the bit-width search phase, uses mutual-information-based diagnostics computed from a small calibration set, and formulates final bit allocation as an integer linear programming problem (Akbulut et al., 6 Aug 2025).

1. Problem setting and formal objective

Mixed-precision quantization assigns different bit-widths to different layers of a trained network. For layers $\ell = 1,\dots,L$ and candidate bit set

$\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$

InfoQ considers choices $b_w^{(\ell)} \in \boldsymbol{B}$ for weights and $b_a^{(\ell)} \in \boldsymbol{B}$ for activations. A full configuration is

$$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$

with search-space size

$|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$

The optimization target is written as

$$\begin{aligned} \boldsymbol{s}^* = \arg\min_{\boldsymbol{s} \in \mathcal{A}} \;& \mathcal{L}\big(f(\boldsymbol{x};\boldsymbol{s}, \boldsymbol{W}_{\boldsymbol{s}}), \boldsymbol{y}\big) \ \text{s.t.}\;& \text{Cost}(\boldsymbol{s}) \le C, \end{aligned}$$

where the cost may represent model size or BitOps (Akbulut et al., 6 Aug 2025).

For model size, the cost is

$$\text{Cost}(\boldsymbol{s}) = \sum_{\ell=1}^L \text{Size}(b_w^{(\ell)}),$$

and for BitOps it is

$$\text{Cost}(\boldsymbol{s}) = \sum_{\ell=1}^L \text{BitOps}\big(b_w^{(\ell)}, b_a^{(\ell)}\big).$$

BitOps are proportional to weight-bit $\times$ activation-bit $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$0 MAC count. Because the configuration space is combinatorial, exhaustive evaluation is infeasible, especially if every candidate requires fine-tuning. InfoQ is positioned against two broad classes of prior methods: search-based approaches that repeatedly train or evaluate candidate configurations, and criterion-based approaches that rely on local layerwise proxies such as Hessian information. The method’s motivating contention is that such local descriptors do not directly capture the cascading forward effects of quantization error.

2. Information-theoretic basis

InfoQ adopts an information-flow perspective grounded in the quantities $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$1 and $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$2 for a layer representation $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$3. The framework does not optimize an Information Bottleneck objective directly. Instead, it treats mutual information as a diagnostic for how much quantizing one layer changes the information carried by later layers about the input and the labels (Akbulut et al., 6 Aug 2025).

The conceptual criterion is simple: if quantizing layer $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$4 causes a large decrease in downstream information, then layer $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$5 is globally sensitive. Mutual information is defined as

$\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$6

Direct computation for high-dimensional activations is intractable, and deterministic-network representations raise further difficulties for $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$7. InfoQ therefore introduces two pragmatic devices.

First, the input is compressed with a pretrained DINOv2 encoder $\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$8:

$\boldsymbol{B} = \{b_0, b_1, \dots, b_{n-1}\},$9

This produces a lower-dimensional, semantically rich representation in place of raw images. Second, the method uses Sliced Mutual Information (SMI), defined for random vectors $b_w^{(\ell)} \in \boldsymbol{B}$0 and $b_w^{(\ell)} \in \boldsymbol{B}$1 as

$b_w^{(\ell)} \in \boldsymbol{B}$2

where $b_w^{(\ell)} \in \boldsymbol{B}$3 and $b_w^{(\ell)} \in \boldsymbol{B}$4 are random unit vectors. Scalar mutual information for each projected pair is estimated with the KSG k-NN estimator through NPEET. This suggests that InfoQ replaces direct high-dimensional estimation with an averaged one-dimensional projection scheme that is computationally tractable while still tied to downstream representational change.

3. Sensitivity estimation and observer layers

The framework measures the impact of quantizing one upstream layer on downstream observer layers. Let $b_w^{(\ell)} \in \boldsymbol{B}$5 denote activations at observer layer $b_w^{(\ell)} \in \boldsymbol{B}$6 when the full model is at 8-bit, and let $b_w^{(\ell)} \in \boldsymbol{B}$7 denote activations at the same observer when only one upstream layer $b_w^{(\ell)} \in \boldsymbol{B}$8 is quantized to bit-width $b_w^{(\ell)} \in \boldsymbol{B}$9 and all other layers remain at 8-bit. With compressed input $b_a^{(\ell)} \in \boldsymbol{B}$0 and labels $b_a^{(\ell)} \in \boldsymbol{B}$1, InfoQ computes two forms of information change (Akbulut et al., 6 Aug 2025).

The first is the input-relevant information drop:

$b_a^{(\ell)} \in \boldsymbol{B}$2

The second is the label-relevant information drop:

$b_a^{(\ell)} \in \boldsymbol{B}$3

Observer layers are selected through a one-time, architecture-specific correlation analysis. For each quantizable layer $b_a^{(\ell)} \in \boldsymbol{B}$4, InfoQ quantizes only that layer to a low bit-width, evaluates the resulting accuracy drop

$b_a^{(\ell)} \in \boldsymbol{B}$5

and computes $b_a^{(\ell)} \in \boldsymbol{B}$6 and $b_a^{(\ell)} \in \boldsymbol{B}$7 for downstream layers. For each candidate observer layer $b_a^{(\ell)} \in \boldsymbol{B}$8, it then correlates $b_a^{(\ell)} \in \boldsymbol{B}$9 with $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$0 using Pearson correlation. Layers satisfying $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$1 are retained as observers, and the paper reports that $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$2 works robustly; ablations indicate that values that are too low are catastrophic, while values that are too high leave too few observers. Strong correlations are reported, including $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$3 at ResNet50’s global pooling and fully connected layers for $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$4.

With observer sets fixed, InfoQ defines a normalized sensitivity score for quantizing layer $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$5 to bit-width $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$6:

$$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$7

where the numerator sums downstream information loss across selected observers and the denominator sums the corresponding baseline 8-bit SMI values. The factor $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$8 penalizes lower bit-widths. Separate scores are computed for weights and activations, denoted $$\boldsymbol{s} = \{(b_w^{(1)}, b_a^{(1)}), \dots, (b_w^{(L)}, b_a^{(L)})\},$$9 and $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$0.

4. Optimization procedure and computational workflow

Once per-layer, per-bit sensitivity scores are available, bit allocation is posed as an integer linear program. Binary decision variables $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$1 and $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$2 indicate the selected bit-width for weights and activations. The optimization minimizes total sensitivity:

$|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$3

subject to exactly one bit choice per layer and a hardware budget constraint linear in the decision variables (Akbulut et al., 6 Aug 2025).

In compact form, the paper writes the optimization as

$|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$4

Because the objective and constraints are linear in binary variables, standard ILP solvers can be used; the implementation described in the paper uses PuLP in Python, with solve times typically in milliseconds.

The scoring phase is explicitly training-free. The total number of forward passes is

$|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$5

consisting of one 8-bit baseline pass and one perturbed pass for each layer-bit pair. This phase uses only inference and mutual-information estimation, with no backpropagation. After the configuration is selected, quantization-aware training with LSQ may optionally be applied to recover further accuracy. The candidate bit-set commonly used in the experiments is

$|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$6

The overall workflow is therefore: select observers once per architecture, compute sensitivity scores on a small calibration set, solve the ILP for a specified budget, and optionally run short QAT.

5. Empirical behavior and reported results

The reported experiments use ImageNet (ILSVRC12), ResNet18, ResNet50, and MobileNetV2, with LSQ as the QAT method and candidate bits $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$7 (Akbulut et al., 6 Aug 2025). The paper presents both post-training evaluations of the selected configurations and results after short QAT, with particular emphasis on high-compression regimes.

For ResNet18 in the weight-only W3A8 setting, full precision is reported at 70.60% top-1, while InfoQ reaches 70.94% top-1 with mixed-precision 3-bit weights and 8-bit activations. Under a 23 GBitOps weight-and-activation mixed-precision budget, InfoQ attains 69.99% top-1 at 23.04 GBitOps, compared with 69.70% at 23.07 GBitOps for LIMPQ; the paper notes that in that comparison the first-layer activation is left unquantized at 32-bit for LIMPQ. For ResNet50 under 12.2× weight compression, full precision is 77.37% top-1 and InfoQ yields 77.03% with 3MP weights and 4MP activations. For MobileNetV2 under 14× weight compression, full precision is 72.02% top-1 and InfoQ reaches 69.83% with mixed-precision weights and 8-bit activations.

Setting	InfoQ result	Reported context
ResNet18, W3A8	70.94% top-1	Full precision: 70.60%
ResNet18, 23.04 GBitOps	69.99% top-1	Weight+activation MPQ
ResNet50, 12.2× compression	77.03% top-1	Full precision: 77.37%
MobileNetV2, 14.00× compression	69.83% top-1	Full precision: 72.02%

The paper also evaluates the quality of the sensitivity metric without retraining. In these post-training comparisons, all methods are near full precision at large model sizes, but in high-compression regimes InfoQ is reported to dominate; for ResNet18 at roughly 4.5MB, it is reported to be about 25% more accurate than the closest competitor. This suggests that the mutual-information-based sensitivity metric is not merely a good initialization for later QAT, but already yields stronger first-shot configurations under tight budgets.

Search efficiency is a central part of the empirical argument. InfoQ uses on the order of 5–6k labeled images for observer selection and SMI estimation, whereas the comparison to LIMPQ is described as using approximately 0.6M samples for its retraining-based search. For a ResNet50 example with 52 layers, 5 candidate bit-widths, and about 10 observers, the framework requires $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$8 forward passes and about 2600 SMI estimates, with total sensitivity analysis finishing in a few hours on a single GPU and the ILP taking milliseconds.

6. Interpretation, limitations, and extensions

InfoQ is presented as a global alternative to local sensitivity criteria. Its rationale is that quantization does not only perturb the local loss landscape around a single layer; it changes how information propagates through the remainder of the network. By measuring changes in $|\mathcal{A}| = (|\boldsymbol{B}|^2)^L.$9 and $$\begin{aligned} \boldsymbol{s}^* = \arg\min_{\boldsymbol{s} \in \mathcal{A}} \;& \mathcal{L}\big(f(\boldsymbol{x};\boldsymbol{s}, \boldsymbol{W}_{\boldsymbol{s}}), \boldsymbol{y}\big) \ \text{s.t.}\;& \text{Cost}(\boldsymbol{s}) \le C, \end{aligned}$$0 at downstream observers, the framework attempts to capture the representational consequences of quantization in a task-relevant way (Akbulut et al., 6 Aug 2025). The reported qualitative behavior is consistent with this design: sensitivity rises as bit-width decreases, and early layers are typically assigned higher precision while deeper layers often receive lower precision. This suggests alignment between the learned allocation and widely used engineering heuristics about early-layer fragility.

The paper also states several limitations. Mutual-information estimation remains costly and noisy even with SMI and compressed inputs. Very large models may still make the $$\begin{aligned} \boldsymbol{s}^* = \arg\min_{\boldsymbol{s} \in \mathcal{A}} \;& \mathcal{L}\big(f(\boldsymbol{x};\boldsymbol{s}, \boldsymbol{W}_{\boldsymbol{s}}), \boldsymbol{y}\big) \ \text{s.t.}\;& \text{Cost}(\boldsymbol{s}) \le C, \end{aligned}$$1 forward-pass schedule substantial, although the procedure remains parallelizable and backpropagation-free. Observer selection is architecture-specific and depends on correlation analysis; if data distributions shift, the selected observer set may become suboptimal. The framework still requires labeled samples for $$\begin{aligned} \boldsymbol{s}^* = \arg\min_{\boldsymbol{s} \in \mathcal{A}} \;& \mathcal{L}\big(f(\boldsymbol{x};\boldsymbol{s}, \boldsymbol{W}_{\boldsymbol{s}}), \boldsymbol{y}\big) \ \text{s.t.}\;& \text{Cost}(\boldsymbol{s}) \le C, \end{aligned}$$2 and for accuracy-drop correlation. The experiments are concentrated on vision convolutional networks, so application to transformer architectures would require further engineering. Finally, the method uses mutual information as a diagnostic rather than as an end-to-end optimization objective.

Several extensions are identified in the source discussion. The ILP could incorporate hardware-specific constraints beyond BitOps and model size, such as energy or latency. The SMI/KSG/NPEET pipeline could be replaced by alternative mutual-information estimators if computationally viable. The same framework could, in principle, be adapted to transformer models or to repeated budget re-solving, since once $$\begin{aligned} \boldsymbol{s}^* = \arg\min_{\boldsymbol{s} \in \mathcal{A}} \;& \mathcal{L}\big(f(\boldsymbol{x};\boldsymbol{s}, \boldsymbol{W}_{\boldsymbol{s}}), \boldsymbol{y}\big) \ \text{s.t.}\;& \text{Cost}(\boldsymbol{s}) \le C, \end{aligned}$$3 has been computed, the ILP can be reused for multiple deployment targets. Within the scope reported, however, InfoQ’s distinguishing feature remains its reformulation of mixed-precision quantization as minimization of global information degradation under explicit hardware constraints.