Hybrid Quantization: Methods & Trade-offs

• Hybrid quantization is a method that integrates diverse quantization rules—such as mixed precision, mixed scheme, and granularity—to balance performance and resource constraints.
• It is applied in various domains including MIMO relay communications, neural network compression, and quantum simulation, with tailored approaches for each context.
• Implementations use strategies like ordered amplitude-phase quantization and adaptive search algorithms to negotiate the trade-off between quantization error and hardware efficiency.

Searching arXiv for recent and relevant papers on hybrid quantization across communications, neural quantization, and hardware-aware mixed-precision methods. Hybrid quantization scheme denotes a class of constructions in which a quantization pipeline deliberately combines different quantization rules, bit-width assignments, granularities, or even mathematical representations within a single system. In the communication setting, the term names a specific relay design that applies ordered amplitude quantization together with uniform phase quantization in a MIMO quantize-forward relay (Kim et al., 18 Feb 2025). In neural-network compression, the same expression is used for layer-wise mixed bit-widths, mixed quantization schemes such as uniform and power-of-two, or client- and operator-specific allocation of PTQ and QAT strategies (Wang et al., 2020, Habi et al., 2020, Liang et al., 2024, Zheng et al., 17 May 2025). In quantum simulation and quantum cosmology, it refers instead to switching between first- and second-quantized encodings or combining loop and Fock quantization (Ku et al., 6 Jul 2025, Martín-Benito et al., 2010). The literature therefore uses the phrase as a family resemblance term rather than as the name of one canonical algorithm.

1. Semantic range and organizing principles

Across the cited literature, hybridization appears along several distinct axes: representation splitting, scheme mixing, precision mixing, granularity mixing, and runtime strategy allocation. In multi-antenna relaying, amplitudes and phases are quantized differently; in CNNs and hybrid ViTs, different operator classes receive different schemes; in post-training neural quantization, some layers are quantized per-tensor and others per-channel; in FL systems, some clients run PTQ while others run QAT; and in quantum chemistry the system converts between first- and second-quantized forms (Kim et al., 18 Feb 2025, GVSL et al., 2020, Saha et al., 5 Jun 2025, Liang et al., 2024, Zheng et al., 17 May 2025, Ku et al., 6 Jul 2025).

Domain	Hybridization mechanism	Representative paper
MIMO relay	Ordered amplitude + uniform phase	(Kim et al., 18 Feb 2025)
DNN compression	Layer-wise mixed bit-widths	(Wang et al., 2020)
Hardware-aware CNN quantization	Mixed bit-width + threshold search	(Habi et al., 2020)
CNN-Transformer hybrids	Uniform CNN weights + $\log_2$ Transformer activations	(Saha et al., 5 Jun 2025)
Data-independent PTQ	Per-tensor + per-channel layer selection	(GVSL et al., 2020)
RWKV PTQ	Scalar + vector quantization routing	(Xu et al., 2 May 2025)
Quantum chemistry	First- + second-quantized conversion	(Ku et al., 6 Jul 2025)
Quantum cosmology	Loop + Fock quantization	(Martín-Benito et al., 2010)

A recurrent design principle is selective complexity. The more expressive or costly quantizer is reserved for the system components that are most sensitive to quantization error, while cheaper rules are applied elsewhere. This is explicit in sensitivity-based per-tensor/per-channel selection via KLD (GVSL et al., 2020), entropy- and moment-based routing between scalar and vector quantization (Xu et al., 2 May 2025), and operator- or layer-wise precision search in mixed-precision systems (Wang et al., 2020, Habi et al., 2020).

2. Hybrid amplitude-phase quantization in multi-antenna relay systems

The most literal formulation of a hybrid quantization scheme in the supplied corpus is the hybrid amplitude-phase quantization (H-APQ) method for a MIMO quantize-forward relay (Kim et al., 18 Feb 2025). The system has $N_S$ source antennas, $N_R$ relay antennas, and $N_D$ destination antennas. In the first timeslot,

$$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$

with $x\in\mathbb C^{N_S}$, $\|x\|^2=1$, i.i.d. Rayleigh channel matrices $H_{\rm SR}$ and $H_{\rm SD}$, and $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$. The per-link SNR is $N_S$0. The relay quantizes $N_S$1, stores the bits, and in the second timeslot transmits $N_S$2, yielding

$N_S$3

The phase component is quantized by a uniform phase quantizer $N_S$4. With $N_S$5 bits per complex sample, $N_S$6 is divided into $N_S$7 bins of width $N_S$8. If $N_S$9 lies in

$N_R$0

then

$N_R$1

Uniform phase quantization alone requires $N_R$2 bits at the relay (Kim et al., 18 Feb 2025).

The amplitude component in H-APQ is quantized by an ordered amplitude quantizer $N_R$3. Let $N_R$4, and sort the amplitudes so that

$N_R$5

Given group size $N_R$6, number of levels $N_R$7, and amplitude set $N_R$8, the $N_R$9 smallest amplitudes are assigned to $N_D$0, the next $N_D$1 to $N_D$2, and so on, with the remaining amplitudes assigned to $N_D$3. The relay power constraint $N_D$4 imposes

$N_D$5

A simple choice is $N_D$6, with

$N_D$7

H-APQ combines ordered amplitude quantization with uniform phase quantization. Phases are quantized with $N_D$8 bits per branch, amplitudes by $N_D$9, and the relay transmits

$$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$0

The total bit budget is

$$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$1

Its computational complexity adds one sort of $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$2 amplitudes, group mapping, and uniform phase quantization; the sorting term is $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$3 (Kim et al., 18 Feb 2025).

The numerical results are explicitly framed as a memory-performance trade-off rather than a proof of optimality. For $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$4 with 4-ary modulation under Rayleigh fading, U-APQ with $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$5 is approximately the AF benchmark, U-PQ with $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$6 has the worst BER, and H-APQ with $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$7 nearly matches U-APQ while saving up to $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$8 bits for $$y_{\rm SR}=H_{\rm SR}x+z_{\rm R}, \qquad y_{\rm SD}=H_{\rm SD}x+z_{\rm D},$$9 (Kim et al., 18 Feb 2025). As $x\in\mathbb C^{N_S}$0 grows, H-APQ requires fewer bits, but BER gently degrades toward U-PQ performance. The letter does not derive closed-form end-to-end MSE or achievable-rate expressions, gives no analytical error-propagation or capacity bounds, and includes no explicit lemmas, theorems, or asymptotic high-SNR analyses (Kim et al., 18 Feb 2025).

3. Architectural and granularity-aware neural quantization

In neural-network compression, hybrid quantization commonly means that different layers or operator classes receive different quantization treatments. One line of work uses a meta-learned generator, MetaQuantNet, that maps a bit-width vector $x\in\mathbb C^{N_S}$1 to real-valued weights $x\in\mathbb C^{N_S}$2, followed by differentiable uniform quantization $x\in\mathbb C^{N_S}$3 at each layer. The training objective is bi-level: first learn $x\in\mathbb C^{N_S}$4 over a distribution of hybrid bit-width vectors, then solve for the best $x\in\mathbb C^{N_S}$5 under a compression constraint $x\in\mathbb C^{N_S}$6 (Wang et al., 2020). The reported CIFAR-10 and CIFAR-100 results show hybrid assignments outperforming uniform bit-width counterparts, and the first and last layers consistently receive the highest bits (Wang et al., 2020).

A second line of work broadens hybridization from bit-width alone to quantization scheme. HMQ searches over discrete pairs $x\in\mathbb C^{N_S}$7, where $x\in\mathbb C^{N_S}$8 is a power-of-two threshold and $x\in\mathbb C^{N_S}$9 is the signed bit-width, by using a Gumbel-Softmax estimator over the finite search space $\|x\|^2=1$0. The resulting quantizers are uniform, symmetric, and hardware-friendly, with scale

$\|x\|^2=1$1

and

$\|x\|^2=1$2

The key hybrid element is that each tensor learns its own bit-width-threshold pair under compression-rate objectives (Habi et al., 2020).

In hybrid vision backbones, structure-aware quantization aligns the rule with the operator family. EfficientQuant applies uniform 8-bit quantization to convolutional weights using layerwise min-max calibration, while applying 8-bit base-2 logarithmic quantization to post-Softmax self-attention activations in transformer blocks (Saha et al., 5 Jun 2025). M$\|x\|^2=1$3-ViT goes further by combining mixed precision and mixed schemes: uniform quantization is used for layers with relatively uniform weights or for memory-bound DWConvs, whereas PoT or APoT is used for compute-bound PWConvs and MSA matrix multiplications; the design assigns $\|x\|^2=1$4 bits to DWConvs and $\|x\|^2=1$5 bits to compute-intensive weights (Liang et al., 2024).

Granularity-aware hybridization appears in data-independent PTQ. The retro-synthesis method defines, for each layer $\|x\|^2=1$6, the KLD sensitivity difference

$\|x\|^2=1$7

where $\|x\|^2=1$8 and $\|x\|^2=1$9 are divergences between the FP32 model and auxiliary models with that layer quantized per-tensor or per-channel. If $H_{\rm SR}$0, the layer is quantized per-channel; otherwise, it is quantized per-tensor (GVSL et al., 2020). On ImageNet ResNet-18, pure per-tensor W8A8 gives $H_{\rm SR}$1, pure per-channel $H_{\rm SR}$2, and hybrid with $H_{\rm SR}$3 gives $H_{\rm SR}$4; with $H_{\rm SR}$5, only one of $H_{\rm SR}$6 layers is per-channel and inference time improves by $H_{\rm SR}$7 over the fully per-channel model (GVSL et al., 2020).

RWKVQuant uses yet another notion of hybridity: a proxy-guided routing between scalar quantization and vector quantization. The coarse proxy is an entropy gap,

$H_{\rm SR}$8

computed from sorted adjacent differences, and the fine proxy aggregates high-order central moments,

$H_{\rm SR}$9

Blocks with large $H_{\rm SD}$0 or $H_{\rm SD}$1 are routed to vector quantization; the rest use scalar quantization. The framework quantizes RWKV-6-14B to about $H_{\rm SD}$2-bit with less than $H_{\rm SD}$3 accuracy loss and $H_{\rm SD}$4 speed up (Xu et al., 2 May 2025).

4. Search, optimization, and adaptive allocation

The search problem in hybrid quantization is usually combinatorial. The literature therefore relies on amortization, differentiable relaxation, reinforcement learning, or constrained greedy recovery.

Meta-learning with genetic search is one example. After one training of MetaQuantNet over random bit-width vectors, a genetic algorithm searches the best hybrid policy satisfying $H_{\rm SD}$5. The reported hyperparameters are population size $H_{\rm SD}$6, top-$H_{\rm SD}$7 retention with $H_{\rm SD}$8, crossover probability $H_{\rm SD}$9, mutation probability $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$0, and stopping after $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$1 generations or no fitness improvement for $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$2 consecutive generations (Wang et al., 2020). This formulation separates fast policy evaluation from the more expensive learning stage.

Differentiable search is represented by HMQ and DQMQ. HMQ uses the Gumbel-Softmax estimator so that each discrete $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$3 choice is replaced by a smooth categorical distribution $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$4, annealed over training cycles before final hard selection (Habi et al., 2020). DQMQ similarly relaxes discrete per-layer bit-width choices $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$5 into a softmax policy $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$6, then combines supervised quantization training with policy-gradient updates so that the chosen bit-widths adapt to mixed-quality inputs (Wang et al., 2023).

In communication-oriented fixed-point design, AHPQ decomposes the problem into integral quantization and fractional quantization. The integer width $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$7 is determined by a PDF-based overflow criterion,

$z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$8

while the fractional widths $z_{\rm R},z_{\rm D}\sim \mathrm{CN}(0,\sigma^2 I)$9 are selected by PPO with BER-based rewards. For the NNA-AMP detector, AHPQ reports average $N_S$00 and average $N_S$01, compared with uniform $N_S$02-$N_S$03-$N_S$04 quantization (Ge et al., 2022).

System-level allocation appears in FL and distributed training. FedHQ treats PTQ and QAT as complementary strategies, estimating each client’s speed significance and accuracy significance and then using a linear threshold

$N_S$05

with $N_S$06, to decide whether the client should use PTQ or QAT (Zheng et al., 17 May 2025). QSync formulates operator precision assignment on hybrid devices as minimizing total perturbation

$N_S$07

subject to memory and throughput constraints, then greedily recovers selected operators from lower to higher precision when the constraints remain satisfied (Zhao et al., 2024).

5. Hardware realization and deployment trade-offs

A central motivation for hybrid quantization is that accuracy-optimal quantization is rarely hardware-optimal. HMQ addresses this directly by restricting quantizers to uniform symmetric forms with power-of-two thresholds, so that scaling reduces to bit-shifts and inference reduces to integer adds, multiplies, and shifts on fixed-point DSPs and low-power accelerators (Habi et al., 2020).

M$N_S$08-ViT couples algorithm and hardware explicitly. Its accelerator contains a Mixed-Precision Multiplication Array for uniform-quantized integer multiplies and a Shifters-and-Adder-Tree engine for APoT weights. On a TSMC 28 nm, 0.5 GHz ASIC, the reported latency changes from $N_S$09 ms to $N_S$10 ms, energy from $N_S$11 mJ to $N_S$12 mJ, and EDP from $N_S$13 mJ$N_S$14ms to $N_S$15 mJ$N_S$16ms, corresponding to approximately $N_S$17 EDP saving with comparable quantization accuracy relative to the cited prior accelerator (Liang et al., 2024).

EfficientQuant targets edge deployment of CNN-Transformer hybrids. On ImageNet-1K, MobileViT$N_S$18 changes from $N_S$19 FP32 to $N_S$20 quantized, and latency on an NVIDIA RTX 3080 is reduced by $N_S$21 to $N_S$22 relative to Q-HyViT, with an example of MobileViTv2$N_S$23 changing from $N_S$24 ms to $N_S$25 ms (Saha et al., 5 Jun 2025). The paper also reports edge-device results for EfficientQuant+TensorRT with batch $N_S$26: RTX 3080 averages $N_S$27 ms and $N_S$28 MiB peak memory, AGX Xavier averages $N_S$29 ms, and Jetson Nano averages $N_S$30 ms with engine load times of $N_S$31 s (Saha et al., 5 Jun 2025).

Hybrid quantization also changes system balance during training. QSync reports predictor error below $N_S$32, accuracy improvements of $N_S$33 over uniform precision on ImageNet from-scratch tasks, and throughput increases of $N_S$34 over dynamic batch sizing in the listed configurations (Zhao et al., 2024). FedHQ reports up to $N_S$35 training acceleration and up to $N_S$36 accuracy improvement with negligible extra overhead across FL settings (Zheng et al., 17 May 2025). AHPQ reports $N_S$37 area reduction, a $N_S$38 throughput-to-area ratio improvement, and $N_S$39 lower energy dissipation than its uniform-quantization counterpart in 65 nm CMOS (Ge et al., 2022).

6. Analysis, limitations, and cross-disciplinary extensions

Theoretical treatment varies sharply by domain. Some schemes provide explicit error controls. EfficientQuant states the uniform weight error bound

$N_S$40

and the log-domain activation reconstruction bound

$N_S$41

with $N_S$42 and $N_S$43 described as small in practice (Saha et al., 5 Jun 2025). Recursive hybrid quantization for reflected BSDEs establishes $N_S$44-error bounds for $N_S$45, and with uniform grids yields

$N_S$46

while preserving a finite-state Markov recursion (Nmeir et al., 2021).

Other schemes are primarily empirical. The H-APQ letter explicitly omits closed-form end-to-end MSE, achievable-rate expressions, analytical error-propagation results, capacity bounds, lemmas, theorems, and asymptotic analyses (Kim et al., 18 Feb 2025). HQ-DiT does provide a quantization-error discussion, reporting that observed per-channel MSE on held-out DiT activations drops from approximately $N_S$47 under INT8 to approximately $N_S$48 under FP4, and that on ImageNet $N_S$49 the sFID rises by only $N_S$50 under W4A4 FP4 (Liu et al., 2024).

The broader literature also shows that hybrid quantization is not restricted to low-bit inference. In quantum chemistry, the phrase refers to a conversion circuit $N_S$51 that switches between first-quantized and second-quantized encodings with gate cost $N_S$52 and qubit count $N_S$53, so that plane-wave Hamiltonian simulation and electron non-conserving operations can be carried out in whichever representation is more efficient (Ku et al., 6 Jul 2025). In loop quantum cosmology, hybrid quantization combines the improved-dynamics loop quantization of the homogeneous Bianchi I background with a Fock quantization of Gowdy $N_S$54 inhomogeneities; the Hamiltonian constraint becomes an evolution equation in discrete volume $N_S$55, with a strictly positive minimum and a well-posed initial-value problem (Martín-Benito et al., 2010).

A frequent misconception is that hybrid quantization always means mixed precision. The cited literature shows a broader taxonomy: mixed precision is only one instance, alongside mixed scheme selection, mixed granularity, mixed training strategy, and mixed physical representation. Another misconception is that hybridization removes the fundamental cost-accuracy trade-off. The source materials consistently retain that trade-off: increasing the H-APQ group size $N_S$56 lowers memory but degrades BER toward U-PQ (Kim et al., 18 Feb 2025); per-channel quantization improves accuracy but incurs more memory lookups and broadcast overhead (GVSL et al., 2020); PTQ is faster but may degrade accuracy relative to QAT in FL (Zheng et al., 17 May 2025). What hybridization changes is not the existence of the trade-off, but the locus at which it is negotiated.