FlexiQ: Adaptive Mixed-Precision Quantization

• FlexiQ is an adaptive mixed-precision quantization scheme that assigns 4-bit and 8-bit computations to feature channels based on their value ranges.
• The paper introduces an evolutionary heuristic for channel selection and effective bit extraction to optimize runtime performance while minimizing accuracy loss.
• FlexiQ’s dynamic adjustment between INT8-like accuracy and INT4-like latency enables significant speedup, outperforming uniform quantization approaches on NPUs and GPUs.

FlexiQ is an adaptive mixed-precision quantization scheme for deep neural networks, with a particular focus on computer vision convolutional and transformer models, that combines selective channel-wise low-bit computation, an efficient bit-lowering method, and a runtime policy for real-time latency/accuracy adjustment on NPUs and GPUs (Kim et al., 3 Oct 2025). Its central premise is that fixed-precision deployment exposes only a single operating point, whereas fluctuating serving load requires a controllable continuum between higher accuracy and lower latency. FlexiQ addresses this by assigning 4-bit computation to feature channels with small value ranges, retaining 8-bit computation elsewhere, and changing the low-bit channel ratio on demand without data movement.

1. Problem setting and design objective

FlexiQ is motivated by the observation that neural-network accelerators are costly and difficult to scale elastically under real-time workload fluctuations. Static quantized models provide fixed latency and accuracy, but they do not exploit short-run flexibility needed to remain within latency targets during bursts. Uniform low-bit quantization, especially INT4, can reduce latency substantially but often harms accuracy; layer-wise mixed precision mitigates this only partially because computing an entire layer at 4-bit can still induce large output deviations that accumulate across depth (Kim et al., 3 Oct 2025).

The method is built around a finer-grained view of sensitivity. Across feature channels in vision models, activation and weight value ranges vary widely, and many channels have sufficiently small ranges that high bits in their 8-bit encodings are effectively unused. This suggests that low-bit computation should be assigned at feature-channel granularity rather than uniformly per layer. A plausible implication is that a small retained fraction of 8-bit channels can stabilize network behavior disproportionately relative to its arithmetic cost.

FlexiQ therefore targets an operating regime in which latency can be modulated continuously by changing the ratio of 4-bit to 8-bit channels. In the reported implementation, this ratio is adjusted in real time in response to measured serving conditions, allowing the model to move between INT8-like accuracy and INT4-like latency without switching models or storing multiple complete precision-specific parameter sets (Kim et al., 3 Oct 2025).

2. Quantization model and channel selection

FlexiQ uses standard uniform quantization. In its explicit bounded form, a real value $x$ is clipped to $[l,u]$, quantized at bitwidth $b$ by

$$x_c = \operatorname{clip}(x,\; l,\; u),\quad \Delta_b = \frac{u-l}{2^b-1},\quad q = \operatorname{round}\!\left(\frac{x_c-l}{\Delta_b}\right),\quad \hat{x} = l + q\,\Delta_b.$$

The affine version is

$$q = \operatorname{round}\!\left(\frac{x}{s}\right) + z,\qquad \hat{x} = s\,(q - z),$$

with per-channel MSE

$$\mathrm{MSE}_c = \mathbb{E}\big[(x - \hat{x})^2\big].$$

The scheme is compatible with channel-wise quantization for weights and per-tensor or per-channel quantization for activations (Kim et al., 3 Oct 2025).

Channel selection is based on a range-derived score. For feature channel $c$,

$$\mathrm{Score}_c \;=\; R^{(a)}_c \cdot \max_{o} R^{(w)}_{o,c},$$

where $R^{(a)}_c$ is the activation range estimated from calibration data and $R^{(w)}_{o,c}$ is the corresponding weight-channel range across output channels. Channels with smaller scores are prioritized for low-bit execution because their narrower ranges imply less information loss under bit lowering (Kim et al., 3 Oct 2025).

The paper also frames selection as a combinatorial optimization:

$[l,u]$0

Because this problem is NP-hard, FlexiQ uses an evolutionary heuristic. Chromosomes are per-channel bit flags, with crossover at layer boundaries and mutations constrained by hardware group sizes: multiples of 32 on GPUs and 64 on the NPU. Fitness is defined as the $[l,u]$1 distance between 8-bit logits and mixed-precision logits,

$[l,u]$2

Inclusive selection ensures that channels selected at smaller low-bit ratios are subsets of those at higher ratios, which is essential for efficient online switching (Kim et al., 3 Oct 2025).

3. Effective bit extraction and adaptive runtime control

The defining algorithmic component of FlexiQ is its bit-lowering, or effective bit extraction, procedure. Instead of naively truncating 8-bit integers to their upper four bits, FlexiQ identifies a contiguous 4-bit window that captures the informative magnitude bits once unused high-order bits are excluded. For a signed 8-bit integer $[l,u]$3 represented in two’s complement, and a per-channel shift index $[l,u]$4, the lowered integer is

$[l,u]$5

During accumulation, FlexiQ aligns the contribution by a compensating shift,

$[l,u]$6

where $[l,u]$7 depends on the extraction window (Kim et al., 3 Oct 2025).

The paper’s example makes the point concretely: a full-precision value $[l,u]$8 maps to $[l,u]$9, and if the absolute channel maximum is below 32, the two top magnitude bits are unused. Naive top-4-bit truncation produces approximately 10% error, whereas extracting bits 3–6 yields less than 4% error. This suggests that the method recovers much of the representational content that would otherwise be lost in direct 8-to-4-bit truncation.

Extraction windows may be static or dynamic. Static extraction fixes a per-channel window from calibration data; dynamic extraction uses a runtime bitwise OR over the channel’s values to locate the highest unset bit position. Dynamic extraction improves accuracy, with reported gains of 0.7–1.1 percentage points for CNNs and 0.1–2.1 percentage points for transformers, at a runtime cost of 2–5% of layer execution time (Kim et al., 3 Oct 2025).

Runtime adaptation is implemented through a per-layer threshold index, max_4bit_ch, indicating the last feature channel computed at 4-bit. Because bit-lowered channels are stored as a subset of the 8-bit encodings, the ratio can be changed without copying tensors or reloading weights. One simple policy increases the 4-bit ratio by 25% when measured latency exceeds a budget or when queue length exceeds a threshold, and decreases it by 25% when latency falls below budget minus a hysteresis margin. Updating max_4bit_ch adjusts the ratio in microseconds and requires no data movement (Kim et al., 3 Oct 2025).

4. Systems realization on NPU and GPU

FlexiQ was implemented on both a custom NPU and GPUs. The NPU implementation is based on DNNWeaver v2 with a $b$0 PE array. Each PE has four 4-bit MACs; 8-bit uses all four MACs, whereas 4-bit uses two MACs to perform two MACs in parallel. Rows map to input channels, columns to output channels, and weight-stationary dataflow is used. In 4-bit mode, PEs produce 64 16-bit partial sums, which are bit-aligned and added to 8-bit accumulators. The implementation reports no pipeline bubbles when switching precisions, with the same bandwidth and latency characteristics across modes (Kim et al., 3 Oct 2025).

On GPUs, FlexiQ uses a CUTLASS-based mixed GEMM with Tensor Cores. The kernel employs a nested global-to-shared-to-register pipeline, Tensor Core MMA for 4-bit groups of 32 feature channels, and CUDA-core bit-shift alignment for mixed accumulation. The shift and alignment overhead is pipelined and reported as hidden by compute. The kernel iterates over feature channels up to max_4bit_ch using 4-bit MMA and then switches to 8-bit MMA, which is the systems mechanism that makes the adaptive ratio practical at serving time (Kim et al., 3 Oct 2025).

A nontrivial systems requirement is layout management. To avoid runtime reordering, FlexiQ statically reorders feature channels so that 4-bit channels are contiguous and nested across supported ratios. Residual connections require an added reorder operator. On the NPU, residual reordering incurs approximately 3% overhead, and loading 8-bit tensors adds 1–2%. The GPU implementation keeps the memory footprint equal to INT8 in order to preserve dynamic-ratio flexibility, although bit extraction increases bandwidth and caching extracted 4-bit blocks would trade extra memory for lower bandwidth demand (Kim et al., 3 Oct 2025).

These implementation choices are central to the method’s hardware efficiency. Mixed precision at feature-channel granularity would otherwise risk incurring enough bookkeeping, shifting, or reordering cost to erase arithmetic gains. FlexiQ’s contribution is therefore as much a runtime and kernel design as a quantization policy.

5. Empirical performance and comparative position

FlexiQ was evaluated on eleven vision backbones: ResNet-18, ResNet-34, ResNet-50, MobileNetV2, ViT-S, ViT-B, DeiT-S, DeiT-B, SwinT-S, and SwinT-B, using ImageNet for evaluation with pretrained weights from TorchVision and timm/HuggingFace. Calibration used 256 samples, with activation ranges tracked by EMA at momentum 0.99. Optional finetuning used SGD for 20 epochs on CNNs and 40 epochs on transformers, with channel-wise quantization for weights and 16-bit floating point for non-linear operations such as normalization and GELU (Kim et al., 3 Oct 2025).

The reported accuracy results position FlexiQ above uniform INT4 and several multi-precision baselines. With finetuning, its 100% 4-bit models achieve on average 6.6% higher accuracy for 4-bit models and outperform four state-of-the-art quantization techniques. At 50% 4-bit / 50% 8-bit, the method incurs only about 0.6% average accuracy loss compared to full precision while achieving 40% of the speedup of the 100% 4-bit model over the 8-bit model. Accuracy degrades gradually as the 4-bit ratio rises to roughly 75%, with sharper degradation near full 4-bit operation, which supports the claim that a residual high-precision fraction stabilizes network outputs efficiently (Kim et al., 3 Oct 2025).

Latency and throughput results were reported on both the NPU and GPUs. On an A6000 using ViT-B, FlexiQ’s 100% 4-bit GEMM runs about 6% slower than an ideal INT4 baseline kernel, but end-to-end latency is on par with INT4 because attention, normalization, and GELU remain in FP16. Speedup versus INT8 is approximately $b$1, and the system supports about $b$2 higher request rates than INT8 at similar p90 latency. In a batch-128 ViT-B configuration, the listed latencies are 91.55 ms for uniform INT8, 65.00 ms for FlexiQ 100%, and 64.74 ms for uniform INT4 (Kim et al., 3 Oct 2025).

Under fluctuating real-world traces, the serving controller preserves near-INT8 accuracy while reducing latency toward INT4 levels during peak demand. For ViT-B with request rates varying from 500 to 1500 requests per second, FlexiQ maintained median latencies of roughly 100–150 ms, whereas INT8 rose to roughly 660 ms. The paper attributes this to the ability to raise the 4-bit ratio dynamically rather than relying on a fixed-precision compromise (Kim et al., 3 Oct 2025).

Relative to prior methods, FlexiQ is compared with RobustQuant, AnyPrecision, PTMQ, and HAWQv3. The paper characterizes its distinguishing features as feature-channel granularity, effective bit extraction, evolutionary inter-layer selection, and runtime ratio control with minimal overhead. PTMQ is identified as the only prior method in the comparison that reports transformer multi-precision, and FlexiQ is reported to outperform it on average bitwidths 4 and 6 (Kim et al., 3 Oct 2025).

6. Scope, limitations, and related nomenclature

FlexiQ is presented primarily for vision backbones. The paper notes that LLMs show similar unused-bit patterns, but also states that the best LLM accuracy would likely require coupling the method with activation-aware techniques such as SmoothQuant. It also depends on per-channel calibration statistics, so selection quality and extraction-window placement are sensitive to the fidelity of range estimates. Post-training quantization works well up to about 75% 4-bit, but fully 4-bit execution benefits from finetuning. Hardware grouping constraints—32 channels on GPU and 64 on the NPU—limit granularity, and very low precisions such as 2-bit require larger groups or PE multiplexing (Kim et al., 3 Oct 2025).

The name should also be distinguished from similarly named systems in other domains. “FlexQ” denotes an INT6 post-training quantization framework for LLM serving that uses Binary Tensor Core equivalents and mixed W6A6/W6A8 execution (Zhang et al., 6 Aug 2025). “Flexion” denotes a flexible, on-demand quantum error correction system for trapped-ion QCCD architectures that uses bare qubits for single-qubit gates and logical qubits for two-qubit gates (Yin et al., 22 Apr 2025). “FlexiQuad” denotes a bioinspired class of soft-frame quadrotors designed to combine agility, squeezability, and collision resilience (Girardi et al., 7 Nov 2025). In power systems, “FlexiQ” has also been interpreted as reliability-aware flexibility quantification at the transmission–distribution interface using P–Q area segmentation (Churkin et al., 2021). This naming overlap is terminological rather than methodological.

Within its intended scope, FlexiQ occupies a specific point in the quantization literature: it is neither a static layer-wise mixed-precision method nor a uniform low-bit method, but a channel-wise adaptive scheme whose principal contribution lies in coupling effective bit extraction with runtime-controllable mixed execution.