Adaptive Mixed-bit Activation Quantization

• Adaptive Mixed-bit Activation Quantization (AMAQ) is a strategy that assigns non-uniform bit-widths to activations based on their sensitivity, dynamic range, and task relevance.
• The method employs learnable gates, quality proxies like validation loss, and activation statistics (e.g., entropy, L2 norms) to dynamically allocate precision under strict bit-budget constraints.
• Empirical results show that AMAQ improves model accuracy, computational efficiency, and communication overhead in split-learning, low-bit LLM inference, and vision transformer settings.

Searching arXiv for the cited papers and closely related mixed-precision activation quantization work. Adaptive Mixed bit Activation Quantization (AMAQ) denotes a class of quantization methods in which activation precision is assigned non-uniformly rather than fixed globally, typically under an explicit communication, memory, or bit-budget constraint. In the most direct usage of the term, AMAQ refers to the server-client split-learning method introduced for collaborative parameter-efficient fine-tuning of LLMs, where activations and gradients are progressively compressed from higher precision to low precision and the effective bit-width is adaptively allocated across features using learnable gates and bit regularization (Song et al., 7 Oct 2025). In a broader research sense, AMAQ also describes a family of activation-quantization strategies that vary precision across layers, tokens, windows, or neurons, often to manage activation outliers, heterogeneous sensitivity, or communication bottlenecks (Zhao et al., 11 Jun 2026, Wang et al., 25 Jul 2025, Kim et al., 2023).

1. Concept and scope

AMAQ is concerned specifically with activation precision, not merely weight quantization. This distinction is important because many mixed-precision schemes for neural networks and LLMs are weight-only, even when they are activation-aware in calibration or bit allocation. By contrast, AMAQ methods explicitly vary the precision used for activations themselves, either during inference, training, or both (Song et al., 7 Oct 2025, Zhao et al., 11 Jun 2026).

The core premise is that activations are heterogeneous. Their quantization difficulty varies across layers, channels, tokens, windows, or neurons because of differences in dynamic range, outliers, sparsity, and downstream sensitivity. A fixed activation bit-width therefore tends to be inefficient: it over-allocates bits to robust components and under-allocates them to fragile ones. AMAQ methods seek to correct this by assigning more bits where precision matters and fewer bits where redundancy or robustness permits it (Wang et al., 25 Jul 2025, Varshney et al., 24 May 2026, He et al., 2 Jun 2025).

This general idea appears in several distinct operational regimes. In split or collaborative training, AMAQ is used to reduce communication of transmitted activations and gradients while maintaining optimization stability (Song et al., 7 Oct 2025, He et al., 2 Jun 2025). In post-training quantization for LLM inference, AMAQ appears as layer-wise mixed-precision activation assignment under a global bit budget, often to make very low-bit activation quantization feasible after suppressing outliers (Zhao et al., 11 Jun 2026). In vision transformers, AMAQ can be intra-layer and input-dependent, assigning different activation bit-widths to different windows based on their importance (Wang et al., 25 Jul 2025). In quantization-aware training, the adaptation may occur at layer, global-network, or neuron level (Gernigon et al., 2024, Varshney et al., 24 May 2026).

2. Formal structure of adaptive mixed-bit activation assignment

Most AMAQ methods share three ingredients: a decision variable for activation precision, a quality proxy or task loss, and a resource constraint or regularizer. The decision variable may be discrete layer bits, continuous relaxed bit parameters, per-token routing, or per-neuron thresholded precision strengths.

A direct and explicit formulation appears in TWLA’s activation module, Inter-Layer Aware Activation Mixed Precision (ILA-AMP), which assigns each layer $\ell$ an activation bit-width $b_\ell \in \mathcal{B}$ with

$\mathcal{B}=\{2,4,6,8\}.$

The assignment vector is $\mathbf{b}=(b_1,\ldots,b_L)$, and the quality signal is validation negative log-likelihood:

$v_{\mathrm{NLL}(\mathbf{b}) = \mathbb{E}_{(\mathbf{x},t)\sim\mathcal{D} \Big[ -\log p_{\boldsymbol{\theta}(x_{t+1}\mid \mathbf{x}_{\le t}; \mathbf{b}) \Big].$

The global allocation objective is

$\min_{\{b_\ell\} \sum_{\ell=1}^L C_\ell(b_\ell) + \sum_{\ell=2}^L K_{\ell-1,\ell}(b_{\ell-1}, b_\ell), \quad \text{s.t. }\sum_{\ell=1}^L b_\ell \le B,$

where $C_\ell(b)$ is isolated layer sensitivity and $K_{\ell-1,\ell}(b',b)$ captures adjacent-layer interaction costs (Zhao et al., 11 Jun 2026). This is a canonical AMAQ formulation: discrete, globally budgeted, and explicitly interaction-aware.

A different formulation appears in collaborative split learning. In AMAQ for collaborative PEFT, effective bit-width is controlled by learnable gates:

$$\text{Bit-width} = min + (max - min)\times \sigma(\alpha \cdot Q).$$

The corresponding bit regularization is

$$\text{Bits\_Loss} = \frac{1}{n}\sum_{i=1}^{n} \sigma(\alpha \cdot q_i)^2,$$

and the total loss is

$b_\ell \in \mathcal{B}$0

This yields a training-time adaptive mechanism in which the task loss implicitly preserves higher precision for important features while $b_\ell \in \mathcal{B}$1 pushes the average bit-width downward (Song et al., 7 Oct 2025).

In neuron-level mixed-precision QAT, each neuron has an activation precision strength $b_\ell \in \mathcal{B}$2 mapped to a discrete bit-width from

$b_\ell \in \mathcal{B}$3

and activation quantization is

$b_\ell \in \mathcal{B}$4

Here $b_\ell \in \mathcal{B}$5 is a neuron-specific clipping range, and the adaptation is static after training but learned jointly with the model (Varshney et al., 24 May 2026).

These variants differ in granularity and optimization, but all instantiate the same AMAQ principle: activation precision becomes an adaptive variable rather than a fixed hyperparameter.

3. Principal methodological patterns

AMAQ methods can be divided by the granularity at which bits are adapted and by the signal used to drive adaptation.

3.1 Layer-wise mixed precision

Layer-wise AMAQ is the most common form in LLM post-training quantization. TWLA’s ILA-AMP is exemplary because it does not only measure isolated sensitivity; it also models adjacent-layer coupling to avoid failure cascades in very low-bit settings (Zhao et al., 11 Jun 2026). This is particularly relevant when an upstream transform changes activation quantizability unevenly across layers.

MetaMix is also layer-wise, though in a QAT/search setting rather than PTQ. It searches per-layer activation precision while holding a single global weight precision fixed, and it emphasizes that activation bit exploration destabilizes activation statistics unless the search is performed on a mixed-precision-aware meta-state (Kim et al., 2023). AdaQAT is related but much coarser: it learns a single global activation bit-width for the whole network rather than layer-wise mixed precision (Gernigon et al., 2024). AdaBits and the 2021 adaptive-bit training work likewise support multiple activation precisions at runtime, but as whole-network switchable modes rather than simultaneous per-layer mixed assignment (Jin et al., 2019, Sun et al., 2021).

3.2 Intra-layer or token/window-wise mixed precision

A stricter AMAQ interpretation often emphasizes heterogeneous precision within a layer. MixA-Q is a direct instance: in Swin-based vision transformers, windows are scored by $b_\ell \in \mathcal{B}$6 norm and split into high-precision and low-precision branches within the same block. Less important windows are processed at lower activation precision, while important windows remain at higher precision (Wang et al., 25 Jul 2025). The method therefore performs mixed-bit activation quantization at window granularity and can be dynamic with respect to the input.

STaMP is similar in spirit but uses a sequence-domain transform first. It applies an invertible sequence transform, typically DWT, to concentrate energy into early transformed tokens, then keeps a small number of those transformed tokens at 8 bits and quantizes the remainder at 4 bits. Its theoretical bound,

$b_\ell \in \mathcal{B}$7

with $b_\ell \in \mathcal{B}$8, formalizes the idea that transformed tokens with larger energy should receive more bits (Federici et al., 30 Oct 2025). This is AMAQ-like, though the deployed selection rule is mostly static after choosing the transform and high-precision token count.

TAH-Quant applies a different intra-sequence strategy for pipeline-parallel training. It computes token entropy

$b_\ell \in \mathcal{B}$9

then assigns top-$\mathcal{B}=\{2,4,6,8\}.$0 tokens to INT4 and the remainder to INT3. Quantization parameters are tile-wise within each token, so bit allocation is token-wise while scale and zero-point are finer-grained (He et al., 2 Jun 2025).

3.3 Neuron-wise or very fine-grained adaptation

NMP-QAT pushes AMAQ to neuron granularity. Each neuron independently learns activation precision and clipping range, starting from 4-bit activations and increasing to 8 or 16 bits only if training signals justify it (Varshney et al., 24 May 2026). This suggests a fine-grained AMAQ regime in which activation heterogeneity is modeled within layers rather than only across them.

A plausible implication is that the main unresolved systems challenge for AMAQ is no longer only algorithmic bit allocation but also hardware support: neuron-wise mixed-bit activation execution is substantially harder to realize efficiently than layer-wise or token-wise schemes. The paper itself notes that sub-8-bit arithmetic and mixed-precision-capable accelerators are required for practical gains (Varshney et al., 24 May 2026).

4. Importance signals and allocation criteria

AMAQ methods differ sharply in what they treat as the relevant measure of activation importance or difficulty.

A first class uses task-level loss proxies. TWLA uses validation NLL measured under candidate bit assignments, both for unary sensitivity and pairwise adjacent-layer interaction terms (Zhao et al., 11 Jun 2026). This is arguably the most principled among current AMAQ approaches because the allocation target is directly tied to model quality.

A second class uses activation statistics. TAH-Quant uses token entropy as a proxy for information distribution (He et al., 2 Jun 2025). MixA-Q uses window $\mathcal{B}=\{2,4,6,8\}.$1 norms to identify important windows (Wang et al., 25 Jul 2025). AD-based mixed-precision quantization uses activation density,

$\mathcal{B}=\{2,4,6,8\}.$2

and updates per-layer bit-width by

$\mathcal{B}=\{2,4,6,8\}.$3

quantizing both activations and weights of a layer to the same bit-width (Vasquez et al., 2021). This is activation-driven but more heuristic and coupled than later AMAQ formulations.

A third class uses learned latent importance. Collaborative AMAQ does not define a handcrafted importance score; instead, the gates $\mathcal{B}=\{2,4,6,8\}.$4 learn which features should retain more bits under task loss plus bit regularization (Song et al., 7 Oct 2025). NMP-QAT similarly lets gradient signals move precision strengths across thresholds rather than assigning bits from a closed-form saliency metric (Varshney et al., 24 May 2026).

A fourth class uses stability-aware training surrogates. MetaMix does not compute importance directly from activation magnitudes; instead, it first learns a mixed-precision-aware meta-state by averaging losses across candidate activation bit-widths,

$\mathcal{B}=\{2,4,6,8\}.$5

then searches per-layer bits on frozen weights (Kim et al., 2023). This treats activation instability itself as the primary obstacle to reliable mixed-bit assignment.

5. Activation outliers, transforms, and quantizability shaping

A persistent obstacle for AMAQ is the heavy-tailed or outlier-dominated nature of activations. Many activation-mixed-precision methods therefore include a transformation or shaping step before bit allocation.

TWLA’s KOTMS learns a Kronecker-structured orthogonal rotation

$\mathcal{B}=\{2,4,6,8\}.$6

to reshape weights into a ternary-friendly distribution and, through the shared orthogonal transform, statistically suppress activation outliers. The paper formalizes the activation-side effect via the peak-to-RMS ratio

$\mathcal{B}=\{2,4,6,8\}.$7

and derives a bound showing that orthogonal mixing reduces the maximum coordinate relative to total energy with high probability (Zhao et al., 11 Jun 2026). AMAQ then becomes more effective because the activation distributions are less dominated by a few extreme channels.

STaMP performs a sequence-domain transform rather than a feature-domain rotation. Its contribution is to show that mixed-bit activation assignment becomes easier when activations are represented in a domain where importance is concentrated and ordered (Federici et al., 30 Oct 2025).

TAH-Quant uses a Hadamard transform with pivot swapping only on tiles whose outlier ratio

$\mathcal{B}=\{2,4,6,8\}.$8

exceeds a threshold $\mathcal{B}=\{2,4,6,8\}.$9. This selective transform reduces tile dynamic range before asymmetric quantization (He et al., 2 Jun 2025).

AQuant, though not mixed-bit, is highly relevant because it treats activation quantization itself as adaptive at fixed bit-width. It replaces the standard rounding border $\mathbf{b}=(b_1,\ldots,b_L)$0 by a value-dependent function. The ideal element-wise border is

$\mathbf{b}=(b_1,\ldots,b_L)$1

and the practical coarse-grained border is

$\mathbf{b}=(b_1,\ldots,b_L)$2

This minimizes output error more directly than scalar nearest rounding and is especially useful in ultra-low-bit activation settings (Li et al., 2022). Although AQuant is not AMAQ in the bit-allocation sense, it shows that activation adaptivity can occur inside a fixed bit-width as well as across different bit-widths.

6. Representative systems and empirical evidence

The most explicit AMAQ paper in the narrow sense is the collaborative split-learning method titled "AMAQ: Adaptive Mixed-bit Activation Quantization for Collaborative Parameter Efficient Fine-tuning" (Song et al., 7 Oct 2025). Under the same bit budgets, it reports about 2.5% higher generation accuracy and about 1.3% better classification accuracy than fixed-precision approaches for models such as LLaMA3-8B and Qwen2.5-7B (Song et al., 7 Oct 2025). In distributed experiments, AMAQ 4 improves over AQ-SGD 4 while adding only modest transmission overhead: for a LoRA-only scenario, transmission increases from 6.8 MB to 7.4 MB per batch while GSM8K accuracy improves from 53.15 to 54.28 (Song et al., 7 Oct 2025).

TWLA provides some of the strongest evidence for layer-wise AMAQ under aggressive PTQ for LLM inference. In the W1.58A4 setting with mixed-precision activations, it reports WikiText2 perplexities such as 8.31 for LLaMA2-7B, 6.68 for LLaMA2-13B, and 4.77 for LLaMA2-70B (Zhao et al., 11 Jun 2026). The activation-side ablation is especially striking: on LLaMA2-13B at A4, E2M-ATQ+KOTMS without ILA-AMP gives C4 perplexity 25.03 / MMLU 27.52, while adding ILA-AMP improves to 10.07 / 38.17 (Zhao et al., 11 Jun 2026). This indicates that in the very low-bit regime, adaptive mixed-bit activation assignment is not ancillary but necessary.

MixA-Q shows that intra-layer AMAQ can improve both efficiency and robustness in vision transformers. On COCO detection with Mask R-CNN and Swin-Tiny under W4A4 QAT, MixA-Q achieves a lossless roughly 1.24–1.25× speedup at effective 3.2-bit activations and 1.35× speedup with less than 1% mAP loss, outperforming uniform W4A3 at similar cost (Wang et al., 25 Jul 2025). It also improves the W4A4 baseline mAP by 0.7%, reducing quantization degradation by 24% (Wang et al., 25 Jul 2025).

MetaMix demonstrates that static per-layer AMAQ can outperform fixed activation precision at similar average activation bit-widths. For MobileNet-v2, MetaMix at 3.98/4 activation/weight precision reaches 72.60% top-1 on ImageNet, exceeding several fixed 4/4 baselines (Kim et al., 2023). Its central empirical claim is that fixed-meta-state search yields better layer assignments and much lower search cost than conventional NAS-style mixed-precision search (Kim et al., 2023).

NMP-QAT provides evidence for fine-grained activation adaptivity in tabular and telecom settings. In the weights+activations regime, average learned activation bits are often around 4–5 bits on several tasks, such as 4.855 on KVS and 4.115 on Higgs, well below 8-bit fixed baselines while improving or preserving accuracy (Varshney et al., 24 May 2026). On attention-heavy TabFormer tasks, adaptive activation precision dramatically outperforms uniform activation quantization baselines, which degrade severely (Varshney et al., 24 May 2026).

TAH-Quant shows that token-adaptive mixed 3/4-bit activation quantization is effective in distributed pipeline training. It reports up to 4.3× end-to-end speedup without compromising convergence, and its adaptive bit allocation ablation shows faster loss decrease than non-adaptive variants (He et al., 2 Jun 2025).

7. Relation to adjacent research and common misconceptions

One common misconception is that any activation-aware quantization method is AMAQ. This is not correct. Several important papers are activation-aware but not activation-mixed-bit methods. CMPQ uses activation distributions to assign weight channel precision, yet activations themselves are not quantized (Chen et al., 2024). RAMP uses activation statistics and Scale Folding to guide weight-only mixed-precision PTQ, while exported inference retains FP16 activations (Gautam et al., 18 Mar 2026). Such methods are related conceptually but should not be classified as AMAQ proper.

A second misconception is that adaptive activation quantization necessarily means per-layer mixed precision. In fact, the literature spans multiple granularities. AdaQAT adapts a single global activation precision during training and is therefore only a coarse precursor to richer AMAQ formulations (Gernigon et al., 2024). AdaBits and related switchable-bit networks support multiple activation precisions at runtime, but as global network modes rather than simultaneous mixed-bit assignments (Jin et al., 2019, Sun et al., 2021). By contrast, MixA-Q, TAH-Quant, STaMP, and NMP-QAT allocate activation precision within a model configuration across windows, tokens, or neurons (Wang et al., 25 Jul 2025, He et al., 2 Jun 2025, Federici et al., 30 Oct 2025, Varshney et al., 24 May 2026).

A third misconception is that the essential difficulty is bit allocation alone. Several papers indicate that activation stability and distribution shaping are equally fundamental. MetaMix identifies activation instability during bit search as a distinct optimization pathology (Kim et al., 2023). TWLA shows that orthogonal activation smoothing is a prerequisite for effective low-bit mixed precision (Zhao et al., 11 Jun 2026). AQuant shows that even with fixed bits, adaptive rounding can substantially reduce activation-induced output error (Li et al., 2022).

8. Limitations and open questions

Despite rapid progress, AMAQ remains constrained by several unresolved issues.

The first is systems realizability. Fine-grained mixed-bit activations are difficult to execute efficiently on current accelerators, especially when precision varies at neuron, token, or window level. Papers such as NMP-QAT acknowledge that realizing theoretical savings requires mixed-precision-capable hardware (Varshney et al., 24 May 2026). MixA-Q and STaMP report BOPs or moderate kernel overheads, but wall-clock gains depend heavily on implementation support (Wang et al., 25 Jul 2025, Federici et al., 30 Oct 2025).

The second is metadata and routing overhead. Token-wise or window-wise AMAQ needs selectors, masks, or branch-specific quantizer metadata. TAH-Quant and MixA-Q demonstrate practicality, but detailed accounting of all control overhead is still limited in many works (He et al., 2 Jun 2025, Wang et al., 25 Jul 2025).

The third is calibration dependence in PTQ settings. TWLA’s ILA-AMP uses 128 WikiText2 calibration samples of sequence length 2048, and the learned mixed-bit assignments are model-specific rather than transferable (Zhao et al., 11 Jun 2026). This suggests that AMAQ for PTQ remains sensitive to calibration data quality and target-model idiosyncrasies.

The fourth is objective mismatch. Many methods optimize bit budgets or average precision rather than hardware-grounded latency or energy. TWLA uses a global bit budget $\mathbf{b}=(b_1,\ldots,b_L)$3 (Zhao et al., 11 Jun 2026). Collaborative AMAQ regularizes gate activations rather than an explicit transmission-cost formula (Song et al., 7 Oct 2025). A plausible implication is that future AMAQ systems will need tighter coupling between bit allocation and actual hardware/runtime cost models.

The fifth is granularity selection. Layer-wise AMAQ is simpler and easier to deploy; token- or neuron-wise AMAQ can be more expressive but harder to train and implement. Current literature does not yet establish a universally preferred granularity across model families and hardware environments.

9. Summary

Adaptive Mixed bit Activation Quantization is the general strategy of assigning activation precision non-uniformly in response to heterogeneous importance, sensitivity, or communication cost. In its narrowest usage, AMAQ refers to collaborative split-learning methods that progressively compress activations and gradients from 6–8 bits toward 3–4 bits, using trainable gates and bit regularization to allocate precision across features (Song et al., 7 Oct 2025). In the broader literature, AMAQ encompasses layer-wise NLL-budgeted activation assignment for low-bit LLM inference (Zhao et al., 11 Jun 2026), window-wise mixed-precision routing in vision transformers (Wang et al., 25 Jul 2025), token-adaptive activation communication compression (He et al., 2 Jun 2025), and neuron-level learned activation precision in QAT (Varshney et al., 24 May 2026).

Across these variants, a consistent picture emerges. Activation quantization is difficult primarily because activations are heterogeneous, dynamic, and often outlier-dominated. Effective AMAQ methods therefore combine adaptive bit allocation with stabilization mechanisms such as transforms, interaction-aware optimization, progressive scheduling, or learned clipping and gating (Zhao et al., 11 Jun 2026, Kim et al., 2023, Federici et al., 30 Oct 2025, Li et al., 2022). The result is a shift from viewing activation precision as a fixed design choice to treating it as a structured optimization variable. In current research, this shift is one of the main enablers of practical low-bit activation quantization in both training and inference.