MoBiQuant: Mixture-Based Quantization & Regression

• MoBiQuant is a framework that integrates adaptive quantization for LLMs using mixture-based bit-slice experts and token-aware routing for continuous precision switching.
• It employs a recursive residual quantization algorithm with theoretical guarantees to maintain stable token error distributions across dynamic bit-widths.
• The approach extends to Mixture-of-Experts binary quantization and censored quantile regression in mobility demand, achieving significant memory reductions and enhanced predictive accuracy.

MoBiQuant encompasses several distinct yet thematically related frameworks for quantization, elastic inference, and censored quantile regression in LLMs, mixture-of-experts (MoE) architectures, and mobility demand modeling. The approaches all share an emphasis on mixture-based representation—of quantization bits, binary experts, or quantile outputs—for flexible, adaptive computation and inference under resource or data constraints.

1. MoBiQuant for Elastic LLM Quantization

MoBiQuant for token-adaptive elastic LLMs introduces a novel mixture-of-bits quantization paradigm aimed at enabling dynamic precision allocation and efficient inference on hardware-constrained environments, notably cloud and edge devices. Static post-training quantization (PTQ) methods such as OPTQ, OmniQuant, or AWQ calibrate a model at a specific bit-width and do not generalize to other precisions, leading to dramatic degradation when runtime bit-width changes. The core issue is "precision-dependent outlier migration," where the subset of tokens most sensitive to quantization error shifts unpredictably across bit-widths, undermining generalization away from the calibration point (Wang et al., 21 Feb 2026).

MoBiQuant addresses this bottleneck through two key mechanisms:

• Many-in-One Recursive Residual Quantization (MoBiSlice): Weight matrices are decomposed into a sequence of bit-slice "experts." Each slice captures a 2-bit residual (in practice), and higher-precision reconstructions are formed by summing prefix slices. This enables continuous-range precision switching (e.g., from 2 to 6 bits) without memory blow-up or re-quantization.
• Token-Aware Router (MoBiRoute): A lightweight, jointly trained MLP allocates the optimal number of bit-slices per token, informed by token embedding sensitivity. Precision allocation becomes token-level adaptive, enabling an explicit average bit budget.

A single calibration covers the full spectrum of bit-widths, removing the requirement for per-precision codebook learning or repeated quantization (Wang et al., 21 Feb 2026).

2. Mixture-of-Bits Quantization Paradigm: Algorithms and Theoretical Guarantees

The MoBiQuant algorithm follows a recursive procedure: $$\mathbf R_1 = \mathbf W; \quad \mathbf W_e = Q(\mathbf R_e\mid\Theta_q, b_e); \quad \mathbf R_{e+1} = \mathbf R_e - \mathbf W_e$$ with each quantized slice using shared parameters $\Theta_q$. At inference, weight reconstructions for arbitrary precision $b$ are computed by summing the first $k$ slices: $\mathbf W^{(b)} = \sum_{e=1}^k \mathbf W_e$ This design enables efficient precision switching and storage reuse.

The routing function outputs a score vector for each token, gated via a temperature-controlled sigmoid annealed across training steps. At inference, a user-tunable threshold or quantile on routing scores determines the actual bit budget per token.

Theoretical analysis demonstrates that adding or removing slices does not alter the already quantized coarser bits (the "half-step bound"): $$\mathbb E[E_p]=0, \qquad |E_p| < 2^{p-1} s_2, \quad \frac{|E_p|}{s_2 2^p} < \frac{1}{2}$$ This ensures token error distributions remain stable between adjacent precisions, curbing outlier migration (Wang et al., 21 Feb 2026).

3. Empirical Evaluation and Kernel Acceleration

Evaluation covers LLaMA2 and LLaMA3 variants with calibration on WikiText2, C4, PTB, and mixed sets. MoBiQuant supports elastic adaptation: when trained at 3 bits, perplexity (PPL) remains monotonic across 2–6 bit evaluations and matches or outperforms static 4-bit PTQ (e.g., for LLaMA3-8B, OmniQuant 4b yields PPL ≈ 7.41, MoBiQuant achieves PPL ≈ 7.31 at 3.99 bits). Zero-shot downstream task accuracy also matches or exceeds static PTQ (Wang et al., 21 Feb 2026).

The approach is robust to calibration set choice, insensitive to random seed or number of calibration examples, and features minimal crossing loss in per-token quantization allocation. Customized GPU kernels pack bit-slices in bit-major order, achieving up to $2.7\times$ FP16 speedup on long sequences.

4. MoBiQuant for Mixture-of-Experts Binary Quantization

A variant of MoBiQuant, rebranded from the MoBiE framework, targets binary quantization of Mixture-of-Experts LLMs (Zhao et al., 8 Apr 2026). The pipeline comprises:

• Cross-Expert Joint SVD Decomposition (CEJD): Concatenate and jointly decompose expert weights. This yields an 8-bit shared backbone (UΣ) and N binarized projections, leveraging expert redundancy.
• Global Loss-Aligned Saliency (GLAS): Saliency combines local Hessian structure with the loss gradient, focusing binarization on weights globally significant for end-task loss rather than only local error.
• Null-Space-Guided Expert-Shift Suppression (NGES): Binarization errors in router weights are projected into the router's null space, minimizing routing distortion and expert assignment shift.

This pipeline admits no fine-tuning and operates on small calibration data. MoBiQuant in this context reduces WikiText-2 PPL by 52.2% versus ARB-LLM, improves zero-shot accuracy by 43.4%, doubles decoding speed, and realizes >90% expert footprint memory reduction (Zhao et al., 8 Apr 2026).

Ablations confirm that CEJD recovers most of the performance, with GLAS and NGES providing additive improvements. NGES maintains a high cosine similarity (≥0.90) between quantized and full-precision routing, evidencing robust expert assignment after quantization.

5. MoBiQuant for Censored Quantile Regression in Mobility Demand

In the context of mobility demand modeling, MoBiQuant refers to a "Multi-Output Censored Quantile Regression Neural Network" (Multi-CQNN) (Hüttel et al., 2021). The framework jointly predicts multiple quantiles of a censored latent demand distribution: $$L(\theta) = \sum_{k=1}^K \sum_{i=1}^N \rho_{\tau_k}\left(y_i - \max\{\tau_i, q_{i, \tau_k}(x_i; \theta)\}\right)$$ with pinball loss $\rho_\tau$ and censoring. All $K$ quantiles are estimated as the last layer of a single neural network (dense or LSTM). Parameter sharing across outputs induces implicit monotonicity and drastically reduces quantile crossing.

Empirical evaluation on synthetic censored data and Copenhagen shared-mobility deployments demonstrates 20–30% lower MAE at low quantiles compared to censorship-unaware baselines, almost order-of-magnitude reductions in quantile crossing, and computational cost nearly identical to single-quantile models. Event coverage (ICP) closely matches nominal coverage in scenarios with partial or full censorship, outperforming QNN and single-quantile CQNNs (Hüttel et al., 2021).

6. Applications, Limitations, and Future Directions

MoBiQuant has been successfully applied to:

• Adaptive quantization of LLMs for efficient cloud/edge deployment with per-token precision budget and smooth run-time flexibility (Wang et al., 21 Feb 2026).
• Binary quantization of MoE LLMs maximizing memory/computation savings while preserving expert routing (Zhao et al., 8 Apr 2026).
• Censored quantile regression in mobility, handling supply-limited demand observations for robust policy and planning analytics (Hüttel et al., 2021).

Limitations include the absence of hard monotonicity in quantile regression, restriction to right censoring, and the need for possible extension to interval or left censoring and spatial structure (e.g., via graph convolution).

A plausible implication is that mixture-based frameworks—whether of bits, experts, or output quantiles—enable flexible, fine-grained trade-offs between efficiency, expressivity, and calibration generalization, particularly in settings with resource or data constraints, heterogeneous sensitivity, and structured uncertainty.