RaBitQ-H: Efficient Randomized Quantization

• RaBitQ-H is a randomized quantization framework that generalizes multi-bit compression for Euclidean vectors and serves as a protocol for quantum qubit-oblivious transfer.
• It utilizes random rotations and grid-based quantization to achieve a provable space–accuracy trade-off, enabling scalable nearest neighbor search on hybrid CPU/GPU systems.
• The framework balances computational efficiency and low memory footprint, and it underpins advanced applications such as post-training neural network quantization and quantum secure protocols.

RaBitQ-H denotes a family of randomized quantization methods and algorithmic frameworks prominent in high-dimensional vector quantization and approximate nearest neighbor search, and, in other contexts, as a protocol for quantum qubit-oblivious transfer. The term encompasses both theoretical constructs with provable error guarantees and practical systems for hardware-efficient search and quantization. This article surveys the major interpretations and technological manifestations of RaBitQ-H, with emphasis on its role as (1) a theoretically optimal multi-bit generalization of RaBitQ for Euclidean vector quantization, (2) a hybrid CPU/GPU implementation pipeline for scalable nearest neighbor search, (3) the randomized quantization core in state-of-the-art post-training neural network quantization, and (4) a protocol in quantum cryptography.

1. Algorithmic Origins and Core Quantization Framework

RaBitQ-H generalizes the original RaBitQ (Randomized Bi-valued Quantization) scheme by supporting $B$-bit codes per coordinate, thus interpolating smoothly between one-bit extreme compression and moderate- to high-precision quantizers. The central design uses a randomized rotation followed by per-coordinate grid quantization on a normalized integer lattice.

Given a vector $o_r \in \mathbb{R}^D$, the steps are:

1. Normalize: $o = (o_r - c)/\|o_r - c\|$.
2. Apply random orthogonal projection $P$: $o' = P^\top o$.
3. Quantize: For codebook grid $$G = \{ [-\frac{2^B-1}{2} + u] : u = 0,\ldots,2^B-1 \}^D$$, select $y \in G$, scaling to unit norm, to maximize the inner product $\langle o', y/\|y\| \rangle$. Efficient search uses a grid of "critical" rescaling factors, yielding complexity $O(D \cdot 2^B \log D)$ for quantization per vector at moderate $B$.

Storage per code is $D \cdot B$ bits plus $O(D)$ for normalization factors. Only the rotation matrix (or a seed) is globally stored, keeping the codebook implicit (Gao et al., 2024).

At query time, for a query $q_r$, similar normalization and projection are applied, and approximate squared Euclidean distances are reconstructed via the unbiased estimator:

$$\hat D = \|o_r - c\|^2 + \|q_r - c\|^2 - 2\|o_r-c\|\|q_r-c\| \left[\frac{\langle y/\|y\|, q \rangle}{\langle y/\|y\|, o \rangle}\right].$$

Empirically, for high-dimensional data, RaBitQ-H achieves sub-10% error with as little as $B=1$–$2$ bits per coordinate, and with $B=5$–$7$ can match or exceed the accuracy of other quantizers at much lower memory footprint. Theoretical results (see §2 below) show this method achieves the information-theoretic space–accuracy trade-off for high-dimensional Euclidean quantization (Gao et al., 2024).

2. Theoretical Guarantees and Optimality

The principal theoretical property of RaBitQ-H is a sharp additive error bound for inner product estimation. The main theorem asserts that, to obtain additive error $\leq \epsilon$ in $\langle o, q \rangle$ with high probability over the random codebook, it suffices to use per-coordinate codewidth

$$B = \Theta\left(\log\left(\frac{\epsilon^{-2} \log(1/\delta)}{D}\right)\right)$$

for ambient dimension $D$ and failure probability $\delta$, matching information-theoretic lower bounds up to constants (Gao et al., 2024).

The inner product and distance estimator is both unbiased and sub-Gaussian with variance scaling as $O(2^{-2B})$, controlling both per-query and worst-case tails. At $B=1$, the framework subsumes the original RaBitQ and provides an $O(D^{-1/2})$ additive error for inner-products, surpassing traditional product quantization and lattice vector quantization which lack such bounds (Gao et al., 2024).

The structure of code splitting—using the most-significant-bit for early pruning and the remainder for refinement—enables efficient two-phase search with fast lower-bounding and precise reranking.

3. Hybrid CPU/GPU ANN Search: IVF-RaBitQ-H Pipeline

IVF-RaBitQ-H refers to a hybrid architecture integrating RaBitQ-H quantization with a cluster-based index (IVF) and division of workload between CPU and GPU (Shi et al., 27 Feb 2026). The system allocates latency-insensitive and control-heavy tasks (clustering, codebook search, grid search for rescaling) to CPUs, and routes throughput-critical distance computation and candidate scanning to GPUs.

The pipeline is summarized as:

• Index Build: CPU performs balanced k-means clustering and codebook learning. Quantization uses either a shared rescaling factor (global) or per-vector factor (for small datasets or dense clusters), where the CPU executes coarse grid search and the GPU performs fine search and encoding.
• Data Storage: Each vector is encoded as a 1-bit code (MSB) for fast pre-prune, plus a $(B-1)$-bit ex-code for higher-precision reranking. Precomputed norm/magnitude factors are stored for fast distance calculation.
• Query/Search: Batch queries are rotated and projected on CPU; cluster selection is by batched GEMM. For each (query,cluster) pair, the GPU executes a fused-kernel: Stage 1 computes rank via 1-bit code (popcount/LUT), prunes early, and Stage 2 re-ranks survivors using full ex-code and refined estimator. Top-$K$ results are reduced/merged on CPU/GPU.

Performance analysis shows that for large datasets, CPU offloading reduces GPU-side quantization by up to 30%, and overall queries-per-second improves by $1.1 \times$–$1.3 \times$ relative to pure-GPU implementations. The recall characteristics are identical to IVF-RaBitQ, and storage remains $1$ bit/dim $+$ $(B-1)$ bits/dim per vector, plus centroids and ID arrays. CPU memory bandwidth and CPU utilization guide mode switching (per-vector versus shared-factor quantization) (Shi et al., 27 Feb 2026).

4. RaBitQ-H in Neural Network Quantization

In post-training quantization (PTQ) of large neural models, RaBitQ-H is the quantization core in the RaanA framework (Yang et al., 29 Mar 2025). Here, RaBitQ-H employs a randomized Hadamard transform (rather than a generic random rotation), reducing the rotation cost from $O(d^2)$ to $O(d\log d)$ per vector.

• Quantization: Each layer’s weight matrix is randomized-column-wise, then quantized to $b$ bits per entry using a learnable scaling per column.
• Allocation: Predictive coefficients $\alpha_k$ estimate each layer's quantization sensitivity; an integer program solves for per-layer bitwidths under a global bit-budget constraint.
• Efficiency: Allocation and codebook construction are CPU-efficient, utilizing zero- or few-shot calibration (e.g., $n_c=1$–$5$ samples), and avoiding Hessian or batch statistic estimation required by OBQ/OPTQ/AWQ/Quip# baselines.
• Performance: Empirically, for LLaMA-7B at 2.1 bits/weight, RaanA attains PPL=13.70 vs. 44.0 (GPTQ), 9.72 (OmniQuant), 9.95 (Quip#₂) on wikitext2; scaling to 70B remains within $0.1$–$0.5$ PPL of state-of-the-art, with quantization times several-fold faster than Quip# (Yang et al., 29 Mar 2025).

The core estimator remains unbiased, and per-layer or per-column bitwidths are flexibly allocated for optimal trade-off between model size and accuracy.

5. Quantum Cryptography: RaBitQ-H as p-Rabin Qubit-Oblivious Transfer

Independently, RaBitQ-H denotes a quantum protocol for p-Rabin qubit-oblivious transfer, leveraging probabilistic teleportation (MeiLing et al., 2018). In this setting:

• Protocol: Alice wishes to transfer an arbitrary pure qubit $|\psi\rangle$ to Bob, with probability $p=2|b|^2<1$ (for some fixed entangled resource). Bob knows whether the transfer succeeded; Alice remains oblivious.
• Mechanism: Utilizing a partially entangled Bell state and Bell measurement, Bob receives a post-measurement state $\lvert\chi_i\rangle$, applies an appropriate local correction, and tests success via an ancilla measurement. Security arguments show blindness (Alice never learns Bob's success), and the transfer only succeeds probabilistically, thus evading the Mayers–Lo–Chau no-go theorems for deterministic two-party quantum OT.
• Reduction: Restricting the input to basis states, RaBitQ-H implements classical-bit Rabin OT as a special case (MeiLing et al., 2018).

A plausible implication is that, while this protocol shares nomenclature with the high-dimensional quantization schemes, it is contextually orthogonal; ambiguity arises only in the literature due to acronym collision.

6. Practical Implementation and Performance Guidelines

RaBitQ-H provides several practical features:

• SIMD and blockwise optimization: Codes are organized as MSB$+$ex-code, enabling fast popcount or LUT-based batch processing (AVX2, AVX512). Early pruning on 1-bit codes discards $>50\%$ of candidates before any expensive calculation (Gao et al., 2024).
• Memory layout: Codes are aligned for register-widths, and queries are batched to maximize reuse.
• Parameter selection: For 95% recall in in-memory vector search, $B=5$ is typically sufficient; $B=7$ achieves near-perfect recall. The quantization pipeline is robust for $B \leq 8$, beyond which codebook search cost can become appreciable (Gao et al., 2024).
• Online flexibility: Codes are deterministic for a fixed $P$; $B$ can be changed post-hoc by re-encoding stored codes.

Integration with libraries (such as NVIDIA cuVS) involves hybrid wrappers for index build and fused-scan, exposing device-selection APIs. Memory bandwidth considerations are negligible, as per-vector or per-batch quantization is CPU-bound; search and reranking are the limiting factor for GPU compute (Shi et al., 27 Feb 2026).

7. Limitations, Open Questions, and Extensions

For large $B$ ($B > 8$), per-vector quantization admits high preprocessing costs due to exponential scaling in $2^B$. Mitigating this issue by approximate grid search or hierarchical codebook design is a prominent avenue for further research (Gao et al., 2024).

Potential applications for generalized RaBitQ-H codebooks include hybrid queries (combining vector and attribute search), and explorations into hierarchical and multi-scale coding for extreme memory regimes. In quantum cryptography, RaBitQ-H's probabilistic approach may inspire further protocols for quantum secure computation resistant to current impossibility results (MeiLing et al., 2018).

In summary, RaBitQ-H defines a unified mathematical and algorithmic foundation for high-precision, hardware-efficient high-dimensional quantization and fast similarity search, with rigorous error analysis and demonstrated scalability. Its quantum OT instantiation exemplifies probabilistic techniques circumventing classical cryptographic barriers.