--- title: SymRaBitQ for AI Search url: https://www.emergentmind.com/topics/symrabitq type: topic --- # SymRaBitQ for AI Search SymRaBitQ is a symmetric vector-quantization scheme designed for approximate inner-product and distance evaluation between two quantized vectors. Introduced in the context of UBASE, ByteDance’s large-scale AI search system, it extends the asymmetric RaBitQ approach from query-to-database scoring to database-to-database comparisons required during graph-index construction, pruning, merging, clustering, and traversal. Its central property is that both operands use the same rotationally transformed, normalized lattice code, allowing their inner product to be estimated from compact codes, per-vector residual norms, and self-correlation terms without reconstructing full-precision vectors [2608.30607]. ## 1. Motivation and scope Large-scale approximate-nearest-neighbor systems require many more operations than query-to-database distance evaluation. Graph construction repeatedly compares stored vectors with one another during candidate discovery, robust pruning, backward-edge repair, and graph merging. If only database vectors are quantized while full-precision queries are retained, these data-to-data comparisons continue to require the original vectors. This distinction becomes significant at high dimensionality and scale. A \(2048\)-dimensional FP32 vector occupies approximately \(8\) KB, so \(100\) million vectors require roughly \(800\) GB before graph metadata or temporary construction state. In UBASE’s production example, graph construction reached \(27\) TB against a \(13.8\) TB DRAM budget. Conventional quantization may reduce serving-time storage while leaving the index builder dependent on full-precision vectors. SymRaBitQ addresses this limitation by applying a common quantized representation to both stored vectors and queries. The same representation is used for: - query-to-database distance evaluation; - database-to-database distance evaluation during graph construction; - robust graph pruning and backward-edge repair; - IVF clustering and list construction; - segment merging; - graph traversal and final candidate refinement. The resulting architecture can discard full-precision vectors from the hot construction and serving path. Raw vectors may remain in remote or durable storage, but index construction and most approximate-nearest-neighbor operations use quantized records and scalar correction terms. SymRaBitQ is not a new graph topology or a coherent quantum protocol. In UBASE, it replaces full-precision distance oracles in DiskANN/Vamana-style graph construction and search with a symmetric quantized estimator. ## 2. Residual representation Let raw database and query vectors be \(\bm{x}_r,\bm{q}_r\in\mathbb{R}^D\), and let \(\bm{c}\) denote a centroid when indexing is organized around IVF or another partitioned structure. UBASE represents the vectors by normalized residuals: \[ \bm{o}=\frac{\bm{x}_r-\bm{c}}{\|\bm{x}_r-\bm{c}\|},\qquad \bm{q}=\frac{\bm{q}_r-\bm{c}}{\|\bm{q}_r-\bm{c}\|}. \] The residual magnitudes are stored as scalars: \[ \rho_{\bm{o}}=\|\bm{x}_r-\bm{c}\|,\qquad \rho_{\bm{q}}=\|\bm{q}_r-\bm{c}\|. \] The raw Euclidean distance can then be reconstructed from an estimate of the residual inner product: \[ \|\bm{x}_r-\bm{q}_r\|^2 = \rho_{\bm{o}}^2+\rho_{\bm{q}}^2 -2\rho_{\bm{o}}\rho_{\bm{q}} \langle\bm{o},\bm{q}\rangle. \] The corresponding maximum-inner-product identity is \[ \langle\bm{x}_r,\bm{q}_r\rangle = \langle\bm{x}_r,\bm{c}\rangle-\|\bm{c}\|^2 +\langle\bm{q}_r,\bm{c}\rangle +\rho_{\bm{o}}\rho_{\bm{q}}\langle\bm{o},\bm{q}\rangle. \] Terms involving the stored vector and centroid can be precomputed. The principal computational task is therefore the estimation of \(\langle\bm{o},\bm{q}\rangle\). The quantization process uses a random orthogonal rotation matrix \(P\) and \(B\) bits per dimension. For a unit residual \(\bm{x}\), where \(\bm{x}\) is either \(\bm{o}\) or \(\bm{q}\), the procedure is: 1. Compute the residual norm \(\rho_{\bm{x}}\). 2. Normalize the residual. 3. Rotate it into the lattice coordinate system: \[ \bm{y}=P^{-1}\bm{x}. \] 4. Find the nearest \(B\)-bit integer-lattice code \(\bm{u}\), with \[ u_i\in\{0,1,\ldots,2^B-1\}. \] 5. Center the code: \[ \tilde{\bm{u}} = \bm{u}-\frac{2^B-1}{2}\bm{1}_D. \] 6. Rotate and normalize the centered code: \[ \bar{\bm{x}} = P\frac{\tilde{\bm{u}}}{\|\tilde{\bm{u}}\|}. \] 7. Store a first-bit/sign projection \(\bar{\bm{x}}_0\) for inexpensive coarse comparisons. 8. Store the self-correlation \[ \gamma_{\bm{x}} = \langle\bar{\bm{x}},\bm{x}\rangle. \] An encoded record is conceptually \[ z_{\bm{x}} = \left( \bar{\bm{x}}, \bar{\bm{x}}_0, \gamma_{\bm{x}}, \rho_{\bm{x}} \right). \] The rotated floating-point vector need not be stored. The integer code \(\bm{u}\), or its centered form, is sufficient because the common rotation cancels in code-to-code inner products. The underlying unrotated codebook is \[ \mathcal{G} = \left\{ -\frac{2^B-1}{2}+u \;\middle|\; u=0,1,\ldots,2^B-1 \right\}^D. \] Although the rotated codebook contains \(2^{BD}\) candidates, orthogonality permits encoding in the unrotated lattice. For two centered codes \(\tilde{\bm{u}}_o\) and \(\tilde{\bm{u}}_q\), \[ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle = \frac{ \langle\tilde{\bm{u}}_o,\tilde{\bm{u}}_q\rangle }{ \|\tilde{\bm{u}}_o\|\,\|\tilde{\bm{u}}_q\| }. \] For \(B=1\), centered codes reduce to signs. Their numerator becomes normalized sign agreement and can be evaluated using bitwise operations, Hamming distance, or population count. UBASE uses \(B=5\) for its higher-precision construction code and reports more than \(98\%\) distance-estimation accuracy. ## 3. Symmetric inner-product estimator SymRaBitQ estimates the inner product of two encoded unit residuals by correcting the code-to-code inner product with both self-correlations: \[ \widehat{\langle\bm{o},\bm{q}\rangle} = \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle }. \] Using the stored scalar terms, \[ \widehat{\langle\bm{o},\bm{q}\rangle} = \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \gamma_{\bm{o}}\gamma_{\bm{q}} }. \] For database-to-database comparisons, both self-correlations are stored or precomputed. For query-to-database comparisons, the database self-correlation is stored and the query self-correlation is computed once when the query is encoded. The resulting approximate squared-distance oracle is \[ \operatorname{SymDist}_{B} (\bar{\bm{o}},\bar{\bm{q}};\bm{c}) = \rho_{\bm{o}}^2+\rho_{\bm{q}}^2 - 2\rho_{\bm{o}}\rho_{\bm{q}} \widehat{\langle\bm{o},\bm{q}\rangle}. \] This oracle is used by graph construction, graph search, IVF, and refinement. ### Relationship to RaBitQ The original RaBitQ-style estimator is asymmetric: \[ \widehat{\langle\bm{o},\bm{q}\rangle}_{\mathrm{RaBitQ}} = \frac{ \langle\bar{\bm{o}},\bm{q}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle }. \] Here, the database residual \(\bm{o}\) is quantized while the query residual \(\bm{q}\) remains full precision. This is suitable for query-time database scoring but insufficient for graph construction, where repeated comparisons involve pairs of stored vectors. SymRaBitQ changes the estimator by quantizing both operands: \[ \widehat{\langle\bm{o},\bm{q}\rangle}_{\mathrm{SymRaBitQ}} = \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle }. \] Its distinguishing feature is therefore not merely a lower bit rate. It combines a shared randomized rotational code construction, normalized residuals, residual norms, self-correlations, and a symmetric code-to-code estimator. ## 4. Accuracy and theoretical properties The stated expectation identity for normalized residuals is \[ \mathbb{E}\left[ \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle } \right] = \langle\bm{o},\bm{q}\rangle. \] The paper also gives a high-probability error bound: \[ \mathbb{P}\left\{ \left| \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle } - \langle\bm{o},\bm{q}\rangle \right| > \Delta_{\bm{o},\bm{q}} \frac{\epsilon_0}{\sqrt{D-1}} \right\} \leq 4e^{-c_0\epsilon_0^2}, \] where \[ \Delta_{\bm{o},\bm{q}} = \sqrt{ \frac{1-\langle\bar{\bm{q}},\bm{q}\rangle^2} {\langle\bar{\bm{q}},\bm{q}\rangle^2} } + \sqrt{ \frac{ 1-\langle\bar{\bm{o}},\bm{o}\rangle^2 }{ \langle\bar{\bm{o}},\bm{o}\rangle^2 \langle\bar{\bm{q}},\bm{q}\rangle^2 } }. \] The paper summarizes the resulting behavior as \[ \left| \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle } - \langle\bm{o},\bm{q}\rangle \right| = O\left(\frac{1}{\sqrt{D}}\right) \] with high probability. The proof decomposes the estimation error into residual terms associated with quantizing the two operands and applies high-dimensional concentration. The random rotation makes directions approximately isotropic and supports concentration of orthogonal projections. The supplied paper text contains a qualification concerning unbiasedness. The main theorem presents exact unbiasedness as an expectation identity, whereas the appendix’s sign-based argument describes approximate unbiasedness in typical high-dimensional settings. For the one-bit case, the appendix uses the sign-agreement identity \[ \mathbb{E}[\xi(X_1,X_2)] = 1-\frac{2\theta}{\pi}, \] where \(\theta\) is the angle between the vectors, and notes that self-correlations are close to \[ \sqrt{\frac{2}{\pi}}. \] Consequently, with \(t=\frac{\pi}{2}-\theta\), \[ \mathbb{E}\left[ \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle } \right] \approx \frac{\pi}{2}-\theta \] and the appendix obtains \[ \mathbb{E}\left[ \frac{ \langle\bar{\bm{o}},\bar{\bm{q}}\rangle }{ \langle\bar{\bm{o}},\bm{o}\rangle \langle\bar{\bm{q}},\bm{q}\rangle } \right] - \langle\bm{o},\bm{q}\rangle = t-\sin(t) = O(t^3). \] Thus, the formal theorem and the appendix explanation emphasize related but distinct aspects: an exact stated expectation guarantee and an approximate-isotropy interpretation for typical high-dimensional inputs. Empirically, GIST comparisons show estimated and actual distances concentrated around the identity line, with fitted slope and intercept close to \(1\) and \(0\). This supports the reported accuracy but does not replace the theoretical bound. ## 5. Quantized index construction UBASE uses DiskANN/Vamana-style graph topology; SymRaBitQ supplies the distance oracle rather than a new graph algorithm. Before construction, each vector assigned to a cluster or centroid is encoded as \[ z_i(\bm{o}) = \left( \bar{\bm{o}}, \bar{\bm{o}}_0, \gamma_{\bm{o}}, \rho_{\bm{o}} \right). \] The graph builder then operates on these records. ### Candidate discovery For an inserted encoded vector \(z_{\bm{o}}\), greedy candidate discovery repeatedly selects the closest unvisited candidate under \[ \operatorname{SymDist}_B(z_j,z_{\bm{o}}), \] expands its outgoing neighbors, and truncates the frontier to search width \(L\). These comparisons do not require fetching raw vectors. ### Robust pruning Robust pruning selects candidates in increasing approximate distance from the target: \[ z^* = \arg\min_{z_j\in\mathcal{V}} \operatorname{SymDist}_B(z_{\bm{o}},z_j). \] It removes a candidate \(z_j\) whenever \[ \alpha\cdot \operatorname{SymDist}_B(z^*,z_j) \leq \operatorname{SymDist}_B(z_{\bm{o}},z_j). \] Both terms involve distances between stored vectors. This is the operation for which asymmetric quantization is inadequate and symmetric code-to-code evaluation is required. ### Backward-edge repair and merging When adding a backward edge would cause a vertex to exceed the degree bound \(R\), UBASE reruns robust pruning. Distances continue to be computed from codes, residual norms, and self-correlations. For clustered construction, the system computes cluster centers on a sample, assigns each vector to its \(l\) nearest centers, constructs local subgraphs, and merges them by unioning edges. The construction process includes initialization of an \(R\)-regular directed graph per cluster, parallel encoded-record insertion, pruning, backward-edge repair, and final subgraph merging. The lifecycle consequence is that the builder no longer needs a full-precision copy after encoding. Construction memory is dominated by compact codes, graph edges, and metadata rather than \(D\)-dimensional FP32 vectors. ## 6. Search, storage, and systems integration ### Graph search Queries are encoded once per searched partition: \[ z_{\bm{q}} = \left( \bar{\bm{q}}, \bar{\bm{q}}_0, \gamma_{\bm{q}}, \rho_{\bm{q}} \right). \] UBASE applies a coarse-to-fine strategy: 1. During graph traversal, it compares first-bit projections using \[ \operatorname{SymDist}_1(z_j,z_{\bm{q}}). \] 2. It uses bit operations and SIMD population count, including `vpopcntq`. 3. It maintains the traversal frontier and retains the closest candidates under the one-bit estimate. 4. It selects the closest \(\beta k\) candidates as a promising set \(\mathcal{S}\). 5. It rescores these candidates using the full \(B\)-bit estimator: \[ \operatorname{SymDist}_B(z_j,z_{\bm{q}}). \] 6. It returns the closest \(k\) candidates. For IVF, the same representation is used for list construction and search. Quantized \(k\)-means and list assignment use \(\operatorname{SymDist}_B\). At query time, the system probes the closest centroids, uses one-bit codes for coarse screening and lower-bound-based rejection, and applies \(B\)-bit FastScan-style refinement to survivors. ### Memory and computation The principal systems benefit is elimination of the temporary full-precision construction copy. UBASE can construct and merge graph indexes directly from quantized records, freeing DRAM for serving caches. On GIST 1M, DiskANN requires \(587\) seconds to build with \(3.75\) GB peak memory, whereas UBASE requires \(194+43=237\) seconds and \(0.75\) GB, corresponding to a \(60\%\) build-time reduction and an \(80\%\) peak-memory reduction. On Cohere 100M, build time decreases from \(23{,}826\) seconds to \(11{,}033+1{,}463=12{,}496\) seconds, a \(48\%\) reduction, while peak memory decreases from \(453\) GB to \(90\) GB, an \(80\%\) reduction. For IVF on GIST 1M, conventional IVF requires \(176\) seconds and \(3.7\) GB peak memory, while SymRaBitQ IVF requires \(129\) seconds and \(1.9\) GB. The corresponding reductions are \(27\%\) in build time and \(49\%\) in peak memory. Quantization incurs an up-front encoding cost. On GIST 1M, \(43\) seconds of UBASE’s \(237\)-second build time is spent on quantization and \(194\) seconds on graph construction in quantized space. At query time, one-bit comparisons use bitwise operations and SIMD population count, while full \(B=5\) comparisons use compact integer dot products and FastScan-style lookup tables on promising candidates. ### Hybrid storage Compact records also support SSD-resident indexes. UBASE stores each graph record as a compact vector payload and bounded neighbor list. A record-level cache maps vertex IDs to records using a 64-bit logical pointer containing residency, state, and offset information. The quantizer does not provide the cache policy or asynchronous scheduler, but compact records make these mechanisms more effective. Resident records are expanded immediately; promising disk-resident records are fetched asynchronously; searches yield while SSD reads are outstanding; worker threads execute other ANN coroutines; and suspended searches resume when records arrive. In the storage ablation, UBASE’s record cache achieves a \(97.7\%\) cache hit rate on GIST. Although a query visits \(105.99\) graph records on average, it requires \(51.90\) SSD accesses, compared with \(80.9\) for DiskANN. At 32 threads, asynchronous execution reaches approximately \(15\)K QPS at approximately \(1.7\) ms average latency, about \(1.8\times\) the synchronous UBASE variant and \(2\times\) DiskANN. ## 7. Evaluation, limitations, and significance The evaluation covers Wiki, with \(1\)M vectors of \(768\) dimensions under cosine distance; GIST, with \(1\)M vectors of \(960\) dimensions under \(L_2\); Cohere, with \(113\)M vectors of \(1024\) dimensions under \(L_2\); a \(100\)M-vector production-derived in-house dataset; and NFCorpus, FIQA, and Quora for hybrid retrieval quality. The controlled testbed uses Intel Xeon Platinum 8457C CPUs, three data nodes, and two coordinators. Baselines include Elasticsearch 8.18, Milvus 2.5, PostgreSQL 17 with VectorChord 1.0, HNSW, and DiskANN. The most direct SymRaBitQ measurements are the graph-construction and IVF ablations. The paper reports less than \(0.01\) recall loss under the same search parameters and more than \(98\%\) distance-estimation accuracy for \(B=5\). The broader UBASE results combine SymRaBitQ with the vector kernel, graph design, SIMD execution, record caching, asynchronous I/O, and OpenSearch integration, and therefore are not quantizer-only measurements. Reported system-level results include: - \(21{,}445\) QPS at approximately \(90\%\) Recall@10 on the \(100\)M-vector in-house dataset; - \(9.5\times\) the throughput of Milvus, \(5.5\times\) PostgreSQL, and \(2.3\times\) Elasticsearch on that dataset; - \(23{,}013\) QPS at approximately \(92\%\) Recall@10 on Cohere; - \(11.7\times\) Milvus, \(5.4\times\) Elasticsearch, and \(3.5\times\) PostgreSQL on Cohere; - approximately \(25\)K QPS at \(87\%\) recall and approximately \(15\)K QPS at \(97\%\) recall on Wiki; - up to \(5\times\) the speed of PostgreSQL and more than \(2\times\) the speed of the second-best Milvus configuration on GIST; - an end-to-end Wiki index footprint of \(1{,}182\) MB, or \(29\%\) of PostgreSQL/VectorChord and \(42\%\) of Milvus. The guarantees assume normalized residual vectors and depend on the quality of the random rotation, high-dimensional concentration, and sufficiently strong self-correlation. The \(O(1/\sqrt{D})\) behavior is most compelling in high dimensions. The approximate-unbiasedness discussion relies on near-orthogonality and typical angles near \(\pi/2\). The paper does not provide a complete finite-dimensional distortion analysis for every data distribution or quantify worst-case graph-quality degradation under adversarial or highly anisotropic data. SymRaBitQ also does not eliminate all full-precision storage. UBASE retains raw vectors in remote or durable storage for durability and offline maintenance, although they are removed from the hot graph-construction and serving path. In addition, the reported system improvements do not constitute a fully isolated factorial analysis separating the effects of quantization, SIMD execution, caching, asynchronous I/O, and graph design. The strongest causal evidence concerns construction comparisons and the IVF ablation. The principal significance of SymRaBitQ is that the representation used to search can also be the representation used to build and maintain the index. By making both operands compact and directly comparable while supplying theoretical control of estimation error, it removes the full-precision vector copy that otherwise dominates ingestion memory. At trillion-vector scale, reducing query-time distance cost alone is insufficient; compression must cover graph construction, pruning, merging, clustering, traversal, and serving. SymRaBitQ provides that lifecycle-wide quantized distance kernel. Source: https://www.emergentmind.com/topics/symrabitq