--- title: 'EigenLI: A Matrix-based Retrieval Model' url: https://www.emergentmind.com/topics/eigenli type: topic --- # EigenLI: A Matrix-based Retrieval Model EigenLI is a training-free spectral approximation framework for compressing late-interaction retrieval representations into document-specific low-dimensional subspaces. Designed for models such as ColBERT, it replaces a document’s many token-level vectors with dominant eigendirections of their second-moment matrix, reducing indexing and scoring costs while retaining retrieval effectiveness. Its multi-vector form, $k$-EigenLI, stores $k$ orthonormal spectral directions per document; its single-vector form, EigenLI-SV, converts the resulting second-order structure into an ANN-compatible vector. The framework is introduced in “EigenLI: Spectral Approximations to Late Interaction” [2609.07561]. ## 1. Late-interaction retrieval and the compression problem Late-interaction models represent a document and a query as sets of token-level vectors rather than as single embeddings. Let $$ D=\{d_1,\ldots,d_m\}\subset\mathbb{R}^d,\qquad Q=\{q_1,\ldots,q_n\}\subset\mathbb{R}^d, $$ where token vectors are typically $L_2$-normalized. The ColBERT interaction function is $$ \operatorname{MaxSim}(Q,D) = \sum_{i=1}^{n}\max_{1\leq j\leq m}\langle q_i,d_j\rangle. $$ Each query token independently selects its most similar document token, after which the selected similarities are summed. This preserves fine-grained matching information because different query tokens can match different document tokens. The same expressivity creates substantial computational and storage requirements. A ColBERTv2 document may contain 512 vectors of dimension 128, while visual late-interaction models may contain approximately 1,000–1,250 patch vectors per document. The resulting costs include large indexes and memory footprints, expensive document encoding and indexing, expensive query–document scoring, and limited compatibility with conventional single-vector approximate-nearest-neighbor (ANN) indexes. EigenLI is based on the observation that the effective geometric complexity of a document’s token representation can be substantially lower than the number of stored vectors suggests. Document token vectors often concentrate near a document-specific low-dimensional subspace. EigenLI therefore compresses the token set by retaining dominant spectral directions rather than selecting or constructing a smaller set of token-like representatives. ## 2. Spectral representation of document tokens For a document $D$, EigenLI forms the unnormalized second-moment matrix $$ M_D=\sum_{i=1}^{m}d_i d_i^T. $$ If the document vectors are arranged as columns of $$ X_D=[d_1\ \cdots\ d_m]\in\mathbb{R}^{d\times m}, $$ then $$ M_D=X_DX_D^T. $$ The eigendecomposition is written as $$ M_Dw_r=\lambda_r w_r, \qquad \lambda_1\geq\lambda_2\geq\cdots\geq\lambda_d\geq0. $$ The leading eigenvectors $w_1,\ldots,w_k$ define the document’s dominant spectral subspace. If the token vectors are unit norm, then $$ \operatorname{tr}(M_D) = \sum_{r=1}^{d}\lambda_r = \sum_{i=1}^{m}\|d_i\|_2^2 = m. $$ Let $$ \Pi_k=\sum_{r=1}^{k}w_rw_r^T $$ be the orthogonal projector onto the dominant $k$-dimensional eigenspace. The standard PCA identity gives $$ \sum_{i=1}^{m}\|d_i-\Pi_kd_i\|_2^2 = \sum_{r\geq k+1}\lambda_r. $$ Consequently, if the leading eigenvalues contain at least a $(1-\epsilon)$ fraction of the total spectral mass, $$ \sum_{r=1}^{k}\lambda_r \geq (1-\epsilon)\sum_{r=1}^{d}\lambda_r, $$ then $$ \frac{1}{m}\sum_{i=1}^{m}\|d_i-\Pi_kd_i\|_2^2 \leq \epsilon. $$ This establishes the geometric premise of EigenLI: a document’s token vectors can be close, on average, to a subspace of dimension $k$ even though they are embedded in $\mathbb{R}^d$. The construction can also be obtained from a singular-value decomposition. If $$ X_D=U\Sigma V^T, $$ then $$ M_D=U\Sigma^2U^T, $$ so the columns of $U$ associated with the largest singular values are the dominant eigenvectors of $M_D$. ## 3. $k$-EigenLI construction and interaction score For each document, $k$-EigenLI performs the following operations: 1. The document is encoded using the original late-interaction model. 2. The second-moment matrix $M_D$ is formed. 3. Its top $k$ eigenvectors are computed. 4. The resulting orthonormal basis is stored instead of the original document-token vectors. The compressed document representation is $$ W_D=[w_1\ \cdots\ w_k]\in\mathbb{R}^{d\times k}. $$ The vectors $w_j$ are not representative document tokens. They are basis vectors for a document-specific subspace and need not correspond to any actual token embedding. Given query vectors $q_1,\ldots,q_n$, EigenLI uses the score $$ s_k(Q,D) = \sum_{i=1}^{n}\sum_{j=1}^{k} \langle q_i,w_j\rangle^2. $$ Equivalently, $$ s_k(Q,D) = \sum_{i=1}^{n}\|\Pi_kq_i\|_2^2. $$ Thus, each query token contributes according to the squared norm of its projection onto the document’s dominant subspace. If $$ S_D=\operatorname{span}\{w_1,\ldots,w_k\}, $$ then $$ s_k(Q,D) = \sum_{i=1}^{n} \max_{\substack{v\in S_D\\\|v\|_2=1}} \langle q_i,v\rangle^2. $$ The score can therefore be interpreted as a subspace-alignment measure: each query token evaluates its strongest squared alignment with any unit direction in the document subspace. A defining property of EigenLI is that the eigenvalues are not used to weight the score. The alternative $$ \sum_{i=1}^{n}\sum_{j=1}^{k} \lambda_j\langle q_i,w_j\rangle^2 $$ performs worse according to the paper. This weighted form approximates the sum of squared similarities to all document tokens rather than the best-match behavior of MaxSim. Omitting $\lambda_j$ makes the method a subspace-presence test rather than a weighted estimate of total document-token similarity. The relationship to MaxSim is therefore indirect. MaxSim selects an individual document token for each query token, whereas EigenLI selects the best direction within a document-specific subspace. If every document token satisfies $$ \|d_\ell-\Pi_kd_\ell\|_2\leq\delta, $$ and $d_{\tau(i)}$ is the MaxSim-selected token for $q_i$, the paper gives the intuition $$ s'_k(Q,D) \geq \operatorname{MaxSim}(Q,D)-n\delta, $$ for the related unsquared score $$ s'_k(Q,D) = \sum_{i=1}^{n} \max_{\substack{v\in S_D\\\|v\|_2=1}} \langle q_i,v\rangle. $$ This relation is not a complete ranking-equivalence theorem: average PCA reconstruction error does not imply a uniform per-token approximation bound. ## 4. Difference from pooling and clustering Training-free late-interaction compression methods such as $k$-means++ pooling and Ward pooling reduce the document representation to a smaller set of vectors. These vectors are generally cluster centroids or hierarchically merged representatives, and retrieval continues to use a MaxSim-like interaction against them. EigenLI changes the compression target. It maps $$ \text{a set of }m\text{ token vectors} \longrightarrow \text{one }k\text{-dimensional subspace}. $$ The retained directions are globally optimal spectral directions for reconstructing the document-token matrix in a squared-error/PCA sense. They are not centroids, and they need not correspond to existing document tokens. Retrieval also uses the subspace-interaction score rather than MaxSim against representative vectors. This distinction means EigenLI is not simply PCA applied independently to individual tokens. PCA is applied across the document’s token set to reduce the number of document representation vectors while preserving the dominant geometry of the set. The offline storage requirement changes from $md$ scalar entries to $kd$ entries per document. For ColBERTv2, the original representation is $512\times128=65{,}536$ scalar entries. At $k=32$, EigenLI stores $32\times128=4{,}096$ entries, corresponding to a nominal $16\times$ reduction before metadata and datatype effects. At query time, EigenLI computes $n\times k$ inner products rather than approximately $n m$ query–document token comparisons. In the reported compression experiments, $k$-means++ required $6.81\times$ the time of EigenLI under the arithmetic mean and $6.47\times$ under the geometric mean across 39 model–dataset pairs. Ward pooling required $17.46\times$ and $16.89\times$, respectively. ## 5. EigenLI-SV and ANN compatibility The same second-order structure yields EigenLI-SV, a single-vector representation compatible with ordinary ANN infrastructure. Using $$ \langle q_i,w_j\rangle^2 = \langle q_i\otimes q_i,\, w_j\otimes w_j\rangle, $$ the multi-vector score becomes $$ s_k(Q,D) = \left\langle \sum_{i=1}^{n}q_i\otimes q_i,\, \sum_{j=1}^{k}w_j\otimes w_j \right\rangle. $$ The query and document can consequently be represented by second-order summaries, $$ z_Q=\sum_i q_i\otimes q_i, \qquad z_D=\sum_j w_j\otimes w_j. $$ Because these summaries are symmetric, EigenLI-SV uses the quadratic feature map $$ K(x) = (x_1^2,\ldots,x_d^2,\ldots,\sqrt{2}x_ix_j,\ldots), $$ whose dimension is $$ \frac{d(d+1)}{2}. $$ The representations are $$ z_Q^{\mathrm{SV}} = \sum_{i=1}^{n}K(q_i), \qquad z_D^{\mathrm{SV}} = \sum_{j=1}^{k}K(w_j), $$ and their dot product exactly equals the EigenLI multi-vector score up to numerical precision. For ColBERTv2, $d=128$, giving $$ \frac{128\cdot129}{2}=8256 $$ dimensions. This is smaller than the 10,240-dimensional fixed-dimensional encoding used by MUVERA in the reported comparison. EigenLI-SV is compatible with HNSW, FAISS, DiskANN, ScaNN, SPANN, product quantization, and related ANN systems. The resulting vector is not a mean embedding, a concatenation of basis vectors, or a first-order pooling statistic. It is the flattened symmetric second-order matrix $$ \sum_{j=1}^{k}w_jw_j^T, $$ with the query represented analogously by $$ \sum_{i=1}^{n}q_iq_i^T. $$ ## 6. Empirical evaluation EigenLI was evaluated with ColBERTv2, AnswerAI-ColBERT-small, and GTE-ModernColBERT on MS MARCO passages and 13 BEIR development datasets: ArguAna, Climate-FEVER, DBPedia-Entity, FEVER, FiQA, HotpotQA, MS MARCO, NQ, Quora, SCIDOCS, SciFact, TREC-COVID, and Webis-Touche2020. It was also evaluated with the Qwen-based visual late-interaction model ColQwen3 VL 4B on eight ViDoRe-v3 datasets. The multi-vector experiments used $k\in\{8,16,32\}$, with full MaxSim as the uncompressed reference. The clustering baselines were $k$-means++ pooling and Ward hierarchical pooling. The single-vector comparison used MUVERA fixed-dimensional encodings with 20 repetitions, projection dimension 16, and 32 partitions, producing 10,240 dimensions. ### ColBERTv2 At $k=32$, EigenLI outperformed both clustering baselines across aggregate BEIR metrics. Relative improvements over $k$-means++ were approximately: - Recall: $+12.9\%$ geometric mean and $+13.8\%$ arithmetic mean; - nDCG: $+13.8\%$ geometric mean and $+15.0\%$ arithmetic mean; - MRR: $+12.5\%$ geometric mean and $+13.4\%$ arithmetic mean. Relative improvements over Ward were smaller but positive, including $+7.6\%$ geometric-mean nDCG and $+7.3\%$ geometric-mean MRR. EigenLI-32 remained below full MaxSim in aggregate, by roughly $3\%$ in nDCG and Recall depending on the averaging method. ### AnswerAI-ColBERT-small The overall pattern was similar. EigenLI-32 improved over $k$-means++ by approximately $10.2\%$ under the geometric mean and $10.9\%$ under the arithmetic mean on the aggregate metric summary. Improvements in nDCG-related measures were approximately $11\%$. The method was modestly better than Ward on average in the reported aggregate table, although the discussion characterizes the two methods as broadly comparable. ### GTE-ModernColBERT GTE-ModernColBERT exhibited a different tradeoff. At $k=32$, EigenLI was competitive with $k$-means++ for deeper Recall but generally trailed both clustering methods in nDCG and MRR. Relative differences versus $k$-means++ included approximately $-2.5\%$ geometric-mean Recall, $-2.8\%$ nDCG, and $-2.6\%$ MRR. Deficits against Ward were approximately $8\%$–$10\%$ for several top-level metrics. The paper associates this behavior with model-dependent geometry and strong anisotropy or degeneracy. On DBPedia-Entity, performance could decline substantially as $k$ increased from 8 to 32. In such cases, additional spectral directions may reflect broad global structure rather than retrieval-relevant distinctions. ### Visual retrieval With ColQwen3 on ViDoRe-v3, EigenLI-32 outperformed both pooling baselines at the aggregate level. Relative improvements over $k$-means++ were approximately $+4.8\%$ in Recall@10 and $+5.5\%$ in nDCG@10; improvements over Ward were approximately $+3.1\%$ and $+3.4\%$, respectively. EigenLI remained below full MaxSim. Performance declined at $k=128$, reinforcing the model dependence of the useful spectral budget. ### EigenLI-SV versus MUVERA EigenLI-SV consistently outperformed MUVERA on the evaluated BEIR datasets for all three text models. For ColBERTv2, the arithmetic-mean relative improvement in nDCG@10 was $+78.9\%$, with a geometric-mean improvement of $+69.6\%$. The corresponding arithmetic-mean improvements were $+178.1\%$ for AnswerAI-ColBERT-small and $+156.0\%$ for GTE-ModernColBERT. The paper cautions that relative improvements can be large when the MUVERA baseline is weak. Nevertheless, absolute per-dataset results consistently favored EigenLI-SV. For example, on ColBERTv2, EigenLI-SV-32 versus MUVERA Recall@10 was 0.6600 versus 0.5000 on ArguAna, 0.4125 versus 0.2399 on FiQA, 0.7960 versus 0.2280 on TREC-COVID, and 0.6685 versus 0.5377 on MS MARCO. The paper also evaluated centering, $$ \bar{x}=\frac{1}{n}\sum_i x_i, \qquad x_i\leftarrow x_i-\bar{x}, $$ excluding padding tokens and without subsequent renormalization. Centering substantially improved MUVERA on AnswerAI-ColBERT-small and GTE-ModernColBERT, while EigenLI-SV generally remained superior. EigenLI-SV was not evaluated on ColQwen3 because its quadratic dimension would be $$ \frac{320\cdot321}{2}=51{,}360. $$ ## 7. Parameter selection, limitations, and significance The principal hyperparameter is $k$, the number of retained spectral directions. Lower values provide stronger compression but can discard retrieval-relevant variation. Increasing $k$ captures more document-token geometry but can also make the document subspace less selective, allowing irrelevant query directions to receive score. The reported operating points are: - $k=8$: strongest compression with potentially greater information loss; - $k=16$: intermediate compression–quality tradeoff; - $k=32$: the most consistently successful setting in the experiments; - larger $k$: not necessarily better and potentially harmful for anisotropic representations. EigenLI is therefore not guaranteed to improve monotonically with $k$. The useful value depends on the backbone and domain, and the paper recommends model-aware validation. The reported ablations show that eigenvalue-weighted scoring performs worse than the unweighted projector score, centering benefits MUVERA more consistently than EigenLI, and model geometry substantially affects effectiveness. EigenLI-SV remains usable after product quantization, but quantization introduces measurable degradation. Against full MaxSim on four BEIR datasets, average relative Recall losses for 1-, 2-, 4-, and 8-bit product quantization were approximately $-8.9\%$, $-5.5\%$, $-5.5\%$, and $-5.2\%$ at @10, respectively. Several limitations remain. The experiments are primarily brute-force retrieval evaluations and do not provide a complete end-to-end ANN pipeline study against PLAID, WARP, optimized Ward indexes, or other production systems. The quadratic dimension of EigenLI-SV becomes costly for large token dimensions, as illustrated by the 51,360-dimensional representation for ColQwen3. Spectral low rank does not ensure retrieval relevance: dominant directions can encode anisotropic or globally shared structure rather than discriminative semantics. The method is training-free and therefore cannot directly optimize its subspaces or scoring function for a target relevance objective. EigenLI’s central significance is the change in compression objective. Instead of approximating a document’s token set with fewer token-like vectors, it represents the document through the dominant eigenspace of $$ M_D=\sum_i d_id_i^T. $$ Its multi-vector score measures query-token energy in that subspace, $$ s_k(Q,D)=\sum_{i=1}^{n}\|\Pi_kq_i\|_2^2, $$ while EigenLI-SV converts the same second-order structure into a single vector for conventional ANN retrieval. The empirical results support EigenLI as a spectral alternative to clustering-based late-interaction compression, particularly for $k\leq32$, while the results for GTE-ModernColBERT demonstrate that its effectiveness depends on the geometry of the underlying model representations. Source: https://www.emergentmind.com/topics/eigenli