Multi-Vector Retrieval for Research in Natural Language Processing

• Multi-Vector Retrieval (MVR) enhances search accuracies, handling complex queries by representing queries and documents with multiple vectors, preserving specific features with balances network bandwidth.
• MVR encompasses various methods including alignment, sparse indexing, optimized candidate retrieval, dimension fixing and storage optimization, foundational in various forms tailored for abilities like recall percentage, speed, and costs.
• Significant applications of MVR include question answering, information retrieval tasks on text and visual documents, and multimodal retrieval where each modality needs a different granularity of representation and interaction model.

Multi-vector retrieval (MVR) represents a query and a document as sets or sequences of vectors rather than as one vector per item. If $Q=\{q_1,\ldots,q_n\}$ and $D=\{d_1,\ldots,d_m\}$, a common late-interaction score is the MaxSim function

$$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$

Each query vector can therefore match a different document vector. This preserves fine-grained evidence concerning entities, terms, passages, image regions, tables, or layout elements, but requires storing and searching many document vectors. MVR research consequently addresses a central accuracy–efficiency problem: how to retain token- or region-level matching while reducing interaction cost, index size, latency, and memory demand.

1. Conceptual foundations and scoring

Single-vector dense retrieval maps a query and document to vectors $\mathbf q,\mathbf d\in\mathbb R^d$ and scores them with an inner product,

$\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$

This representation is efficient because approximate-nearest-neighbor and maximum-inner-product-search systems operate directly on one vector per document. Its limitation is representational capacity: entities, facts, topics, and possible query–document relations must be compressed into one point.

MVR retains multiple contextualized representations. In ColBERT-style retrieval, the score is

$$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$

The maximum is computed independently for every query token, allowing different query components to align with different document tokens. This late-interaction structure avoids running a cross-encoder for every query–document pair while preserving more local matching information than a dual encoder.

A more general formulation introduces an alignment matrix $A$:

$$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$

The matrix specifies which token pairs contribute and with what weight. In ColBERT, each query token selects its highest-scoring document token, so $A$ is sparse and has at most one nonzero entry per query token. “Multi-Vector Retrieval as Sparse Alignment” presents this alignment perspective as a unifying framework for dual encoders, lexical systems, and late-interaction models (Qian et al., 2022).

MVR should not be equated with exhaustive all-to-all interaction in every implementation. Several systems retain the MaxSim objective while approximating candidate generation, pruning document vectors, restricting eligible interactions, or replacing the original representation with a fixed-dimensional proxy.

2. Sparse alignment, salience, and routing

MVR systems differ in how they determine which query–document vector pairs interact. Pairwise sparsity reduces the number of token-to-token links; unary salience determines whether a token participates in any link.

AligneR factorizes the alignment matrix as

$$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$

where $D=\{d_1,\ldots,d_m\}$0 is a pairwise alignment matrix, $D=\{d_1,\ldots,d_m\}$1 and $D=\{d_1,\ldots,d_m\}$2 contain query- and document-token unary saliences, $D=\{d_1,\ldots,d_m\}$3 is the outer product, and $D=\{d_1,\ldots,d_m\}$4 is elementwise multiplication. Pairwise alignment answers which query token should match which document token, whereas unary salience answers whether a token needs to participate at all. A zero unary salience masks an entire row or column.

Pairwise alignment can be controlled with top-$D=\{d_1,\ldots,d_m\}$5 or top-$D=\{d_1,\ldots,d_m\}$6 rules. Under top-$D=\{d_1,\ldots,d_m\}$7, every query token aligns with its $D=\{d_1,\ldots,d_m\}$8 highest-scoring document tokens; top-1 is the ColBERT rule. Under top-$D=\{d_1,\ldots,d_m\}$9, the number of alignments is proportional to document length. The optimal density is task-dependent: factoid question answering and duplicate-question retrieval generally favor sparse alignment, whereas argument retrieval can benefit from several alignments per query token. Few-shot alignment adaptation selects among candidate sparsity settings without changing encoder parameters. In the reported experiments, ArguAna increased from $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$0 nDCG@10 with top-1 to $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$1 with top-$$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$2, while NQ, FiQA, and Quora often deteriorated as $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$3 increased (Qian et al., 2022).

CITADEL applies a different form of sparsity through dynamic lexical routing. A contextualized token vector is mapped to scores over vocabulary-sized learned lexical keys:

$$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$4

Each query token is routed by default to one key and each document token to up to five keys. Only tokens sharing a key interact. The keys are associated with dimensions of the BERT masked-language-modeling head rather than necessarily with surface tokens, permitting contextual relations such as “car” and “automobile” to be connected while retaining an inverted-index structure. CITADEL combines contrastive retrieval loss, router contrastive loss, $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$5 sparsity regularization, and load-balancing regularization. Its reported headline result was $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$6 ms/query and nearly 40 times lower latency than ColBERT-v2 under the specified implementation and hardware configuration (Li et al., 2022).

Sparse coding provides another route to indexable MVR. SSR uses a Sparse Autoencoder to map a token embedding $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$7 to a high-dimensional sparse vector $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$8, with $$\operatorname{sim}(Q,D)=\sum_{i=1}^{n}\max_{1\leq j\leq m} q_i^\top d_j.$$9 and only $\mathbf q,\mathbf d\in\mathbb R^d$0 active neurons. In the controlled configuration, $\mathbf q,\mathbf d\in\mathbb R^d$1, $\mathbf q,\mathbf d\in\mathbb R^d$2, and $\mathbf q,\mathbf d\in\mathbb R^d$3. The active neuron IDs act as semantic inverted-index keys, eliminating the K-means stage used by many dense MVR systems. Sparse token similarity is computed only over overlapping active neurons:

$\mathbf q,\mathbf d\in\mathbb R^d$4

SSR therefore remains a multi-vector MaxSim system, but its token vectors are sparse semantic representations rather than dense vectors. SSR-CLS adds a global $\mathbf q,\mathbf d\in\mathbb R^d$5 similarity. The reported SSR-CLS result was $\mathbf q,\mathbf d\in\mathbb R^d$6 average nDCG@10, compared with $\mathbf q,\mathbf d\in\mathbb R^d$7 for PLAID and $\mathbf q,\mathbf d\in\mathbb R^d$8 for SPLADE-v3, with indexing reduced from $\mathbf q,\mathbf d\in\mathbb R^d$9 hours for ColBERTv2 to approximately $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$0 hours in the displayed experiment (Guo et al., 28 May 2026).

3. Candidate generation and approximate late interaction

The main systems challenge is that exact MaxSim cannot normally be evaluated against every document in a large corpus. A conventional ColBERT pipeline retrieves document-token candidates, gathers all token vectors for their source documents, and recomputes the full late-interaction score. The gathering and refinement stages can dominate latency.

XTR changes the training objective so that token retrieval itself is optimized. During training, document tokens compete across all documents in a minibatch, and only tokens surviving a top-$\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$1 selection contribute to the score. At inference, XTR reuses retrieved token similarities and imputes missing per-query-token similarities with the score of the last retrieved token:

$\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$2

Documents are scored from retrieved evidence rather than by loading all document vectors. This removes document gathering and makes the scoring stage two to three orders of magnitude cheaper than ColBERT in the reported analysis. With $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$3, $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$4, $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$5, $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$6, and $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$7, the illustrative calculation reported approximately $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$8 FLOPs for ColBERT and $\operatorname{sim}(Q,D)=\mathbf q^\top\mathbf d.$9 FLOPs for XTR-style scoring. XTR achieved $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$0 average nDCG@10 on the 13-dataset BEIR evaluation with the XXL model, compared with $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$1 for ColBERTv2 (Lee et al., 2023).

SLIM makes sparse token representations compatible with standard lexical infrastructure. Contextualized token embeddings are projected into a sparse vocabulary space using the masked-language-modeling head:

$$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$2

The exact sparsified late-interaction score remains

$$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$3

For first-stage retrieval, SLIM pools document token vectors by elementwise maximum and uses lower- and upper-bound constructions to form a sequence-level sparse representation indexed by Lucene. The top candidates are then refined using the original sparse token vectors. The two-stage design preserves token-level late interaction while using standard inverted files and CPU search. SLIM++ obtained $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$4 MS MARCO MRR@10 and used $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$5 GB in the reported table, compared with $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$6 and $$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$7 GB for ColBERT-v2 (Li et al., 2023).

WARP is an execution engine designed primarily for XTR-style retrieval. It combines dynamic similarity imputation through WARP$$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$8, implicit decompression of centroid-plus-residual codes, and two-stage sparse score reduction. Instead of reconstructing a document vector, it computes

$$\operatorname{sim}(Q,D)= \sum_{i=1}^{n}\max_{1\leq j\leq m}\mathbf q_i^\top\mathbf d_j.$$9

where the centroid-query contribution is reused and the residual contribution is obtained through lookup tables. WARP aggregates first by document and query token, then across query tokens without materializing a dense score matrix. On LoTTE Pooled, it reduced latency from more than six seconds for the XTR reference implementation to approximately $A$0 ms, and achieved a reported threefold speedup over the ColBERTv2/PLAID engine (Scheerer et al., 29 Jan 2025).

4. Fixed-dimensional reductions and graph-based indexing

A second family of methods reduces set-to-set retrieval to ordinary single-vector search.

MUVERA constructs Fixed Dimensional Encodings (FDEs) for query and document vector sets. For the Chamfer score

$A$1

the query vectors are summed within locality-sensitive buckets, while document vectors are averaged within the corresponding buckets. SimHash supplies the bucket partition, and empty document buckets can be filled with vectors having nearby hash codes. Independent repetitions and optional random projections produce a fixed-dimensional encoding $A$2 such that

$A$3

Under the stated normalization and parameter conditions, MUVERA provides an additive-$A$4 approximation guarantee for the normalized Chamfer score and an additive approximate-nearest-neighbor guarantee. In experiments, FDEs retrieved two to five times fewer candidates than the deduplicated single-vector heuristic at comparable recall. Across six BEIR datasets, MUVERA reported approximately 10% higher average recall and 90% lower average latency than the compared prior methods (Dhulipala et al., 2024).

LEMUR also reduces MVR to single-vector maximum-inner-product search, but learns its latent space directly from MaxSim targets. For each document $A$5, it defines the token-level contribution

$A$6

An MLP approximates the multi-document function $A$7, and the query representation is

$A$8

The learned score for document $A$9 becomes

$$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$0

The rows $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$1 of the output layer become document vectors in a task-specific latent space. A single-vector ANNS index retrieves $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$2 candidates, which are then reranked with exact MaxSim. At 80% recall, LEMUR reported five- to eleven-fold higher QPS than the best listed baselines across the evaluated ColBERTv2 datasets. Unlike MUVERA, LEMUR learns the proxy specifically for the corpus and MaxSim function, but it still requires the original document token vectors for exact reranking (Jääsaari et al., 29 Jan 2026).

GEM instead builds a graph directly over vector sets. It uses EMD for graph construction and Chamfer for final relevance, deliberately decoupling the graph metric from the retrieval metric. Its index uses set-level clustering, TF-IDF-guided cluster assignment, local proximity graphs, cross-cluster bridges, semantic shortcuts, and quantized distance estimation. Query-time search begins from multiple cluster-specific entry points and uses beam expansion, cluster pruning, and final Chamfer reranking. The reported operating points included MSMARCO $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$3 at approximately $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$4 ms and LoTTE $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$5 at approximately $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$6 ms. The supplied material does not provide the complete numerical comparison or hardware configuration, so GEM’s broader speedup claims remain dependent on the full implementation details (Tian et al., 20 Mar 2026).

5. Storage, compression, and representation reduction

MVR index size is commonly proportional to the number of stored vectors, their dimension, and their numeric precision. If a document has $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$7 vectors of dimension $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$8, raw storage is approximately $$\operatorname{rel}(D,Q) = \frac{1}{Z} \sum_{i=1}^{m}\sum_{j=1}^{n} d_i^\top q_j A_{ij}, \qquad Z=\sum_{i,j}A_{ij}.$$9, excluding metadata. ESPN addresses this cost by moving the complete re-ranking embedding table from DRAM to SSD while retaining the candidate-generation structures in memory. It uses asynchronous GPUDirect Storage, ANN-guided prefetching, early reranking, and optional partial reranking. The reported memory reduction was between fivefold and sixteenfold, with prefetch hit rates exceeding 90% at suitable prefetch steps. On MS MARCO v2, ESPN reduced latency relative to mmap from approximately $A$0 ms to $A$1 ms at a 32 GB memory limit (Shrestha et al., 2023).

ConstBERT reduces the number of document vectors rather than compressing each token vector. A document with token embeddings $A$2 is transformed into $A$3 learned vectors,

$A$4

The resulting score remains MaxSim,

$A$5

The vectors are not tied one-to-one to input tokens. With $A$6, the reported MSMARCO index decreased from 22 GB for ColBERT to 11 GB while retaining MRR@10 of $A$7 versus $A$8. The approach provides fixed-size records and therefore potentially simpler memory allocation and paging behavior, but requires retraining for each configured value of $A$9 and reduces token-level interpretability (MacAvaney et al., 2 Apr 2025).

MarginMerge performs post-hoc compression for visual document retrievers. It selects coverage-aware anchors using a weighted maximum-coverage objective over query prototypes, assigns document patches to geometrically similar anchors, and synthesizes one representative per cluster with a lightweight shared network. If $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$0 representatives are retained, standard MaxSim is applied to the compressed set. At 5% retention, MarginMerge preserved approximately 97% of full-index average nDCG@5 on both ColQwen2.5 and ColPali; at 10% retention it preserved approximately 99%, corresponding to 95% and 90% vector reductions, respectively. Its design targets diverse future query support rather than independent patch salience (Mahdizadeh et al., 4 Aug 2026).

ReinPool converts token- or patch-level representations to a single vector through learned selection followed by mean or max pooling. A policy network samples binary keep/discard actions, and reinforcement learning uses inverse retrieval with NDCG@3 rewards. On Vidore V2, ReinPool-Mean recovered approximately 76% of full multi-vector performance for Tomoro-ColQwen-4b and 81% for ColQwen2.5-3b, while producing nominal compression ratios of $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$1 and $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$2. The NeMo-ColEmbed-3b result recovered approximately 35%, demonstrating that one-vector compressibility depends strongly on the embedding model (Cha et al., 12 Jan 2026).

MM-Matryoshka introduces elasticity along two axes: encoder depth $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$3 and vector width $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$4. It trains intermediate layers through score-level distillation and trains coordinate prefixes with retrieval loss. A deployment can therefore select a pair $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$5 without training a separate model. For PaliGemma, the $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$6 configuration provided approximately fourfold storage reduction and reported approximately $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$7 online speedup, while the $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$8 configuration provided approximately $$A=\widetilde A\odot(\mathbf u^q\otimes\mathbf u^d),$$9 speedup. Unlike token pruning or merging, MM-Matryoshka does not reduce the number of visual vectors; it changes how wide they are and how deeply they are computed (Xiang et al., 3 Jun 2026).

6. Efficiency frontiers and multimodal extensions

MVR efficiency techniques can be organized by the computation or storage axis they modify:

Strategy	Primary mechanism	Representative systems
Alignment sparsification	Restrict token-to-token links or suppress tokens	AligneR, CITADEL
Sparse semantic indexing	Replace dense vectors with sparse learned coordinates	SLIM, SSR
Candidate-generation redesign	Train or approximate retrieval from selected evidence	XTR, WARP
Fixed-dimensional proxy	Reduce vector-set similarity to single-vector search	MUVERA, LEMUR
Compression and storage scheduling	Quantize vectors or move them outside DRAM	EMVB, ESPN
Vector-count or budget reduction	Pool, merge, or select a fixed number of vectors	ConstBERT, MarginMerge, ReinPool, MM-Matryoshka

EMVB improves CPU retrieval through thresholded centroid membership, stacked bit vectors, SIMD centroid interaction, product quantization, and per-document-term filtering. On MS MARCO, its $D=\{d_1,\ldots,d_m\}$00 configuration reduced latency from $D=\{d_1,\ldots,d_m\}$01 ms for PLAID to $D=\{d_1,\ldots,d_m\}$02 ms at $D=\{d_1,\ldots,d_m\}$03, reduced measured storage from 36 to 20 bytes per embedding, and preserved the reported effectiveness. On LoTTE, the $D=\{d_1,\ldots,d_m\}$04 configuration reached a reported $D=\{d_1,\ldots,d_m\}$05 speedup at $D=\{d_1,\ldots,d_m\}$06 with a small quality decrease (Nardini et al., 2024).

WARP and EMVB illustrate that arithmetic optimization and data layout can be as important as representation learning. LEMUR and MUVERA show that an MVR objective can sometimes be searched through ordinary single-vector infrastructure. ESPN shows that storage placement and asynchronous access can make multi-vector indexes feasible even when their re-ranking tables do not fit in DRAM. These approaches address different bottlenecks and can be combined in principle, although the supplied experiments do not establish every possible combination.

Multimodal MVR applies the same fine-grained principle to image patches, rendered document regions, and visual tokens. HiMIR decomposes a query into semantic sub-queries and represents each image at multiple segmentation granularities. Each query component selects its best image segment and hierarchy level, allowing one object to match coarsely and another finely. Cross-hierarchy candidate reduction, Kendall’s $D=\{d_1,\ldots,d_m\}$07 early exit, and offline hierarchy sparsity pruning reduce redundant computation. On the reported CREPE, MS COCO, NoCaps, and Flickr experiments, HiMIR achieved an average $D=\{d_1,\ldots,d_m\}$08 speedup over POQD with higher NDCG@10, although its query-decomposition cost was excluded from the throughput comparison (Li et al., 10 Oct 2025).

The common limitation across these methods is candidate-generation error. If a relevant document or region is discarded before exact or higher-fidelity scoring, later refinement cannot recover it. Sparse routing can miss a relevant token whose key is absent; FDEs can blur dissimilar vectors within one bucket; learned latent proxies can misestimate MaxSim; pruning and quantization can remove evidence; and early-exit schedules can terminate before a later representation improves the ranking. Reported speedups are therefore implementation-, hardware-, corpus-, and recall-dependent rather than universal properties of MVR.

The emerging design principle is a coarse-to-fine architecture in which cheap structures preserve broad candidate coverage and expensive token-level operations are restricted to a small subset. The relevant axes include pairwise alignment density, unary token salience, routing-key selectivity, vector width, vector count, encoder depth, quantization precision, candidate-list size, and storage location. “Incorporating Token Importance in Multi-Vector Retrieval” further shows that the aggregation rule itself can be adapted: weighted Chamfer assigns different coefficients to query-token contributions while leaving the frozen ColBERTv2 representations unchanged. IDF weighting improved average Recall@10 by $D=\{d_1,\ldots,d_m\}$09 zero-shot, and few-shot learned weights improved it by $D=\{d_1,\ldots,d_m\}$10 in the reported BM25 re-ranking experiment (S et al., 20 Nov 2025).

MVR is consequently not a single architecture but a design space spanning representation, alignment, indexing, approximation, and systems execution. Its defining property is the preservation of multiple query- and document-side vectors with a fine-grained interaction function. Current research seeks to retain that representational advantage while replacing exhaustive comparison with sparse alignment, learned routing, fixed-dimensional surrogates, graph navigation, hardware-aware execution, and retrieval-optimized compression.