Papers
Topics
Authors
Recent
Search
2000 character limit reached

Vector-Adapted Retrieval Scoring (VARS)

Updated 5 July 2026
  • VARS is a framework that adapts retrieval scoring to the structure of vector representations by aligning scores with semantic topical coverage, verification utility, or user preferences.
  • It employs diverse methodologies—set-based similarity, distance-aware verification, and fixed-dimensional encodings—to closely approximate task-specific retrieval objectives.
  • Demonstrated across document retrieval, image search, and conversational agents, VARS emphasizes calibration challenges and effective score adaptation under various constraints.

Searching arXiv for the cited papers and VARS-related context. Vector-Adapted Retrieval Scoring (VARS) denotes a family of retrieval-scoring constructions in which the score used for ranking, filtering, or reranking is adapted to the structure of vector representations rather than treated as a generic nearest-neighbor surrogate. In the literature considered here, the term appears explicitly as a personalization framework for conversational agents, and it also functions as a broader organizing description for methods that score query–document pairs as set-to-set vector similarities, distance-conditioned verification utilities, or fixed-dimensional proxies for late-interaction objectives (Hao et al., 21 Mar 2026, Roy et al., 2016, Szilvasy et al., 2024, Dhulipala et al., 2024). Across these settings, the common aim is to align retrieval-time scoring with the quantity that matters downstream: semantic topical coverage, expected positive yield under a verification budget, fidelity to multi-vector interaction, or user-specific preference relevance.

1. Conceptual scope

In the broad sense suggested by these works, VARS replaces a generic retrieval score with a score explicitly matched to vector semantics or deployment constraints. The adapted target may be the similarity between sets of embedded words rather than between single document vectors, the expected utility of verifying a candidate at a given distance, the late-interaction score of a multi-vector retriever, or a user-conditioned residual added to a frozen reranker (Roy et al., 2016, Szilvasy et al., 2024, Dhulipala et al., 2024, Hao et al., 21 Mar 2026).

This usage is broader than a single algorithm. In the document-retrieval setting, queries and documents are represented as sets of word vectors, and scoring is defined over query vectors and document centroids rather than over a single document embedding (Roy et al., 2016). In small-radius vector search, scoring is tied to the probability that a candidate passes post-verification at a given distance, so ranking is budget-aware and distance-aware rather than based on recall@k alone (Szilvasy et al., 2024). In late-interaction retrieval, fixed-dimensional encodings are constructed so that a single inner product approximates the multi-vector Chamfer or MaxSim objective (Dhulipala et al., 2024). In conversational agents, the score is adapted to user state by adding a dot-product bonus between a learned per-user vector and retrieved preference memories (Hao et al., 21 Mar 2026).

A plausible implication is that VARS is best understood as a design principle rather than a single canonical formula. What remains invariant is the decision to make the retrieval score reflect vector structure that would otherwise be discarded by standard top-kk or single-vector scoring.

2. Core scoring forms

The papers instantiate VARS through several mathematically distinct scoring rules.

Setting Score Adaptation target
Set-based IR sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k Query-to-document centroid similarity
Small-radius search S(d)=f(d2)S(d) = f(d^2) and VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|) Expected verification utility
Multi-vector retrieval F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle Proxy for late interaction
Personalized reranking s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle User-aware preference relevance

In the set-based IR formulation, the score is an average-link similarity between query word embeddings and document cluster centroids. In the small-radius formulation, the score is the sum of distance-conditioned utilities over a shortlist, with the default choice S(d)=f(d2)S(d)=f(d^2) and total VARS equal to RSM when S=fS=f. In MUVERA, the score is adapted by constructing fixed-dimensional encodings whose inner product approximates the late-interaction objective. In the conversational-agent formulation, the score is adapted by a low-rank residual determined by long-term and short-term user vectors (Roy et al., 2016, Szilvasy et al., 2024, Dhulipala et al., 2024, Hao et al., 21 Mar 2026).

These forms differ in surface syntax but share two structural properties. First, the score is not merely a raw embedding similarity; it is calibrated to a task-specific object. Second, the adapted score is intended to be usable at retrieval or reranking time without requiring a full end-to-end evaluation for every candidate set.

3. Set-based document scoring in classical information retrieval

A canonical early instance of this broader VARS pattern is the method in “Representing Documents and Queries as Sets of Word Embedded Vectors for Information Retrieval” (Roy et al., 2016). The paper argues that “a vector addition for composition does not scale well for a larger unit of text, such as passages or full documents, because of the broad context present within a whole document,” and it therefore represents both document and query as sets of word vectors rather than compressing a document into one embedding.

Formally, if the bag-of-words representation of a document is Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}, then the bag-of-vectors representation is Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|} with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k0. The query is likewise represented as observed query points sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k1. Each document is viewed as a mixture distribution over topics, operationalized by K-means clustering. After pre-clustering the collection vocabulary, words in a document are grouped by global cluster id and averaged into document-specific centroids

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k2

The resulting document representation typically contains far fewer than sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k3 clusters, with the paper reporting typically sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k4–sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k5 clusters per document.

The core score is the average-link similarity

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k6

This vector score is combined with a language-model score using

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k7

with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k8 selected by grid search and sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k9 instantiated as a Jelinek–Mercer LLM with S(d)=f(d2)S(d) = f(d^2)0. The embeddings are 200-dimensional CBOW word2vec vectors with negative sampling, trained on a pre-processed TREC collection.

The reported experimental setting uses TREC News collections and evaluates MAP, GMAP, Recall, and S(d)=f(d2)S(d) = f(d^2)1. The paper’s abstract states that the method “improves MAP by up to S(d)=f(d2)S(d) = f(d^2)2, in comparison to standard text-based LLM similarity, on the TREC 6, 7, 8 and Robust ad-hoc test collections.” On TREC-8 initial retrieval, LM MAP is reported as S(d)=f(d2)S(d) = f(d^2)3, while LM+wvsim_kmeans reaches S(d)=f(d2)S(d) = f(d^2)4 and LM+wvsim_one_cluster reaches S(d)=f(d2)S(d) = f(d^2)5; on Robust, LM MAP is S(d)=f(d2)S(d) = f(d^2)6 and LM+wvsim_kmeans reaches S(d)=f(d2)S(d) = f(d^2)7 (Roy et al., 2016).

The method also illustrates a recurrent VARS trade-off. Its vector component preserves topical granularity better than a single-vector document embedding, but the paper notes a scale mismatch between LM probabilities and vector similarities, handled only implicitly through S(d)=f(d2)S(d) = f(d^2)8. This suggests that calibration is a central issue whenever heterogeneous retrieval signals are combined within a single score.

A second, more explicitly utility-theoretic VARS formulation appears in “Vector search with small radiuses” (Szilvasy et al., 2024). The paper begins from the observation that the dominant vector-search metric, recall of a fixed-size top-S(d)=f(d2)S(d) = f(d^2)9 list, is “distantly related to the end-to-end accuracy of a full system that integrates vector search.” Its motivating use case is a two-stage pipeline in which vector search produces candidates and a costly post-verification stage determines whether a query image matches a database image.

The paper therefore frames retrieval as range search. For a query VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)0, a database VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)1, and a radius VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)2, the range set is

VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)3

The key modeling move is to define utility by the probability that a pair passes post-verification given its distance:

VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)4

with VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)5 monotonically decreasing. For a shortlist VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)6, the expected number of positives is

VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)7

and over multiple queries the paper defines

VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)8

with VARS(q)=xshortlist(q)S(qx)VARS(q) = \sum_{x \in shortlist(q)} S(\|q-x\|)9. The score is therefore distance-aware, and the optimal budgeted strategy is to select the globally F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle0 smallest distances across all queries, equivalently setting a global radius that yields F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle1 candidates in total.

The function F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle2 is estimated nonparametrically by isotonic regression on labeled pairs F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle3 with F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle4 and F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle5, solving

F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle6

after sorting distances increasingly. The paper states that the pool-adjacent-violators algorithm solves this in linear time. Operationally, the VARS-style score is then taken as F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle7, and per-query VARS is

F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle8

with total VARS equal to RSM when F(Q),F(P)qQmaxpPq,p\langle F(Q), F(P)\rangle \approx \sum_{q\in Q}\max_{p\in P}\langle q,p\rangle9 (Szilvasy et al., 2024).

The empirical setting uses 85M filtered images derived from YFCC100M, with s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle0k queries, s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle1M database images, and 75M training images. Embeddings are 512-dimensional SSCD descriptors with squared s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle2 distance. Verification uses KAZE keypoints and a RANSAC similarity transform via OpenCV. Budgets are s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle3 in the strict setting and s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle4 in the relaxed setting, with throughput around 85 geometric verifications per second.

The paper reports that recall@k and RSM can favor different index designs. For IVF with s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle5 centroids, RSM in the strict setting saturates after visiting about s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle6 clusters, while in the relaxed setting there is no RSM improvement beyond s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle7, which is less than s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle8 of clusters. Fast-scan PQ centroid search achieves the best RSM, HNSW saturates in high-RSM regimes, and compact codes are comparatively effective because “RSM is dominated by very close neighbors that small codes already capture.” For example, with strict budget s(ut,mi;U)=s0(ut,mi)+zU,teff,vmis(u_t,m_i;U)=s_0(u_t,m_i)+\langle z_{U,t}^{\mathrm{eff}},v_{m_i}\rangle9, residual PQ16S(d)=f(d2)S(d)=f(d^2)08 reaches S(d)=f(d2)S(d)=f(d^2)1 (in units of S(d)=f(d2)S(d)=f(d^2)2), close to exact at S(d)=f(d2)S(d)=f(d^2)3; in the relaxed setting, PQ32S(d)=f(d2)S(d)=f(d^2)48 reaches S(d)=f(d2)S(d)=f(d^2)5 versus exact S(d)=f(d2)S(d)=f(d^2)6 (Szilvasy et al., 2024).

This formulation directly challenges a common misconception that maximizing top-S(d)=f(d2)S(d)=f(d^2)7 recall is equivalent to maximizing end-to-end retrieval quality. Here, the relevant objective is expected positives per verification budget, and the adapted score is built to estimate exactly that quantity.

5. Fixed-dimensional encodings for late-interaction retrieval

A third VARS instantiation arises in multi-vector retrieval, particularly in “MUVERA: MUlti-VEctor Retrieval Algorithm” (Dhulipala et al., 2024). The paper does not define VARS explicitly; however, it states that, interpreted as “choosing a retrieval-time scoring function adapted to the vector representation,” MUVERA is exactly a VARS method. Its target is the ColBERT-style late-interaction objective

S(d)=f(d2)S(d)=f(d^2)8

with normalized form

S(d)=f(d2)S(d)=f(d^2)9

MUVERA constructs Fixed Dimensional Encodings (FDEs) S=fS=f0 and S=fS=f1 such that

S=fS=f2

The construction has three steps. First, token vectors are partitioned by angular LSH using SimHash with S=fS=f3 Gaussian hyperplanes, yielding S=fS=f4 clusters. Second, within each cluster, query blocks are sums of query vectors, while document blocks are centroids of document vectors; if a document cluster is empty, the method fills it with the document token whose SimHash code has minimum Hamming distance to that cluster id. Third, within-block random projections reduce dimension from S=fS=f5 to S=fS=f6, and the whole process is repeated S=fS=f7 times and concatenated, giving FDE dimension

S=fS=f8

Theoretical guarantees are stated for unit-normalized token embeddings. With no inner projection, the paper reports a one-sided estimator:

S=fS=f9

deterministically. Its main approximation theorem states that with Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}0, Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}1, and Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}2, the FDE inner product yields an Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}3-additive approximation to normalized Chamfer with probability at least Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}4 and also in expectation. A second theorem states that with Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}5, top-1 retrieval under FDE dot product returns an Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}6-approximate nearest neighbor under normalized Chamfer with high probability (Dhulipala et al., 2024).

Systemically, the consequence is that multi-vector retrieval can be reduced to a single-vector MIPS stage followed by exact reranking under the true Chamfer score. MUVERA indexes document FDEs in a standard MIPS or ANN system; the paper uses DiskANN, applies “ball carving” to reduce the number of query token groups for reranking, and compresses FDEs with PQ-256-8. The reported results state that FDEs achieve the same recall as prior state-of-the-art heuristics while retrieving Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}7–Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}8 fewer candidates, and that MUVERA achieves an average of about Wd={wi}i=1dW_d=\{w_i\}_{i=1}^{|d|}9 improved recall with about Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}0 lower latency than PLAID across six BEIR datasets. On MS MARCO, a 10240-dimensional FDE with PQ-256-8 matches PLAID within Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}1 Recall@Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}2, while offline experiments report that FDE-10240 needs Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}3 candidates to reach Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}4 recall, compared with Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}5 for the deduplicated SV heuristic and Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}6 for the non-deduplicated SV heuristic (Dhulipala et al., 2024).

The conceptual significance for VARS is precise: the retrieval-time inner product is engineered to be a mathematically coupled proxy for the score that the late-interaction model actually cares about, rather than a heuristic prefilter only loosely related to that objective.

6. User-aware VARS in conversational retrieval-augmented agents

The term VARS is used explicitly in “User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction” (Hao et al., 21 Mar 2026). There, VARS is a “pipeline-agnostic, frozen-backbone framework” that learns a compact per-user representation as long-term and short-term vectors in a shared preference space and uses these vectors to bias retrieval scoring over structured preference memory.

The system stores preference “cards” containing a structured preference tuple, a short note, an is_global flag, source metadata, and embeddings from a frozen model, Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}7, specifically Qwen3-Embedding-8B with raw dimension Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}8. A PCA projection produces a shared item space of dimension Vd={xi}i=1dV_d=\{x_i\}_{i=1}^{|d|}9, and each memory card obtains an item vector

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k00

For each user sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k01, the long-term vector sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k02 persists across sessions, while the short-term vector sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k03 is reset each session and decayed between turns. The effective user vector is

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k04

with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k05 and sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k06.

Conditional preference cards are first retrieved by dense cosine retrieval. A frozen cross-encoder then assigns a base score

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k07

and VARS adds a user-conditioned bonus:

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k08

A softmax policy over candidates is defined with temperature sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k09, the top-sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k10 cards are injected into the prompt with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k11, and global cards bypass retrieval and are directly injected up to a cap of sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k12.

Online learning proceeds from weak scalar rewards sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k13 obtained from the next user turn. The reward combines keyword sentiment and topic coherence, is clipped to sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k14, and is further scaled by a heuristic retrieval-attribution gate sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k15. A per-user EMA baseline is updated as

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k16

with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k17. Letting sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k18, the long-term and short-term vectors are updated by REINFORCE-style rules

sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k19

with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k20, sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k21, and short-term decay sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k22 (Hao et al., 21 Mar 2026).

Evaluation is conducted on MultiSessionCollab with 60 user profiles, 60 sessions per profile, up to 10 turns per session, and three task domains: math-hard, math-500, and bigcodebench. Across 3600 sessions per method, VARS reports Success sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k23, Timeout sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k24, and User tokens sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k25, compared with Reflection at sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k26, sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k27, and sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k28. Relative to Reflection, success improves by sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k29 percentage points with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k30 and is not significant, while timeout decreases by sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k31 percentage points with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k32 and user tokens decrease by sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k33 with sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k34. The paper therefore concludes that, under frozen backbones, the principal benefit of user-aware retrieval is improved interaction efficiency rather than large gains in raw task accuracy (Hao et al., 21 Mar 2026).

This explicit VARS formulation makes personalization a scoring problem rather than a backbone fine-tuning problem. The only learned state is two 256-dimensional per-user vectors plus a baseline scalar, and the added inference cost is a single 256-dimensional dot product per candidate.

7. Assumptions, limitations, and recurrent issues

Several assumptions recur across these VARS formulations. In the small-radius setting, RSM assumes stationarity of the learned sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k35, monotonic decrease of utility with distance, and comparability of distances across queries; it is also sensitive to sharp radius effects in high dimensions and to encoding distortion from PQ or ITQ (Szilvasy et al., 2024). In MUVERA, the theoretical guarantees assume unit-normalized token embeddings and depend on SimHash collisions, fill-empty behavior, and bounded random-projection error (Dhulipala et al., 2024). In the conversational-agent framework, reward quality and reward attribution are central, since removing gating causes “454% vector-norm inflation and directional drift in sensitivity tests” (Hao et al., 21 Mar 2026). In the set-based IR method, the mixed score combines a probability-like LM component with an inner-product similarity, and the paper identifies calibration as an unresolved issue (Roy et al., 2016).

These works also clarify several misconceptions. VARS is not identical to top-sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k36 nearest-neighbor retrieval: the small-radius formulation shows directly that recall@sim(q,d)=1Kqikqiμksim(q,d) = \frac{1}{K|q|}\sum_i\sum_k q_i \cdot \mu_k37 may improve while RSM does not (Szilvasy et al., 2024). VARS is not synonymous with single-vector document embeddings: the set-based IR method was motivated precisely by the claim that single-vector composition does not scale well to larger text units (Roy et al., 2016). VARS is not restricted to learned end-to-end retrievers: MUVERA uses data-oblivious random partitions and off-the-shelf MIPS infrastructure, while the conversational framework keeps chat, embedding, and reranker backbones frozen (Dhulipala et al., 2024, Hao et al., 21 Mar 2026).

Open directions listed in the papers are correspondingly diverse. The small-radius work identifies joint training with losses that normalize distances, adaptive per-query budgets, and extensions incorporating additional signals or multi-stage utility (Szilvasy et al., 2024). The conversational work proposes richer feedback signals, improved gating, multi-objective preference modeling, and partial backbone adaptation or adapter layers (Hao et al., 21 Mar 2026). The set-based IR paper suggests alternative distance measures, multi-sense embeddings, and further work on feedback and expansion (Roy et al., 2016). MUVERA’s results suggest further exploration of dimensionality–approximation trade-offs and compression–latency trade-offs within a single-vector proxy regime (Dhulipala et al., 2024).

Taken together, these directions indicate that the central unresolved problem in VARS is not whether vector adaptation is useful, but how to calibrate adapted scores so that they remain faithful under approximation, budget constraints, personalization noise, and domain shift.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Vector-Adapted Retrieval Scoring (VARS).