Multi-Vector Embeddings are Provably More Expressive than Single Vector Embeddings

Abstract: Multi-vector (MV) embeddings have become a powerful paradigm in neural information retrieval (IR), achieving high retrieval accuracy by representing data with multiple vectors and scoring them via the non-linear Chamfer similarity. Despite their widely perceived superiority over single-vector (SV) embeddings which use inner product similarity, to date there is no formal proof that SV similarities cannot approximate MV similarities with the same representation size. Specifically, we ask the following: for any bounded dataset size $n \leq 2^{poly(m)}$, what is the smallest dimension $D$ so that given any collection of MV embeddings $Q_1,\dots,Q_n,X_1,\dots,X_n \subset \mathbb{R}^d$ containing at most $m$ vectors each, there always exist $q_1,\dots,q_n$, $d_1,\dots,d_n \in \mathbb{R}^{D}$ satisfying $|\langle q_i, d_j \rangle - \texttt{Chamfer}(Q_i,X_j)| \leq ε$ for all $i,j$? Recently, the MUVERA algorithm demonstrated that $D = m^{O(1/ε^2)}$ is possible. If improved to $D = md$, this would imply that MV embeddings are no more expressive than SV embeddings. In this paper, we rule out this scenario. Specifically, we prove the existence of a collection of MV embeddings in $\mathbb{R}^d$, each containing at most $m$ vectors, which require single-vector dimension of $D =(ε^2 m)^{Ω(1/ε)}$ to approximate, establishing a strong separation in representation size between MV and SV embeddings. Our proof leverages the Pattern Matrix Method by constructing a hard instance whose Chamfer similarity matrix encodes the $NAND_k$ boolean function. Our results confirm a long-held belief in the IR community: at a fixed representation size, multi-vector embeddings can express similarities which cannot even be approximately represented by single vector embeddings.

• The paper proves that multi-vector embeddings are provably more expressive than single-vector embeddings by establishing an exponential lower bound on the required dimension.
• It leverages the pattern matrix method and NAND function hardness to construct lower bounds that nearly match the existing MUVERA upper bound.
• The results provide practical guidance for designing modern retrieval systems and motivate further research in non-linear, multi-representation embedding techniques.

Provable Expressiveness Separation Between Multi-Vector and Single-Vector Embeddings

Background and Motivation

Single-vector (SV) embeddings have been the prevailing paradigm in neural information retrieval (IR), representing datapoints as single vectors and employing inner product similarity. Despite their efficiency and scalability, SV embeddings face constraints in expressiveness, particularly when handling complex geometric relationships among queries and documents. The emergence of multi-vector (MV) embeddings, exemplified by models such as ColBERT, has demonstrated marked improvements in retrieval accuracy by modeling each datapoint as a set of vectors and leveraging non-linear, asymmetric scoring mechanisms like the Chamfer similarity.

Experimentally, MV embeddings consistently outperform SV counterparts, especially on multimodal and complex retrieval benchmarks. However, prior to this work, the community lacked a formal proof establishing that MV embeddings, at a fixed representation size, are strictly more expressive than SV embeddings. The central question becomes whether SV embeddings of equivalent size can approximate MV similarities to arbitrary precision, implying no expressiveness gap.

Theoretical Contributions

This paper rigorously answers the expressiveness question by proving that MV embeddings are fundamentally more expressive than SV embeddings for fixed representation sizes. The authors formulate and resolve the long-standing open problem: Given two sets of MV embeddings (each containing at most $m$ vectors in $\mathbb{R}^d$), what is the minimal SV embedding dimension $D$ required to approximate all pairwise Chamfer similarities to additive error $\epsilon$?

Main Lower Bound

Leveraging the Pattern Matrix Method and hardness construction via the NAND$_k$ Boolean function, the paper shows that there exist MV embedding collections where any SV embedding requires dimension

$(\epsilon^2 m)^{2^{2(1/\epsilon)}}$

to achieve approximation within error $\epsilon$. This separation is exponential in $m$ and $1/\epsilon$, dramatically larger than the naive $md$ bound (the total number of parameters for MV embeddings), refuting the possibility that SV embeddings can universally approximate MV embeddings at equal representation size.

Comparison with Existing Upper Bounds

The MUVERA algorithm previously demonstrated an upper bound of $\mathbb{R}^d$0 for SV embedding dimension needed to approximate MV similarities. The new lower bound nearly matches this upper bound, indicating that substantial improvements in SV embedding-based approximations are unattainable, even for data-dependent (non-oblivious) mappings. This strongly justifies the design and continued development of MV retrieval models.

Extension to MAX-IP and Snowflake Embeddings

The separation applies not only to the Chamfer similarity but also to the MAX-IP metric—a restriction where queries are single vectors and documents are sets. The paper further explores "Snowflake Embedding" techniques for MAX-ABS-IP (maximum absolute inner product), enabling order-preserving embeddings into SV space. In particular, mappings exist such that the $\mathbb{R}^d$1-power of the SV embedding inner products approximates MAX-ABS-IP similarities, with dimension $\mathbb{R}^d$2.

Methodology

The proof technique relies on constructing a hard instance where the Chamfer similarity matrix encodes the pattern matrix of the NAND$\mathbb{R}^d$3 Boolean function. The approximate rank of this matrix, lower bounded by the approximate polynomial degree of NAND$\mathbb{R}^d$4, yields the requisite SV embedding dimension. The rigorous use of polynomial degree arguments and symmetry properties of Boolean functions provides tight separation bounds.

Relation to Prior Work

While previous works have experimentally demonstrated MV superiority and highlighted SV limitations (e.g., restricted nearest-neighbor orderings, poor average pooling mappings), none have established explicit lower bounds on representation size or provided formal separation in expressiveness. This work decisively closes that gap by quantifying the dimensional barrier, including applicability to both oblivious and data-dependent embeddings.

Implications and Future Directions

Practical Implications

The result provides robust theoretical support for the use of MV embeddings in IR and related domains. For system designers, it becomes clear that MV models are not only empirically superior but are provably necessary for capturing complex similarities at scale. Attempts to encode MV similarity structures via SV embeddings, even with sophisticated mapping functions, face insurmountable dimensionality requirements.

Theoretical Implications

The separation suggests intrinsic limits of inner product-based SV representations, guiding future research toward MV architectures and non-linear geometric similarity measures. The snowflake embedding approach hints at potential order-preserving reductions for retrieval ranking tasks, though not for direct similarity matching.

Open Problems

The precise dependency on $\mathbb{R}^d$5 in the lower bound remains open; tightening the separation to completely match the upper bound from MUVERA may yield further insights. Extensions to broader classes of similarity metrics or alternative MV scoring functions (beyond Chamfer/Max-IP) are promising avenues.

Conclusion

This paper establishes a tight, formal separation in expressiveness between MV and SV embeddings in neural IR, demonstrating that SV embeddings require exponential dimension to approximate MV similarities. By grounding the separation in hardness constructions and polynomial degree lower bounds, the research provides both theoretical clarity and practical guidance for model design in information retrieval and related fields. The results mark a decisive advance in understanding the geometric and expressive power of learned representations, with direct consequences on the deployment and evolution of large-scale retrieval systems (2606.23475).