G-Vendi Score: A Spectral Diversity Measure

• G-Vendi Score is a spectral diversity measure that quantifies the effective number of distinct items through eigenvalue dispersion of a normalized kernel matrix.
• It employs generalized entropy methods, using Rényi and Shannon orders, to adjust sensitivity toward rare versus dominant features in data.
• Practical variants integrate weighting, abundance, and quality adjustments, with applications in retrieval augmentation, image generation, and experimental design.

The G-Vendi Score denotes a family of similarity-based diversity measures built from the spectrum of a normalized positive-semidefinite kernel matrix. In the literature, the term is used in several closely related senses: as the order-$q$ generalization of the original Vendi Score, as weighted or quality-aware extensions, and as domain-specific instantiations such as Geo-diversity Vendi Score. Across these variants, the central interpretation is stable: the score is an effective number of distinct items under a user-chosen similarity notion, equal to $1$ for a completely collapsed set and increasing toward $n$ or $\mathrm{rank}(K)$ as the set becomes more spectrally spread out (Friedman et al., 2022, Pasarkar et al., 2023).

1. Nomenclature and canonical spectral construction

The original Vendi Score is defined for a finite collection $X=\{x_1,\dots,x_n\}$ and a positive-semidefinite similarity kernel $k$ with $k(x,x)=1$. Writing $K_{ij}=k(x_i,x_j)$ and normalizing by trace, or equivalently by $n$ when the kernel has unit diagonal, yields a density matrix $\rho$ whose eigenvalues $1$0 are nonnegative and sum to $1$1. The Shannon-order Vendi Score is

$1$2

This is the form introduced as the Vendi Score in machine learning and later re-expressed as one member of a broader generalized family (Friedman et al., 2022, Pasarkar et al., 2023).

The generalized, or G-Vendi, family replaces Shannon entropy by the Hill-number or quantum Rényi construction. For $1$3,

$1$4

with the continuous limit at $1$5 recovering the Shannon-order score. Several later papers use “G-Vendi” to emphasize either this order-$1$6 generalization, a kernel-weighted variant, or a feature-specific instantiation such as gradient-space or geographic-diversity Vendi. The shared object is always spectral: diversity is read off from eigenvalue mass dispersion rather than from pairwise dissimilarity averages alone (Pasarkar et al., 2023, Nguyen et al., 13 May 2025).

2. Weighted, abundance-aware, and quality-aware variants

A major extension introduces weights before spectral normalization. One formulation uses a diagonal weight matrix $1$7 and replaces $1$8 by $1$9, then normalizes to trace $n$0 and computes the same entropy-exponential. In Vendi-RAG, this is described as the weighted-kernel version underlying the Quality-Weighted Vendi Score; the broader point is that weighting changes the spectrum before entropy is computed, rather than post-processing the final score (Rezaei et al., 16 Feb 2025).

In genomic epidemiology, the G–Vendi Score is defined with abundance weights $n$1, $n$2, and $n$3. After normalization $n$4, the score becomes

$n$5

or

$n$6

where $n$7 are the eigenvalues of $n$8. Here $n$9 controls rare-versus-abundant spectral sensitivity, while $\mathrm{rank}(K)$0 controls the extent to which empirical abundance affects the kernel weighting (Nielsen et al., 26 Sep 2025).

A distinct quality-aware construction multiplies diversity by an average quality term. In “Quality-Weighted Vendi Scores And Their Application To Diverse Experimental Design,” the quality-weighted score is

$\mathrm{rank}(K)$1

with the order-$\mathrm{rank}(K)$2 version

$\mathrm{rank}(K)$3

This construction does not alter the spectrum directly; instead it composes a scalar quality summary with the spectral diversity term. As a result, the literature contains two different mechanisms for incorporating “quality”: spectral reweighting before eigendecomposition, and multiplicative coupling after spectral evaluation (Nguyen et al., 2024).

3. Interpretation, limit cases, and relation to classical diversity

The central interpretation is that G-Vendi measures an effective number of independent or dissimilar directions encoded by the kernel spectrum. If all items are identical, the normalized matrix has one eigenvalue equal to $\mathrm{rank}(K)$4 and the rest $\mathrm{rank}(K)$5, so every order yields value $\mathrm{rank}(K)$6. If items are mutually orthogonal, the spectrum is uniform and the score reaches $\mathrm{rank}(K)$7 for a sample of size $\mathrm{rank}(K)$8. Between these extremes, spectral flattening raises the score and spectral concentration lowers it (Friedman et al., 2022, Pasarkar et al., 2023).

The order parameter $\mathrm{rank}(K)$9 determines what kinds of structure are emphasized. For $X=\{x_1,\dots,x_n\}$0, smaller eigenvalues receive more weight, making the score more sensitive to rare, weak, or emerging modes. At $X=\{x_1,\dots,x_n\}$1, the weighting is Shannon-type. For $X=\{x_1,\dots,x_n\}$2, dominant eigen-directions matter more strongly, making the score more sensitive to duplication, dominance, or coarse clustering. The order limits are explicit: $X=\{x_1,\dots,x_n\}$3 is the rank of the kernel, $X=\{x_1,\dots,x_n\}$4 is the exponential of spectral entropy, and $X=\{x_1,\dots,x_n\}$5 (Pasarkar et al., 2023).

This spectral family reduces to classical Hill numbers when similarity degenerates to exact identity. In the genomic formulation, if $X=\{x_1,\dots,x_n\}$6 and $X=\{x_1,\dots,x_n\}$7, then $X=\{x_1,\dots,x_n\}$8 becomes the usual Hill number on abundance proportions, with Shannon effective number at $X=\{x_1,\dots,x_n\}$9 and Simpson reciprocal at $k$0. This reduction is important because it shows that G-Vendi is not a separate axiomatic system but an extension of effective-species counting to settings where pairwise similarity matters (Nielsen et al., 26 Sep 2025).

Several structural properties recur across papers. The score is permutation invariant, insensitive to duplicates in the sense that repeated identical elements do not artificially create new effective types, and interpretable on a bounded scale as an “effective number.” For the generalized family, monotonicity in $k$1 gives

$k$2

so increasing $k$3 never increases the effective count (Friedman et al., 2022, Pasarkar et al., 2023).

4. Algorithms, complexity, and optimization regimes

Exact computation is straightforward but cubic in the set size. One forms the $k$4 kernel matrix, normalizes it, eigendecomposes it, and evaluates either spectral entropy or the Rényi-order expression. This costs $k$5 time and $k$6 memory in the naive regime. When a linear kernel on $k$7-dimensional embeddings is available with $k$8, the nonzero spectrum can be obtained from a $k$9 covariance or density matrix at cost $k(x,x)=1$0 rather than $k(x,x)=1$1 (Friedman et al., 2022, Pasarkar et al., 10 Feb 2026).

Approximation strategies vary by use case. The generalized-family paper describes low-rank or subspace approximations, including randomized SVD, Nyström, landmark methods, and embedding-based Rayleigh–Ritz projections, while noting that if only $k(x,x)=1$2 is needed then only the largest eigenvalue is required (Pasarkar et al., 2023). In Vendi-RAG, the working set is intentionally small—typically $k(x,x)=1$3 to $k(x,x)=1$4—so the eigendecomposition is described as negligible in practice (Rezaei et al., 16 Feb 2025).

Recent work has shifted from evaluation to direct optimization. In dataset appraisal, the log-Vendi score is treated as a matrix-spectral set function, and secular-equation rank-one updates avoid repeated full eigendecompositions during greedy selection. The reported empirical speedup is about $k(x,x)=1$5 on ImageNet-1K-scale optimization, making direct Vendi maximization feasible in a regime where oracle eigencalls would be prohibitive (Bilmes et al., 28 May 2026). In pretraining data selection, the Spokes framework instead relaxes subset selection to weights $k(x,x)=1$6 and maximizes a joint quality-diversity objective with exponentiated gradient descent, with convergence typically reached in fewer than $k(x,x)=1$7 steps (Lee et al., 13 Jun 2026).

These optimization regimes expose an important distinction. Some papers optimize the score itself over subsets or simplex weights; others use the score as a control signal inside a larger algorithm, such as diffusion guidance, retrieval reranking, or OOD novelty scoring. The score is therefore both a standalone diversity functional and a modular spectral primitive.

5. Domain-specific instantiations and empirical uses

In retrieval-augmented generation, Vendi-RAG integrates the Vendi Score into an iterative retriever that balances diversity and relevance. The retriever uses

$k(x,x)=1$8

starting with a high diversity weight and then adjusting $k(x,x)=1$9 according to an LLM-judge score on candidate answers. The framework is designed for multi-hop QA, where redundancy among top-ranked passages is especially costly. On HotpotQA, 2WikiMultiHopQA, and MuSiQue, the reported accuracy gains over Adaptive-RAG reach $K_{ij}=k(x_i,x_j)$0, $K_{ij}=k(x_i,x_j)$1, and $K_{ij}=k(x_i,x_j)$2, respectively, with improvements becoming more pronounced as the number of retrieved documents increases and remaining consistent across GPT-3.5, GPT-4, and GPT-4o-mini (Rezaei et al., 16 Feb 2025).

In text-to-image generation, the Geo-diversity Vendi Score is used inside contextualized Vendi Score Guidance. The method maintains a fake memory bank of generated images and a small set of real exemplar images, then perturbs the diffusion score by adding a gradient that increases diversity relative to the fake bank and subtracts a gradient that limits drift from real exemplars. On GeoDE, the reported average F1 improves from $K_{ij}=k(x_i,x_j)$3 for the baseline LDM to $K_{ij}=k(x_i,x_j)$4 for c-VSG, while worst-region F1 rises from $K_{ij}=k(x_i,x_j)$5 to $K_{ij}=k(x_i,x_j)$6; on DollarStreet, c-VSG reaches average F1 $K_{ij}=k(x_i,x_j)$7 and worst-region F1 $K_{ij}=k(x_i,x_j)$8 while keeping precision and CLIPScore within $K_{ij}=k(x_i,x_j)$9–$n$0 of baseline (Hemmat et al., 2024).

In large-scale data curation, Spokes defines G-Vendi on cosine similarities of normalized per-example gradients under a proxy model. The score is interpreted as the effective number of independent gradient directions. On $n$1k-document subsets, diversity-only Spokes raises G-Vendi from approximately $n$2 to approximately $n$3 on DCLM, a $n$4 increase over random sampling. Joint optimization of quality and diversity yields the strongest downstream results, improving average benchmark accuracy by $n$5 points on DCLM and $n$6 points on FineWeb relative to random sampling (Lee et al., 13 Jun 2026).

In out-of-distribution detection, Vendi Novelty Score uses the change induced by a test example in the Vendi diversity of in-distribution features. The method combines a class-conditional local term based on $n$7 with a dataset-level global term based on $n$8. The global component is computable from the top eigenpair of a density matrix and requires only $n$9 time per example at test time. Empirically, including the global term can reduce FPR@95 by up to ten points on ImageNet-1K OOD benchmarks, and the detector retains its performance when the density matrix is estimated from only $\rho$0 of the training data (Pasarkar et al., 10 Feb 2026).

In experimental design, quality-weighted Vendi scores are used to construct diverse active search and diverse Bayesian optimization policies in drug discovery, materials discovery, and reinforcement learning. The reported outcome is a $\rho$1–$\rho$2 increase in the number of effective discoveries compared to baselines (Nguyen et al., 2024). In genomic epidemiology, abundance-aware G–Vendi scoring on SARS-CoV-2 sequence windows and phylogenetic clades is used to characterize circulating viral diversity and to detect emerging variants; on 2021-12-05, the Omicron BA.1 clade appeared as the lowest-diversity subtree before formal lineage assignment (Nielsen et al., 26 Sep 2025).

6. Conceptual boundaries, neighboring objectives, and open questions

A recurring misconception is that G-Vendi is a single fixed metric. The surveyed literature instead uses the label for a family of closely related spectral constructions whose behavior depends on three design choices: the kernel, the normalization or weighting scheme, and the entropy order $\rho$3. Kernel choice is especially consequential because it determines what counts as “difference”: semantic cosine similarity, gradient cosine similarity, Hamming-derived genomic similarity, and CLIP-feature similarity all produce different diversity geometries even when the spectral functional is identical (Friedman et al., 2022, Lee et al., 13 Jun 2026).

Another misconception is that maximizing G-Vendi is equivalent to maximizing downstream utility. Large-scale dataset appraisal gives a more qualified picture. The Vendi Score is shown there to be a special case of a broader class of matrix-spectral objectives, and common neural-scaling-law objectives and the Vendi Score are shown to be submodular in that framework. Yet the same study finds that, across fixed-size, class-balanced, and fixed-budget regimes, facility location correlates much better with held-out performance than Vendi, and that pushing Vendi to very high values can produce poorer subsets; Vendi is described as predictive only over moderate score ranges (Bilmes et al., 28 May 2026).

Theoretical status also depends on which variant is under discussion. For the standard log-Vendi objective in the matrix-spectral setting, submodularity follows from the matrix-monotonicity criterion for $\rho$4 with $\rho$5 (Bilmes et al., 28 May 2026). By contrast, the quality-weighted design paper states that a formal proof of submodularity for $\rho$6, and more generally for VS or its Rényi-$\rho$7 variants in that setting, remained open (Nguyen et al., 2024). The generalized Rényi forms are likewise identified as an open direction for submodularity theory in the dataset-worth analysis (Bilmes et al., 28 May 2026).

The framework has also been extended beyond diversity as such. Vendi Information Gain defines Vendi entropy as $\rho$8 and then measures information gain as the difference between marginal and conditional Vendi entropy. This construction is asymmetric, sample-based rather than density-based, and reduces to mutual information when all samples are completely dissimilar, so that the kernel matrix becomes diagonal and the normalized eigenvalues coincide with sample probabilities (Nguyen et al., 13 May 2025). A plausible implication is that G-Vendi now functions less as a single metric than as a spectral design pattern: once a PSD similarity structure is specified, the same eigenvalue calculus can be repurposed for diversity evaluation, novelty detection, acquisition, retrieval control, and information-theoretic scoring.