Covariance Similarity: Methods and Applications

• Covariance similarity is a framework that defines similarity based on second-order statistical structure, using covariance matrices to capture variability, anisotropy, and uncertainty.
• It applies methods like Bhattacharyya distance and Mahalanobis cosine similarity to compare multivariate distributions, graphs, and neural representations across diverse domains.
• Key challenges include sensitivity to covariance estimation, model assumptions, and terminological ambiguities that necessitate robust regularization and adaptive parameter tuning.

Covariance similarity denotes a class of methods in which similarity is mediated by second-order structure: covariance matrices, covariance components, or covariance-weighted inner products. In multivariate distribution comparison, covariance enters through overlap measures such as the Bhattacharyya distance; in graph analysis, covariance matrices of structural descriptors become graph signatures; in representation learning, covariance reweights angular or Mahalanobis notions of alignment; and in neural data analysis, representational similarity is cast as covariance component estimation (Kashyap, 2016, Shrivastava et al., 2014, Figueroa et al., 2017, Ying et al., 17 Jun 2026). Taken together, these formulations suggest not a single canonical metric, but a family of constructions in which similarity is defined by how variability, anisotropy, co-occurrence, and uncertainty are organized across dimensions.

1. Core mathematical forms

A foundational statistical form appears in the Bhattacharyya distance between multivariate Gaussians. For $X\sim N(\mu_1,\Sigma_1)$ and $Y\sim N(\mu_2,\Sigma_2)$, with $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$, the distance is

$$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$

Its first term is a generalized Mahalanobis distance weighted by the inverse average covariance; its second term is a log-volume comparison that is $0$ when $\Sigma_1=\Sigma_2$ and grows with differences in eigenstructure. In this sense, covariance encodes similarity both through uncertainty-weighted mean separation and through direct comparison of shape, scale, and anisotropy (Kashyap, 2016).

A second canonical form replaces the Euclidean inner product itself. In variance-adjusted cosine distance, whitening by a covariance factor yields

$$\cos_{\Sigma}(x,y) = \frac{x^\top \Sigma^{-1} y} {\sqrt{x^\top \Sigma^{-1} x}\,\sqrt{y^\top \Sigma^{-1} y}},$$

which is cosine under the Mahalanobis geometry induced by $\Sigma^{-1}$. In Mahalanobis cosine similarity for linear probes, the inner product is instead reweighted by test covariance,

$$\mathrm{MCS}(u,v) = \frac{u^\top \Sigma_{\text{test}} v} {\sqrt{u^\top \Sigma_{\text{test}} u}\,\sqrt{v^\top \Sigma_{\text{test}} v}},$$

and is equivalent to ordinary cosine after scaling directions by $\Sigma_{\text{test}}^{1/2}$ (Sahoo et al., 4 Feb 2025, Ying et al., 17 Jun 2026). The two constructions differ in whether covariance is used in primal or dual space, but both make similarity explicitly task- or distribution-aware.

A third form concerns the null behavior of cosine itself. For centered data, the asymptotic variance of cosine similarity satisfies

$Y\sim N(\mu_2,\Sigma_2)$0

and this variance is minimized, for fixed trace, when the eigenvalues of $Y\sim N(\mu_2,\Sigma_2)$1 are equal. This establishes an isotropy principle: covariance structure controls the baseline dispersion of cosine similarity, and whitening or sphering can therefore be understood as a covariance calibration of similarity scores (Smith et al., 2023).

2. Distributional comparison and probabilistic overlap

One systematic program for covariance similarity compares entire datasets by combining the Bhattacharyya distance with the Johnson–Lindenstrauss lemma. The procedure applies randomized linear projections to raw data matrices when dimensions differ, estimates means and covariance matrices in the common projected space, and then computes Gaussian or truncated-normal Bhattacharyya distances. The same work develops a generic extension of Stein’s lemma and derives identities in which covariance and the Bhattacharyya coefficient $Y\sim N(\mu_2,\Sigma_2)$2 enter the same expression, making explicit how distributional overlap and co-movement are linked (Kashyap, 2016).

A distinct probabilistic use arises for Gaussian process predictive distributions. There, similarity between two predictive laws over a common comparison set $Y\sim N(\mu_2,\Sigma_2)$3 is defined as

$Y\sim N(\mu_2,\Sigma_2)$4

where $Y\sim N(\mu_2,\Sigma_2)$5 is a matrix distance between predictive covariance matrices and $Y\sim N(\mu_2,\Sigma_2)$6 is an affine alignment of predictive means. In this formulation, covariance similarity is one component of a broader comparison that jointly assesses mean shape, uncertainty, and correlation structure, with the covariance term becoming especially relevant when uncertainty profiles matter for downstream decisions (Asencio-Martín et al., 2021).

A third probabilistic strand replaces distances by similarities before taking covariance. With pairwise similarities

$Y\sim N(\mu_2,\Sigma_2)$7

triple-centering yields matrices $Y\sim N(\mu_2,\Sigma_2)$8 and $Y\sim N(\mu_2,\Sigma_2)$9, and similarity covariance is defined by

$\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$0

Similarity correlation then maximizes the normalized covariance over scale parameters $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$1. This construction is asymptotically equivalent to distance correlation for large scales, but it emphasizes small distances and hence local structure. The resulting notion of covariance similarity is not covariance of random variables in the usual sense; it is covariance of centered similarity matrices (Pascual-Marqui et al., 2013).

3. Structured objects, kernels, and covariance manifolds

For graphs, covariance similarity can be built from power iterations of the adjacency matrix. Given a simple undirected graph with adjacency $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$2 and all-ones vector $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$3, define normalized vectors $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$4, and let $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$5 be the covariance matrix of these vectors across vertices. Its entries satisfy

$\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$6

This $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$7 is symmetric positive semidefinite, is a graph invariant under relabeling, encodes spectral information and weighted walk statistics, and serves as a fixed-dimensional representation of the graph. Similarity between two graphs is then defined by the Bhattacharyya similarity between their covariance matrix representations; for fixed small $\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$8, both the representation and the similarity can be computed in operations linear in the number of edges (Shrivastava et al., 2014).

A related graph-theoretic formulation defines covariance and correlation kernels over nodes by sampling paths from a generalized bag-of-paths distribution. Nodes are considered similar when they often appear together on weighted paths, and closed-form expressions for co-presence and co-occurrence yield covariance kernels

$\Sigma_{\text{avg}}=(\Sigma_1+\Sigma_2)/2$9

and correlation kernels

$$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$0

These kernels are positive semidefinite because they are covariance or correlation matrices of path-space features, and they provide node similarities that incorporate both path structure and path costs (Guex et al., 2019).

For covariance matrices themselves, transform-invariant similarity has been developed on the SPD cone. The Spectral Polytope Covariance Matrix similarity constructs a spectral polytope from eigenvectors scaled by $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$1, compares homothety ratios across dimensions, and defines a dissimilarity by the variance of those ratios. The dissimilarity is symmetric, non-negative, and vanishes exactly when two covariance matrices are equivalent up to orthogonal rotation and uniform scaling; its bounded form maps the result to $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$2. This makes similarity explicitly invariant to rotations and isotropic scale changes, unlike standard SPD distances such as AIRM, LERM, KLDM, or JBLD (Figueroa et al., 2017).

4. Covariance-aware machine learning metrics

In few-shot prompt learning for vision-LLMs, covariance similarity appears as a replacement for cosine similarity under a generative Gaussian view. Each class $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$3 is modeled with anisotropic covariance $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$4, text features $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$5 are treated as class means, and the cross-modal distance becomes

$$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$6

Because few-shot covariances are often rank-deficient, shrinkage is applied to obtain a full-rank estimator, followed by classwise normalization. The resulting covariance-aware softmax uses the inverse Mahalanobis distance rather than cosine, and multiple diverse prompts induce a multi-centered covariance model with more flexible decision boundaries (Dong et al., 3 Mar 2025).

Few-shot segmentation uses a related idea through learned Gaussian-process kernels. Instead of forming a 4D cost volume with cosine similarity, covariance similarity is defined by kernel evaluations

$$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$7

where $$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$8 is typically an ARD squared-exponential kernel learned by maximizing GP marginal likelihood on hard query examples. The learned covariance kernel functions are then aggregated by a doubly deformable 4D Transformer, so similarity is both covariance-aware and spatially adaptive (Xiong et al., 2022).

For comparing linear probes, covariance similarity takes the form of Mahalanobis cosine similarity. Under balanced Gaussian projections, OOD AUROC is

$$D_B(X,Y) = \frac{1}{8}(\mu_2-\mu_1)^\top \Sigma_{\text{avg}}^{-1}(\mu_2-\mu_1) + \frac{1}{2}\ln\frac{|\Sigma_{\text{avg}}|}{\sqrt{|\Sigma_1||\Sigma_2|}}.$$9

while MCS to a reference probe is a covariance-weighted cosine. The analysis shows that both AUROC and MCS are sigmoid-shaped functions of the same test-data SNR, explaining why MCS to an OOD reference probe can linearly predict OOD AUROC across models, layers, and domains; the same theory also specifies failure modes, including wrong covariance choice, class imbalance, heteroscedasticity, and a mis-specified reference probe (Ying et al., 17 Jun 2026).

5. Representational similarity and covariance component models

In representational similarity analysis, covariance similarity appears through the error structure of dissimilarity estimates. The whitened unbiased RDM cosine similarity compares an unbiased vector of crossvalidated dissimilarities $0$0 to a model RDM $0$1 using the covariance $0$2 of RDM estimation errors:

$0$3

This criterion uses analytical expressions for the covariance of biased and unbiased squared Euclidean or Mahalanobis distances, whitens the dependence among pairwise dissimilarities, and yields near-optimal model selection while preserving the meaningful zero-point of crossvalidated distances (Diedrichsen et al., 2020).

A complementary formulation treats similarity directly as covariance component structure. In variational RSA for M/EEG, the condition-by-condition covariance matrix is modeled as

$0$4

with $0$5 representing hypothesized similarity effects and $0$6 their weights. Variational Bayes estimates the posterior on the log-weights, supports time-resolved hypothesis testing by introducing time-specific covariance components $0$7, and enables model comparison by free energy and Bayesian model reduction. Here covariance similarity is not merely a metric; it is the latent object whose decomposition encodes representational hypotheses (Lepauvre et al., 3 Nov 2025).

These two RSA formulations occupy different points on the same spectrum. WUC corrects similarity comparison by modeling the covariance of estimated dissimilarities, whereas vRSA replaces dissimilarity-first workflows with a direct covariance decomposition. This suggests a broad second-order view of representational analysis in which similarity is either whitened by covariance or defined as covariance structure itself.

6. Estimation under nonstationarity and high dimension

In financial time series, covariance similarity is used to weight historical observations according to how close past market dependence structures are to the present. A probe correlation matrix $0$8 is estimated on a short window, and similarity between times $0$9 is defined by the matrix $\Sigma_1=\Sigma_2$0-norm

$\Sigma_1=\Sigma_2$1

After normalization and an optional top-$\Sigma_1=\Sigma_2$2 restriction, these similarities become weights in a similarity-weighted estimator

$\Sigma_1=\Sigma_2$3

The resulting covariance and correlation estimators are reported to be less biased and to show lower variance than either unweighted or exponentially weighted estimators, especially when correlation structures vary over time (Münnix et al., 2010).

A more explicit high-dimensional regression version models covariance directly as a sparse linear combination of similarity matrices:

$\Sigma_1=\Sigma_2$4

The loss

$\Sigma_1=\Sigma_2$5

is combined with Lasso, SCAD, or MCP penalties to select informative predictors and estimate coefficients simultaneously. The theory establishes non-asymptotic error bounds for the Lasso estimator and a strong oracle property for the folded concave penalized estimator, despite the fact that the matrix entries of $\Sigma_1=\Sigma_2$6 are not i.i.d. In practice, positive semidefiniteness is enforced by eigenvalue clipping after estimation (Gao et al., 2024).

These estimation problems underscore a recurring practical feature of covariance similarity: the quality of the similarity notion depends as much on covariance estimation as on the nominal metric. Shrinkage, normalization, support restriction, and spectral regularization are therefore not secondary implementation details but part of the definition of a usable covariance-aware similarity.

7. Limitations, failure modes, and terminological ambiguity

Most covariance-similarity constructions are model-contingent. Closed-form Bhattacharyya decompositions and many covariance–distance identities rely on Gaussian or truncated-Gaussian assumptions; when data are far from normal, the same work recommends truncated normals, mixtures, discrete moment-matched approximations, or nonparametric alternatives such as energy distance and distance correlation (Kashyap, 2016). Transform-invariant SPD similarity is explicitly invariant to rotations and uniform scalings, but not to anisotropic scalings, and its dissimilarity is a semimetric rather than a metric because it does not satisfy the triangle inequality (Figueroa et al., 2017). Similarity correlation based on kernels depends on scale parameters $\Sigma_1=\Sigma_2$7; the method maximizes over them, but the paper presents this as a preliminary empirical study and does not establish the full independence-characterization known for distance correlation (Pascual-Marqui et al., 2013).

Covariance estimation itself is often the dominant source of instability. Few-shot covariance-aware prompt learning explicitly shrinks and normalizes classwise covariance matrices, and still notes sensitivity when sample sizes are extremely small or under domain shift (Dong et al., 3 Mar 2025). Mahalanobis cosine similarity for probes is theoretically sharp under balanced Gaussian projections, but the predicted linearity with AUROC can fail with the wrong covariance, class imbalance, heteroscedasticity, or a poor reference probe (Ying et al., 17 Jun 2026). These are not incidental pathologies; they delimit the regime in which covariance similarity is a faithful surrogate for task performance.

A separate source of confusion is terminological. In theoretical physics and in convolutional neural networks, “covariance” often means that equations or feature maps transform consistently under coordinate changes or group actions. Formally, a law $\Sigma_1=\Sigma_2$8 is covariant if $\Sigma_1=\Sigma_2$9, and a map $$\cos_{\Sigma}(x,y) = \frac{x^\top \Sigma^{-1} y} {\sqrt{x^\top \Sigma^{-1} x}\,\sqrt{y^\top \Sigma^{-1} y}},$$0 is equivariant if $$\cos_{\Sigma}(x,y) = \frac{x^\top \Sigma^{-1} y} {\sqrt{x^\top \Sigma^{-1} x}\,\sqrt{y^\top \Sigma^{-1} y}},$$1 for all $$\cos_{\Sigma}(x,y) = \frac{x^\top \Sigma^{-1} y} {\sqrt{x^\top \Sigma^{-1} x}\,\sqrt{y^\top \Sigma^{-1} y}},$$2 (Cheng et al., 2019). This usage concerns symmetry of operators, not covariance matrices or covariance-based similarity measures. The coexistence of these meanings is historically entrenched, and distinguishing them is essential when “covariance similarity” is used across statistics, geometry, machine learning, and physics.

In that broader sense, covariance similarity is best understood as a second-order methodology: similarity is defined not only by position, angle, or adjacency, but by how variability and dependence are organized. Whether instantiated through Bhattacharyya overlap, covariance kernels, sparse covariance regression, or covariance-weighted cosine, the unifying principle is that second-order structure is treated as a primary carrier of relational information rather than as a nuisance parameter.