CoVariance Neural Networks: Theory & Applications

• CoVariance Neural Networks are specialized models that use sample covariance matrices as graph shift operators to perform polynomial convolutions.
• They achieve improved stability and interpretability by applying spectral filtering in the principal component domain, addressing key PCA limitations.
• VNNs have demonstrated state-of-the-art performance in applications such as brain age prediction, fairness optimization, infinite-dimensional signal processing, and spatiotemporal modeling.

CoVariance Neural Networks (VNNs) are a specialized class of neural architectures that generalize graph neural networks (GNNs) to operate on sample covariance matrices as data-driven graph shift operators. By replacing canonical graph adjacency or Laplacian matrices with empirically estimated covariance, VNNs perform polynomial graph convolutions in the principal component (PCA) basis of the data, yielding models with distinctive stability, interpretability, and cross-domain transfer properties. This paradigm has led to new state-of-the-art models in biomedical regression (brain age prediction), fair machine learning, infinite-dimensional signal processing, and covariance-based time-series analysis.

1. Mathematical Foundations and Layer Structure

Given $n$ i.i.d. samples $x_i\in\mathbb{R}^m$, VNNs construct the sample covariance

$$C = \frac{1}{n-1}\sum_{i=1}^n (x_i - \mu)(x_i - \mu)^\top, \quad \mu = \frac{1}{n}\sum_i x_i,$$

and treat $C$ as a graph shift operator. Instead of learning dense $m\times m$ weight matrices, a VNN layer learns a small number of scalar filter taps $\{h_k\}_{k=0}^K$ and applies the covariance polynomial

$H(C) = \sum_{k=0}^K h_k C^k,$

to each input signal. In general, with $F_{\ell-1}$ input and $F_\ell$ output channels at layer $\ell$,

$$x^{(\ell)}_{[f]} = \sigma\left( \sum_{g=1}^{F_{\ell-1}} H_{\ell,fg}(C) x^{(\ell-1)}_{[g]} \right),\ H_{\ell,fg}(C) = \sum_{k=0}^K h_{\ell,fg}[k] C^k,$$

where $\sigma(\cdot)$ is a nonlinear activation (ReLU, tanh). The model output after $L$ layers is an $m\times F$ matrix $\Phi(x; C,H)$. For regression (e.g., brain age), a permutation-invariant readout is typically used: $$\hat{y} = \frac{1}{m} \sum_{j=1}^m \left[ \frac{1}{F} \sum_{f=1}^F \Phi(x;C,H)_{j,f} \right].$$ This architecture naturally extends to multiple layers and supports both single-input–single-output and multi-channel filtering (Sihag et al., 12 Feb 2024, Sihag et al., 2022).

2. Spectral Properties and Theoretical Advantages

Polynomial filtering in the covariance eigenspace (i.e., learned $h(\lambda) = \sum_{k=0}^K h_k \lambda^k$) enables VNNs to generalize principal component analysis (PCA) by weighting or suppressing regions of the spectrum in a continuous and learnable fashion. This approach directly addresses two core limitations of PCA-based approaches:

• Stability to Covariance Perturbations: Theorems in (Sihag et al., 2022, Cavallo et al., 12 Nov 2025) demonstrate that VNN outputs are $O(n^{-1/2})$-stable with respect to sampling noise in $C$, under mild Lipschitz constraints on $h(\lambda)$. In contrast, PCA projections can be arbitrarily unstable for nearly degenerate eigenvalues, due to eigenvector rotation. VNNs regularize this sensitivity by spectral averaging.
• Expressiveness Beyond PCA: Stacked polynomial filters and nonlinearities in VNNs can approximate a broad function class over the covariance spectrum, allowing modeling of high-, band-, or low-pass spectral features not accessible to linear projections.

Empirical validations confirm that VNNs systematically outperform PCA-based regression/classification in terms of robustness to subsampling noise and cross-dataset transferability (Sihag et al., 2022, Cavallo et al., 12 Nov 2025).

3. Applications and Model Instantiations

3.1 Brain Age Regression (NeuroVNN)

For cortical thickness–based brain age prediction, the NeuroVNN model instantiates a 3-layer VNN with F=35 channels and filter orders $K=1$ (for layers 1-2), $K=8$ (for layer 3) (Sihag et al., 12 Feb 2024). The model is pre-trained to regress chronological age with MSE loss and achieves test MAE ≈9.24±0.59 years, Pearson $r ≈ 0.79$ on held-out data. A linear bias-correction on residuals yields the brain-age gap: $$\hat{y}_B = \hat{y} - (\alpha y + \beta),\quad \Delta \text{Age} = \hat{y}_B - y.$$ The output can be anatomically interpreted at the regional level (salience maps), and $\Delta$Age correlates with clinical metrics such as Alzheimer’s Progression Score and clinical diagnosis in independent datasets. The same trained filter taps are "scale-free" and transferable across brain atlases of different dimensionality ($m'=68, 100, 148, 200, 400$) without retraining, yielding outputs with $r > 0.97$ inter-atlas correlation.

3.2 Fairness and Bias Mitigation (FVNN)

Fair coVariance Neural Networks extend VNNs with bias-mitigated covariance estimates (group-wise reweightings, linear de-biasings) and end-to-end fairness regularizers. The FVNN trains to minimize

$$\Theta^* = \arg\min_\Theta \left[ \gamma \mathcal{L}_\text{task} + (1-\gamma) \Delta(\Theta) \right],$$

where $\Delta(\Theta)$ penalizes group-wise loss disparities. FVNNs maintain intrinsic stability advantages over fair PCA, especially in imbalanced/low-sample settings (Cavallo et al., 13 Sep 2024).

3.3 Infinite-Dimensional Signal Processing (Hilbert CoVariance Networks)

Hilbert coVariance Networks (HVNs) generalize VNNs to infinite-dimensional Hilbert spaces by filtering with functional polynomials in the empirical covariance operator. Discretization via bounded sampling operators preserves commutativity with polynomial filtering, linking this framework to exact functional PCA. HVNs deliver superior robustness and transferability in classification of large-scale functional/time series data compared to function-agnostic MLPs and FPCA (Battiloro et al., 16 Sep 2025).

3.4 Spatiotemporal Modeling

Spatiotemporal coVariance Neural Networks (STVNNs) extend the VNN approach with joint space-time convolutions, utilizing lag-0 covariance shifts and learnable causal temporal memories. Online covariance updates enable adaptation to non-stationary data streams, maintaining $O(1/\sqrt{t})$ filter stability and outperforming temporal PCA and recurrent baselines in streaming multivariate forecasting tasks (Cavallo et al., 16 Sep 2024).

4. Interpretability and Biomarker Extraction

The design of VNN readout as an unweighted mean over output channels and features supports direct anatomical mapping of the prediction to individual features (e.g., brain regions). Back-projecting local output contributions $p_j$ identifies regions most influential in prediction, with high $p_j$ driving elevated brain age gap. These salience maps are independently validated against clinical correlates and anatomical knowledge, e.g., in Alzheimer's disease, VNNs consistently identify medial temporal and parahippocampal regions as leading factors (Sihag et al., 12 Feb 2024, Sihag et al., 2023, Sihag et al., 2022). The spectral form $H(C) = V h(\Lambda) V^\top$, where $V$ is the eigenbasis of $C$, makes VNNs methodologically transparent: learned filters directly modulate variance along PCA directions, providing a built-in mechanism to attribute model decisions to biologically meaningful latent factors.

5. Transferability, Scale-Free Structure, and Generalization

VNN filter taps $\{h_k\}$ are independent of feature dimension $m$; thus, once trained, a VNN may be applied without modification to any new dataset represented by a different feature set and corresponding covariance, provided the underlying "graphon" (limit object) of the population covariances is similar. Theoretical results show outputs converge with network width/depth scaling under standard continuity and smoothness assumptions (Sihag et al., 2023). This scale-free property is empirically confirmed: models trained on one atlas (e.g., $m=100$) yield commensurate outputs ($r>0.97$) on others ($m'=68, 148, 200,400$), preserving not only predictive accuracy (MAE $\approx$ 9–11 years, $r \approx 0.8$) but also anatomical interpretability.

6. Limitations, Practical Considerations, and Future Directions

Limitations of VNNs include:

• Dependence on a high-quality covariance estimate, which may be sample-limited or contaminated by group bias.
• Complexity growing with the product of polynomial order $K$, number of channels $F$, and number of layers $L$, though this is typically less severe than fully dense parameterizations.
• Fixed-graph assumption: the learned model cannot natively adapt to changing network structure beyond the empirical covariance used.

Future work encompasses:

• Multimodal extensions where VNNs process or fuse different measurement modalities (e.g., fMRI, PET) through multiple covariance graphs.
• Adaptive graph learning, where both the covariance matrix and filter coefficients are optimized.
• Extension to online and high-dimensional/functional settings, leveraging infinite-dimensional HVN theory (Battiloro et al., 16 Sep 2025).
• Deployment of fairness-aware variants for broad scientific and clinical impact (Cavallo et al., 13 Sep 2024).

7. Summary Table: Core VNN Properties and Results

Property	VNNs	Numerical Metrics
Filter type	Covariance polynomials $\sum h_k C^k$	$K=1,2,8$ (NeuroVNN layers)
Readout	Unweighted mean, global pooling	$\hat{y} = \frac{1}{m}\sum p_j$
Interpretability	Anatomical regions and spectral modes	$r(\Delta$Age, APS$)=0.43$ PREVENT-AD
Stability to perturbations	$O(n^{-1/2})$ for $n$ samples, Lipschitz $h(\cdot)$
Transferability (“scale-free”)	Model applies to any $m'$ with same filter taps	$r > 0.97$ cross-atlas
Age regression (brain age)	Test MAE $\approx$ 9–11 yrs, Pearson $r\approx 0.8$	(Sihag et al., 12 Feb 2024)
Cross-pop. biomarker validity	$\Delta$Age correlates with disease/progression	ANCOVA $p\approx 10^{-3}$

In summary, CoVariance Neural Networks (VNNs) constitute a robust, interpretable, and transfer-friendly framework for neural learning in domains where pairwise feature dependencies, as encoded in the sample covariance, drive task-relevant information. The architecture and theory underpinning VNNs deliver provable stability and superior generalization over PCA, modular integration with fairness objectives, and demonstrable utility in foundation models for biomedical applications (Sihag et al., 12 Feb 2024, Sihag et al., 2023, Cavallo et al., 13 Sep 2024, Sihag et al., 2022).