Brain-Model Representational Similarity

• Brain-model representational similarity is the approach that quantifies the alignment between biological neural systems and ANNs using geometric and topological summaries like RDMs.
• It leverages techniques such as sliding-window time-resolved RDMs, topological RSA, and unbiased estimators to capture dynamic, high-dimensional data across sensory and cognitive processes.
• This framework aids in model selection and neuroAI advancement by systematically evaluating task-driven, behavioral, and representational convergence between brains and models.

Brain-model representational similarity refers to the extent to which the internal representations of biological neural systems (such as the human brain) and artificial neural networks (ANNs) are aligned, as evaluated by formal comparison of their responses to sets of stimuli. This concept underpins a major current in computational neuroscience and neuroAI, aiming to not only match models to observable behavior but also to ensure convergence in the latent, population-level structure of sensory and cognitive representations. Central to this approach is the use of representational dissimilarity matrices (RDMs) and related techniques, which abstract neural population patterns or deep model activations into geometric or topological summaries, enabling systematic cross-domain comparison.

1. Fundamental Principles and Methodological Framework

At the core of brain-model representational similarity is the abstraction of raw high-dimensional neural or model responses into RDMs, where each RDM entry encodes the dissimilarity (e.g., 1 minus correlation, Euclidean, or Mahalanobis distance) between the activity patterns evoked by different experimental conditions or stimulus pairs (Wardle et al., 2015, Blanchard et al., 2018, Diedrichsen et al., 2020). This summary encodes the “geometry” of representation space—how different percepts or concepts are differentiated by neural activity or by a model’s latent features—and is robust to differences in feature dimensionality or basis set.

The comparison proceeds by calculating a statistical similarity (often Spearman’s or Pearson’s rank correlation) between vectorized forms of two RDMs: one from the brain (fMRI, MEG, EEG, or multi-unit recordings) and one from a computational model. Variants of Representational Similarity Analysis (RSA) extend to cross-modal and cross-species comparisons, interpretability in LLMs (Abnar et al., 2019, Abdou et al., 2019), temporally resolved phenomena (Lin et al., 2019), and different dissimilarity measures, some of which must be unbiased to avoid noise-induced artifacts (Diedrichsen et al., 2020).

A generalized formula for the RDM entry is, for features $v_i$ and $v_j$,

$$RDM(i, j) = 1 - \frac{(v_i - \bar{v}_i) \cdot (v_j - \bar{v}_j)}{||v_i - \bar{v}_i||_2 \, ||v_j - \bar{v}_j||_2}$$

Comparison between any two representational spaces (across modalities, organisms, or models) is typically conducted as:

$$RSA(System~A, System~B) = \text{corr}(\text{vec}(RDM_A), \text{vec}(RDM_B))$$

Recent advances such as Topological RSA (“tRSA”) have further abstracted away from strict geometric distances, emphasizing neighborhood or topological relationships to increase robustness to measurement noise and inter-individual variability (Lin et al., 2023, Lin, 21 Aug 2024).

2. Temporal, Hierarchical, and Task-Driven Dynamics

Dynamic stimuli and temporally resolved neural data (e.g., MEG, time-resolved fMRI, neural spiking) reveal that representational similarity between brain and models evolves on sub-second timescales, often in stages. For example, when viewing structured visual patterns, the earliest neural representations (<80 ms post-stimulus) reflect low-level retinotopic organization, which are well-predicted by models encoding local visual features (Wardle et al., 2015, Lin et al., 2019). Thereafter (100–200 ms), the neural dissimilarity structure converges with that determined by human perceptual similarity (Gestalt), suggesting a dynamic shift from raw feature encoding to global perceptual organization.

Tracking these processes across space and time requires technical extensions such as:

• Sliding-window, time-resolved RDMs (“RDM movies”) with Multidimensional Scaling and Procrustes alignment to visualize the evolution and convergence of category-specific representations, capturing trajectories from undifferentiated to segregated clusters and, later, oscillatory or recurrent dynamics (Lin et al., 2019).
• Temporal Topological Data Analysis (tTDA) for neural population or single-cell developmental data, preserving both spatial and temporal organization (Lin, 21 Aug 2024).

These methodologies show that both brain and models often follow hierarchical, staged transformations: from sensory-dominated spaces aligned with input (pixels, word tokens), to increasingly abstract, task-relevant, or categorical representations (Toosi, 2023, Marcos-Manchón et al., 18 Jul 2025).

3. Task Optimization, Representational Constraints, and Model Alignment

Task demands shape representational similarity. Neural and ANN systems trained for similar tasks (e.g., object classification, language understanding) exhibit aligned intermediate representations, due to constraints imposed by the abstraction trajectory from input to output spaces (Toosi, 2023, Marcos-Manchón et al., 18 Jul 2025). This is formalized by mapping neural and ANN layers into an “abstraction space” with axes corresponding to similarity to pixel space and to class (category) space; proximity in this space robustly predicts representational similarity (as quantified by CKA or RSA).

Experiments reveal that models explicitly trained to match higher-order behavioral or neural similarity (e.g., using human fMRI as a metric—HMS; (Blanchard et al., 2018)), or to project into shared semantic embedding spaces (e.g., LLM embeddings) better recapitulate both behavioral and neural similarity structure than models optimized only for canonical supervised objectives (Golan et al., 2022, Simkova et al., 29 Jul 2025).

Adversarial robustness, dataset scaling, and pretraining, while affecting model-model alignment, do not necessarily close the gap in task-relevant dimensions between brains and models; decision variable correlation (DVC) analyses show that model-model DVC can match intra-brain DVC, but model-brain DVC is lower and may decrease with increasing ImageNet performance (Yu et al., 2 Jun 2025).

4. Statistical, Topological, and Computational Advances

Emergent challenges in brain-model representational similarity concern noise bias, statistical dependencies between dissimilarity estimates, and limitations of finite neuron sampling:

• Crossvalidated unbiased estimators for RDMs are introduced to remove positive biases introduced by measurement noise, accepting increased variance as a tradeoff (Diedrichsen et al., 2020).
• Whitening procedures (e.g., the whitened unbiased RDM cosine similarity WUC) further mitigate correlated error structure in RDMs, improving model selection and aligning with near-optimal likelihood-ratio discriminators.
• Random Matrix Theory–based spectral analyses of similarity metrics (CCA, CKA) reveal that limited neuron sampling induces eigenvector delocalization, suppressing observed brain-model similarity. Novel denoising procedures leveraging estimated population eigenvalues and overlap matrices can recover true population-level representational similarity from small neural samples (Kang et al., 27 Feb 2025).

Topological RSA (tRSA) moves beyond geometric dissimilarity by applying nonlinear, monotonic geo-topological transforms to the RDM, focusing on neighborhood relationships or the “shape” of representation space. This approach increases robustness to noise and idiosyncratic variability and may outperform full RDM-based methods in high-variability or low-sample regimes (Lin et al., 2023, Lin, 21 Aug 2024).

5. Behavioral, Semantic, and Conceptual Convergence

Behavioral tasks (e.g., similarity judgments via multi-arrangement methods) and multimodal modeling (image-language pairs) reveal that the human perceptual system, LLMs, and the brain all encode similarity in a shared, high-dimensional representational space. Human similarity judgments for images and captions form RDMs that are highly correlated (e.g., fixed-effects Spearman ≈ 0.78), and both explain fMRI similarity patterns in mid- and high-level occipitotemporal cortex (Simkova et al., 29 Jul 2025). LLM-embedding–optimized visual models outperform category-trained networks and AlexNet in recapitulating this behavioral and neural structure.

Such findings support theoretical frameworks of “second-order isomorphism”—that internal representations preserve stable, relational properties of the environment, not merely the arbitrary product of low-level sensory or architectural bias.

6. Empirical Indistinguishability and the NeuroAI Turing Test

Recent theoretical work contends that a model can only be deemed truly brain-like if its internal representations are statistically indistinguishable, up to the level of individual variation, from those measured in real biological systems (Feather et al., 22 Feb 2025). The NeuroAI Turing Test framework formalizes this in a two-tiered structure:

• Behavioral alignment (classic Turing test): Model matches human behavioral outputs.
• Representational alignment: Model activation distances to brain data are not statistically greater than the distances among brains (“empirical indistinguishability”).

Metrics such as RSA, CKA, or mapping-predictivity are noise-corrected and normalized for inter-individual variance, and hypothesis testing (e.g., permutation tests) is performed. If the model–brain distribution matches the brain–brain distribution, the model is said to have achieved “representational convergence.”

This framework sets a systematic, testable standard for evaluating computational models of intelligence, requiring robust architectural, optimization, and data-driven advances to close the remaining representational gaps.

7. Practical Applications and Future Directions

Brain-model representational similarity forms the methodological backbone for:

• Model selection and neurobiological plausibility evaluation (e.g., via HMS, WUC, tRSA, DVC).
• Early-stopping and online selection in automated neural architecture search using neural alignment metrics (Blanchard et al., 2018).
• Designing controversial stimuli (via Bayesian Optimal Experimental Design) to efficiently adjudicate between models otherwise indistinguishable on typical datasets (Golan et al., 2022).
• Guiding real-time brain-computer interface development, brain decoding pipelines, and multimodal semantic image reconstruction grounded in neural constraints (Ferrante et al., 2022).
• Bridging vision and language in shared, conceptually organized representation spaces, with implications for neuro-symbolic and cross-modal AI architectures (Simkova et al., 29 Jul 2025).

Ongoing challenges include closing the gap in task-relevant dimensions (as highlighted by low DVCs despite high superficial or geometric similarity), robust handling of noise and inter-individual variability, and extending similarity-based model evaluation to developmental, temporal, and multi-scale neuroscientific data.

---

In sum, brain-model representational similarity is defined by the formal alignment, at the level of internal representational structure, between neural systems and computational models. Leveraging statistical, topological, and geometric frameworks, current research has demonstrated both convergent solutions and persistent divergences, providing not just a toolbox for model evaluation but also deepening our understanding of how biology and computation jointly constrain the emergence of intelligence.