Brain Networks and Intelligence: A Graph Neural Network Based Approach to Resting State fMRI Data (2311.03520v3)
Abstract: Resting-state functional magnetic resonance imaging (rsfMRI) is a powerful tool for investigating the relationship between brain function and cognitive processes as it allows for the functional organization of the brain to be captured without relying on a specific task or stimuli. In this paper, we present a novel modeling architecture called BrainRGIN for predicting intelligence (fluid, crystallized, and total intelligence) using graph neural networks on rsfMRI derived static functional network connectivity matrices. Extending from the existing graph convolution networks, our approach incorporates a clustering-based embedding and graph isomorphism network in the graph convolutional layer to reflect the nature of the brain sub-network organization and efficient network expression, in combination with TopK pooling and attention-based readout functions. We evaluated our proposed architecture on a large dataset, specifically the Adolescent Brain Cognitive Development Dataset, and demonstrated its effectiveness in predicting individual differences in intelligence. Our model achieved lower mean squared errors and higher correlation scores than existing relevant graph architectures and other traditional machine learning models for all of the intelligence prediction tasks. The middle frontal gyrus exhibited a significant contribution to both fluid and crystallized intelligence, suggesting their pivotal role in these cognitive processes. Total composite scores identified a diverse set of brain regions to be relevant which underscores the complex nature of total intelligence.
- Fluid ability (gf) and complex problem solving (CPS). Journal of Intelligence, 5(3):28, July 2017. doi:10.3390/jintelligence5030028. URL https://doi.org/10.3390/jintelligence5030028.
- Effective training strategy for NN models of working memory classification with limited samples. In 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI). IEEE, April 2023. doi:10.1109/isbi53787.2023.10230722. URL https://doi.org/10.1109/isbi53787.2023.10230722.
- Replication and refinement of brain age model for adolescent development. August 2023. doi:10.1101/2023.08.16.553472. URL https://doi.org/10.1101/2023.08.16.553472.
- Fluid and flexible minds: Intelligence reflects synchrony in the brain’s intrinsic network architecture. Network Neuroscience, 1(2):192–207, June 2017. doi:10.1162/netn_a_00010. URL https://doi.org/10.1162/netn_a_00010.
- Deep neural networks and kernel regression achieve comparable accuracies for functional connectivity prediction of behavior and demographics. NeuroImage, 206:116276, February 2020. doi:10.1016/j.neuroimage.2019.116276. URL https://doi.org/10.1016/j.neuroimage.2019.116276.
- A distributed brain network predicts general intelligence from resting-state human neuroimaging data. Philosophical Transactions of the Royal Society B: Biological Sciences, 373(1756):20170284, August 2018. doi:10.1098/rstb.2017.0284. URL https://doi.org/10.1098/rstb.2017.0284.
- On the prediction of human intelligence from neuroimaging: A systematic review of methods and reporting. Intelligence, 93:101654, July 2022. doi:10.1016/j.intell.2022.101654. URL https://doi.org/10.1016/j.intell.2022.101654.
- Resting-state fMRI: A review of methods and clinical applications. American Journal of Neuroradiology, 34(10):1866–1872, August 2012. doi:10.3174/ajnr.a3263. URL https://doi.org/10.3174/ajnr.a3263.
- Functional brain networks related to individual differences in human intelligence at rest. Scientific Reports, 6(1), August 2016. doi:10.1038/srep32328. URL https://doi.org/10.1038/srep32328.
- MRI-based intelligence quotient (IQ) estimation with sparse learning. PLOS ONE, 10(3):e0117295, March 2015. doi:10.1371/journal.pone.0117295. URL https://doi.org/10.1371/journal.pone.0117295.
- A deep network model on dynamic functional connectivity with applications to gender classification and intelligence prediction. Frontiers in Neuroscience, 14, August 2020. doi:10.3389/fnins.2020.00881. URL https://doi.org/10.3389/fnins.2020.00881.
- Accounting for temporal variability in functional magnetic resonance imaging improves prediction of intelligence. Human Brain Mapping, 44(13):4772–4791, July 2023. doi:10.1002/hbm.26415. URL https://doi.org/10.1002/hbm.26415.
- Understanding graph isomorphism network for rs-fMRI functional connectivity analysis. Frontiers in Neuroscience, 14, June 2020. doi:10.3389/fnins.2020.00630. URL https://doi.org/10.3389/fnins.2020.00630.
- Differentiable graph module (DGM) for graph convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2):1606–1617, February 2023. doi:10.1109/tpami.2022.3170249. URL https://doi.org/10.1109/tpami.2022.3170249.
- Automated eloquent cortex localization in brain tumor patients using multi-task graph neural networks. Medical Image Analysis, 74:102203, December 2021. doi:10.1016/j.media.2021.102203. URL https://doi.org/10.1016/j.media.2021.102203.
- Metric learning with spectral graph convolutions on brain connectivity networks. NeuroImage, 169:431–442, April 2018. doi:10.1016/j.neuroimage.2017.12.052. URL https://doi.org/10.1016/j.neuroimage.2017.12.052.
- Human brain structural connectivity matrices–ready for modelling. Scientific Data, 9(1), August 2022. doi:10.1038/s41597-022-01596-9. URL https://doi.org/10.1038/s41597-022-01596-9.
- Connectome-based individual prediction of cognitive behaviors via graph propagation network reveals directed brain network topology. Journal of Neural Engineering, 18(4):0460a3, July 2021. doi:10.1088/1741-2552/ac0f4d. URL https://doi.org/10.1088/1741-2552/ac0f4d.
- Similarity learning with higher-order graph convolutions for brain network analysis, 2018. URL https://arxiv.org/abs/1811.02662.
- Graph saliency maps through spectral convolutional networks: Application to sex classification with brain connectivity, 2018. URL https://arxiv.org/abs/1806.01764.
- Ia-gcn: Interpretable attention based graph convolutional network for disease prediction, 2021. URL https://arxiv.org/abs/2103.15587.
- Functional connections between and within brain subnetworks under resting-state. Scientific Reports, 10(1), February 2020. doi:10.1038/s41598-020-60406-7. URL https://doi.org/10.1038/s41598-020-60406-7.
- BrainGNN: Interpretable brain graph neural network for fMRI analysis. Medical Image Analysis, 74:102233, December 2021. doi:10.1016/j.media.2021.102233. URL https://doi.org/10.1016/j.media.2021.102233.
- How powerful are graph neural networks?, 2018. URL https://arxiv.org/abs/1810.00826.
- Fbnetgen: Task-aware gnn-based fmri analysis via functional brain network generation, 2022a. URL https://arxiv.org/abs/2205.12465.
- A generalization of transformer networks to graphs, 2020. URL https://arxiv.org/abs/2012.09699.
- Brain network transformer, 2022b. URL https://arxiv.org/abs/2210.06681.
- Interpretable graph neural networks for connectome-based brain disorder analysis, 2022. URL https://arxiv.org/abs/2207.00813.
- Exploiting edge features in graph neural networks, 2018. URL https://arxiv.org/abs/1809.02709.
- Squeeze-and-excitation networks, 2017. URL https://arxiv.org/abs/1709.01507.
- Learning dynamic graph representation of brain connectome with spatio-temporal attention, 2021. URL https://arxiv.org/abs/2105.13495.
- Attention is all you need, 2017. URL https://arxiv.org/abs/1706.03762.
- Recalibrating fully convolutional networks with spatial and channel “squeeze and excitation” blocks. IEEE Transactions on Medical Imaging, 38(2):540–549, February 2019. doi:10.1109/tmi.2018.2867261. URL https://doi.org/10.1109/tmi.2018.2867261.
- Pytorch: An imperative style, high-performance deep learning library, 2019. URL https://arxiv.org/abs/1912.01703.
- Fast graph representation learning with pytorch geometric, 2019. URL https://arxiv.org/abs/1903.02428.
- NeuroMark: An automated and adaptive ICA based pipeline to identify reproducible fMRI markers of brain disorders. NeuroImage: Clinical, 28:102375, 2020. doi:10.1016/j.nicl.2020.102375. URL https://doi.org/10.1016/j.nicl.2020.102375.
- The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology, 106(3):1125–1165, September 2011. doi:10.1152/jn.00338.2011. URL https://doi.org/10.1152/jn.00338.2011.
- Brainnetcnn: Convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage, 146:1038–1049, February 2017. ISSN 1053-8119. doi:10.1016/j.neuroimage.2016.09.046. URL http://dx.doi.org/10.1016/j.neuroimage.2016.09.046.
- Semi-supervised classification with graph convolutional networks, 2016. URL https://arxiv.org/abs/1609.02907.
- Graph attention networks, 2017. URL https://arxiv.org/abs/1710.10903.
- Inductive representation learning on large graphs, 2017. URL https://arxiv.org/abs/1706.02216.
- The WU-minn human connectome project: An overview. NeuroImage, 80:62–79, October 2013. doi:10.1016/j.neuroimage.2013.05.041. URL https://doi.org/10.1016/j.neuroimage.2013.05.041.
- The minimal preprocessing pipelines for the human connectome project. NeuroImage, 80:105–124, October 2013. doi:10.1016/j.neuroimage.2013.04.127. URL https://doi.org/10.1016/j.neuroimage.2013.04.127.
- Automatic denoising of functional MRI data: Combining independent component analysis and hierarchical fusion of classifiers. NeuroImage, 90:449–468, April 2014. doi:10.1016/j.neuroimage.2013.11.046. URL https://doi.org/10.1016/j.neuroimage.2013.11.046.
- FSL. NeuroImage, 62(2):782–790, August 2012. doi:10.1016/j.neuroimage.2011.09.015. URL https://doi.org/10.1016/j.neuroimage.2011.09.015.
- Dorsolateral prefrontal contributions to human working memory. Cortex, 49(5):1195–1205, May 2013. doi:10.1016/j.cortex.2012.05.022. URL https://doi.org/10.1016/j.cortex.2012.05.022.
- The role of the left dorsolateral prefrontal cortex in attentional bias. Neuropsychologia, 148:107631, November 2020. doi:10.1016/j.neuropsychologia.2020.107631. URL https://doi.org/10.1016/j.neuropsychologia.2020.107631.
- The role of dorsolateral and ventromedial prefrontal cortex in the processing of emotional dimensions. Scientific Reports, 11(1), January 2021. doi:10.1038/s41598-021-81454-7. URL https://doi.org/10.1038/s41598-021-81454-7.
- Where do you know what you know? the representation of semantic knowledge in the human brain. Nature Reviews Neuroscience, 8(12):976–987, December 2007. doi:10.1038/nrn2277. URL https://doi.org/10.1038/nrn2277.
- The posterior middle temporal gyrus serves as a hub in syntactic comprehension: A model on the syntactic neural network. Brain and Language, 232:105162, September 2022. doi:10.1016/j.bandl.2022.105162. URL https://doi.org/10.1016/j.bandl.2022.105162.
- The neural architecture of the language comprehension network: Converging evidence from lesion and connectivity analyses. Frontiers in System Neuroscience, 5, 2011. doi:10.3389/fnsys.2011.00001. URL https://doi.org/10.3389/fnsys.2011.00001.
- Where is the semantic system? a critical review and meta-analysis of 120 functional neuroimaging studies. Cerebral Cortex, 19(12):2767–2796, March 2009. doi:10.1093/cercor/bhp055. URL https://doi.org/10.1093/cercor/bhp055.
- Spatiotemporal dissociation of fMRI activity in the caudate nucleus underlies human de novo motor skill learning. Proceedings of the National Academy of Sciences, 117(38):23886–23897, September 2020. doi:10.1073/pnas.2003963117. URL https://doi.org/10.1073/pnas.2003963117.
- Hippocampal–caudate nucleus interactions support exceptional memory performance. Brain Structure and Function, November 2017. doi:10.1007/s00429-017-1556-2. URL https://doi.org/10.1007/s00429-017-1556-2.
- Subcortical intelligence: Caudate volume predicts IQ in healthy adults. Human Brain Mapping, 36(4):1407–1416, December 2014. doi:10.1002/hbm.22710. URL https://doi.org/10.1002/hbm.22710.
- DiGuiseppi J;Tadi. Neuroanatomy, postcentral gyrus. URL https://pubmed.ncbi.nlm.nih.gov/31751015/.
- BrainMadeSimple, Feb 2022. URL https://brainmadesimple.com/occipital-lobe/.
- The inferior occipital gyrus is a major cortical source of the face-evoked n170: Evidence from simultaneous scalp and intracerebral human recordings. Human Brain Mapping, 40(5):1403–1418, November 2018. doi:10.1002/hbm.24455. URL https://doi.org/10.1002/hbm.24455.
- Role for supplementary motor area cells in planning several movements ahead. Nature, 371(6496):413–416, September 1994. doi:10.1038/371413a0. URL https://doi.org/10.1038/371413a0.
- Fronto-parietal network: flexible hub of cognitive control. Trends in Cognitive Sciences, 17(12):602–603, December 2013. doi:10.1016/j.tics.2013.10.001. URL https://doi.org/10.1016/j.tics.2013.10.001.
- Marcus E. Raichle. The brain's default mode network. Annual Review of Neuroscience, 38(1):433–447, July 2015. doi:10.1146/annurev-neuro-071013-014030. URL https://doi.org/10.1146/annurev-neuro-071013-014030.
- Medial prefrontal cortex and self-referential mental activity: Relation to a default mode of brain function. Proceedings of the National Academy of Sciences, 98(7):4259–4264, March 2001. doi:10.1073/pnas.071043098. URL https://doi.org/10.1073/pnas.071043098.
- Coherent spontaneous activity identifies a hippocampal-parietal memory network. Journal of Neurophysiology, 96(6):3517–3531, December 2006. doi:10.1152/jn.00048.2006. URL https://doi.org/10.1152/jn.00048.2006.
- From movement to thought: Executive function, embodied cognition, and the cerebellum. The Cerebellum, 11(2):505–525, November 2011. doi:10.1007/s12311-011-0321-y. URL https://doi.org/10.1007/s12311-011-0321-y.
- Functional boundaries in the human cerebellum revealed by a multi-domain task battery. Nature Neuroscience, 22(8):1371–1378, July 2019. doi:10.1038/s41593-019-0436-x. URL https://doi.org/10.1038/s41593-019-0436-x.