Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Universal Differential Equations as a Common Modeling Language for Neuroscience (2403.14510v1)

Published 21 Mar 2024 in cs.CE

Abstract: The unprecedented availability of large-scale datasets in neuroscience has spurred the exploration of artificial deep neural networks (DNNs) both as empirical tools and as models of natural neural systems. Their appeal lies in their ability to approximate arbitrary functions directly from observations, circumventing the need for cumbersome mechanistic modeling. However, without appropriate constraints, DNNs risk producing implausible models, diminishing their scientific value. Moreover, the interpretability of DNNs poses a significant challenge, particularly with the adoption of more complex expressive architectures. In this perspective, we argue for universal differential equations (UDEs) as a unifying approach for model development and validation in neuroscience. UDEs view differential equations as parameterizable, differentiable mathematical objects that can be augmented and trained with scalable deep learning techniques. This synergy facilitates the integration of decades of extensive literature in calculus, numerical analysis, and neural modeling with emerging advancements in AI into a potent framework. We provide a primer on this burgeoning topic in scientific machine learning and demonstrate how UDEs fill in a critical gap between mechanistic, phenomenological, and data-driven models in neuroscience. We outline a flexible recipe for modeling neural systems with UDEs and discuss how they can offer principled solutions to inherent challenges across diverse neuroscience applications such as understanding neural computation, controlling neural systems, neural decoding, and normative modeling.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (185)
  1. Large-Scale Neural Recordings Call for New Insights to Link Brain and Behavior. \JournalTitleNature Neuroscience 25, 11–19, DOI: 10.1038/s41593-021-00980-9 (2022).
  2. Blending Computational and Experimental Neuroscience. \JournalTitleNature Reviews Neuroscience 17, 667–668, DOI: 10.1038/nrn.2016.114 (2016).
  3. Ring Attractor Dynamics in the Drosophila Central Brain. \JournalTitleScience 853, 849–853 (2017).
  4. Large-Scale Model of Mammalian Thalamocortical Systems. \JournalTitleProceedings of the National Academy of Sciences 105, 3593–3598 (2008).
  5. Distributed Hierarchical Processing in the Primate Cerebral Cortex. \JournalTitleCerebral Cortex 1, 1–47 (1991).
  6. A Synaptic Model of Memory: Long-Term Potentiation in the Hippocampus. \JournalTitleNature 361, 31–39 (1993).
  7. Van Gelder, T. The dynamical hypothesis in cognitive science. \JournalTitleBehavioral and brain sciences 21, 615–628 (1998).
  8. Friston, K. J. The Free-Energy Principle: A Rough Guide to the Brain? \JournalTitleTrends in Cognitive Sciences 13, 293–301 (2009).
  9. Hawkins, J. A Thousand Brains: A New Theory of Intelligence (Basic Books, 2021).
  10. An Integrative Theory of Prefrontal Cortex Function. \JournalTitleAnnual Review of Neuroscience 24, 167–202 (2001).
  11. Steinmetz, N. A. et al. Neuropixels 2.0: A Miniaturized High-Density Probe for Stable, Long-Term Brain Recordings. \JournalTitleScience 372, eabf4588, DOI: 10.1126/science.abf4588 (2021).
  12. Large-scale neural recordings call for new insights to link brain and behavior. \JournalTitleNature neuroscience 25, 11–19 (2022).
  13. Multiregion Neuronal Activity: The Forest and the Trees. \JournalTitleNature Reviews Neuroscience 23, 683–704 (2022).
  14. Deisseroth, K. Optogenetics: 10 Years of Microbial Opsins in Neuroscience. \JournalTitleNature Neuroscience 18, 1213–1225, DOI: 10.1038/nn.4091 (2015).
  15. Lozano, A. M. et al. Deep Brain Stimulation: Current Challenges and Future Directions. \JournalTitleNature Reviews Neurology 15, 148–160 (2019).
  16. Blumberger, D. M. et al. Effectiveness of Theta Burst Versus High-Frequency Repetitive Transcranial Magnetic Stimulation in Patients with Depression (THREE-D): A Randomised Non-Inferiority Trial. \JournalTitleThe Lancet 391, 1683–1692 (2018).
  17. Rackauckas, C. et al. Universal differential equations for scientific machine learning. \JournalTitlearXiv preprint arXiv:2001.04385 (2020).
  18. Izhikevich, E. M. Dynamical systems in neuroscience (MIT press, 2007).
  19. Favela, L. H. The Dynamical Renaissance in Neuroscience. \JournalTitleSynthese 199, 2103–2127 (2021).
  20. Reconstructing computational system dynamics from neural data with recurrent neural networks. \JournalTitleNature Reviews Neuroscience 1–18 (2023).
  21. Differentiable biology: using deep learning for biophysics-based and data-driven modeling of molecular mechanisms. \JournalTitleNature methods 18, 1169–1180 (2021).
  22. Structural identification with physics-informed neural ordinary differential equations. \JournalTitleJournal of Sound and Vibration 508, 116196 (2021).
  23. Karniadakis, G. E. et al. Physics-informed machine learning. \JournalTitleNature Reviews Physics 3, 422–440 (2021).
  24. The Dynamic Brain: From Spiking Neurons to Neural Masses and Cortical Fields. \JournalTitlePLoS Computational Biology 4, e1000092 (2008).
  25. Breakspear, M. Dynamic Models of Large-Scale Brain Activity. \JournalTitleNature Neuroscience 20, 340–352 (2017).
  26. A Quantitative Description of Membrane Current and Its Application to Conduction and Excitation in Nerve. \JournalTitleThe Journal of Physiology 117, 500 (1952).
  27. FitzHugh, R. Impulses and Physiological States in Theoretical Models of Nerve Membrane. \JournalTitleBiophysical Journal 1, 445–466 (1961).
  28. Izhikevich, E. M. Simple Model of Spiking Neurons. \JournalTitleIEEE Transactions on Neural Networks 14, 1569–1572 (2003).
  29. Ongoing Cortical Activity at Rest: Criticality, Multistability, and Ghost Attractors. \JournalTitleJournal of Neuroscience 32, 3366–3375 (2012).
  30. Attractor and integrator networks in the brain. \JournalTitleNature Reviews Neuroscience 23, 744–766 (2022).
  31. Excitatory and inhibitory interactions in localized populations of model neurons. \JournalTitleBiophysical Journal 12, 1–24 (1972).
  32. Kuramoto, Y. Self-entrainment of a population of coupled non-linear oscillators. In International Symposium on Mathematical Problems in Theoretical Physics: January 23–29, 1975, Kyoto University, Kyoto/Japan, 420–422 (Springer, 1975).
  33. Neuronal Oscillations in Cortical Networks. \JournalTitleScience 304, 1926–1929 (2004).
  34. Ramezanian-Panahi, M. et al. Generative models of brain dynamics. \JournalTitleFrontiers in Artificial Intelligence 5, 807406 (2022).
  35. Landhuis, E. Big Brain, Big Data. \JournalTitleNature 541 (2017).
  36. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (Cambridge University Press, 2019).
  37. Data-Driven Models in Human Neuroscience and Neuroengineering. \JournalTitleCurrent Opinion in Neurobiology 58, 21–29 (2019).
  38. Computation through neural population dynamics. \JournalTitleAnnual Review of Neuroscience 43, 249–275 (2020).
  39. Cortical Control of Arm Movements: A Dynamical Systems Perspective. \JournalTitleAnnual Review of Neuroscience 36, 337–359, DOI: 10.1146/annurev-neuro-062111-150509 (2013).
  40. Sussillo, D. Neural circuits as computational dynamical systems. \JournalTitleCurrent Opinion in Neurobiology 25, 156–163 (2014).
  41. Barak, O. Recurrent neural networks as versatile tools of neuroscience research. \JournalTitleCurrent Opinion in Neurobiology 46, 1–6 (2017).
  42. No free lunch from deep learning in neuroscience: A case study through models of the entorhinal-hippocampal circuit. \JournalTitleAdvances in Neural Information Processing Systems 35, 16052–16067 (2022).
  43. Context-dependent computation by recurrent dynamics in prefrontal cortex. \JournalTitleNature 503, 78–84 (2013).
  44. Colloquium: Control of dynamics in brain networks. \JournalTitleReviews of Modern Physics 90, 031003 (2018).
  45. Brain modeling for control: A review. \JournalTitleFrontiers in Control Engineering 3, 1046764 (2022).
  46. A control-theoretic system identification framework and a real-time closed-loop clinical simulation testbed for electrical brain stimulation. \JournalTitleJournal of Neural Engineering 15, 066007 (2018).
  47. State-space optimal feedback control of optogenetically driven neural activity. \JournalTitleJournal of Neural Engineering 18, 036006 (2021).
  48. An in-silico framework for modeling optimal control of neural systems. \JournalTitleFrontiers in Neuroscience 17, 1141884 (2023).
  49. Willett, F. R. et al. A high-performance speech neuroprosthesis. \JournalTitleNature 1–6 (2023).
  50. Metzger, S. L. et al. A high-performance neuroprosthesis for speech decoding and avatar control. \JournalTitleNature 1–10 (2023).
  51. Speech synthesis from neural decoding of spoken sentences. \JournalTitleNature 568, 493–498 (2019).
  52. Zhang, X. et al. A survey on deep learning based brain computer interface: Recent advances and new frontiers. \JournalTitlearXiv preprint arXiv:1905.04149 66 (2019).
  53. Deep learning approaches for neural decoding across architectures and recording modalities. \JournalTitleBriefings in Bioinformatics 22, 1577–1591 (2021).
  54. Computational non-linear dynamical psychiatry: A new methodological paradigm for diagnosis and course of illness. \JournalTitleJournal of Psychiatric Research 46, 428–435 (2012).
  55. Clinical applications of stochastic dynamic models of the brain, part i: A primer. \JournalTitleBiological psychiatry: cognitive neuroscience and neuroimaging 2, 216–224 (2017).
  56. Clinical applications of stochastic dynamic models of the brain, part ii: A review. \JournalTitleBiological psychiatry: cognitive neuroscience and neuroimaging 2, 225–234 (2017).
  57. Psychiatric illnesses as disorders of network dynamics. \JournalTitleBiological Psychiatry: Cognitive Neuroscience and Neuroimaging 6, 865–876 (2021).
  58. Building population models for large-scale neural recordings: Opportunities and pitfalls. \JournalTitleCurrent Opinion in Neurobiology 70, 64–73 (2021).
  59. Kao, J. C. Considerations in using recurrent neural networks to probe neural dynamics. \JournalTitleJournal of Neurophysiology 122, 2504–2521 (2019).
  60. Alber, M. et al. Integrating machine learning and multiscale modeling—perspectives, challenges, and opportunities in the biological, biomedical, and behavioral sciences. \JournalTitleNPJ Digital Medicine 2, 115 (2019).
  61. False discoveries occur early on the lasso path. \JournalTitleThe Annals of Statistics 2133–2150 (2017).
  62. Universality and individuality in neural dynamics across large populations of recurrent networks. \JournalTitleAdvances in Neural Information Processing Systems 32 (2019).
  63. System Identification of Neural Systems: If We Got It Right, Would We Know? In International Conference on Machine Learning, 12430–12444 (PMLR, 2023).
  64. Neural ordinary differential equations. \JournalTitleAdvances in Neural Information Processing Systems 31 (2018).
  65. Kidger, P. On neural differential equations. \JournalTitlearXiv preprint arXiv:2202.02435 (2022).
  66. Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit. \JournalTitlearXiv preprint arXiv:1905.09883 (2019).
  67. Neural rough differential equations for long time series. In International Conference on Machine Learning, 7829–7838 (PMLR, 2021).
  68. Li, Z. et al. Fourier neural operator for parametric partial differential equations. \JournalTitlearXiv preprint arXiv:2010.08895 (2020).
  69. Neural jump stochastic differential equations. \JournalTitleAdvances in Neural Information Processing Systems 32 (2019).
  70. Poli, M. et al. Graph neural ordinary differential equations. \JournalTitlearXiv preprint arXiv:1911.07532 (2019).
  71. Neural controlled differential equations for irregular time series. \JournalTitleAdvances in Neural Information Processing Systems 33, 6696–6707 (2020).
  72. Inferring latent dynamics underlying neural population activity via neural differential equations. In International Conference on Machine Learning, 5551–5561 (PMLR, 2021).
  73. Expressive architectures enhance interpretability of dynamics-based neural population models. \JournalTitlearXiv preprint arXiv:2212.03771 (2022).
  74. Geenjaar, E. et al. Learning low-dimensional dynamics from whole-brain data improves task capture. \JournalTitlearXiv preprint arXiv:2305.14369 (2023).
  75. Expressive dynamics models with nonlinear injective readouts enable reliable recovery of latent features from neural activity. \JournalTitlearXiv preprint arXiv:2309.06402 (2023).
  76. Shen, C. et al. Differentiable modelling to unify machine learning and physical models for geosciences. \JournalTitleNature Reviews Earth & Environment 4, 552–567 (2023).
  77. Wang, H. et al. Scientific discovery in the age of artificial intelligence. \JournalTitleNature 620, 47–60 (2023).
  78. Cybenko, G. Approximation by superpositions of a sigmoidal function. \JournalTitleMathematics of control, signals and systems 2, 303–314 (1989).
  79. Multilayer feedforward networks are universal approximators. \JournalTitleNeural Networks 2, 359–366 (1989).
  80. Oksendal, B. Stochastic differential equations: an introduction with applications (Springer Science & Business Media, 2013).
  81. Applied stochastic differential equations, vol. 10 (Cambridge University Press, 2019).
  82. A comparison of automatic differentiation and continuous sensitivity analysis for derivatives of differential equation solutions. In 2021 IEEE High Performance Extreme Computing Conference (HPEC), 1–9 (IEEE, 2021).
  83. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, 3870–3882 (PMLR, 2020).
  84. Neural sdes as infinite-dimensional gans. In International Conference on Machine Learning, 5453–5463 (PMLR, 2021).
  85. Theoretical Guarantees for Sampling and Inference in Generative Models with Latent Diffusions. In Conference on Learning Theory, 3084–3114 (PMLR, 2019).
  86. Generative ode modeling with known unknowns. In Proceedings of the Conference on Health, Inference, and Learning, 79–94 (ACM, 2021).
  87. How to learn and generalize from three minutes of data: Physics-constrained and uncertainty-aware neural stochastic differential equations. \JournalTitlearXiv preprint arXiv:2306.06335 (2023).
  88. Abrevaya, G. et al. Effective Latent Differential Equation Models via Attention and Multiple Shooting. In The Symbiosis of Deep Learning and Differential Equations III (2023).
  89. Stochastic methods in neuroscience (OUP Oxford, 2009).
  90. Hille, B. Ionic channels in excitable membranes. current problems and biophysical approaches. \JournalTitleBiophysical journal 22, 283–294 (1978).
  91. Sakmann, B. Single-channel recording (Springer Science & Business Media, 2013).
  92. Tuckwell, H. C. Introduction to theoretical neurobiology: linear cable theory and dendritic structure, vol. 1 (Cambridge University Press, 1988).
  93. The noisy brain: stochastic dynamics as a principle of brain function (Oxford academic, 2010).
  94. Noise in the nervous system. \JournalTitleNature reviews neuroscience 9, 292–303 (2008).
  95. Black-Box Variational Inference for Stochastic Differential Equations. In International Conference on Machine Learning, 4423–4432 (PMLR, 2018).
  96. State Estimation of a Physical System with Unknown Governing Equations. \JournalTitleNature 622, 261–267 (2023).
  97. On the choice of interpolation scheme for neural cdes. \JournalTitleTransactions on Machine Learning Research 2022 (2022).
  98. Interpolation in time series: An introductive overview of existing methods, their performance criteria and uncertainty assessment. \JournalTitleWater 9, 796 (2017).
  99. Dissecting neural odes. \JournalTitleAdvances in Neural Information Processing Systems 33, 3952–3963 (2020).
  100. Latent Ordinary Differential Equations for Irregularly-Sampled Time Series. \JournalTitleAdvances in Neural Information Processing Systems 32 (2019).
  101. Auto-encoding variational bayes. \JournalTitlearXiv preprint arXiv:1312.6114 (2013).
  102. Learning structured output representation using deep conditional generative models. \JournalTitleAdvances in Neural Information Processing Systems 28 (2015).
  103. Constrained generation of semantically valid graphs via regularizing variational autoencoders. \JournalTitleAdvances in Neural Information Processing Systems 31 (2018).
  104. Neural system identification for large populations separating “what” and “where”. \JournalTitleAdvances in Neural Information Processing Systems 30 (2017).
  105. Seeliger, K. et al. End-to-end neural system identification with neural information flow. \JournalTitlePLoS Computational Biology 17, e1008558 (2021).
  106. Heeger, D. et al. Poisson model of spike generation. \JournalTitleHandout, University of Standford 5, 76 (2000).
  107. Herreras, O. Local field potentials: Myths and misunderstandings. \JournalTitleFrontiers in Neural Circuits 10, 101 (2016).
  108. Nonlinear responses in fmri: the balloon model, volterra kernels, and other hemodynamics. \JournalTitleNeuroImage 12, 466–477 (2000).
  109. Inferring neuronal dynamics from calcium imaging data using biophysical models and bayesian inference. \JournalTitlePLoS computational biology 12, e1004736 (2016).
  110. Churchland, M. M. et al. Neural population dynamics during reaching. \JournalTitleNature 487, 51–56 (2012).
  111. Sadtler, P. T. et al. Neural constraints on learning. \JournalTitleNature 512, 423–426 (2014).
  112. Structure in neural population recordings: An expected byproduct of simpler phenomena? \JournalTitleNature Neuroscience 20, 1310–1318 (2017).
  113. Yu, B. M. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. \JournalTitleAdvances in Neural Information Processing Systems 21 (2008).
  114. Dimensionality reduction for large-scale neural recordings. \JournalTitleNature Neuroscience 17, 1500–1509 (2014).
  115. Recurrent switching dynamical systems models for multiple interacting neural populations. \JournalTitleAdvances in Neural Information Processing Systems 33, 14867–14878 (2020).
  116. Hurwitz, C. et al. Targeted neural dynamical modeling. \JournalTitleAdvances in Neural Information Processing Systems 34, 29379–29392 (2021).
  117. Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-vae. \JournalTitleAdvances in Neural Information Processing Systems 33, 7234–7247 (2020).
  118. Learnable latent embeddings for joint behavioural and neural analysis. \JournalTitleNature 1–9 (2023).
  119. Structural inference of networked dynamical systems with universal differential equations. \JournalTitleChaos: An Interdisciplinary Journal of Nonlinear Science 33 (2023).
  120. Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks. \JournalTitleNeural Computation 25, 626–649 (2013).
  121. Linderman, S. et al. Bayesian learning and inference in recurrent switching linear dynamical systems. In Artificial Intelligence and Statistics, 914–922 (PMLR, 2017).
  122. Pandarinath, C. et al. Inferring single-trial neural population dynamics using sequential auto-encoders. \JournalTitleNature methods 15, 805–815 (2018).
  123. Grimmer, J. An introduction to bayesian inference via variational approximations. \JournalTitlePolitical Analysis 19, 32–47 (2011).
  124. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. \JournalTitleProceedings of the national academy of sciences 113, 3932–3937 (2016).
  125. Distilling free-form natural laws from experimental data. \JournalTitlescience 324, 81–85 (2009).
  126. Cranmer, M. et al. Discovering symbolic models from deep learning with inductive biases. \JournalTitleAdvances in Neural Information Processing Systems 33, 17429–17442 (2020).
  127. Raissi, M. Deep hidden physics models: Deep learning of nonlinear partial differential equations. \JournalTitleThe Journal of Machine Learning Research 19, 932–955 (2018).
  128. Keith, J. A. et al. Combining machine learning and computational chemistry for predictive insights into chemical systems. \JournalTitleChemical Reviews 121, 9816–9872 (2021).
  129. Davies, A. et al. Advancing mathematics by guiding human intuition with ai. \JournalTitleNature 600, 70–74 (2021).
  130. Turbulence modeling in the age of data. \JournalTitleAnnual Review of Fluid Mechanics 51, 357–377 (2019).
  131. Grammar variational autoencoder. In International Conference on Machine Learning, 1945–1954 (PMLR, 2017).
  132. Choudhary, K. et al. Recent advances and applications of deep learning methods in materials science. \JournalTitlenpj Computational Materials 8, 59 (2022).
  133. Schiff, S. J. Neural Control Engineering: The Emerging Intersection Between Control Theory and Neuroscience (MIT Press, 2011).
  134. Ramirez-Zamora, A. et al. Evolving applications, technological challenges and future opportunities in neuromodulation: Proceedings of the fifth annual deep brain stimulation think tank. \JournalTitleFrontiers in Neuroscience 734 (2018).
  135. Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge University Press, 2015).
  136. Moerland, T. M. et al. Model-based reinforcement learning: A survey. \JournalTitleFoundations and Trends in Machine Learning 16, 1–118 (2023).
  137. Continuous-time model-based reinforcement learning. In International Conference on Machine Learning, 12009–12018 (PMLR, 2021).
  138. Optimal Control (John Wiley & Sons, 2012).
  139. Spikes: Exploring the Neural Code (MIT Press, 1999).
  140. Neural decoding of visual imagery during sleep. \JournalTitleScience 340, 639–642 (2013).
  141. Generative adversarial networks for reconstructing natural images from brain activity. \JournalTitleNeuroImage 181, 775–785 (2018).
  142. Donoghue, J. P. Connecting cortex to machines: recent advances in brain interfaces. \JournalTitleNature Neuroscience 5, 1085–1088 (2002).
  143. Decoding visual information from a population of retinal ganglion cells. \JournalTitleJournal of Neurophysiology 78, 2336–2350 (1997).
  144. Model-based decoding, information estimation, and change-point detection techniques for multineuron spike trains. \JournalTitleNeural Computation 23, 1–45 (2011).
  145. Statistical models for neural encoding, decoding, and optimal stimulus design. \JournalTitleProgress in Brain Research 165, 493–507 (2007).
  146. Interpreting encoding and decoding models. \JournalTitleCurrent Opinion in Neurobiology 55, 167–179 (2019).
  147. ilqr-vae: control-based learning of input-driven dynamics with applications to neural data. \JournalTitlebioRxiv 2021–10 (2021).
  148. Beyond lumping and splitting: a review of computational approaches for stratifying psychiatric disorders. \JournalTitleBiological psychiatry: cognitive neuroscience and neuroimaging 1, 433–447 (2016).
  149. Insel, T. et al. Research domain criteria (rdoc): toward a new classification framework for research on mental disorders (2010).
  150. Bethlehem, R. A. et al. Brain charts for the human lifespan. \JournalTitleNature 604, 525–533 (2022).
  151. Why has it taken so long for biological psychiatry to develop clinical tests and what to do about it? \JournalTitleMolecular psychiatry 17, 1174–1179 (2012).
  152. Wolfers, T. et al. Individual differences v. the average patient: mapping the heterogeneity in adhd using normative models. \JournalTitlePsychological medicine 50, 314–323 (2020).
  153. Zabihi, M. et al. Dissecting the heterogeneous cortical anatomy of autism spectrum disorder using normative models. \JournalTitleBiological Psychiatry: Cognitive Neuroscience and Neuroimaging 4, 567–578 (2019).
  154. Rutherford, S. et al. Evidence for embracing normative modeling. \JournalTitleElife 12, e85082 (2023).
  155. Marquand, A. F. et al. Conceptualizing mental disorders as deviations from normative functioning. \JournalTitleMolecular psychiatry 24, 1415–1424 (2019).
  156. Brodersen, K. H. et al. Dissecting psychiatric spectrum disorders by generative embedding. \JournalTitleNeuroImage: Clinical 4, 98–111 (2014).
  157. Rutherford, S. et al. The normative modeling framework for computational psychiatry. \JournalTitleNature protocols 17, 1711–1734 (2022).
  158. Improving the diagnosis of psychiatric disorders with self-supervised graph state space models. \JournalTitlearXiv preprint arXiv:2206.03331 (2022).
  159. On a quantitative approach to clinical neuroscience in psychiatry: lessons from the kuramoto model. \JournalTitleHarvard Review of Psychiatry 29, 318–326 (2021).
  160. Segal, A. et al. Regional, circuit and network heterogeneity of brain abnormalities in psychiatric disorders. \JournalTitleNature Neuroscience 26, 1613–1629 (2023).
  161. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. \JournalTitleJournal of Computational Physics 378, 686–707 (2019).
  162. Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems. \JournalTitleACM Computing Surveys 55, 1–37 (2022).
  163. Cuomo, S. et al. Scientific machine learning through physics-informed neural networks: Where we are and what’s next. \JournalTitleJournal of Scientific Computing 92, 88 (2022).
  164. Deep learning for physical processes: Incorporating prior scientific knowledge. \JournalTitleJournal of Statistical Mechanics: Theory and Experiment 2019, 124009 (2019).
  165. Li, Z. et al. Neural operator: Graph kernel network for partial differential equations. \JournalTitlearXiv preprint arXiv:2003.03485 (2020).
  166. Fagin, J. et al. Latent stochastic differential equations for modeling quasar variability and inferring black hole properties. \JournalTitlearXiv preprint arXiv:2304.04277 (2023).
  167. Learning fractional white noises in neural stochastic differential equations. \JournalTitleAdvances in Neural Information Processing Systems 35, 37660–37675 (2022).
  168. Variational inference for sdes driven by fractional noise. \JournalTitlearXiv preprint arXiv:2310.12975 (2023).
  169. Latent sdes on homogeneous spaces. \JournalTitlearXiv preprint arXiv:2306.16248 (2023).
  170. Efficient and accurate gradients for neural sdes. \JournalTitleAdvances in Neural Information Processing Systems 34, 18747–18761 (2021).
  171. Automatic differentiation of programs with discrete randomness. \JournalTitleAdvances in Neural Information Processing Systems 35, 10435–10447 (2022).
  172. Differentialequations. jl–a performant and feature-rich ecosystem for solving differential equations in julia. \JournalTitleJournal of open research software 5 (2017).
  173. Rackauckas, C. et al. Diffeqflux. jl-a julia library for neural differential equations. \JournalTitlearXiv preprint arXiv:1902.02376 (2019).
  174. torchode: A parallel ode solver for pytorch. \JournalTitlearXiv preprint arXiv:2210.12375 (2022).
  175. Tuor, A. et al. Neuromancer: Neural modules with adaptive nonlinear constraints and efficient regularizations. \JournalTitleURL https://github. com/pnnl/neuromancer (2021).
  176. Equinox: neural networks in jax via callable pytrees and filtered transformations. \JournalTitlearXiv preprint arXiv:2111.00254 (2021).
  177. The pricing of options and corporate liabilities. \JournalTitleJournal of political economy 81, 637–654 (1973).
  178. A theory of the term structure of interest rates. In Theory of valuation, 129–164 (World Scientific, 2005).
  179. The Langevin equation: with applications to stochastic problems in physics, chemistry and electrical engineering, vol. 27 (World Scientific, 2012).
  180. Arató, M. A famous nonlinear stochastic equation (lotka-volterra model with diffusion). \JournalTitleMathematical and Computer Modelling 38, 709–726 (2003).
  181. Ewens, W. J. Mathematical population genetics: theoretical introduction, vol. 27 (Springer, 2004).
  182. A monte carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems. \JournalTitleSIAM Journal on Control and Optimization 29, 1216–1249 (1991).
  183. Path integral methods for stochastic differential equations. \JournalTitleThe Journal of Mathematical Neuroscience (JMN) 5, 1–35 (2015).
  184. Sensitivity analysis using itô–malliavin calculus and martingales, and application to stochastic optimal control. \JournalTitleSIAM Journal on Control and Optimization 43, 1676–1713 (2005).
  185. Girsanov, I. V. On transforming a certain class of stochastic processes by absolutely continuous substitution of measures. \JournalTitleTheory of Probability & Its Applications 5, 285–301 (1960).
Citations (2)
View on Semantic Scholar

Summary

  • The paper demonstrates that universal differential equations combine mechanistic frameworks with trainable neural components to improve model interpretability in neuroscience.
  • It employs stochastic processes and Bayesian inference to robustly address uncertainties in neural system identification.
  • UDEs offer promising advances in neural decoding, control, and normative modeling for both research and clinical applications.

Universal Differential Equations as a Common Modeling Language for Neuroscience

The paper proposes universal differential equations (UDEs) as a unifying framework for neuroscience modeling, arguing that they integrate the strengths of both traditional mechanistic models and modern data-driven approaches, notably artificial deep neural networks (DNNs). The proliferation of large-scale datasets in neuroscience has positioned DNNs as valuable tools, but their application comes with challenges, notably the risk of producing scientifically implausible models and the opacity of their interpretiveness. The authors suggest that UDEs can mitigate these issues by leveraging the parameterization of differential equations with trainable components, allowing the integration of domain-specific constraints and facilitating greater model transparency.

A UDE marries the classic differential equation techniques with the flexibility of neural networks, positioning them as powerful universal approximators capable of modeling complex dynamical systems within neuroscience. This approach aims to seamlessly blend existing calculus and numerical analysis literature with novel AI advancements, creating a hybrid methodology that can adjust to data availability and complexity of neural dynamics.

The authors structure the paper around key components common to neural system identification problems, with an emphasis on stochastic processes and Bayesian inference using variational methods. Given the inherent stochasticity present in neural systems and the uncertainties due to limited data, UDEs provide a structured yet flexible modeling paradigm. They present a range of scenarios, from traditional differential equations with clear unknowns to complex systems where neural networks model unexplored dynamics, illustrating the spectrum of UDE applications from white-box to black-box models.

The implications of this research are vast, suggesting substantial advances across key areas in neuroscience:

  1. Understanding Cognitive and Behavioral Functions: Leveraging UDEs may standardize how low-dimensional latent dynamics in neural populations are analyzed, moving beyond stereotyped tasks to more naturalistic behavior scenarios. UDEs promise enhanced model expressiveness and improved inference on neural dynamics through structured latent spaces, potentially advancing cognitive neuroscience research.
  2. Neural Control: UDEs present an opportunity to revolutionize neural control systems, particularly in brain-computer interfaces (BCI). By incorporating stochastic dynamical models, they could improve robustness and facilitate the development of closed-loop neurostimulation paradigms, enhancing both clinical and research applications.
  3. Neural Decoding: The potential of UDEs in neural decoding is profound, allowing for accurate reconstruction and interpretation of external stimuli from neural recordings. The ability to train models as encoding systems that adapt to ever-growing datasets could provide improved neural decoding performance across varied modalities.
  4. Normative Modeling: The authors suggest UDEs as ideal for normative modeling in clinical neuroscience, by representing high-dimensional neuroimaging data with reduced dimensionality dynamics. This approach might identify normative population structures and detect deviations indicative of disorders, refining diagnostic tools and predictions.

The paper makes a compelling case for UDEs as a common modeling language across multiple scales and modalities, offering a robust, data-driven, and interpretative approach that could bridge current methodological gaps in neuroscience. However, the authors acknowledge remaining challenges, such as ensuring efficient simulation of UDEs under computational constraints and accurately addressing the stochastic nature of complex neural data.

Future developments in AI, combined with the mathematically rigorous underpinnings of differential equations, might further expand the capabilities of UDEs, offering practitioners the means to achieve nuanced, analytically tractable models of neural processes. As key challenges in scalability and optimization are addressed, UDEs could redefine computational neuroscience as a field, aligning it with modern data-centric methodologies while embedding a deeper appreciation for the complex variability inherent in neural systems.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown