Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exact minimax entropy models of large-scale neuronal activity (2402.00007v1)

Published 18 Dec 2023 in physics.bio-ph, cond-mat.dis-nn, cond-mat.stat-mech, and q-bio.NC

Abstract: In the brain, fine-scale correlations combine to produce macroscopic patterns of activity. However, as experiments record from larger and larger populations, we approach a fundamental bottleneck: the number of correlations one would like to include in a model grows larger than the available data. In this undersampled regime, one must focus on a sparse subset of correlations; the optimal choice contains the maximum information about patterns of activity or, equivalently, minimizes the entropy of the inferred maximum entropy model. Applying this ``minimax entropy" principle is generally intractable, but here we present an exact and scalable solution for pairwise correlations that combine to form a tree (a network without loops). Applying our method to over one thousand neurons in the mouse hippocampus, we find that the optimal tree of correlations reduces our uncertainty about the population activity by 14% (over 50 times more than a random tree). Despite containing only 0.1% of all pairwise correlations, this minimax entropy model accurately predicts the observed large-scale synchrony in neural activity and becomes even more accurate as the population grows. The inferred Ising model is almost entirely ferromagnetic (with positive interactions) and exhibits signatures of thermodynamic criticality. These results suggest that a sparse backbone of excitatory interactions may play an important role in driving collective neuronal activity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (44)
  1. Norbert Wiener, Nonlinear Problems in Random Theory (MIT Press, Cambridge MA, 1958).
  2. Leon N Cooper, “A possible organization of animal memory and learning,” in Collective Properties of Physical Systems: Proceedings of Nobel Symposium 24, edited by B Lundqvist and S Lundqvist (Academic Press, New York, 1973) pp. 252–264.
  3. William A Little, “The existence of persistent states in the brain,” Math. Biosci. 19, 101–120 (1974).
  4. John J Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. U.S.A. 79, 2554–2558 (1982).
  5. Daniel J Amit, Modeling Brain Function: The World of Attractor Neural Networks (Cambridge University Press, 1989).
  6. Ronen Segev, Joe Goodhouse, Jason Puchalla,  and Michael J Berry II, “Recording spikes from a large fraction of the ganglion cells in a retinal patch,” Nat. Neurosci. 7, 1155–1162 (2004).
  7. AM Litke, N Bezayiff, EJ Chichilnisky, W Cunningham, W Dabrowski, AA Grillo, M Grivich, P Grybos, P Hottowy, S Kachiguine, et al., “What does the eye tell the brain?: Development of a system for the large-scale recording of retinal output activity,” IEEE Trans. Nucl. Sci. 51, 1434–1440 (2004).
  8. Jason E Chung, Hannah R Joo, Jiang Lan Fan, Daniel F Liu, Alex H Barnett, Supin Chen, Charlotte Geaghan-Breiner, Mattias P Karlsson, Magnus Karlsson, Kye Y Lee, et al., “High-density, long-lasting, and multi-region electrophysiological recordings using polymer electrode arrays,” Neuron 101, 21–31 (2019).
  9. Daniel A Dombeck, Christopher D Harvey, Lin Tian, Loren L Looger,  and David W Tank, “Functional imaging of hippocampal place cells at cellular resolution during virtual navigation,” Nat. Neurosci. 13, 1433–1440 (2010).
  10. Lin Tian, Jasper Akerboom, Eric R Schreiter,  and Loren L Looger, “Neural activity imaging with genetically encoded calcium indicators,” Prog. Brain Res. 196, 79–94 (2012).
  11. Jeffrey Demas, Jason Manley, Frank Tejera, Kevin Barber, Hyewon Kim, Francisca Martínez Traub, Brandon Chen,  and Alipasha Vaziri, “High-speed, cortex-wide volumetric recording of neuroactivity at cellular resolution using light beads microscopy,” Nat. Methods 18, 1103–1111 (2021).
  12. Nicholas A Steinmetz, Cagatay Aydin, Anna Lebedeva, Michael Okun, Marius Pachitariu, Marius Bauza, Maxime Beau, Jai Bhagat, Claudia Böhm, Martijn Broux, et al., “Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings,” Science 372, eabf4588 (2021).
  13. Edwin T Jaynes, “Information theory and statistical mechanics,” Phys. Rev. 106, 620 (1957).
  14. Elad Schneidman, Michael J Berry II, Ronen Segev,  and William Bialek, “Weak pairwise correlations imply strongly correlated network states in a neural population,” Nature 440, 1007 (2006).
  15. Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank,  and William Bialek, “Collective behavior of place and non-place neurons in the hippocampal network,” Neuron 96, 1178–1191 (2017).
  16. Leenoy Meshulam, Jeffrey L Gauthier, Carlos D Brody, David W Tank,  and William Bialek, “Successes and failures of simplified models for a network of real neurons,” Preprint: arxiv.org/abs/2112.14735  (2021).
  17. Gašper Tkačik, Thierry Mora, Olivier Marre, Dario Amodei, Stephanie E Palmer, Michael J Berry II,  and William Bialek, “Thermodynamics and signatures of criticality in a network of neurons,” Proc. Natl. Acad. Sci. U.S.A. 112, 11508–11513 (2015).
  18. Timothy R Lezon, Jayanth R Banavar, Marek Cieplak, Amos Maritan,  and Nina V Fedoroff, “Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns,” Proc. Natl. Acad. Sci. U.S.A. 103, 19033–19038 (2006).
  19. Martin Weigt, Robert A White, Hendrik Szurmant, James A Hoch,  and Terence Hwa, “Identification of direct residue contacts in protein–protein interaction by message passing,” Proc. Natl. Acad. Sci. U.S.A. 106, 67–72 (2009).
  20. Debora S Marks, Lucy J Colwell, Robert Sheridan, Thomas A Hopf, Andrea Pagnani, Riccardo Zecchina,  and Chris Sander, “Protein 3D structure computed from evolutionary sequence variation,” PLoS One 6, e28766 (2011).
  21. Alan Lapedes, Bertrand Giraud,  and Christopher Jarzynski, “Using sequence alignments to predict protein structure and stability with high accuracy,” arXiv preprint arXiv:1207.2484  (2012).
  22. William Bialek, Andrea Cavagna, Irene Giardina, Thierry Mora, Edmondo Silvestri, Massimiliano Viale,  and Aleksandra M Walczak, “Statistical mechanics for natural flocks of birds,” Proc. Natl. Acad. Sci. U.S.A. 109, 4786–4791 (2012).
  23. William P Russ, Matteo Figliuzzi, Christian Stocker, Pierre Barrat-Charlaix, Michael Socolich, Peter Kast, Donald Hilvert, Remi Monasson, Simona Cocco, Martin Weigt, et al., “An evolution-based model for designing chorismate mutase enzymes,” Science 369, 440–445 (2020).
  24. Christopher W Lynn, Lia Papadopoulos, Daniel D Lee,  and Danielle S Bassett, “Surges of collective human activity emerge from simple pairwise correlations,” Phys. Rev. X 9, 011022 (2019).
  25. Song Chun Zhu, Ying Nian Wu,  and David Mumford, “Minimax entropy principle and its application to texture modeling,” Neural Comput. 9, 1627–1660 (1997).
  26. Jason S. Prentice, Olivier Marre, Mark L. Ioffe, Adrianna R. Loback, Gašper Tkačik,  and Michael J. Berry II, “Error–robust modes of the retinal population code,” PLoS Comput. Biol. 12, e1005148 (2016).
  27. CKCN Chow and Cong Liu, “Approximating discrete probability distributions with dependence trees,” IEEE Trans. Inf. Theory 14, 462–467 (1968).
  28. Christopher W Lynn, Qiwei Yu, Rich Pang, William Bialek,  and Stephanie E Palmer, “Exactly solvable statistical physics models for large neuronal populations,” Preprint: arxiv.org/abs/2310.10860  (2023).
  29. Jeffrey L Gauthier and David W Tank, “A dedicated population for reward coding in the hippocampus,” Neuron 99, 179–193 (2018).
  30. Claude E Shannon, “A mathematical theory of communication,” Bell Syst. Tech. J. 27, 379–423 (1948).
  31. Elad Schneidman, Susanne Still, Michael J Berry II,  and William Bialek, “Network information and connected correlations,” Phys. Rev. Lett. 91, 238701 (2003).
  32. Cristopher Moore and Stephan Mertens, The Nature of Computation (OUP Oxford, 2011).
  33. Sen Song, Per Jesper Sjöström, Markus Reigl, Sacha Nelson,  and Dmitri B Chklovskii, “Highly nonrandom features of synaptic connectivity in local cortical circuits,” PLoS Biol. 3, e68 (2005).
  34. Christopher W Lynn and Danielle S Bassett, “The physics of brain network structure, function and control,” Nat. Rev. Phys. 1, 318 (2019).
  35. Albert Lin, Runzhe Yang, Sven Dorkenwald, Arie Matsliah, Amy R Sterling, Philipp Schlegel, Szi-chieh Yu, Claire E McKellar, Marta Costa, Katharina Eichler, et al., “Network statistics of the whole-brain connectome of Drosophila,” Preprint: biorxiv.org/content/10.1101/2023.07.29.551086  (2023).
  36. Réka Albert and Albert-László Barabási, “Statistical mechanics of complex networks,” Rev. Mod. Phys. 74, 47 (2002).
  37. Julia J Harris, Renaud Jolivet,  and David Attwell, “Synaptic energy use and supply,” Neuron 75, 762–777 (2012).
  38. Simon B Laughlin and Terrence J Sejnowski, “Communication in neuronal networks,” Science 301, 1870–1874 (2003).
  39. Duncan J Watts and Steven H Strogatz, “Collective dynamics of ‘small–world’ networks,” Nature 393, 440–442 (1998).
  40. Stefan Schnabel, Daniel T Seaton, David P Landau,  and Michael Bachmann, “Microcanonical entropy inflection points: Key to systematic understanding of transitions in finite systems,” Phys. Rev. E 84, 011127 (2011).
  41. Rudolf Peierls, “On Ising’s model of ferromagnetism,” in Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 32 (Cambridge University Press, 1936) pp. 477–481.
  42. Rodney J Baxter, Exactly solved models in statistical mechanics (Elsevier, 2016).
  43. Christopher W Lynn, Caroline M Holmes,  and Stephanie E Palmer, “Heavy–tailed neuronal connectivity arises from Hebbian self–organization,” Preprint: biorxiv.org/content/10.1101/2022.05.30.494086  (2022).
  44. Steven P Strong, Roland Koberle, Rob R de Ruyter van Steveninck,  and William Bialek, “Entropy and information in neural spike trains,” Phys. Rev. Lett. 80, 197 (1998).
Citations (1)
View on Semantic Scholar

Summary

  • The paper develops an exact minimax entropy framework to capture the most informative pairwise correlations in neuronal networks using tree structures.
  • It shows that minimal pairwise correlations can reduce uncertainty by 14% in a dataset of over 1,480 hippocampal neurons.
  • The approach significantly outperforms random and proximity-based models in predicting neuronal synchrony, emphasizing its scalability and practical application in neuroscience.

Overview of "Exact Minimax Entropy Models of Large-Scale Neuronal Activity"

The paper "Exact Minimax Entropy Models of Large-Scale Neuronal Activity" addresses the complex problem of modeling large-scale neuronal networks. The authors focus on developing a framework based on the minimax entropy principle to infer the most informative pairwise correlations in the context of neuronal activity. As datasets grow larger, this approach is pivotal in bypassing the combinatorial explosion associated with considering all possible interactions, which quickly becomes computationally infeasible.

Central to this work is the application of minimax entropy models that leverage tree structures of neuronal interactions. The choice of tree structures, which lack loops, simplifies the problem significantly and allows for an efficient solution that reduces the entropy (or uncertainty) of the probabilistic model. Trees are chosen via an exact, scalable method, solving a minimum spanning tree problem to identify the most informative correlations.

Key Contributions

  1. Minimax Entropy Framework: The paper introduces an exact and scalable solution for modeling large-scale neuronal activity by focusing on minimizing entropy through the selection of an optimal tree of pairwise correlations. This approach is grounded in the minimax entropy principle, offering a method for prioritizing the most informative features.
  2. Application to Neuronal Data: The framework is applied to a dataset comprising over 1,480 neurons from the mouse hippocampus. The results indicate that the optimal tree captures significantly more information than expected by chance, reducing uncertainty by 14% with just 0.1% of pairwise correlations.
  3. Typical Ising Model Features and Criticality: The inferred model reveals predominantly ferromagnetic interactions, with neurons exhibiting excitatory (positive) interactions. The paper suggests that these topological features contribute to collective behavior, potentially implicating the model's criticality properties as relevant to understanding neuronal activity at a large scale.
  4. Empirical Evaluation: The paper provides detailed empirical results, contrasting the minimax model against models derived from random or minimal physical distance trees. The minimax model shows a marked improvement in predicting observed neuronal synchrony and patterns of large-scale activity.
  5. Scalability and Applicability: The methodology demonstrates scalability and enhances its utility for even larger neuronal systems. As population size increases, the minimax model maintains accuracy, revealing superextensive growth in information capture, indicating its robustness and practical benefits.

Implications and Future Directions

The theoretical contributions and empirical validations presented suggest that the minimax entropy principle, when applied using tree-based structures, can serve as a robust foundation for modeling interactions in complex and large-scale systems like neural networks. This approach not only draws from statistical mechanics but also provides insights that might contribute to better understanding information processing in the brain.

In future work, extending the model to encompass networks beyond trees could address a broader scope of interactions seen in real-world networks, including loopy graphs. Additionally, applying this framework to other biological systems could provide insights benefiting a wide array of fields dependent on diagnosing collective behavior from limited experimental observations.

In conclusion, this paper successfully bridges theoretical concepts from statistical mechanics and graph theory with practical applications in neuroscience, offering an innovative pathway for understanding and modeling complex systems reliably and efficiently.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown