Information thermodynamics: from physics to neuroscience (2409.17599v1)

Abstract: This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in neuroscience have been treated separately, in contrast to physics approaches, where the relationship of entropy production with heat is a central idea. It is argued here that also in neural systems information and energy can be considered within the same theoretical framework. Starting from basic ideas of thermodynamics and information theory on a classic Brownian particle, it is shown how noisy neural networks can infer its probabilistic motion. The decoding of the particle motion by neurons is performed with some accuracy and it has some energy cost, and both can be determined using information thermodynamics. In a similar fashion, we also discuss how neural networks in the brain can learn the particle velocity, and maintain that information in the weights of plastic synapses from a physical point of view. Generally, it is shown how the framework of stochastic and information thermodynamics can be used practically to study neural inference, learning, and information storing.

• The paper establishes a unified framework that integrates information thermodynamics with neural computation, linking energy dissipation to information processing.
• It quantifies the energetic cost of decoding external stimuli, demonstrating that neural accuracy is closely tied to entropy production and mutual information.
• The research employs a stochastic model of synaptic plasticity to show how energy dissipation underpins memory storage and synaptic state transitions.

Integration of Information Thermodynamics into Neuroscience

The exploration of information thermodynamics as a bridge between physics and theoretical neuroscience presents a compelling perspective on the intersection of information, energy, and biological processes. The paper by Jan Karbowski posits that concepts from information thermodynamics, originally developed in nonequilibrium statistical physics, can be effectively applied to theoretical neuroscience to understand neural information processing, learning, and memory storage. This approach represents a paradigm shift from treating energy and information as distinct entities in neuroscience to considering them within a unified theoretical framework.

Overview and Methodology

The paper adopts tools from stochastic thermodynamics, typically applied to microscopic systems, to model neural systems characterized by stochastic fluctuations. By extending thermodynamic concepts to noisy neural networks, the research investigates how neurons can decode probabilistic motions, such as those of a Brownian particle, with a consideration of both accuracy and energetic costs. The methodology involves using stochastic equations, akin to Langevin equations in physics, to describe neural activities and synaptic plasticity, encapsulating the energetic dynamics of neural computations.

Key Findings

1. Unified Framework for Information and Energy: The research establishes that the principles of information thermodynamics allow for a combined view of information and energy in neural computations, echoing ideas in thermodynamic theories where entropy production is correlated with heat dissipation.
2. Energetic Considerations in Neural Decoding: The decoding accuracy of external stimuli by neural populations, such as estimating the velocity of observed stimuli, is shown to involve significant energy expenditure, quantified via entropy production. The mutual information between stimulus and neural response underscores the relationship between energy use and information gain.
3. Synaptic Plasticity and Memory: The research utilizes a stochastic model of the BCM rule to demonstrate that synapses can store information about external stimuli as memories. The model shows that synaptic weights exhibit bistability, with energy costs associated with transitions between these states linked to the amount of learned information. Information storage and retention are proportional to energy dissipation over time.

Practical and Theoretical Implications

Practical Insights

The paper's integration of thermodynamic principles into neuroscience could propel the development of more energy-efficient neural computations, both artificial and biological. Understanding the energetic constraints can be vital for neuromorphic engineering, where synaptic plasticity and information processing are replicated in silicon-based architectures.

Theoretical Contributions

The application of stochastic thermodynamics in neuroscience offers a unified framework that captures the complex interplay of stochasticity, energy, and information in biological systems. This framework may inspire new models that align neural network theories with fundamental physical laws, enhancing their biological plausibility.

Speculative Future Developments

Looking forward, principles from information thermodynamics could aid in unraveling the energetic constraints that shape brain evolution and functionality. As computational models grow more intricate, incorporating stochastic thermodynamic insights might refine our understanding of brain efficiency and the mechanisms underlying persistent neural coding and memory formation.

Conclusion

Jan Karbowski's paper introduces a transformative approach to neural computation through the lens of information thermodynamics, highlighting the intricate connection between energy use and information processing in neural networks. The propositions set the stage for future interdisciplinary research that could bridge theoretical neuroscience with physical sciences, offering a robust framework to explore the brain's efficiency in processing and storing information.