Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
93 tokens/sec
Gemini 2.5 Pro Premium
49 tokens/sec
GPT-5 Medium
24 tokens/sec
GPT-5 High Premium
32 tokens/sec
GPT-4o
93 tokens/sec
DeepSeek R1 via Azure Premium
75 tokens/sec
GPT OSS 120B via Groq Premium
475 tokens/sec
Kimi K2 via Groq Premium
82 tokens/sec
2000 character limit reached

The foundations of statistical physics: entropy, irreversibility, and inference (2310.06070v1)

Published 9 Oct 2023 in cond-mat.stat-mech

Abstract: Statistical physics aims to describe properties of macroscale systems in terms of distributions of their microscale agents. Its central tool is the maximization of entropy, a variational principle. We review the history of this principle, first considered as a law of nature, more recently as a procedure for inference in model-making. And while equilibria (EQ) have long been grounded in the principle of Maximum Entropy (MaxEnt), until recently no equally foundational generative principle has been known for non-equilibria (NEQ). We review evidence that the variational principle for NEQ is Maximum Caliber. It entails maximizing \textit{path entropies}, not \textit{state entropies}. We also describe the role of entropy in characterizing irreversibility, and describe the relationship between MaxCal and other prominent approaches to NEQ physics, including Stochastic Thermodynamics (ST), Large Deviations Theory (LDT), Macroscopic Fluctuation Theory (MFT), and non-extensive entropies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (112)
  1. Huang, K. Statistical Mechanics (Wiley, New York, 1987).
  2. Hill, T. L. Statistical Mechanics: Principles and Selected Applications (Dover Publications, New York, 1987).
  3. McQuarrie, D. A. Statistical Mechanics (University Science Books, Sausalito, Calif, 2000).
  4. Molecular Driving Forces: Statistical Thermodynamics in Biology, Chemistry, Physics, and Nanoscience, 2nd Edition (Garland Science, Boca Raton, Florida, 2010).
  5. Statistical Mechanics (Academic Press, London San Diego Cambridge, MA Oxford, 2021).
  6. An Introduction to the Maximum Entropy Approach and its Application to Inference Problems in Biology. \JournalTitleHeliyon 4 (2018).
  7. Maximum Information Entropy: A Foundation for Ecological Theory. \JournalTitleTrends in Ecology & Evolution 29, 384–389 (2014).
  8. Information Theory: A Foundation for Complexity Science. \JournalTitleProceedings of the National Academy of Sciences 119, e2119089119 (2022).
  9. Irreversibility and Biased Ensembles in Active Matter: Insights from Stochastic Thermodynamics. \JournalTitleAnnual Review of Condensed Matter Physics 13 (2022).
  10. Path Sampling of Recurrent Neural Networks by Incorporating Known Physics. \JournalTitleNat Commun 13, 7231 (2022).
  11. Principles of Maximum Entropy and Maximum Caliber in Statistical Physics. \JournalTitleRev. Mod. Phys. 85, 1115–1141 (2013).
  12. The Maximum Caliber Variational Principle for Nonequilibria. \JournalTitleAnnual Review of Physical Chemistry 71, 213–238 (2020).
  13. Bagchi, B. Statistical Mechanics for Chemistry and Materials Science (CRC Press, 2018).
  14. Reichl, L. A Modern Course in Statistical Physics (Wiley, 2016).
  15. Kubo, R. Statistical-Mechanical Theory of Irreversible Processes. I. General Theory and Simple Applications to Magnetic and Conduction Problems. \JournalTitleJournal of the Physical Society of Japan 12, 570–586 (1957).
  16. Statistical Thermodynamics and Data ad Infinitum: Conjugate Variables as Forces, and their Statistical Variations (2023). ArXiv:2205.09321 [cond-mat].
  17. Kolmogorov, A. N. Foundations of the Theory of Probability: Second English Edition (Dover Publications, Muneola, New York, 2018).
  18. Inaba, H. The Development of Ensemble Theory. \JournalTitleThe European Physical Journal H 40, 489–526 (2015).
  19. Touchette, H. The Large Deviation Approach to Statistical Mechanics. \JournalTitlePhysics Reports 478, 1–69 (2009).
  20. Emergence and Breaking of Duality Symmetry in Generalized Fundamental Thermodynamic Relations. \JournalTitlePhys. Rev. Lett. 128, 150603 (2022).
  21. Sanov, I. N. On the Probability of Large Deviations of Random Variables. \JournalTitleMat. Sbornik 42, 11–44 (1958).
  22. Statistical Physics, Third Edition, Part 1: Volume 5 (Butterworth-Heinemann, Amsterdam u.a, 1980).
  23. Conditional Limit Theorems under Markov Conditioning. \JournalTitleIEEE Trans. Inform. Theory 33, 788–801 (1987).
  24. Nonequilibrium Microcanonical and Canonical Ensembles and Their Equivalence. \JournalTitlePhys. Rev. Lett. 111, 120601 (2013).
  25. Jaynes, E. T. Information Theory and Statistical Mechanics. \JournalTitlePhys. Rev. 106, 620–630 (1957).
  26. Entropy in Relation to Incomplete Knowledge (Cambridge University Press, 1985).
  27. Axiomatic Derivation of the Principle of Maximum Entropy and the Principle of Minimum Cross-entropy. \JournalTitleIEEE Trans. Inf. Theor. 26, 26–37 (1980).
  28. Bak, P. How Nature Works: The Science of Self-organized Criticality (Copernicus, New York, NY, USA, 1996).
  29. Tsallis, C. Nonextensive Statistical Mechanics: Construction and Physical Interpretation. In Nonextensive Entropy: Interdisciplinary Applications (Oxford University Press, 2004).
  30. Nonadditive Entropies Yield Probability Distributions with Biases not Warranted by the Data. \JournalTitlePhys. Rev. Lett. 111, 180604 (2013).
  31. Tsallis, C. Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems. \JournalTitleEntropy 17, 2853–2861 (2015).
  32. Reply to C. Tsallis’ “Conceptual Inadequacy of the Shore and Johnson Axioms for Wide Classes of Complex Systems”. \JournalTitleEntropy 17, 5043–5046 (2015).
  33. Maximum Entropy Principle in Statistical Inference: Case for Non-Shannonian Entropies. \JournalTitlePhys. Rev. Lett. 122, 120601 (2019).
  34. Caticha, A. Entropy, Information, and the Updating of Probabilities. \JournalTitleEntropy 23, 895 (2021).
  35. A Maximum Entropy Framework for Nonexponential Distributions. \JournalTitleProceedings of the National Academy of Sciences 110, 20380–20385 (2013).
  36. Davis, S. Conditional Maximum Entropy and Superstatistics. \JournalTitleJournal of Physics A: Mathematical and Theoretical 53, 445006 (2020).
  37. Davis, S. et al. A Derivation of the Kappa Distribution in Non-equilibrium, Steady-state Plasmas (2023). ArXiv:2304.13792 [cond-mat, physics:physics].
  38. Davis, S. The q𝑞qitalic_q-canonical Ensemble as a Consequence of Bayesian Superstatistics (2022). ArXiv:2112.11496 [cond-mat].
  39. Superstatistical Distributions from a Maximum Entropy Principle. \JournalTitlePhysical Review E 78, 051101 (2008).
  40. Ramshaw, J. D. Maximum Entropy and Constraints in Composite Systems. \JournalTitlePhysical Review E 105, 024138 (2022).
  41. Maximum Entropy Approach to H𝐻Hitalic_H-theory: Statistical Mechanics of Hierarchical Systems. \JournalTitlePhysical Review E 97, 022104 (2018).
  42. Nonequilibrium Statistical Physics beyond the Ideal Heat Bath Approximation. \JournalTitlePhys. Rev. E 107, 014131 (2023).
  43. Fluctuation Theorems in q𝑞qitalic_q-canonical Ensembles. \JournalTitlePhysica A: Statistical Mechanics and its Applications 563, 125337 (2021).
  44. Bercher, J. F. Tsallis Distribution as a Standard Maximum Entropy Solution with ‘Tail’ Constraint. \JournalTitlePhysics Letters A 372, 5657–5659 (2008).
  45. Variational Principle Underlying Scale Invariant Social Systems. \JournalTitleThe European Physical Journal B 85, 293 (2012).
  46. Jarzynski, C. Equalities and Inequalities: Irreversibility and the Second Law of Thermodynamics at the Nanoscale. \JournalTitleAnnual Review of Condensed Matter Physics 2, 329–351 (2011).
  47. Elements of Information Theory 2nd Edition (Wiley-Interscience, Hoboken, N.J, 2006).
  48. Schnakenberg, J. Network Theory of Microscopic and Macroscopic Behavior of Master Equation Systems. \JournalTitleRev. Mod. Phys. 48, 571–585 (1976).
  49. Thermodynamics of Information. \JournalTitleNature Phys 11, 131–139 (2015).
  50. Amari, S.-i. Information Geometry and Its Applications. Applied Mathematical Sciences (Springer Japan, 2016).
  51. Crooks, G. E. Measuring Thermodynamic Length. \JournalTitlePhys. Rev. Lett. 99, 100602 (2007).
  52. Time-information Uncertainty Relations in Thermodynamics. \JournalTitleNat Phys 1–5 (2020).
  53. Stochastic Time Evolution, Information Geometry, and the Cramer-Rao Bound. \JournalTitlePhys. Rev. X 10, 021056 (2020).
  54. Random Perturbations of Dynamical Systems. Grundlehren der mathematischen Wissenschaften (Springer-Verlag, New York, 1984).
  55. Strogatz, S. H. Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry, and Engineering (CRC Press, 2018).
  56. Gardiner, C. Stochastic Methods: A Handbook for the Natural and Social Sciences. Springer Series in Synergetics (Springer-Verlag, Berlin Heidelberg, 2009).
  57. Balls, Cups, and Quasi-potentials: Quantifying Stability in Stochastic Systems. \JournalTitleEcology 15–1047.1 (2015).
  58. Nonequilibrium Physics in Biology. \JournalTitleRev. Mod. Phys. 91, 045004 (2019).
  59. How Do Cells Adapt? Stories Told in Landscapes. \JournalTitleAnnual Review of Chemical and Biomolecular Engineering 11, 155–182 (2020).
  60. Kinematic Basis of Emergent Energetics of Complex Dynamics. \JournalTitleEPL 131, 50002 (2020).
  61. Potentials of Continuous Markov Processes and Random Perturbations. \JournalTitleJ. Phys. A: Math. Theor. 54, 195001 (2021).
  62. Seifert, U. From Stochastic Thermodynamics to Thermodynamic Inference. \JournalTitleAnnual Review of Condensed Matter Physics 10, 171–192 (2019).
  63. Stochastic Thermodynamics: An Introduction (Princeton University Press, 2021).
  64. Chemical Thermodynamics (Wiley, 1962).
  65. Jarzynski, C. Hamiltonian Derivation of a Detailed Fluctuation Theorem. \JournalTitleJ. Stat. Phys. 98, 77–102 (2000).
  66. Mathematical Theory of Nonequilibrium Steady States: On the Frontier of Probability and Dynamical Systems (Springer, Berlin ; New York, 2004).
  67. Bivectorial Nonequilibrium Thermodynamics: Cycle Affinity, Vorticity Potential, and Onsager’s Principle. \JournalTitleJ Stat Phys 182, 46 (2021).
  68. Ge, H. Extended Forms of the Second Law for General Time-dependent Stochastic Processes. \JournalTitlePhys. Rev. E 80, 021137 (2009).
  69. Three Detailed Fluctuation Theorems. \JournalTitlePhys. Rev. Lett. 104 (2010).
  70. Crooks, G. E. Path-ensemble Averages in Systems Driven far from Equilibrium. \JournalTitlePhys. Rev. E 61, 2361–2366 (2000).
  71. Unified Formalism for Entropy Production and Fluctuation Relations. \JournalTitlePhys. Rev. E 101, 022129 (2020).
  72. Time-translational Symmetry in Statistical Dynamics Dictates Einstein Relation, Green-Kubo Formula, and their Generalizations. \JournalTitlePhys. Rev. E 107, 024110 (2023).
  73. Graham, R. Covariant Stochastic Calculus in the Sense of Itô. \JournalTitlePhysics Letters A 109, 209–212 (1985).
  74. Ao, P. Potential in Stochastic Differential Equations: Novel Construction. \JournalTitleJ. Phys. A: Math. Gen. 37, L25–L30 (2004).
  75. Potential Landscape and Flux Framework of Nonequilibrium Networks: Robustness, Dissipation, and Coherence of Biochemical Oscillations. \JournalTitlePNAS 105, 12271–12276 (2008).
  76. Stochastic Thermodynamics in a Non-markovian Dynamical System. \JournalTitlePhysical Review E 105, 064124 (2022).
  77. Esposito, M. Stochastic Thermodynamics under Coarse Graining. \JournalTitlePhys. Rev. E 85, 041125 (2012).
  78. Sekimoto, K. Stochastic Energetics (Springer, Heidelberg ; New York, 2010).
  79. Seifert, U. Stochastic Thermodynamics: From Principles to the Cost of Precision. \JournalTitlePhysica A: Statistical Mechanics and its Applications 504, 176–191 (2018).
  80. Thermodynamic Uncertainty Relation for Biomolecular Processes. \JournalTitlePhys. Rev. Lett. 114, 158101 (2015).
  81. Thermodynamic Uncertainty Relations Constrain Non-equilibrium Fluctuations. \JournalTitleNat Phys 16, 15–20 (2020).
  82. Local Detailed Balance across Scales: From Diffusions to Jump processes and beyond. \JournalTitlePhysical Review E 103, 042114 (2021).
  83. Violation of Local Detailed Balance Despite a Clear Time-Scale Separation (2022). ArXiv:2111.14734 [cond-mat, physics:physics].
  84. Wasnik, V. Revisiting Multiple Thermal Reservoir Stochastic Thermodynamics (2023). ArXiv:2303.14949 [cond-mat].
  85. Variational Path Optimization and Upper and Lower Bounds to Free Energy Changes via Finite Time Minimization of External Work. \JournalTitleThe Journal of Chemical Physics 97, 1599–1601 (1992).
  86. A Finite-time Variational Method for Determining Optimal Paths and Obtaining Bounds on Free Energy Changes from Computer Simulations. \JournalTitleThe Journal of Chemical Physics 99, 6856–6864 (1993).
  87. Jarzynski, C. Nonequilibrium Equality for Free Energy Differences. \JournalTitlePhys. Rev. Lett. 78, 2690–2693 (1997).
  88. Seifert, U. First and Second Law of Thermodynamics at Strong Coupling. \JournalTitlePhys. Rev. Lett. 116, 020601 (2016).
  89. Jarzynski, C. Stochastic and Macroscopic Thermodynamics of Strongly Coupled Systems. \JournalTitlePhys. Rev. X 7, 011008 (2017).
  90. Entropy Production and Time Asymmetry in the Presence of Strong Interactions. \JournalTitlePhys. Rev. E 95, 062123 (2017).
  91. Crooks, G. E. Nonequilibrium Measurements of Free Energy Differences for Microscopically Reversible Markovian Systems. \JournalTitleJ. Stat. Phys. 90, 1481–1487 (1998).
  92. Fluctuation Relations: A Pedagogical Overview. \JournalTitleNonequilibrium Statistical Physics of Small Systems: Fluctuation Relations and Beyond 3–56 (2013).
  93. Macroscopic Fluctuation Theory. \JournalTitleRev. Mod. Phys. 87, 593–636 (2015).
  94. Principle of Minimum Entropy Production. \JournalTitlePhys. Rev. 96, 250–255 (1954).
  95. Non-Equilibrium Thermodynamics (Dover Publications, New York, 2011).
  96. Dewar, R. C. Maximum Entropy Production as an Inference Algorithm that Translates Physical Assumptions into Macroscopic Predictions: Don’t Shoot the Messenger. \JournalTitleEntropy 11, 931–944 (2009).
  97. Martyushev, L. M. Maximum Entropy Production Principle: History and Current Status. \JournalTitlePhys. Usp. 64, 558–583 (2021).
  98. Stochastic Dynamics and Non-equilibrium Thermodynamics of a Bistable Chemical System: the Schlögl Model Revisited. \JournalTitleJournal of The Royal Society Interface 6, 925–940 (2008).
  99. Jaynes, E. T. The Minimum Entropy Production Principle. \JournalTitleAnnual Review of Physical Chemistry 31, 579–601 (1980).
  100. Communication: Maximum Caliber is a General Variational Principle for Nonequilibrium Statistical Mechanics. \JournalTitleThe Journal of Chemical Physics 143, 051104 (2015).
  101. Markov Processes Follow from the Principle of Maximum Caliber. \JournalTitleThe Journal of Chemical Physics 136, 064108 (2012).
  102. A Derivation of the Master Equation from Path Entropy Maximization. \JournalTitleThe Journal of Chemical Physics 137, 074103 (2012).
  103. Analytical Mechanics in Stochastic Dynamics: Most Probable Path, Large-deviation Rate Function and Hamilton-Jacobi Equation. \JournalTitleInt. J. Mod. Phys. B 26, 1230012 (2012).
  104. Newtonian Dynamics from the Principle of Maximum Caliber. \JournalTitleFoundations of Physics 44, 923–931 (2014).
  105. General, I. J. Principle of Maximum Caliber and Quantum Physics. \JournalTitlePhysical Review E 98, 012110 (2018).
  106. E. Schrödinger’s 1931 paper “On the Reversal of the Laws of Nature” [“Über dieUmkehrung der Naturgesetze”,Sitzungsberichte der preussischen Akademie der Wissenschaften, physikalische mathematische Klasse, 8 N9 144-153]. \JournalTitlearXiv:2105.12617 [cond-mat, physics:physics, physics:quant-ph] (2021).
  107. A Maximum Caliber Approach for Continuum Path Ensembles. \JournalTitleThe European Physical Journal B 94, 188 (2021).
  108. Maximum Entropy Principle and the Higgs Boson Mass. \JournalTitlePhysica A: Statistical Mechanics and its Applications 420, 1–7 (2015).
  109. Sattin, F. Bayesian Approach to Superstatistics. \JournalTitleThe European Physical Journal B - Condensed Matter and Complex Systems 49, 219–224 (2006).
  110. Dixit, P. D. A Maximum Entropy Thermodynamics of Small Systems. \JournalTitleThe Journal of Chemical Physics 138, 184111 (2013).
  111. Minimal Constraints for Maximum Caliber Analysis of Dissipative Steady-state Systems. \JournalTitlePhys. Rev. E 100, 010105 (2019).
  112. Caticha, A. Entropic Inference: Some Pitfalls and Paradoxes We Can Avoid. \JournalTitleAIP Conference Proceedings 1553, 200–211 (2013).

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets