Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

An Exact Theory of Causal Emergence for Linear Stochastic Iteration Systems (2405.09207v1)

Published 15 May 2024 in cs.IT, cs.SY, eess.SY, and math.IT

Abstract: After coarse-graining a complex system, the dynamics of its macro-state may exhibit more pronounced causal effects than those of its micro-state. This phenomenon, known as causal emergence, is quantified by the indicator of effective information. However, two challenges confront this theory: the absence of well-developed frameworks in continuous stochastic dynamical systems and the reliance on coarse-graining methodologies. In this study, we introduce an exact theoretic framework for causal emergence within linear stochastic iteration systems featuring continuous state spaces and Gaussian noise. Building upon this foundation, we derive an analytical expression for effective information across general dynamics and identify optimal linear coarse-graining strategies that maximize the degree of causal emergence when the dimension averaged uncertainty eliminated by coarse-graining has an upper bound. Our investigation reveals that the maximal causal emergence and the optimal coarse-graining methods are primarily determined by the principal eigenvalues and eigenvectors of the dynamic system's parameter matrix, with the latter not being unique. To validate our propositions, we apply our analytical models to three simplified physical systems, comparing the outcomes with numerical simulations, and consistently achieve congruent results.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (61)
  1. J. H. Holland, Emergence: From chaos to order. OUP Oxford, 2000.
  2. Penguin, 2018.
  3. R. Li, L. Dong, J. Zhang, X. Wang, W.-X. Wang, Z. Di, and H. E. Stanley, “Simple spatial scaling rules behind complex cities,” Nature communications, vol. 8, no. 1, p. 1841, 2017.
  4. L. Dong, Z. Huang, J. Zhang, and Y. Liu, “Understanding the mesoscopic scaling patterns within cities,” Scientific reports, vol. 10, no. 1, p. 21201, 2020.
  5. J. Zhang, C. P. Kempes, M. J. Hamilton, R. Tao, and G. B. West, “Scaling laws and a general theory for the growth of public companies,” arXiv preprint arXiv:2109.10379, 2021.
  6. Y. Zhang, K. Liu, and W. Gui, “Bayesian and e-bayesian estimations of bathtub-shaped distribution under generalized type-i hybrid censoring,” Entropy, vol. 23, no. 8, p. 934, 2021.
  7. J. Xu, X. Chen, L. Wen, and J. Zhang, “Company scaling and its deviations: New indicators for enterprise evaluation and bankruptcy prediction,” Plos one, vol. 18, no. 10, p. e0287105, 2023.
  8. R. T. Wicks, S. C. Chapman, and R. Dendy, “Mutual information as a tool for identifying phase transitions in dynamical complex systems with limited data,” Physical Review E, vol. 75, no. 5, p. 051125, 2007.
  9. C. Hartman and B. Benes, “Autonomous boids,” Computer Animation and Virtual Worlds, vol. 17, no. 3-4, pp. 199–206, 2006.
  10. D. Liu, W. Liu, X. Yuan, and Y. Jiang, “Conscious and unconscious processing of ensemble statistics oppositely modulate perceptual decision-making.,” American Psychologist, vol. 78, no. 3, p. 346, 2023.
  11. L. Wang and Y. Jiang, “Action observation network: domain-specific or domain-general?,” Trends in Cognitive Sciences, 2023.
  12. R. Wang, V. Gates, Y. Shen, P. Tino, and Z. Kourtzi, “Flexible structure learning under uncertainty,” Available at SSRN 4014051.
  13. S. Jingnan, J. He, and X. Gao, “Neurofeedback training of control network improves ssvep-based bci performance in children,” 2021.
  14. O. Sporns, J. Faskowitz, A. S. Teixeira, S. A. Cutts, and R. F. Betzel, “Dynamic expression of brain functional systems disclosed by fine-scale analysis of edge time series,” Network Neuroscience, vol. 5, no. 2, pp. 405–433, 2021.
  15. T. F. Varley, Uncovering Higher-Order Structures in Complex Systems with Multivariate Information Theory. Indiana University, 2023.
  16. Z. Zhao, Y. Zhou, B. Liu, J. He, J. Zhao, Y. Cai, J. Fan, X. Li, Z. Wang, Z. Lu, et al., “Two-photon synthetic aperture microscopy for minimally invasive fast 3d imaging of native subcellular behaviors in deep tissue,” Cell, vol. 186, no. 11, pp. 2475–2491, 2023.
  17. T. Dong, J. He, S. Wang, L. Wang, Y. Cheng, and Y. Zhong, “Inability to activate rac1-dependent forgetting contributes to behavioral inflexibility in mutants of multiple autism-risk genes,” Proceedings of the National Academy of Sciences, vol. 113, no. 27, pp. 7644–7649, 2016.
  18. B. Yuan, J. Zhang, A. Lyu, J. Wu, Z. Wang, M. Yang, K. Liu, M. Mou, and P. Cui, “Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies,” Entropy, vol. 26, no. 2, p. 108, 2024.
  19. J. T. Oden, T. Belytschko, I. Babuska, and T. Hughes, “Research directions in computational mechanics,” Computer Methods in Applied Mechanics and Engineering, vol. 192, no. 7-8, pp. 913–922, 2003.
  20. A. K. Seth, “Measuring emergence via nonlinear granger causality.,” in alife, vol. 2008, pp. 545–552, 2008.
  21. F. E. Rosas, P. A. M. Mediano, H. J. Jensen, A. K. Seth, A. B. Barrett, R. L. Carhart-Harris, and D. Bor, “Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data,” PLOS Computational Biology, vol. 16, p. e1008289, Dec 2020.
  22. E. P. Hoel, L. Albantakis, and G. Tononi, “Quantifying causal emergence shows that macro can beat micro,” Proceedings of the National Academy of Sciences, vol. 110, no. 49, pp. 19790–19795, 2013.
  23. E. P. Hoel, L. Albantakis, W. Marshall, and G. Tononi, “Can the macro beat the micro? integrated information across spatiotemporal scales,” Neuroscience of Consciousness, vol. 2016, no. 1, p. niw012, 2016.
  24. F. E. Rosas, B. C. Geiger, A. I. Luppi, A. K. Seth, D. Polani, M. Gastpar, and P. A. Mediano, “Software in the natural world: A computational approach to emergence in complex multi-level systems,” arXiv preprint arXiv:2402.09090, 2024.
  25. E. P. Hoel, “When the map is better than the territory,” Entropy, vol. 19, no. 5, p. 188, 2017.
  26. P. L. Williams and R. D. Beer, “Nonnegative decomposition of multivariate information,” arXiv preprint arXiv:1004.2515, 2010.
  27. B. Rassouli, F. E. Rosas, and D. Gündüz, “Data disclosure under perfect sample privacy,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 2012–2025, 2019.
  28. K. Liu, X. Lü, F. Gao, and J. Zhang, “Expectation-maximizing network reconstruction and most applicable network types based on binary time series data,” Physica D: Nonlinear Phenomena, vol. 454, p. 133834, 2023.
  29. J. Zhang, R. Tao, and B. Yuan, “Dynamical reversibility and a new theory of causal emergence,” arXiv preprint arXiv:2402.15054, 2024.
  30. P. Chvykov and E. Hoel, “Causal geometry,” Entropy, vol. 23, no. 1, p. 24, 2020.
  31. M. K. Transtrum, B. B. Machta, K. S. Brown, B. C. Daniels, C. R. Myers, and J. P. Sethna, “Perspective: Sloppiness and emergent theories in physics, biology, and beyond,” The Journal of chemical physics, vol. 143, no. 1, 2015.
  32. B. Klein and E. Hoel, “The emergence of informative higher scales in complex networks,” Complexity, vol. 2020, pp. 1–12, 2020.
  33. R. Comolatti and E. Hoel, “Causal emergence is widespread across measures of causation,” arXiv preprint arXiv:2202.01854, 2022.
  34. K. Liu and Y. Zhang, “The e-bayesian estimation for lomax distribution based on generalized type-i hybrid censoring scheme,” Mathematical Problems in Engineering, vol. 2021, pp. 1–19, 2021.
  35. M. Yang, Z. Wang, K. Liu, Y. Rong, B. Yuan, and J. Zhang, “Finding emergence in data: causal emergence inspired dynamics learning,” arXiv preprint arXiv:2308.09952, 2023.
  36. L. P. Kadanoff, “Scaling laws for ising models near t c,” Physics Physique Fizika, vol. 2, no. 6, p. 263, 1966.
  37. C. d. Villemagne and R. E. Skelton, “Model reductions using a projection formulation,” International Journal of Control, vol. 46, no. 6, pp. 2141–2169, 1987.
  38. D. L. Boley, “Krylov space methods on state-space control models,” Circuits, Systems and Signal Processing, vol. 13, pp. 733–758, 1994.
  39. K. Gallivan, E. Grimme, and P. Van Dooren, “Asymptotic waveform evaluation via a lanczos method,” Applied Mathematics Letters, vol. 7, no. 5, pp. 75–80, 1994.
  40. S. Gugercin, “An iterative svd-krylov based method for model reduction of large-scale dynamical systems,” Linear Algebra and its Applications, vol. 428, no. 8-9, pp. 1964–1986, 2008.
  41. A. C. Antoulas, “An overview of approximation methods for large-scale dynamical systems,” Annual reviews in Control, vol. 29, no. 2, pp. 181–190, 2005.
  42. H.-Y. Hu, S.-H. Li, L. Wang, and Y.-Z. You, “Machine learning holographic mapping by neural network renormalization group,” Physical Review Research, p. 023369, 2020.
  43. J. Zhang and K. Liu, “Neural information squeezer for causal emergence,” Entropy, vol. 25, no. 1, p. 26, 2022.
  44. B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio, “Toward causal representation learning,” Proceedings of the IEEE, vol. 109, no. 5, pp. 612–634, 2021.
  45. K. A. Murphy and D. S. Bassett, “Information decomposition to identify relevant variation in complex systems with machine learning,” arXiv preprint arXiv:2307.04755, 2023.
  46. K. A. Murphy and D. S. Bassett, “Information decomposition in complex systems via machine learning,” Proceedings of the National Academy of Sciences, vol. 121, no. 13, p. e2312988121, 2024.
  47. W. Dunsmuir and E. J. Hannan, “Vector linear time series models,” Advances in Applied Probability, vol. 8, no. 2, pp. 339–364, 1976.
  48. E. J. Hannan and L. Kavalieris, “Multivariate linear time series models,” Advances in Applied Probability, vol. 16, no. 3, pp. 492–561, 1984.
  49. J. C. A. Barata and M. S. Hussein, “The moore–penrose pseudoinverse: A tutorial review of the theory,” Brazilian Journal of Physics, vol. 42, pp. 146–165, 2012.
  50. R. A. Horn and C. R. Johnson, Matrix analysis. Cambridge university press, 2012.
  51. J. Pearl, “Causal inference in statistics: An overview,” 2009.
  52. P. Venkatesh, C. Bennett, S. Gale, T. Ramirez, G. Heller, S. Durand, S. Olsen, and S. Mihalas, “Gaussian partial information decomposition: Bias correction and application to high-dimensional data,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  53. Cambridge University Press, 2010.
  54. D. W. Peaceman and H. H. Rachford, Jr, “The numerical solution of parabolic and elliptic differential equations,” Journal of the Society for industrial and Applied Mathematics, vol. 3, no. 1, pp. 28–41, 1955.
  55. T. L. Bergman, Fundamentals of heat and mass transfer. John Wiley & Sons, 2011.
  56. S. Patankar, Numerical heat transfer and fluid flow. CRC press, 2018.
  57. R. J. LeVeque, Finite difference methods for ordinary and partial differential equations: steady-state and time-dependent problems. SIAM, 2007.
  58. N. R. Roy and P. Chandra, “Energy dissipation model for wireless sensor networks: a survey,” International Journal of Information Technology, vol. 12, no. 4, pp. 1343–1353, 2020.
  59. E. W. Weisstein, “Rotation matrix,” https://mathworld. wolfram. com/, 2003.
  60. W. Qiu, Analytic Geometry. Peking University Press, 1996.
  61. C. B. Boyer, History of analytic geometry. Courier Corporation, 2012.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now