Emergence and Causality in Complex Systems: A Survey on Causal Emergence and Related Quantitative Studies (2312.16815v3)
Abstract: Emergence and causality are two fundamental concepts for understanding complex systems. They are interconnected. On one hand, emergence refers to the phenomenon where macroscopic properties cannot be solely attributed to the cause of individual properties. On the other hand, causality can exhibit emergence, meaning that new causal laws may arise as we increase the level of abstraction. Causal emergence theory aims to bridge these two concepts and even employs measures of causality to quantify emergence. This paper provides a comprehensive review of recent advancements in quantitative theories and applications of causal emergence. Two key problems are addressed: quantifying causal emergence and identifying it in data. Addressing the latter requires the use of machine learning techniques, thus establishing a connection between causal emergence and artificial intelligence. We highlighted that the architectures used for identifying causal emergence are shared by causal representation learning, causal model abstraction, and world model-based reinforcement learning. Consequently, progress in any of these areas can benefit the others. Potential applications and future perspectives are also discussed in the final section of the review.
- Ledford, H. How to solve the world’s biggest problems. Nature 2015, 525, 308–311.
- Systems analysis approach for complex global challenges; Springer, 2018.
- Bar-Yam, Y. General features of complex systems. Encyclopedia of Life Support Systems (EOLSS), UNESCO, EOLSS Publishers, Oxford, UK 2002, 1.
- Holland, J.H. Emergence: From chaos to order; OUP Oxford, 2000.
- From the origin of life to pandemics: Emergent phenomena in complex systems, 2022.
- The emergence of human consciousness: from fetal to neonatal life. Pediatric research 2009, 65, 255–260.
- Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 2022.
- Anderson, P.W. More is different: broken symmetry and the nature of the hierarchical structure of science. Science 1972, 177, 393–396.
- The concept of emergence. Minnesota studies in the philosophy of science 1956, 1, 239–252.
- Bedau, M.A. Weak emergence. Philosophical perspectives 1997, 11, 375–399.
- Statistical mechanics of complex networks. Reviews of modern physics 2002, 74, 47.
- Wikipedia. Butterfly effect — Wikipedia, The Free Encyclopedia, 2023. [Online; accessed 27-September-2023].
- Wikipedia. Homeostasis — Wikipedia, The Free Encyclopedia, 2023. [Online; accessed 4-October-2023].
- Granger, C.W. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society 1969, pp. 424–438.
- Pearl, J.; et al. Models, reasoning and inference. Cambridge, UK: CambridgeUniversityPress 2000, 19.
- Pearl, J. Causality; Cambridge university press, 2009.
- Kim, J. ‘Downward causation’in emergentism and nonreductive physicalism. Emergence or reduction 1992, pp. 119–138.
- Fromm, J. Types and forms of emergence. arXiv preprint nlin/0506028 2005.
- Quantifying causal emergence shows that macro can beat micro. Proceedings of the National Academy of Sciences 2013, 110, 19790–19795.
- Inferring statistical complexity. Physical review letters 1989, 63, 105.
- Seth, A.K. Measuring emergence via nonlinear Granger causality. In Proceedings of the alife, 2008, Vol. 2008, pp. 545–552.
- Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data. PLoS computational biology 2020, 16, e1008289.
- Hoel, E.P. When the Map Is Better Than the Territory. Entropy 2017, 19. https://doi.org/10.3390/e19050188.
- Causal geometry. Entropy 2020, 23, 24.
- The emergence of informative higher scales in complex networks. Complexity 2020, 2020, 1–12.
- Causal emergence is widespread across measures of causation. arXiv preprint arXiv:2202.01854 2022.
- Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515 2010.
- Beyond integrated information: A taxonomy of information dynamics phenomena. arXiv preprint arXiv:1909.02297 2019.
- Neural information squeezer for causal emergence. Entropy 2022, 25, 26.
- Detection and emergence. arXiv preprint arXiv:1108.4279 2011.
- Finding emergence in data: causal emergence inspired dynamics learning. arXiv preprint arXiv:2308.09952 2023.
- Toward causal representation learning. Proceedings of the IEEE 2021, 109, 612–634.
- Approximate causal abstractions. In Proceedings of the Uncertainty in Artificial Intelligence. PMLR, 2020, pp. 606–615.
- World models. arXiv preprint arXiv:1803.10122 2018.
- Examining the Causal Structures of Deep Neural Networks Using Information Theory. Entropy 2020, 22, 1429.
- Causality and model abstraction. Artificial intelligence 1994, 67, 143–194.
- Sucar, L.E. Probabilistic graphical models. Advances in Computer Vision and Pattern Recognition. London: Springer London. doi 2015, 10, 1.
- Rubin, D.B. Causal inference using potential outcomes: Design, modeling, decisions. Journal of the American Statistical Association 2005, 100, 322–331.
- Causal discovery algorithms: A practical guide. Philosophy Compass 2018, 13, e12470.
- Causation Prediction and Search 2nd Edition, 2000.
- Chickering, D.M. Learning equivalence classes of Bayesian-network structures. The Journal of Machine Learning Research 2002, 2, 445–498.
- Hume, D. An enquiry concerning human understanding. In Seven masterpieces of philosophy; Routledge, 2016; pp. 183–276.
- Eells, E. Probabilistic causality; Vol. 1, Cambridge University Press, 1991.
- Suppes, P. A probabilistic theory of causality. British Journal for the Philosophy of Science 1973, 24.
- Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control. Communicative & Integrative Biology 2020, 13, 108–118.
- Pepper, S.C. Emergence. The Journal of Philosophy 1926, 23, 241–245.
- Complexity, control and goal-directedness in biological systems. The Routledge handbook of emergence 2019, p. 134.
- Hendry, R.F. Substance and structure. The Routledge handbook of emergence 2019, p. 339.
- Huxley, T.H. Evolution and Ethics 1893-1943; Pilot Press, 1947.
- In A System of Logic, Ratiocinative and Inductive: Being a Connected View of the Principles of Evidence, and the Methods of Scientific Investigation; Cambridge University Press, 2011; Vol. 1, Cambridge Library Collection - Philosophy, p. 425–436. https://doi.org/10.1017/CBO9781139149839.022.
- The Routledge handbook of emergence; Routledge, 2019.
- The Works of Aristotle: Metaphysica, by WD Ross; Vol. 8, Clarendon Press, 1908.
- Holland, J.H. Hidden order: How adaptation builds complexity; Addison Wesley Longman Publishing Co., Inc., 1996.
- Emergence: Contemporary readings in philosophy and science; MIT press, 2008.
- Bedau, M. Downward causation and the autonomy of weak emergence. Principia: an international journal of epistemology 2002, 6, 5–50.
- Wolfram, S.; et al. A new kind of science; Vol. 5, Wolfram media Champaign, 2002.
- Reynolds, C.W. Flocks, herds and schools: A distributed behavioral model. In Proceedings of the Proceedings of the 14th annual conference on Computer graphics and interactive techniques, 1987, pp. 25–34.
- Merton, R.K. The sociology of science: Theoretical and empirical investigations; University of Chicago press, 1973.
- Yurchenko, S.B. Can there be a synergistic core emerging in the brain hierarchy to control neural activity by downward causation? 2023.
- Harré, R. The philosophies of science 1985.
- Baas, N.A. Emergence, hierarchies, and hyperstructures. Artificial Life III, SFI Studies in the Science of Complexity, XVII 1994, pp. 515–537.
- Newman, D.V. Emergence and strange attractors. Philosophy of Science 1996, 63, 245–261.
- O’Connor, T. Emergent properties. American Philosophical Quarterly 1994, 31, 91–104.
- In defense of explanatory ecumenism. Economics & Philosophy 1992, 8, 1–21.
- Kim, J. Making sense of emergence. Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition 1999, 95, 3–36.
- Dewhurst, J. Causal emergence from effective information: Neither causal nor emergent? Thought: A Journal of Philosophy 2021, 10, 158–168.
- Causal emergence: When distortions in a map obscure the territory. Philosophies 2022, 7, 30.
- Crutchfield, J.P. The calculi of emergence: computation, dynamics and induction. Physica D: Nonlinear Phenomena 1994, 75, 11–54.
- Shalizi, C.R. Causal architecture, complexity and self-organization in time series and cellular automata; The University of Wisconsin-Madison, 2001.
- What is a macrostate? Subjective observations and objective dynamics. arXiv preprint cond-mat/0303625 2003.
- Quantitative emergence. Organic Computing—A Paradigm Shift for Complex Systems 2011, pp. 39–52.
- Information entropy-based metrics for measuring emergences in artificial societies. Entropy 2014, 16, 4583–4602.
- Quantitative emergence–A refined approach based on divergence measures. In Proceedings of the 2010 Fourth IEEE International Conference on Self-Adaptive and Self-Organizing Systems. IEEE Computer Society, 2010, pp. 94–103.
- Techniques for knowledge acquisition in dynamically changing environments. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 2012, 7, 1–25.
- Divergence measures as a generalised approach to quantitative emergence. Organic Computing—A Paradigm Shift for Complex Systems 2011, pp. 53–66.
- On autonomy and emergence in self-organizing systems. In Proceedings of the International Workshop on Self-Organizing Systems. Springer, 2008, pp. 157–169.
- Methods for approximations of quantitative measures in self-organizing systems. In Proceedings of the Self-Organizing Systems: 5th International Workshop, IWSOS 2011, Karlsruhe, Germany, February 23-24. 2011. Proceedings 5. Springer, 2011, pp. 1–15.
- Formalization of emergence in multi-agent systems. In Proceedings of the Proceedings of the 1st ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, 2013, pp. 231–240.
- Formalization of weak emergence in multiagent systems. ACM Transactions on Modeling and Computer Simulation (TOMACS) 2015, 26, 1–25.
- Complexity and criticality; Vol. 1, World Scientific Publishing Company, 2005.
- McComb, W.D. Renormalization methods: a guide for beginners; OUP Oxford, 2003.
- Finite Markov Chains: With a New Appendix "Generalization of a Fundamental Matrix"; Undergraduate Texts in Mathematics, Springer New York, 1983.
- Comparison between different methods of level identification. Advances in Complex Systems 2014, 17, 1450007.
- Balanced truncation for a class of stochastic jump linear systems and model reduction for hidden Markov models. IEEE Transactions on Automatic Control 2008, 53, 2543–2557.
- Lumpable hidden Markov models-model reduction and reduced complexity filtering. IEEE Transactions on Automatic Control 2000, 45, 2297–2306.
- Optimal high-level descriptions of dynamical systems. arXiv preprint arXiv:1409.7403 2014.
- Varley, T.F. Flickering emergences: The question of locality in information-theoretic approaches to emergence. Entropy 2022, 25, 54.
- Varley, T.F. Causal emergence in discrete and continuous dynamical systems. arXiv preprint arXiv:2003.13075 2020.
- Finding the right scale of a network: efficient identification of causal emergence through spectral clustering. arXiv preprint arXiv:1908.07565 2019.
- Exploring noise, degeneracy and determinism in biological networks with the einet package. Methods in Ecology and Evolution 2022, 13, 799–804.
- Evolution and emergence: higher order information structure in protein interactomes across the tree of life. Integrative Biology 2021, 13, 283–294.
- Interactions and information: exploring task allocation in ant colonies using network analysis. Animal Behaviour 2022, 189, 69–81.
- From the phenomenology to the mechanisms of consciousness: integrated information theory 3.0. PLoS computational biology 2014, 10, e1003588.
- A synergistic core for human brain evolution and cognition. Nature Neuroscience 2022, 25, 771–782.
- Emergence as the conversion of information: a unifying theory. Philosophical Transactions of the Royal Society A 2022, 380, 20210150.
- Mastering the game of Go with deep neural networks and tree search. nature 2016, 529, 484–489.
- A distributional code for value in dopamine-based reinforcement learning. Nature 2020, 577, 671–675.
- Improved protein structure prediction using potentials from deep learning. Nature 2020, 577, 706–710.
- Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural networks 1991, 4, 251–257.
- Coupling-based invertible neural networks are universal diffeomorphism approximators. Advances in Neural Information Processing Systems 2020, 33, 3362–3373.
- Rosenblatt, M. Remarks on some nonparametric estimates of a density function. The annals of mathematical statistics 1956, pp. 832–837.
- Causal feature learning: an overview. Behaviormetrika 2017, 44, 137–164.
- Visual causal feature learning. In Proceedings of the Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, 2015, pp. 181–190.
- Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 2013, 35, 1798–1828.
- Deep learning. nature 2015, 521, 436–444.
- Deep learning; MIT press, 2016.
- Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions. Journal of big Data 2021, 8, 1–74.
- Gradient-based learning applied to document recognition. Proceedings of the IEEE 1998, 86, 2278–2324.
- Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 5265–5274.
- Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 2013.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018.
- Attention is all you need. Advances in neural information processing systems 2017, 30.
- Human-level control through deep reinforcement learning. nature 2015, 518, 529–533.
- Stable learning establishes some common ground between causal inference and machine learning. Nature Machine Intelligence 2022, 4, 110–115.
- Deep stable learning for out-of-distribution generalization. In Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 5372–5382.
- Disentangled representation learning. arXiv preprint arXiv:2211.11695 2022.
- Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems 2016, 29.
- Variational autoencoders and nonlinear ica: A unifying framework. In Proceedings of the International Conference on Artificial Intelligence and Statistics. PMLR, 2020, pp. 2207–2217.
- Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516 2014.
- Causal discovery with general non-linear relationships using non-linear ica. In Proceedings of the Uncertainty in artificial intelligence. PMLR, 2020, pp. 186–195.
- Causal inference by using invariant prediction: identification and confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology 2016, 78, 947–1012.
- Stable prediction across unknown environments. In Proceedings of the proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018, pp. 1617–1626.
- Invariant risk minimization. arXiv preprint arXiv:1907.02893 2019.
- Towards a theoretical framework of out-of-distribution generalization. Advances in Neural Information Processing Systems 2021, 34, 23519–23531.
- The Missing Invariance Principle found–the Reciprocal Twin of Invariant Risk Minimization. Advances in Neural Information Processing Systems 2022, 35, 23023–23035.
- On learning invariant representations for domain adaptation. In Proceedings of the International conference on machine learning. PMLR, 2019, pp. 7523–7532.
- Domain-adversarial training of neural networks. The journal of machine learning research 2016, 17, 2096–2030.
- Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering 2022.
- Domain generalization via conditional invariant representations. In Proceedings of the Proceedings of the AAAI conference on artificial intelligence, 2018, Vol. 32.
- Out-of-distribution generalization with maximal invariant predictor 2020.
- Invariance principle meets information bottleneck for out-of-distribution generalization. Advances in Neural Information Processing Systems 2021, 34, 3438–3450.
- Modeling the data-generating process is necessary for out-of-distribution generalization. arXiv preprint arXiv:2206.07837 2022.
- Stable learning via sample reweighting. In Proceedings of the Proceedings of the AAAI Conference on Artificial Intelligence, 2020, Vol. 34, pp. 5692–5699.
- Stable learning via differentiated variable decorrelation. In Proceedings of the Proceedings of the 26th acm sigkdd international conference on knowledge discovery & data mining, 2020, pp. 2185–2193.
- Causal consistency of structural equation models. arXiv preprint arXiv:1707.00819 2017.
- Multi-level cause-effect systems. In Proceedings of the Artificial intelligence and statistics. PMLR, 2016, pp. 361–369.
- Abstracting causal models. In Proceedings of the Proceedings of the aaai conference on artificial intelligence, 2019, Vol. 33, pp. 2678–2685.
- Beckers, S. Equivalent causal models. In Proceedings of the Proceedings of the AAAI Conference on Artificial Intelligence, 2021, Vol. 35, pp. 6202–6209.
- Unsupervised discovery of El Nino using causal feature learning on microlevel climate data. arXiv preprint arXiv:1605.09370 2016.
- Convolutional neural network pruning with structural redundancy reduction. In Proceedings of the Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 14913–14922.
- An abstraction-refinement approach to verification of artificial neural networks. In Proceedings of the Computer Aided Verification: 22nd International Conference, CAV 2010, Edinburgh, UK, July 15-19, 2010. Proceedings 22. Springer, 2010, pp. 243–257.
- Causal abstractions of neural networks. Advances in Neural Information Processing Systems 2021, 34, 9574–9586.
- Antoulas, A.C. An overview of approximation methods for large-scale dynamical systems. Annual reviews in Control 2005, 29, 181–190.
- Reinforcement learning: An introduction; MIT press, 2018.
- A Survey on Causal Reinforcement Learning. arXiv preprint arXiv:2302.05209 2023.
- Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 2013.
- Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104 2023.
- LeCun, Y. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review 2022, 62.
- Learning latent dynamics for planning from pixels. In Proceedings of the International conference on machine learning. PMLR, 2019, pp. 2555–2565.
- Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603 2019.
- Mastering atari with discrete world models. arXiv preprint arXiv:2010.02193 2020.
- Planning to explore via self-supervised world models. In Proceedings of the International Conference on Machine Learning. PMLR, 2020, pp. 8583–8592.
- Invariant causal prediction for block mdps. In Proceedings of the International Conference on Machine Learning. PMLR, 2020, pp. 11214–11224.
- Laser: Learning a latent action space for efficient reinforcement learning. In Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 6650–6656.
- Efficient planning in a compact latent action space. arXiv preprint arXiv:2208.10291 2022.
- Learning invariant representations for reinforcement learning without reconstruction. arXiv preprint arXiv:2006.10742 2020.
- Causal dynamics learning for task-independent state abstraction. arXiv preprint arXiv:2206.13452 2022.
- Towards robust bisimulation metric learning. Advances in Neural Information Processing Systems 2021, 34, 4764–4777.
- The fractal geometry of nature; Vol. 1, WH freeman New York, 1982.
- Detecting emergence in engineered systems: A literature review and synthesis approach. Systems Engineering 2023.
- Decentralised detection of emergence in complex adaptive systems. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 2017, 12, 1–31.
- Towards decentralised detection of emergence in complex adaptive systems. In Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems. IEEE, 2014, pp. 60–69.
- The emergence of objectness: Learning zero-shot segmentation from videos. Advances in Neural Information Processing Systems 2021, 34, 13137–13152.
- A Survey of Graph-based Deep Learning for Anomaly Detection in Distributed Systems. arXiv preprint arXiv:2206.04149 2022.
- Review of causal discovery methods based on graphical models. Frontiers in genetics 2019, 10, 524.
- A survey on causal discovery: theory and practice. International Journal of Approximate Reasoning 2022, 151, 101–129.
- The backbone of the climate network. Europhysics Letters 2009, 87, 48007.
- Model-free reconstruction of excitatory neuronal connectivity from calcium imaging signals 2012.
- Estimating high-dimensional directed acyclic graphs with the PC-algorithm. Journal of Machine Learning Research 2007, 8.
- Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems 2018, 31.
- Causal discovery with reinforcement learning. arXiv preprint arXiv:1906.04477 2019.
- Richardson, T.S. A discovery algorithm for directed cyclic graphs. arXiv preprint arXiv:1302.3599 2013.
- Discovering cyclic causal models by independent components analysis. arXiv preprint arXiv:1206.3273 2012.
- Constraint-based causal discovery for non-linear structural causal models with cycles and latent confounders. arXiv preprint arXiv:1807.03024 2018.
- Learning optimal cyclic causal graphs from interventional data. In Proceedings of the International Conference on Probabilistic Graphical Models. PMLR, 2020, pp. 365–376.
- A general deep learning framework for network reconstruction and dynamics learning. Applied Network Science 2019, 4, 1–17.
- Dynotears: Structure learning from time-series data. In Proceedings of the International Conference on Artificial Intelligence and Statistics. PMLR, 2020, pp. 1595–1605.
- D’ya like dags? a survey on structure learning and causal discovery. ACM Computing Surveys 2022, 55, 1–36.
- CUTS+: High-dimensional Causal Discovery from Irregular Time-series. arXiv preprint arXiv:2305.05890 2023.
- Hierarchical graph neural networks for causal discovery and root cause localization. arXiv preprint arXiv:2302.01987 2023.
- Wavelet Multiscale Granger Causality Analysis Based on State Space Models. Symmetry 2023, 15, 1286.
- Interpretable Multi-Scale Neural Network for Granger Causality Discovery. In Proceedings of the ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5.
- Neural network attributions: A causal perspective. In Proceedings of the International Conference on Machine Learning. PMLR, 2019, pp. 981–990.
- Defining and quantifying the emergence of sparse concepts in dnns. In Proceedings of the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 20280–20289.
- The quantization model of neural scaling. arXiv preprint arXiv:2303.13506 2023.
- Information closure theory of consciousness. Frontiers in Psychology 2020, 11, 1504.
- Machine learning hidden symmetries. Physical Review Letters 2022, 128, 180201.
- Emergence, Complexity and Computation.
- Emergent control. Organic Computing—A Paradigm Shift for Complex Systems 2011, pp. 67–78.
- Revisiting the edge of chaos: Evolving cellular automata to perform computations. arXiv preprint adap-org/9303003 1993.
- Growing Isotropic Neural Cellular Automata. In Proceedings of the Artificial Life Conference Proceedings 34. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info …, 2022, Vol. 2022, p. 65.
- E (n)-equivariant Graph Neural Cellular Automata. arXiv preprint arXiv:2301.10497 2023.
- A Relational Macrostate Theory Guides Artificial Intelligence to Learn Macro and Design Micro. arXiv preprint arXiv:2210.07374 2022.
- Generative adversarial nets. Advances in neural information processing systems 2014, 27.
- A survey on generative diffusion model. arXiv preprint arXiv:2209.02646 2022.
- Diffusion models: A comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796 2022.
- Normalizing flows for probabilistic modeling and inference. The Journal of Machine Learning Research 2021, 22, 2617–2680.
- Normalizing flows: An introduction and review of current methods. IEEE transactions on pattern analysis and machine intelligence 2020, 43, 3964–3979.
- Consciousness: here, there and everywhere? Philosophical Transactions of the Royal Society B: Biological Sciences 2015, 370, 20140167.
- Walker, S.I. Top-down causation and the rise of information in the emergence of life. Information 2014, 5, 424–439.
- Elder-Vass, D. The causal power of social structures: Emergence, structure and agency; Cambridge University Press, 2010.
- Integrated information theory: from consciousness to its physical substrate. Nature Reviews Neuroscience 2016, 17, 450–461.
- Holland, J.H. Hidden order. Business Week-Domestic Edition 1995, p. 21.
- Holland, J.H. Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence; MIT press, 1992.