Active Inference and Intentional Behaviour (2312.07547v2)
Abstract: Recent advances in theoretical biology suggest that basal cognition and sentient behaviour are emergent properties of in vitro cell cultures and neuronal networks, respectively. Such neuronal networks spontaneously learn structured behaviours in the absence of reward or reinforcement. In this paper, we characterise this kind of self-organisation through the lens of the free energy principle, i.e., as self-evidencing. We do this by first discussing the definitions of reactive and sentient behaviour in the setting of active inference, which describes the behaviour of agents that model the consequences of their actions. We then introduce a formal account of intentional behaviour, that describes agents as driven by a preferred endpoint or goal in latent state-spaces. We then investigate these forms of (reactive, sentient, and intentional) behaviour using simulations. First, we simulate the aforementioned in vitro experiments, in which neuronal cultures spontaneously learn to play Pong, by implementing nested, free energy minimising processes. The simulations are then used to deconstruct the ensuing predictive behaviour, leading to the distinction between merely reactive, sentient, and intentional behaviour, with the latter formalised in terms of inductive planning. This distinction is further studied using simple machine learning benchmarks (navigation in a grid world and the Tower of Hanoi problem), that show how quickly and efficiently adaptive behaviour emerges under an inductive form of active inference.
- “In vitro neurons learn and exhibit sentience when embodied in a simulated game-world” In Neuron, 2022
- Chris Fields, James F Glazebrook and Michael Levin “Minimal physicalism as a scale-free substrate for cognition and consciousness” In Neuroscience of Consciousness 2021, 2021, pp. niab013
- M. Levin “The Computational Boundary of a "Self": Developmental Bioelectricity Drives Multicellularity and Scale-Free Cognition” In Frontiers in Psychology 10, 2019, pp. 2688
- “Modeling somatic computation with non-neural bioelectric networks” Scientific Reports 9, 18612, 2019
- “Emergence of sense-making behavior by the Stimulus Avoidance Principle: Experiments on a robot behavior controlled by cultured neuronal cells” In Artificial Life Conference Proceedings, 2015, pp. 373–380 MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info …
- “In vitro neural networks minimise variational free energy” In Scientific Reports 8, 2018, pp. 16926
- “Experimental validation of the free-energy principle with in vitro neural networks” In Nature Communications 14, 2023, pp. 4547
- “Sophisticated inference” In Neural Computation 33, 2021, pp. 713–763
- “The emergence of synchrony in networks of mutually inferring neurons” In Scientific Reports 9, 2019, pp. 6412
- “Variational message passing” In Journal of Machine Learning Research 6, 2005, pp. 661–694
- Jakob Hohwy “The self-evidencing brain” In Noûs 50.2 Wiley Online Library, 2016, pp. 259–285
- “Human-level control through deep reinforcement learning” In Nature 518, 2015, pp. 529–533
- “Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model” arXiv:1911.08265, 2019
- Fuat Balci “A response to claims of emergent intelligence and sentience in a dish” In Neuron 111, 2023, pp. 604–605
- Hagai Attias “Planning by probabilistic inference” In International workshop on artificial intelligence and statistics, 2003, pp. 9–16 PMLR
- “Planning as inference” In Trends in cognitive sciences 16.10 Elsevier, 2012, pp. 485–488
- Sander G Van Dijk and Daniel Polani “Informational constraints-driven organization in goal-directed behavior” In Advances in Complex Systems 16.02n03 World Scientific, 2013, pp. 1350016
- “Active inference on discrete state-spaces: A synthesis” In Journal of Mathematical Psychology 99, 2020, pp. 102447
- “On Bayesian mechanics: a physics of and by beliefs” In Interface Focus 13.3 The Royal Society, 2023, pp. 20220029
- “The free energy principle made simpler but not too simple” In Physics Reports 1024, 2023, pp. 1–29
- “Path integrals, particular kinds, and strange things” arXiv:2210.12761, 2022
- Colin F Camerer “Progress in behavioral game theory” In Journal of Economic Perspectives 11, 1997, pp. 167–188
- C. Hure, H. Pham and X. Warin “Deep Backward Schemes for High-Dimensional Nonlinear Pdes” In Mathematics of Computation 89, 2020, pp. 1547–1579
- Richard Bellman “On the Theory of Dynamic Programming” In Proc Natl Acad Sci U S A 38, 1952, pp. 716–719
- “The relationship between dynamic programming and active inference: the discrete, finite-horizon case” In arXiv preprint arXiv:2009.08111, 2020
- R.S. Sutton, D. Precup and S. Singh “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning” In Artificial Intelligence 112, 1999, pp. 181–211
- “On efficient computation in active inference” In arXiv preprint arXiv:2307.00504, 2023
- G.H. Harman “The inference to the best explanation” In Philosophical Review 74, 1965, pp. 88–95
- A.K. Seth “Inference to the Best Prediction” In Open MIND Frankfurt am Main: MIND Group, 2015
- R.A. Howard “Dynamic Programming and Markov Processes” Cambridge, MA: MIT Press, 1960
- “On Bayesian Mechanics: A Physics of and by Beliefs” arXiv:2205.11543, 2022
- Andy Clark, Karl Friston and Sam Wilkinson “Bayesing Qualia: Consciousness as Inference, Not Raw Datum” In Journal of Consciousness Studies 26, 2019, pp. 19–33
- “Towards a computational phenomenology of mental action: modelling meta-awareness and attentional control with deep parametric active inference” In Neuroscience of Consciousness 2021, 2021, pp. niab018
- Ryan Smith, Maxwell J.D. Ramstead and Alex Kiefer “Active Inference Models Do Not Contradict Folk Psychology” In Synthese 200.2 Springer Verlag, 2022, pp. 1–37 DOI: 10.1007/s11229-022-03480-w
- Matthew J Beal “Variational Algorithms for Approximate Bayesian Inference”, 2003
- Randy L Buckner “The role of the hippocampus in prediction and imagination” In Annual review of psychology 61 Annual Reviews, 2010, pp. 27–48
- Kenway Louie and Matthew A Wilson “Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep” In Neuron 29.1 Elsevier, 2001, pp. 145–156
- Will D Penny, Peter Zeidman and Neil Burgess “Forward and backward inference in spatial cognition” In PLoS computational biology 9.12 Public Library of Science San Francisco, USA, 2013, pp. e1003383
- “Internally generated sequences in learning and executing goal-directed behavior” In Trends in cognitive sciences 18.12 Elsevier, 2014, pp. 647–657
- Thomas Parr, Giovanni Pezzulo and Karl J Friston “Active inference: the free energy principle in mind, brain, and behavior” MIT Press, 2022
- “Active inference and epistemic value” In Cognitive neuroscience 6.4 Taylor & Francis, 2015, pp. 187–214
- “Predictive information and explorative behavior of autonomous robots” In European Physical Journal B 63, 2008, pp. 329–339
- Horace B Barlow “Possible principles underlying the transformation of sensory messages” In Sensory communication 1.01, 1961, pp. 217–233
- R. Linsker “Perceptual Neural Organization - Some Approaches Based on Network Models and Information-Theory” In Annual Review of Neuroscience 13, 1990, pp. 257–281
- “Emergence of simple-cell receptive field properties by learning a sparse code for natural images” In Nature 381, 1996, pp. 607–609
- “Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons” In Nature communications 12.1 Nature Publishing Group UK London, 2021, pp. 6456
- Eduardo Hugo Sanchez, Mathieu Serrurier and Mathias Ortner “Learning disentangled representations via mutual information estimation” In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, 2020, pp. 205–221 Springer
- C. Gros “Cognitive computation with autonomously active neural networks: An emerging field” In Cognitive Computation 1, 2009, pp. 77–90
- Dalton AR Sakthivadivel “Weak Markov blankets in high-dimensional, sparsely-coupled random dynamical systems” In arXiv preprint arXiv:2207.07620, 2022
- Michael E Tipping “Sparse Bayesian learning and the relevance vector machine” In Journal of machine learning research 1.Jun, 2001, pp. 211–244
- “Computational mechanisms of curiosity and goal-directed exploration” In elife 8 eLife Sciences Publications, Ltd, 2019, pp. e41703
- “Bayesian Surprise Attracts Human Attention” In Vision Research 49, 2009, pp. 1295–1306
- D.V. Lindley “On a Measure of the Information Provided by an Experiment” In Annals of Mathematical Statistics 27, 1956, pp. 986–1005
- J. Schmidhuber “Curious model-building control systems” In International Joint Conference on Neural Networks 2, 1991, pp. 1458–1463 IEEE
- “An information-theoretic approach to curiosity-driven reinforcement learning” In Theory in Biosciences 131, 2012, pp. 139–148
- Ronald A Howard “Information value theory” In IEEE Transactions on systems science and cybernetics 2.1 IEEE, 1966, pp. 22–26
- Alexander S Klyubin, Daniel Polani and Chrystopher L Nehaniv “Empowerment: A universal agent-centric measure of control” In 2005 ieee congress on evolutionary computation 1, 2005, pp. 128–135 IEEE
- Bart van den Broek, Wim Wiegerinck and Hilbert Kappen “Risk sensitive path integral control” In arXiv preprint arXiv:1203.3523, 2012
- “Path integral control and bounded rationality” In 2011 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), 2011, pp. 202–209 IEEE
- Pedro A Ortega and Daniel A Braun “Thermodynamics as a theory of decision-making with information-processing costs” In Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 469.2153 The Royal Society Publishing, 2013, pp. 20120683
- “Bayesian reinforcement learning: A survey” In Foundations and Trends® in Machine Learning 8.5-6 Now Publishers, Inc., 2015, pp. 359–483
- James O Berger “Statistical decision theory and Bayesian analysis” Springer Science & Business Media, 2013
- Colin F Camerer, Teck-Hua Ho and Juin-Kuan Chong “A cognitive hierarchy model of games” In Quarterly Journal of Economics 119, 2004, pp. 861–898
- D Gowanlock R Tervo, Joshua B Tenenbaum and Samuel J Gershman “Toward the neural implementation of structure learning” In Current opinion in neurobiology 37 Elsevier, 2016, pp. 99–105
- Horace B Barlow “Inductive inference, coding, perception, and language” In Perception 3, 1974, pp. 123–134
- J. Hawthorne “Inductive Logic” Stanford Encyclopedia of Philosophy, 2021
- Alex Kiefer “Literal Perceptual Inference” In Philosophy and predictive processing, 2017
- Emanuel Todorov “Linearly-solvable Markov decision problems” In Advances in neural information processing systems 19, 2006
- “Homo heuristicus: why biased minds make better inferences” In Topics in Cognitive Science 1, 2009, pp. 107–143
- W. Mansell “Control of perception should be operationalized as a fundamental property of the nervous system” In Topics in Cognitive Science 3, 2011, pp. 257–261
- “Winnerless competition principle and prediction of the transient dynamics in a Lotka-Volterra model” In Chaos 18.043103, 2008
- M. Rabinovich, R. Huerta and G. Laurent “Transient dynamics for neural processing” In Science 321, 2008, pp. 48–50
- José Fonollosa, Emre Neftci and Mikhail Rabinovich “Learning of chunking sequences in cognition and behavior” In PLoS Computational Biology 11, 2015, pp. e1004592
- Chris L Baker, Rebecca Saxe and Joshua B Tenenbaum “Action understanding as inverse planning” In Cognition 113, 2009, pp. 329–349
- “The misbehavior of value and the discipline of the will” In Neural Networks 19, 2006, pp. 1153–1160
- “Planning and navigation as active inference” In Biological Cybernetics 112, 2018, pp. 323–343
- Francesco Donnarumma, Domenico Maisto and Giovanni Pezzulo “Problem solving as probabilistic inference with subgoaling: explaining human successes and pitfalls in the tower of hanoi” In PLoS computational biology 12.4 Public Library of Science San Francisco, CA USA, 2016, pp. e1004864
- “Supervised structure learning” In arXiv preprint arXiv:2311.10300, 2023
- Frédéric Colas, Julien Diard and Pierre Bessière “Common Bayesian models for common cognitive issues” In Acta Biotheoretica 58, 2010, pp. 191–216
- “PDDL2.1: An extension to PDDL for expressing temporal planning domains” In Journal of Artificial Intelligence Research 20, 2003, pp. 61–124
- M. Gilead, Y. Trope and N. Liberman “Above and beyond the concrete: The diverse representational substrates of the predictive brain” In Behavioral and Brain Sciences 43, 2019, pp. e121
- Chris Fields “Control flow in active inference systems—part II: tensor networks as general models of control flow” In IEEE Transactions on Molecular, Biological and Multi-Scale Communications 9, 2023, pp. 246–256
- “Theory of quantum error-correcting codes” In Physical Review A 55, 1997, pp. 900–911
- “Reward Maximization Through Discrete Active Inference” In Neural Computation 35.5 MIT Press, 2023, pp. 807–852
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.