Inverse decision-making using neural amortized Bayesian actors (2409.03710v2)
Abstract: Bayesian observer and actor models have provided normative explanations for many behavioral phenomena in perception, sensorimotor control, and other areas of cognitive science and neuroscience. They attribute behavioral variability and biases to interpretable entities such as perceptual and motor uncertainty, prior beliefs, and behavioral costs. However, when extending these models to more naturalistic tasks with continuous actions, solving the Bayesian decision-making problem is often analytically intractable. Inverse decision-making, i.e. performing inference over the parameters of such models given behavioral data, is computationally even more difficult. Therefore, researchers typically constrain their models to easily tractable components, such as Gaussian distributions or quadratic cost functions, or resort to numerical approximations. To overcome these limitations, we amortize the Bayesian actor using a neural network trained on a wide range of parameter settings in an unsupervised fashion. Using the pre-trained neural network enables performing efficient gradient-based Bayesian inference of the Bayesian actor model's parameters. We show on synthetic data that the inferred posterior distributions are in close alignment with those obtained using analytical solutions where they exist. Where no analytical solution is available, we recover posterior distributions close to the ground truth. We then show how our method allows for principled model comparison and how it can be used to disentangle factors that may lead to unidentifiabilities between priors and costs. Finally, we apply our method to empirical data from three sensorimotor tasks and compare model fits with different cost functions to show that it can explain individuals' behavioral patterns.
- Wei Ji Ma. Bayesian decision models: A primer. Neuron, 104(1):164–175, 2019.
- Motion illusions as optimal percepts. Nature neuroscience, 5(6):598–604, 2002.
- Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415(6870):429–433, 2002.
- A bayesian observer model constrained by efficient coding can explain’anti-bayesian’percepts. Nature neuroscience, 18(10):1509–1517, 2015.
- Object perception as bayesian inference. Annu. Rev. Psychol., 55:271–304, 2004.
- Bayesian integration in sensorimotor learning. Nature, 427(6971):244–247, 2004.
- Optimal feedback control as a theory of motor coordination. Nature neuroscience, 5(11):1226–1235, 2002.
- Optimal predictions in everyday cognition. Psychological science, 17(9):767–773, 2006.
- Word learning as bayesian inference. Psychological Review, 114(2):245, 2007.
- Learning the value of information in an uncertain world. Nature neuroscience, 10(9):1214–1221, 2007.
- Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science, 331(6013):83–87, 2011.
- The Role of Execution Noise in Movement Variability. Journal of Neurophysiology, 91(2):1050–1063, 2004.
- Decision making, movement planning and statistical decision theory. Trends in cognitive sciences, 12(8):291–297, 2008.
- Signal-dependent noise determines motor planning. Nature, 394(6695):780–784, 1998.
- Ernst Heinrich Weber. De Pulsu, resorptione, auditu et tactu: Annotationes anatomicae et physiologicae. Koehler, 1831.
- The loss function of sensorimotor learning. Proceedings of the National Academy of Sciences, 101(26):9839–9842, 2004.
- Chris R Sims. The cost of misremembering: Inferring the loss function in visual working memory. Journal of vision, 15(3):2–2, 2015.
- Computational rationality: Linking mechanism and behavior through bounded utility maximization. Topics in cognitive science, 6(2):279–311, 2014.
- Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. Behavioral and brain sciences, 43:e1, 2020.
- Computational rationality: A converging paradigm for intelligence in brains, minds, and machines. Science, 349(6245):273–278, 2015.
- Learning rational temporal eye movement strategies. Proceedings of the National Academy of Sciences, 113(29):8332–8337, 2016.
- Putting perception into action with inverse optimal control for continuous psychophysics. Elife, 11:e76635, 2022.
- Jacob Feldman. Tuning your priors to the world. Topics in cognitive science, 5(1):13–34, 2013.
- Herbert A Simon. A behavioral model of rational choice. The quarterly journal of economics, pages 99–118, 1955.
- John R Anderson. Is human cognition adaptive? Behavioral and brain sciences, 14(3):471–485, 1991.
- Doubly bayesian analysis of confidence in perceptual decision-making. PLoS computational biology, 11(10):e1004519, 2015.
- Cognitive tomography reveals complex, task-independent mental representations. Current Biology, 23(21):2169–2175, 2013.
- Modular inverse reinforcement learning for visuomotor behavior. Biological cybernetics, 107:477–490, 2013.
- Learning strategies in table tennis using inverse reinforcement learning. Biological cybernetics, 108:603–619, 2014.
- Inverse rational control with partially observable continuous nonlinear dynamics. Advances in neural information processing systems, 33:7898–7909, 2020.
- Inverse optimal control adapted to the noise characteristics of the human sensorimotor system. Advances in Neural Information Processing Systems, 34:9429–9442, 2021.
- Signal detection theory and psychophysics, volume 1. Wiley New York, 1966.
- Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty, 5:297–323, 1992.
- Cardinal rules: visual orientation perception reflects knowledge of environmental statistics. Nature neuroscience, 14(7):926–932, 2011.
- Noise characteristics and prior expectations in human visual speed perception. Nature neuroscience, 9(4):578–585, 2006.
- Validating model-based bayesian integration using prior–cost metamers. Proceedings of the National Academy of Sciences, 118(25):e2021531118, 2021.
- A framework for testing identifiability of bayesian models of perception. Advances in neural information processing systems, 27, 2014.
- Inferring perceptual decision making parameters from behavior in production and reproduction tasks. arXiv preprint arXiv:2112.15521, 2021.
- Likelihood approximation networks (lans) for fast inference of simulation models in cognitive neuroscience. Elife, 10:e65074, 2021.
- Automatic posterior transformation for likelihood-free inference. In International Conference on Machine Learning, pages 2404–2414. PMLR, 2019.
- Fast inference of spinal neuromodulation for motor control using amortized neural networks. Journal of neural engineering, 19(5):056037, 2022.
- Bayesflow: Learning complex stochastic models with invertible neural networks. IEEE transactions on neural networks and learning systems, 33(4):1452–1466, 2020.
- James O Berger. Statistical Decision Theory and Bayesian Analysis. Springer Science & Business Media, 1985.
- Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2014.
- Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural networks, 107:3–11, 2018.
- The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res., 15(1):1593–1623, 2014.
- Compiling machine learning programs via high-level tracing. Systems for Machine Learning, 4(9), 2018.
- Equinox: neural networks in JAX via callable PyTrees and filtered transformations. Differentiable Programming workshop at Neural Information Processing Systems 2021, 2021.
- Composable effects for flexible and accelerated probabilistic programming in numpyro. arXiv preprint arXiv:1912.11554, 2019.
- Gustav Theodor Fechner. Elemente der psychophysik, volume 2. Breitkopf u. Härtel, 1860.
- Linlin Yi. Do rats represent time logarithmically or linearly? Behavioural Processes, 81(2):274–279, 2009.
- William A. Roberts. Evidence that pigeons represent both time and number on a logarithmic scale. Behavioural Processes, 72(3):207–214, 2006.
- Spatial attention and the mental number line: Evidence for characteristic biases and compression. Neuropsychologia, 45(7):1400–1407, 2007.
- A framework for Bayesian optimality of psychophysical laws. Journal of Mathematical Psychology, 56(6):495–501, 2012.
- Stanislas Dehaene. The neural basis of the Weber –Fechner law: a logarithmic mental number line. TRENDS in Cognitive Sciences, 7(4):145–147, 2003.
- GG Sutton and K Sykes. The variation of hand tremor with force in healthy subjects. The Journal of physiology, 191(3):699–711, 1967.
- Motor-output variability: a theory for the accuracy of rapid motor acts. Psychological review, 86(5):415, 1979.
- Inference from iterative simulation using multiple sequences. Statistical science, 7(4):457–472, 1992.
- Long-term motor learning: Effects of varied and specific practice. Vision research, 152:10–16, 2018.
- Continuous decisions. Philosophical Transactions of the Royal Society B, 376(1819):20190664, 2021.
- Approximate inference for the loss-calibrated bayesian. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 416–424. JMLR Workshop and Conference Proceedings, 2011.
- Variational bayesian decision-making for continuous utilities. Advances in Neural Information Processing Systems, 32, 2019.