Astrocyte Regulated Neuromorphic Central Pattern Generator Control of Legged Robotic Locomotion (2312.15805v2)
Abstract: Neuromorphic computing systems, where information is transmitted through action potentials in a bio-plausible fashion, is gaining increasing interest due to its promise of low-power event-driven computing. Application of neuromorphic computing in robotic locomotion research have largely focused on Central Pattern Generators (CPGs) for bionics robotic control algorithms - inspired from neural circuits governing the collaboration of the limb muscles in animal movement. Implementation of artificial CPGs on neuromorphic hardware platforms can potentially enable adaptive and energy-efficient edge robotics applications in resource constrained environments. However, underlying rewiring mechanisms in CPG for gait emergence process is not well understood. This work addresses the missing gap in literature pertaining to CPG plasticity and underscores the critical homeostatic functionality of astrocytes - a cellular component in the brain that is believed to play a major role in multiple brain functions. This paper introduces an astrocyte regulated Spiking Neural Network (SNN)-based CPG for learning locomotion gait through Reward-Modulated STDP for quadruped robots, where the astrocytes help build inhibitory connections among the artificial motor neurons in different limbs. The SNN-based CPG is simulated on a multi-object physics simulation platform resulting in the emergence of a trotting gait while running the robot on flat ground. $23.3\times$ computational power savings is observed in comparison to a state-of-the-art reinforcement learning based robot control algorithm. Such a neuroscience-algorithm co-design approach can potentially enable a quantum leap in the functionality of neuromorphic systems incorporating glial cell functionality.
- Modulation of spinal motor networks by astrocyte-derived adenosine is dependent on D1-like dopamine receptor signaling. Journal of Neurophysiology, 120(3): 998–1009.
- Gliotransmission and adenosinergic modulation: insights from mammalian spinal motor networks. Journal of Neurophysiology, 118(6): 3311–3327.
- A computational model to investigate astrocytic glutamate uptake influence on synaptic transmission and neuronal spiking. Frontiers in computational neuroscience, 6: 70.
- Computational modeling of spinal locomotor circuitry in the age of molecular genetics. International Journal of Molecular Sciences, 22(13): 6835.
- Coupled nonlinear oscillators and the symmetries of animal gaits. Journal of Nonlinear science, 3: 349–392.
- Dale Purves; George J. Augustine; David Fitzpatrick; William C. Hall; Anthony-Samuel LaMantia; Richard D. Mooney; Michael L. Platt; and Leonard E. White, eds. 2018. Neuroscience. Oxford University Press, 6th edition. ISBN 978-1-60535-380-7.
- Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in computational neuroscience, 9: 99.
- Effect of biologically-motivated energy constraints on liquid state machine dynamics and classification performance. Neuromorphic Computing and Engineering, 2(2): 024005.
- Minimizing energy consumption leads to the emergence of gaits in legged robots. arXiv preprint arXiv:2111.01674.
- Current principles of motor control, with special reference to vertebrate locomotion. Physiological reviews.
- The CPGs for limbed locomotion–facts and fiction. International Journal of Molecular Sciences, 22(11): 5882.
- How detailed is the central pattern generation for locomotion? Brain research, 88(2): 367–371.
- NeuroPod: a real-time neuromorphic spiking CPG applied to robotics. Neurocomputing, 381: 10–19.
- Learning to walk via deep reinforcement learning. arXiv preprint arXiv:1812.11103.
- AstroNet: When Astrocyte Meets Artificial Neural Network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 20258–20268.
- Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28.
- Astromorphic Self-Repair of Neuromorphic Hardware Systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 7821–7829.
- Izhikevich, E. M. 2007. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral cortex, 17(10): 2443–2452.
- Hebbian plasticity in CPG controllers facilitates self-synchronization for human-robot handshaking. Frontiers in neurorobotics, 12: 29.
- Kiehn, O. 2016. Decoding the organization of spinal circuits that control locomotion. Nature Reviews Neuroscience, 17(4): 224–238.
- Building Transformers from Neurons and Astrocytes. bioRxiv, 2022–10.
- Gait adaptation in a quadruped robot. Autonomous robots, 12: 301–312.
- Equations for InsP3 receptor-mediated [Ca2+] i oscillations derived from a detailed kinetic model: a Hodgkin-Huxley like formalism. Journal of theoretical Biology, 166(4): 461–473.
- CPG driven locomotion control of quadruped robot. In 2009 IEEE International Conference on Systems, Man and Cybernetics, 2368–2373. IEEE.
- SPANNER: A self-repairing spiking neural network hardware architecture. IEEE transactions on neural networks and learning systems, 29(4): 1287–1300.
- Exploring self-repair in a coupled spiking astrocyte neural network. IEEE transactions on neural networks and learning systems, 30(3): 865–875.
- A brain basis of dynamical intelligence for AI and computational neuroscience. arXiv preprint arXiv:2105.07284.
- Isometric force production by motor units of extensor digitorum communis muscle in man. Journal of neurophysiology, 40(6): 1432–1443.
- Glia accumulate evidence that actions are futile and suppress unsuccessful behavior. Cell, 178(1): 27–43.
- An astrocyte-modulated neuromorphic central pattern generator for hexapod robot locomotion on intel’s loihi. In International Conference on Neuromorphic Systems 2020, 1–9.
- On the self-repair role of astrocytes in STDP enabled unsupervised SNNs. Frontiers in Neuroscience, 14: 603796.
- mujoco-py. https://github.com/openai/mujoco-py.
- Dynamic hebbian learning in adaptive frequency oscillators. Physica D: Nonlinear Phenomena, 216(2): 269–281.
- A CPG system based on spiking neurons for hexapod robot locomotion. Neurocomputing, 170: 47–54.
- Neuronal and astrocytic localization of the cannabinoid receptor-1 in the dorsal horn of the rat spinal cord. Neuroscience letters, 329(1): 13–16.
- Sandamirskaya, Y. 2022. Rethinking computing hardware for robots. Science Robotics, 7(67): eabq3909.
- Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in neuroscience, 13.
- Sim-to-real: Learning agile locomotion for quadruped robots. arXiv preprint arXiv:1804.10332.
- MuJoCo: A physics engine for model-based control. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 5026–5033. IEEE.
- Astrocytes mediate analogous memory in a multi-layer neuron–astrocyte network. Neural Computing and Applications, 34(11): 9147–9160.
- Physiology of astroglia. Physiological reviews, 98(1): 239–389.
- Self-repair in a bidirectionally coupled astrocyte-neuron (AN) system based on retrograde signaling. Frontiers in computational neuroscience, 76.
- Wang, X. 2020. Unitree A1. https://www.unitree.com/.
- Whelan, P. J. 1996. Control of locomotion in the decerebrate cat. Progress in neurobiology, 49(5): 481–515.
- Trot gait design and CPG method for a quadruped robot. Journal of Bionic Engineering, 11(1): 18–25.
- V1 and v2b interneurons secure the alternating flexor-extensor motor activity mice require for limbed locomotion. Neuron, 82(1): 138–150.