Learning Feedforward and Recurrent Deterministic Spiking Neuron Network Feedback Controllers (1708.02603v2)

Abstract: We address the problem of learning feedback control where the controller is a network constructed solely of deterministic spiking neurons. In contrast to previous investigations that were based on a spike rate model of the neuron, the control signal here is determined by the precise temporal positions of spikes generated by the output neurons of the network. We model the problem formally as a hybrid dynamical system comprised of a closed loop between a plant and a spiking neuron network. We derive a novel synaptic weight update rule via which the spiking neuron network controller learns to hold process variables at desired set points. The controller achieves its learning objective based solely on access to the plant's process variables and their derivatives with respect to changing control signals; in particular, it requires no internal model of the plant. We demonstrate the efficacy of the rule by applying it to the classical control problem of the cart-pole (inverted pendulum) and a model of fish locomotion. Experiments show that the proposed controller has a stability region comparable to a traditional PID controller, its trajectories differ qualitatively from those of a PID controller, and in many instances the controller achieves its objective using very sparse spike train outputs.

• The paper introduces a synaptic weight update rule that enables SNN controllers to stabilize dynamic systems without relying on internal plant models.
• Experiments on cart-pole and fish locomotion systems demonstrate that these controllers achieve stability similar to PID controllers while using sparse spike trains for energy efficiency.
• The study paves the way for neuromorphic and bio-inspired feedback systems in robotics, offering a promising alternative for adaptive, low-power control.

Overview of Learning Feedforward and Recurrent Deterministic Spiking Neuron Network Feedback Controllers

The paper presents an in-depth paper of feedback control systems implemented via deterministic spiking neuron networks (SNNs), aiming to address the challenge of learning feedback control without relying on an internal model of the plant. By leveraging deterministic spiking neurons as the sole components of feedback controllers, this work diverges from traditional controllers that utilize continuous control signals. The spiking neuron networks considered in this paper directly map precise spike timings to control signals, introducing a novel modality within hybrid dynamical systems.

Problem Definition and Methodology

The paper formulates the learning control problem in a hybrid dynamical system encompassing a plant and an SNN controller. The pivotal contribution is the derivation of a synaptic weight update rule enabling the SNN controller to maintain process variables at desired set points. This rule is developed under the assumption that the controller has access solely to the process variables of the plant and their derivatives with respect to control signals. The SNN controller is devoid of any internal plant model, presenting a nuanced approach to real-time feedback control.

This framework is explored through experiments involving two classic control problems:

1. The cart-pole (inverted pendulum).
2. A simulated fish locomotion model.

The results are indicative of the SNN controller's stability regions being similar to traditional PID controllers, with the added advantage of producing sparse spike train outputs, potentially enhancing energy efficiency.

Experimental Findings

Several insights emerge from the experiments:

• Cart-Pole Experiment: Spiking neuron network configurations with varying complexities (feedforward with single or multiple kernels and recurrent architectures) successfully controlled the inverted pendulum, although the coverage and stability were dependent on the network's complexity. Recurrent networks exhibited notably sparse spike activity while maintaining effective control.
• Fish Locomotion Experiment: The SNN controller managed to guide a simulated fish towards a target, showcasing the system's capability to coordinate multiple control signals, such as yaw, pitch, and roll synergistically. The applicability in both 2D and 3D spaces further highlights the versatility of the SNN controllers.

Implications and Future Directions

This research posits significant implications in both theoretical and practical realms. Theoretically, it opens avenues for revisiting control systems through the lens of neuromorphic computing, inferring new possibilities in how control tasks can be learned and executed in the spiking domain. Practically, such controllers may find applications in bio-inspired robotics where energy-efficient computations are pivotal.

Future developments may explore the incorporation of kernel adaptations for filtering process variables and broader error functions tailored to specific control cost objectives. These extensions could refine the controllers' adaptability and robustness, potentially broadening the scope of applications in real-world scenarios.

Overall, the paper invites further exploration into leveraging the inherent sparsity and energy efficiency of spiking neuron networks in feedback control systems, suggesting a paradigm shift toward biologically plausible controllers for complex dynamical environments.