Nonleaky Neuron Models: Integration and Sparsity

• Nonleaky neuron models are idealized spiking units that integrate synaptic inputs without decay, accumulating signals perfectly until a threshold is reached.
• They exhibit an all-pass frequency response and allow closed-form gradient backpropagation, enhancing trainability with simplified dynamics.
• Empirical studies show these models achieve maximal sparsity and energy efficiency, though they are more vulnerable to input noise compared to leaky variants.

Nonleaky neuron models comprise a class of spiking neuron abstractions that omit any leak (decay) path in their membrane potential dynamics. This nonleaky idealization, most commonly instantiated as the nonleaky integrate-and-fire (IF) model, allows the neuron's internal voltage to integrate input events perfectly over time until an emission threshold is crossed, at which point the voltage is reset. Owing to the absence of low-pass filtering, these models forgo biological realism in favor of computational simplicity and maximal event sparsity, while exhibiting distinct statistical and dynamical properties compared to both their leaky integrate-and-fire (LIF) and their more biofidelic stochastic analogs.

1. Mathematical Formulation of Nonleaky Neuron Models

The canonical discrete-time nonleaky IF neuron updates its membrane potential $U[t]$ according to: $U[t+1] = U[t] + W\,O_\mathrm{in}[t+1]$ where $O_\mathrm{in}[t+1]$ denotes the input spike train (or its weighted sum), and $W$ is the synaptic weight. When $U[t] \ge V_\mathrm{th}$ (threshold), the neuron fires ($O[t]=1$) and resets $U[t]\leftarrow 0$.

In continuous time, nonleaky IF neurons correspond to $\tau_m \to \infty$ in the generic LIF equation, thus eliminating the decay (leak) term. The generalization for a population of $N$ nonleaky neurons (stochastic interaction model) yields a piecewise-deterministic Markov process (PDMP) with the state vector $U(t) = (U_1(t), ..., U_N(t)) \in \mathbb{R}_+^N$. Between spikes, the evolution follows: $\frac{dU_i(t)}{dt} = -\lambda (U_i(t) - \bar U(t))$ where $\bar U(t)$ is the mean potential, capturing the effect of electrical synapses. Spike events (point processes with rate $\varphi(U_i)$) induce resets and instant "kicks" according to the synaptic coupling.

In the single-spike, temporally coded regime, the nonleaky membrane equation collapses to: $$\frac{dv_j(t)}{dt} = \sum_{i=1}^N w_{ji}\,g(t-t_i),\quad v_j(0)=0$$ Here, $g(\cdot)$ is the synaptic kernel, and the solution for the output spike time $t_j$ becomes a closed-form affine or simple log-ratio function of input spike times, depending on $g$.

2. Comparative Dynamics: Nonleaky vs. Leaky IF Neurons

The absence of membrane leak in IF neurons ($\alpha=1$) turns them into perfect integrators. Unlike their LIF counterparts, which "forget" inputs exponentially, nonleaky IF neurons sum all prior synaptic inputs since the last reset. They lack intrinsic low-pass filtering and are thus highly sensitive to rapid fluctuations and input noise.

A critical frequency-domain distinction is evident through their transfer functions: $$H_\mathrm{LIF}(j\omega) = \frac{1}{1+j\omega\tau_m} \qquad H_\mathrm{IF}(j\omega) = 1$$ The LIF neuron acts as a first-order low-pass filter, suppressing input above $1/\tau_m$, whereas the IF neuron is all-pass—transmitting all input frequencies to the spike-triggering mechanism.

Coherence analysis confirms that for IF models, the output spike train's statistical fidelity to the input is constant across all frequencies (i.e., $C_\mathrm{IF}(\omega)\equiv1$), in contrast to the decaying coherence at high frequencies for LIF models.

3. Trainability and Functional Complexity

Nonleaky IF neurons, especially in the single-spike, temporally coded setting, possess input–output relations of low algebraic complexity. With, for example, a unit-step synaptic kernel, the output spike time $t_j$ is exactly affine in the set of input spike times. For an exponential kernel, $t_j$ resolves to a log-ratio of linear combinations: $$t_j = \tau\ln\!\left(\frac{\sum_{i\in\mathcal{C}_j}w_{ji}\,e^{t_i/\tau}}{\sum_{i\in\mathcal{C}_j}w_{ji}-v_0/\tau}\right)$$ This leads to a well-behaved (constant) Jacobian and enables straightforward analytic backpropagation of gradients, facilitating efficient optimization with standard algorithms such as Adam. In contrast, leaky counterparts require solving transcendental equations (e.g., involving the Lambert-W function), yielding highly nonlinear, parameter-sensitive mappings and non-constant Jacobians. This hierarchy in complexity is directly implicated in the relative trainability and stability of networks constructed from these units.

4. Computational and Statistical Properties

Extensive experimentation on tasks such as CIFAR-10 and SVHN demonstrates that nonleaky IF SNNs achieve maximal activity sparsity and reduced synaptic operation counts compared to LIF networks. For instance, in time-unrolled SNNs (100 ms, $\Delta t=1$ ms):

• IF models (infinite $\tau_m$) attain spike activity rates as low as $4.94\%$ (CIFAR-10) and $11.85\%$ (SVHN), with up to $7.18\times10^8$ synaptic operations, compared to LIF's higher rates.
• However, IF models display pronounced fragility under increasing input noise—classification accuracy deteriorates rapidly beyond the leaky regime.
• At equal training loss, IF models show higher test set error, indicative of greater overfitting and compromised generalization.

A summary table highlights quantitative metrics:

$\tau_m$ (ms)	CIFAR-10 Acc. (%)	CIFAR-10 Spikes (%)	CIFAR-10 SynOps
30	89.65	9.45	$1.59\times10^9$
100	90.19	5.26	$7.92\times10^8$
$\infty$	90.30	4.94	$7.18\times10^8$

Analogous patterns are found for SVHN. The trend is that increasing leak decreases sparsity and increases computational cost, while improving robustness and generalization.

5. Dynamical Systems and Ergodicity in Nonleaky Populations

In networked settings, nonleaky neuron models governed by stochastic spiking (rates modulated by $\varphi(U_i)$) and electrical synapse coupling ($-\lambda(U_i-\bar U(t))$) maintain persistent, ergodic spiking dynamics, provided the synaptic interaction graph is connected. Specifically:

• There exists a unique invariant measure $\pi$ on $\mathbb{R}_+^N$, indicating maintenance of activity and absence of extinction.
• The system converges exponentially quickly to this invariant law, with explicit (though implicit in terms of parameters) bounds on the rate: $$\|P_u(U(t)\in\cdot) - \pi(\cdot)\|_\mathrm{TV} \leq C(1+\|u\|)e^{-\rho t}$$ Connectivity and the positivity of all interaction weights are essential; the presence of leak ($\alpha>0$) extinguishes activity almost surely.

This suggests that, for recurrently coupled populations, the nonleaky idealization ensures sustained collective activity, which is otherwise disrupted by any nonzero leak term.

6. Application Domains and Empirical Performance

Empirical results from temporally coded, nonleaky SNNs on standard network intrusion detection tasks (e.g., NSL-KDD and AWID) show superior discriminative accuracy compared to conventional DNNs and legacy machine learning models. Reported results include:

• NSL-KDD: SNN accuracy of $0.9931$ (resampled) versus $0.8834$ (DNN) and $0.9564$ (1D-CNN).
• AWID: SNN accuracy of $0.9984$ (resampled) versus $0.9585$ (DNN) and $0.9528$ (1D-CNN). Nonleaky SNNs leveraged their closed-form functional mappings and analytically tractable gradients to enable efficient and accurate training pipelines without surrogate-gradient approximations.

Practical guidelines emphasize using nonleaky IF neurons when energy efficiency and maximal sparsity are prioritized under low-noise conditions, or when network trainability is paramount due to their closed-form Jacobians and reduced nonlinearity. Conversely, in noise-prone environments where robustness or generalization supersede sparsity, leaky models (moderate $\tau_m$) offer considerable advantages in performance and stability.

7. Limitations, Design Trade-offs, and Model Selection

While nonleaky IF neurons confer maximum event sparsity and minimal algebraic complexity, they are intrinsically vulnerable to input noise and high-frequency perturbations, owing to their all-pass filtering property. They also tend to overfit statistical structure in data, reflected in higher generalization error for equivalent training error. Leaky models mitigate these issues via built-in low-pass filtering and improved regularization at the expense of increased synaptic event density and parameterization complexity (e.g., careful selection of $\tau_m$ to avoid vanishing spike rates).

Model selection in SNN architecture design thus crucially depends on the application's requirements in the sparsity–robustness–trainability space. Nonleaky IF neurons are optimal for low-noise, energy-constrained settings or when functional simplicity is essential for gradient-based learning, while leaky neurons remain preferable in the presence of substantial input variability or when generalization is a priority.

8. Conclusion

Nonleaky neuron models represent a non-biological but computationally favorable limiting case of spiking neuron dynamics. Characterized by perfect integration, minimal functional complexity, and maximal activity sparsity, these models offer unique advantages in learnability and computational efficiency, particularly in deep, temporally coded SNNs. However, the absence of leak precludes intrinsic stability against noise and overfitting, delimiting the contexts in which these models can be usefully deployed. Their mathematical tractability in both deterministic and stochastic regimes makes them a valuable tool for both theoretical analysis and practical SNN engineering, as documented across multiple recent works (Chowdhury et al., 2020, Zhou et al., 2020, Duarte et al., 2014).