Temporal Membrane Potential Regularization

• TMPR is a training methodology for spiking neural networks that leverages time-varying regularization to stabilize membrane potentials and sustain gradient flow.
• It integrates with Complemented Ternary Spiking Neurons in CTSN architectures to counteract issues like vanishing gradients, over-resetting, and information loss.
• Experimental evaluations on vision benchmarks show TMPR improves accuracy by up to 0.7 percentage points while maintaining low computational overhead.

Temporal Membrane Potential Regularization (TMPR) is a training methodology designed for spiking neural networks (SNNs), specifically those employing Complemented Ternary Spiking Neurons (CTSN). TMPR introduces a time-varying regularization scheme that leverages the temporal dynamics of membrane potential distributions, augmenting standard optimization procedures with gradient pathways that target the stability and informativeness of neuron states over time. This approach is developed in the context of addressing inherent limitations in conventional ternary and binary spiking neuron models, including information loss, vanishing gradients, and membrane potential irregularity, ultimately supporting enhanced accuracy and robustness on static and neuromorphic vision benchmarks (Zhang et al., 22 Jan 2026).

1. Rationale and Formulation

TMPR is motivated by the observation that spiking neuron models, even those with ternary excitatory/inhibitory outputs, exhibit temporal instabilities. The standard cross-entropy objective $\mathcal{L}_{CE}$ fails to address phenomena such as over-resetting and vanishing backpropagation signals. TMPR rectifies these deficiencies by introducing a secondary loss term,

$$\mathcal{L}_{TMPR} = \frac{1}{T\,L}\, \sum_{t=1}^{T}\frac{\lambda}{t}\sum_{l=1}^{L} \frac{[\tilde u^l(t)]^2}{B\,D^l}$$

where $T$ denotes the total timesteps, $L$ the number of layers, $B$ the batch size, $D^l$ the channel dimension per layer, and $\lambda$ is a global regularization coefficient. The early-timestep emphasis $(\lambda/t)$ applies stronger regulation when sequences are most susceptible to instability, while scaling across layers and batches ensures balanced penalization.

2. Mechanism and Gradient Propagation

TMPR directly modulates the training dynamics by injecting residual-like gradients into the membrane potential of each neuron at every timestep and layer: $$\frac{\partial\mathcal{L}_{TMPR}}{\partial \tilde u^l(t)} = \frac{2\,\lambda}{t\,T\,L\,B\,D^l}\,\tilde u^l(t)$$ This mechanism circumvents conventional temporal vanishing-gradient effects, especially pronounced when recurrent operations and resets are governed by leak constants ($\tau < 1$). By maintaining strong gradient signals throughout early timesteps and layering, TMPR creates additional backpropagation paths that facilitate stable learning of neuron parameters and reinforce temporal consistency.

3. Interaction with CTSN Architecture

TMPR is specifically designed for use with the CTSN framework, which augments standard ternary spiking neurons with a learnable complemental memory term $h(t)$: $\tilde u(t) = h(t) + x(t)$ where $x(t)$ is the synaptic input at time $t$. The update strategy for $h(t)$ differs for static-image and neuromorphic modalities, utilizing learnable scalars $\alpha$, $\beta$, $\gamma$ in $(0,1)$ (sigmoid-parametrized) to control retention and update rates. Reintroducing $h(t)$ at each timestep preserves information across hard resets and amplifies the effect of residual gradients injected by TMPR, allowing for enhanced retention and smoother membrane potential evolution.

4. Experimental Evaluation and Performance Gains

Extensive benchmarking on both static (CIFAR-10, CIFAR-100, ImageNet-100) and neuromorphic (CIFAR10-DVS) datasets demonstrates that integrating TMPR with CTSN yields quantitative improvements over prior ternary and binary SNN baselines:

Dataset	Backbone	Ternary Baseline (%)	CTSN + TMPR (%)
CIFAR-10	ResNet19 (T=4)	96.28	96.46
CIFAR-100	ResNet19 (T=4)	79.68	81.19
ImageNet-100	SEW-ResNet34 (T=4)	83.27	85.06
CIFAR10-DVS	ResNet20 (T=10)	80.30	81.23
CIFAR10-DVS	VGGSNN	—	83.20
CIFAR-100 Ablation	ResNet20 (T=6)	74.48 (CTSN only)	75.12

The addition of TMPR consistently contributes 0.3–0.7 percentage point improvements over CTSN-only configurations and surpasses best-reported SNN variants in the literature.

5. Biological Plausibility and Computational Impact

TMPR, in conjunction with CTSN, advances biological plausibility in SNN design through several facets:

• Ternary output spikes $\{-1,0,1\}$ emulate excitation–inhibition balance observed in cortical circuits.
• The complemental term $h(t)$ models sustained depolarization, akin to graded persistent activity documented in entorhinal cortex neurons.
• Layer-wise learnable parameters ($\alpha, \beta, \gamma$) induce neural heterogeneity, mirroring the diverse time constants and memory profiles inherent in biological neural populations.

From a computational standpoint, TMPR introduces lightweight regularization without altering inference-time complexity or hardware mapping. Each neuron only incurs three additional scalar multiplications per step (from CTSN) and a simple penalty computation (from TMPR), maintaining the event-driven and multiplication-free character of SNNs.

6. Implications and Future Directions

The integration of TMPR allows for explicit control of temporal membrane potential variability, counteracting instability and supporting effective gradient flow throughout spatiotemporal sequences. This suggests applicability in broader settings—anywhere membrane potential evolution is a limiting factor in performance or convergence. A plausible implication is that similar regularization strategies might enable more robust training in SNN-based reinforcement learning and neuromorphic processing, provided the temporal dynamics are sufficiently informative.

7. Summary and Context

Temporal Membrane Potential Regularization provides a direct mechanism for controlling and optimizing the temporal trajectory of neuron membrane potentials in spiking neural networks. Coupled with complemented memory dynamics (as in CTSN), TMPR establishes new state-of-the-art performance on vision benchmarks by preserving historical input information, sustaining gradients through time, and enhancing the regularity of neuron activation. It achieves this with minimal computational overhead, retaining the core efficiency advantages of SNNs while improving both accuracy and robustness (Zhang et al., 22 Jan 2026).