Create and explore imaginative 3D worlds with our new product Dreamfield (opens in a new tab)
Papers
Topics
Authors
Recent
Search
2000 character limit reached

Differentiable Spike-Time Solvers

Updated 4 July 2026
  • Differentiable spike-time solvers are methods that compute gradients via implicit differentiation of threshold equations, enabling effective training of spiking neural networks.
  • They employ various formulations—timing-based, event-driven, and representation-based—to retain temporal coding while addressing the discontinuity of spike generation.
  • These methods enhance SNN performance across neuron models by balancing exact gradient computation with computational efficiency and model generality.

Differentiable spike-time solvers are methods for training spiking neural networks by propagating gradients through the quantities that define spiking dynamics: threshold-crossing times, reset maps, event sequences, or continuous representations of spike trains. In the exact event-driven setting, spike times are treated as implicit functions of parameters and differentiated through threshold equations, causal dependencies, or numerical root-finding; in related sub-differentiable formulations, entire spike trains are mapped to continuous layer representations so that standard backpropagation can proceed without per-time-step surrogate derivatives (Lee et al., 2022, Yamamoto et al., 2022, König et al., 9 Mar 2026, Meng et al., 2022, Holberg et al., 2024). Across these lines of work, the central objective is to retain temporal coding while avoiding the usual incompatibility between hard thresholding and gradient-based optimization.

1. Conceptual scope and historical lines

The modern literature contains three closely related formulations of differentiability in spiking systems. One line derives exact gradients directly from spike times and threshold conditions. Another line differentiates numerical solvers that integrate continuous dynamics and locate discrete events. A third line avoids direct spike-time differentiation by encoding a spike train into a continuous spike representation and proving that the resulting layer map is sub-differentiable (Banerjee, 2014, Lee et al., 2022, Meng et al., 2022, König et al., 9 Mar 2026).

Formulation Differentiated object Representative papers
Timing-based exact methods Implicit spike times and reset paths (Banerjee, 2014, Lee et al., 2022, Yamamoto et al., 2022)
Representation-based methods Layerwise spike representation and clamp map (Meng et al., 2022)
Event-driven autodiff solvers ODE/SDE trajectories, root-finding, resets (König et al., 9 Mar 2026, Holberg et al., 2024)

A recurrent misconception in the field has been that exact gradients for SNNs with respect to weights do not exist because spike generation is discontinuous. The forward-propagation framework explicitly challenges this view: by applying the implicit function theorem to discrete spike times, it proves that SNNs have well-defined gradients with respect to their weights, even though they are non-differentiable in time (Lee et al., 2022). Earlier timing-based work had already shown that precise spike-train-to-spike-train learning can be handled in multilayer feedforward networks by differentiating output spike times and propagating those sensitivities through intermediate spikes (Banerjee, 2014).

This suggests that “differentiability” in spiking models should not be identified solely with smooth membrane trajectories. In the literature surveyed here, differentiability is often piecewise, implicit, event-conditioned, or numerical-solver-relative rather than globally smooth in the elementary ANN sense.

2. Deterministic event formulations

A canonical starting point is the leaky integrate-and-fire formulation used in timing-based backpropagation without single-spike restrictions. For neuron ii,

τmdVi(t)dt=−Vi(t)+Ii(t),\tau_m \frac{dV_i(t)}{dt} = -V_i(t) + I_i(t),

with postsynaptic current

Ii(t)=∑jwji∑kκ(t−tjk),κ(t)=βIe−t/τIθ(t).I_i(t) = \sum_j w_{ji}\sum_k \kappa(t-t_j^k), \qquad \kappa(t)=\beta_I e^{-t/\tau_I}\theta(t).

A spike is emitted at the smallest tikt_i^k such that Vi(tik)=VthV_i(t_i^k)=V_{th}, after which Vi(tik+)←0V_i(t_i^{k+})\leftarrow 0, while Ii(t)I_i(t) does not reset, so past spikes continue to contribute to future current (Yamamoto et al., 2022).

The forward-propagation framework recasts continuous-time LIF neurons in a post-synaptic-kernel form. If Fi={ti1,ti2,… }\mathcal F_i=\{t_i^1,t_i^2,\dots\} denotes the sorted output spike times of neuron ii, then

yji(t)=∑f∈Fjhji(t−f),hji(t)=e−αjt−e−βitβi−αju(t),y_{ji}(t)=\sum_{f\in\mathcal F_j} h_{ji}(t-f), \qquad h_{ji}(t)=\frac{e^{-\alpha_j t}-e^{-\beta_i t}}{\beta_i-\alpha_j}u(t),

and the membrane potential becomes

Vi(t)=∑j∈NiWjiyji(t)−θi∑f∈Fihin(t−f),V_i(t)=\sum_{j\in\mathcal N_i}W_{ji}y_{ji}(t)-\theta_i\sum_{f\in\mathcal F_i}h_i^n(t-f),

with each output spike time satisfying the threshold condition Vi(tik)=θiV_i(t_i^k)=\theta_i (Lee et al., 2022).

An earlier multilayer feedforward formulation based on the Spike-Response Model employs a similar event-centered perspective but assigns virtual weights to spikes rather than synapses. Its membrane potential is written as

P(t)=∑i∈Γ∑j∈Fiwi,j ξi(t−ti,jI)+∑k∈FOη(t−tkO),P(t)=\sum_{i\in\Gamma}\sum_{j\in\mathcal F_i} w_{i,j}\,\xi_i(t-t_{i,j}^I) +\sum_{k\in\mathcal F_O}\eta(t-t_k^O),

where ξi\xi_i is the PSP kernel and η\eta is the afterhyperpolarization kernel. This device permits a perturbation analysis of individual spike times and weights, with online-style updates derived from first-order Taylor expansions (Banerjee, 2014).

Across these deterministic formulations, spike times are not auxiliary outputs; they are the primary state variables through which synaptic interactions, resets, and learning signals are routed.

3. Exact gradient mechanisms

The basic exact-gradient identity in threshold-based SNNs follows from differentiating the firing equation ϕi,k(W,t)=Vi(t)−θi=0\phi_{i,k}(W,t)=V_i(t)-\theta_i=0 at t=tikt=t_i^k. For a weight ww,

∂tik∂w=−(∂Vi/∂w∂Vi/∂t)t=tik.\frac{\partial t_i^k}{\partial w} = - \left( \frac{\partial V_i/\partial w}{\partial V_i/\partial t} \right)_{t=t_i^k}.

For w=Wjiw=W_{ji}, the numerator is yji(tik)y_{ji}(t_i^k), and the denominator is the time derivative of the full membrane potential, including reset contributions from previous self-spikes. The resulting derivative has a closed form in terms of smooth exponential kernels (Lee et al., 2022).

Banerjee’s multilayer spike-train learning framework begins from an error functional that compares desired and observed spike trains by their putative impact on a virtual postsynaptic neuron. The kernel family is

fβ,τ(t)=1τexp⁡ ⁣(−βte−t/τ),f_{\beta,\tau}(t)=\frac{1}{\tau}\exp\!\Bigl(-\frac{\beta}{t}e^{-t/\tau}\Bigr),

and integrating out the nuisance parameters (β,τ)(\beta,\tau) yields a closed-form spike-train discrepancy E(sD,sO)E(s^D,s^O) with E≥0E\ge 0 and E=0E=0 iff the two spike-time vectors coincide. Gradients are then decomposed into ∂E/∂tiO\partial E/\partial t_i^O, ∂tlO/∂wi,j\partial t_l^O/\partial w_{i,j}, and ∂tlO/∂ti,jI\partial t_l^O/\partial t_{i,j}^I, where sums over k<lk<l capture indirect through-spike effects (Banerjee, 2014).

Yamamoto et al. derive timing-based backpropagation by changing variables from spike time to

zjk≔etjk/τI.z_j^k \coloneqq e^{t_j^k/\tau_I}.

Within each inter-spike interval, the membrane potential admits the closed form

Vj(z)=Ajz−Bjzp,p=τI/τV.V_j(z)=A_j z - B_j z^p, \qquad p=\tau_I/\tau_V.

The next spike time zjk+1z_j^{k+1} is the smallest root of Vj(z)=VthV_j(z)=V_{th}, and for p=2p=2,

zjk+1=Aj−Aj2−4VthBj2Vth.z_j^{k+1} = \frac{A_j-\sqrt{A_j^2-4V_{th}B_j}}{2V_{th}}.

Backpropagation proceeds through downstream spike times and output quantities by the chain rule, with

∂L∂t=1τIz∂L∂z.\frac{\partial L}{\partial t} = \frac{1}{\tau_I} z \frac{\partial L}{\partial z}.

Weight gradients are then expressed as ∂L/∂wij=∑k(∂L/∂tjk)(∂tjk/∂wij)\partial L/\partial w_{ij}=\sum_k (\partial L/\partial t_j^k)(\partial t_j^k/\partial w_{ij}), where ∂tjk/∂wij\partial t_j^k/\partial w_{ij} is obtained by differentiating the spike-time root formula (Yamamoto et al., 2022).

These derivations share a structural property: the discontinuity of the spike itself is not differentiated directly as a Heaviside surrogate. Instead, the threshold equation defining the event is differentiated, and reset-induced dependencies are handled explicitly.

4. Causality, multiple spikes, and memory structure

A major limitation of many conventional timing-based methods is the single-spike restriction, often associated with time-to-first-spike coding. Yamamoto et al. remove this restriction: each neuron jj may fire Mj≥0M_j\ge 0 times, indexed by tj1,…,tjMjt_j^1,\dots,t_j^{M_j}. Every term in the spike-time coefficients AA and BB, and in their derivatives, sums over all previous presynaptic spikes and all previous self-spikes nj<kn_j<k. Gradient backpropagation likewise sums reset paths from each spike to all later spikes of the same neuron, and synaptic paths into many postsynaptic spikes (Yamamoto et al., 2022).

The forward-propagation algorithm organizes these dependencies through a time-sorted event list and a causality graph. For each new spike s=(i,k)s=(i,k), it records the set of input spikes that actually contributed and the last reset time of the neuron. The local linear system

L⋅(∂F∂W)=−(∂V∂W)L\cdot \Bigl(\frac{\partial \mathcal F}{\partial W}\Bigr) = - \Bigl(\frac{\partial \mathbb V}{\partial W}\Bigr)

is solved by back-substitution for the new row because LL is strictly lower-triangular plus diagonal. This yields forward-time exact gradient accumulation without full matrix inversion (Lee et al., 2022).

These exact timing-based methods incur characteristic path-combinatorial costs. Naïve timing-based backprop explores O(#spikes2)O(\#\text{spikes}^2) reset paths. In Yamamoto et al., a two-stage loop prunes neurons with no further spikes after N1N_1 steps to reduce memory; with two-stage partitioning, memory drops from O(J⋅M⋅M)O(J\cdot M\cdot M) to roughly O(J⋅N1⋅M+#active⋅(N2−N1)⋅M)O(J\cdot N_1\cdot M + \#active\cdot (N_2-N_1)\cdot M), and the implementation is GPU-parallelizable over j,m,kj,m,k loops (Yamamoto et al., 2022).

The empirical interpretation offered in the same work is that allowing multiple spikes naturally improves the computational capacity of SNNs. A plausible implication is that differentiable spike-time solvers become more expressive when they model not only the first threshold crossing but also the entire reset-mediated event cascade of each neuron.

5. Differentiable numerical solvers for ODE and SDE neuron models

Closed-form spike-time expressions are available only for restricted neuron classes and parameter regimes. Eventax addresses this limitation by combining differentiable numerical ODE solvers with event-based spike handling in JAX. In this framework, each neuron is treated as a hybrid dynamical system whose continuous state x(t)∈Rdx(t)\in\mathbb R^d evolves according to

dxdt=f(x(t),t,u(t);θ),\frac{dx}{dt}=f(x(t),t,u(t);\theta),

while spikes are defined by an event function gj(x(t),t)=Vj(t)−θj=0g_j(x(t),t)=V_j(t)-\theta_j=0. Diffrax monitors g(t,y)g(t,y) at each step, and if it switches from negative to positive, a built-in root-finder using bisection plus secant locates teventt_{event} to any desired tolerance, such as 10−6 ms10^{-6}\,\mathrm{ms}. At the exact event time, a model-specific reset mapping x(tevent+)=R(x(tevent−))x(t_{event}^+)=R(x(t_{event}^-)) is applied, and reverse-mode autodiff propagates through the ODE integration, root-finding, and reset callbacks exactly, without surrogate approximations (König et al., 9 Mar 2026).

Eventax exposes this mechanism through a NeuronModel interface in Equinox with methods such as init_state(n), dynamics(t, y), spike_condition(t, y), input_spike(y, w), and reset_spiked(y, mask). Feed-forward and recurrent networks are assembled through classes such as FFEvNN and EvRNN, while .ttfs(in_spikes) and .state_at_t(in_spikes, T) provide task-level interfaces. The framework supports Leaky Integrate-and-fire, Quadratic Integrate-and-fire, Exponential Integrate-and-fire, Izhikevich, Event-based Gated Recurrent Unit, and a multi-compartment neuron with a model of dendritic spikes in human layer 2/3 cortical Pyramidal neurons (König et al., 9 Mar 2026).

The stochastic extension replaces ODE dynamics by Event SDEs driven by càdlàg rough paths. A neuron state yt∈Rey_t\in\mathbb R^e evolves, between spike times, according to

dytn=μ(ytn) dt+σ(ytn) ⋄dXt,dy_t^n=\mu(y_t^n)\,dt+\sigma(y_t^n)\,\diamond dX_t,

with event times

τn+1=inf⁡{t>τn:E(ytn)=0},yτnn=T(yτnn−1).\tau_{n+1}=\inf\{t>\tau_n:\mathcal E(y_t^n)=0\}, \qquad y_{\tau_n}^n=\mathcal T(y_{\tau_n}^{n-1}).

Under the standing assumptions μ∈Lip1\mu\in \mathrm{Lip}^1, σ∈Lip2+ϵ\sigma\in \mathrm{Lip}^{2+\epsilon}, ∇E(y)σ(y)=0\nabla\mathcal E(y)\sigma(y)=0 on {E=0}\{\mathcal E=0\}, ∇E(y)μ(y)≠0\nabla\mathcal E(y)\mu(y)\neq 0 on {E=0}\{\mathcal E=0\}, and σ(T(y))=(∇T)(y)σ(y)\sigma(\mathcal T(y))=(\nabla \mathcal T)(y)\sigma(y) on {E=0}\{\mathcal E=0\}, one obtains almost sure differentiability of (τn,ytn)(\tau_n,y_t^n) with respect to the initial condition. The event-time gradient satisfies

∂τn=−∇E(yτn n−1) ∂yτn n−1∇E(yτn n−1) μ(yτn n−1),\partial \tau_n = - \frac{\nabla \mathcal E(y_{\tau_n}^{\,n-1})\,\partial y_{\tau_n}^{\,n-1}} {\nabla \mathcal E(y_{\tau_n}^{\,n-1})\,\mu(y_{\tau_n}^{\,n-1})},

and the implementation is provided as part of the diffrax library (Holberg et al., 2024).

Together, these developments move differentiable spike-time solvers from analytically tractable LIF settings toward arbitrary neuron models defined by ODEs and stochastic spiking neural networks with noise affecting both spike timing and network dynamics.

6. Representation-based alternatives and the exact-versus-surrogate debate

Differentiation on Spike Representation (DSR) is not a spike-time solver in the narrow event-equation sense, but it is directly relevant because it replaces non-differentiable spike trains by continuous spike representations and shows that the induced layer mapping is sub-differentiable. For a spike train si[n]∈{0,1}s^i[n]\in\{0,1\}, DSR defines a layer representation oi=r(si)\mathbf o^i={\mathrm r}(\mathbf s^i). In the IF model this is a simple average rate, while in the LIF model it is an exponentially weighted rate with λi=exp⁡(−Δt/τi)\lambda_i=\exp(-\Delta t/\tau^i). Under mild stationarity and bounded-membrane-voltage assumptions, as N→∞N\to\infty,

zi≈gWi(zi−1)=clamp(Wizi−1,0,bi),\mathbf z^i \approx g_{W^i}(\mathbf z^{i-1})=\mathrm{clamp}(W^i\mathbf z^{i-1},0,b_i),

where bi=Vthib_i=V_{th}^i for IF and bi=Vthi/Δtb_i=V_{th}^i/\Delta t for LIF, and the clamp is piecewise-linear and everywhere sub-differentiable (Meng et al., 2022).

This framework explicitly contrasts with both back-propagation through time and per-step surrogate gradients. DSR trains weights and per-layer thresholds through gradients of the clamp mapping, analyzes finite-NN representation error into quantization error eqe_q and deviation error ede_d, and establishes the bound

∣oji−[Wizi−1]j∣≤biN.\left|o_j^i-[W^i\mathbf z^{i-1}]_j\right|\le \frac{b_i}{N}.

To reduce quantization error, it trains the threshold VthiV_{th}^i in each layer and introduces a firing threshold offset α∈[0,1]\alpha\in[0,1], replacing s[n]=H(U[n]−Vth)s[n]=H(U[n]-V_{th}) with s[n]=H(U[n]−αVth)s[n]=H(U[n]-\alpha V_{th}). In the IF constant-input case, α=0.5\alpha=0.5 makes the firing-rate quantization error minimal (Meng et al., 2022).

The relationship between exact event-time methods and surrogate gradients is treated explicitly in the forward-propagation work. There, surrogate-gradient methods are described as replacing the hard threshold 1V>θ\mathbf 1_{V>\theta} with a smooth proxy σ(V−θ)\sigma(V-\theta) during backprop, yielding an approximate ∂V/∂t\partial V/\partial t, whereas exact methods obtain the needed sensitivity from the actual post-synaptic and refractory kernels. The same source notes that in single-spike or low-rate regimes, many surrogate shapes coincide roughly with the actual implicit-function denominator and numerator, which helps explain why some surrogates perform well (Lee et al., 2022).

Accordingly, the controversy is not simply exactness versus usefulness. Exact event-time solvers offer analytical or solver-relative correctness, but their scope may depend on model structure, regularity assumptions, or event-detection overhead. Representation-based and surrogate approaches relax different parts of the problem, often improving applicability or latency.

7. Empirical behavior, operating regimes, and limitations

Timing-based exact methods have demonstrated competitive empirical performance. On MNIST with a 784 ⁣− ⁣400 ⁣− ⁣10784\!-\!400\!-\!10 feedforward architecture, the multi-spike timing-based SNN of Yamamoto et al. reaches 98.43%±0.05%98.43\%\pm 0.05\%, outperforming its single-spike TTFS variant at approximately 97.96%97.96\% and matching the best rate-coded SNNs. In practice, τI≈τV≈1.0\tau_I\approx\tau_V\approx 1.0 yields approximately 1 ⁣− ⁣21\!-\!2 spikes per neuron on MNIST; very small τ\tau leads to explosion of spikes, very large τ\tau yields nearly single-spike behavior, and empirically τI≈0.8\tau_I\approx 0.8 achieves best test accuracy. The same study reports that allowing multiple spikes produces an optimal τI\tau_I at which performance peaks, which was not seen in conventional single-spike TTFS coding (Yamamoto et al., 2022).

Eventax reports that accuracy on Yin–Yang and MNIST matches or exceeds prior continuous-time event-based methods and outperforms discrete-time surrogate-gradient methods when temporal precision matters. LIF, QIF, EIF, and Izhikevich models all train to greater than 97%97\% on Yin–Yang; QIF with integral loss reaches greater than 99%99\%; and latency-encoded MNIST reaches 97.5%97.5\%. Throughput scales linearly with batch size via jax.vmap, while smaller ODE step sizes increase accuracy but cost more CPU/GPU time. Using Diffrax adaptive solvers and recursive-checkpoint adjoint lowers memory overhead compared to naïve backprop-through-time, at the cost of solver-call overhead (König et al., 9 Mar 2026).

DSR emphasizes the latency–accuracy regime. Because the representation error shrinks as $1/N$, DSR-trained SNNs can reach ANN-comparable accuracy, for example within 0.1%0.1\%, with only N≈20N\approx 20 time steps on CIFAR-10, CIFAR-100, and ImageNet; typical measured firing sparsity is at most 10%10\% across layers; and empirically DSR models lose less than 1%1\% accuracy when going from N=20N=20 down to N=5N=5. Its analysis also states the principal limitations: the infinite-time mapping is exact only as N→∞N\to\infty, quantization error may impair learning for very small NN such as 2 ⁣− ⁣32\!-\!3, and one must choose Δt≪τi\Delta t\ll \tau^i to justify the LIF averaging (Meng et al., 2022).

The earlier spike-train-to-spike-train learning results are more diagnostic than benchmark-oriented. In a single-layer setting with one neuron and 10 synapses, homogeneous 10 Hz Poisson inputs led to approximately 93%93\% of random witness-learning pairs improving MAPE after 50 k updates, while inhomogeneous 0 ⁣− ⁣100\!-\!10 Hz inputs yielded approximately 99%99\% improvement after 10 k updates and complete convergence on 50 test neurons. In a two-layer 5 ⁣→ ⁣5 ⁣→ ⁣15\!\to\!5\!\to\!1 network with 30 synapses, 32 out of 50 witness-learning pairs fully converged and 18 diverged; the reported causes were non-informative spike-rate extremes, with too high a rate leading to non-uniqueness and too low a rate producing vanishing gradients. The same experiments identified an excitatory–inhibitory asymmetry: when some hidden neurons were inhibitory, their synapses converged at least 5×5\times slower because inhibitory PSPs rise slowly at the threshold crossing, making ∣∂tO/∂tH∣|\partial t^O/\partial t^H| smaller (Banerjee, 2014).

For stochastic spiking systems, the rough-path framework states a different class of limitations. Exact pathwise gradients are guaranteed only under the stated regularity, transversality, threshold-orthogonality, and reset-diffusion commutation assumptions. Within those conditions, the framework provides exact pathwise gradients of solution trajectories and event times, a general-purpose loss function based on signature kernels indexed on càdlàg rough paths, and an end-to-end autodifferentiable solver for Event SDEs. It is described as, to the authors’ knowledge, the first framework enabling gradient-based training of stochastic spiking neural networks with noise affecting both spike timing and network dynamics (Holberg et al., 2024).

Taken together, these results show that differentiable spike-time solvers now span exact analytical timing gradients, differentiable event-aware ODE solvers, and Event SDE formulations, while adjacent spike-representation methods provide a sub-differentiable alternative for low-latency regimes. A plausible implication is that future work will continue to negotiate the same three-way trade-off visible across the literature: event-level exactness, neuron-model generality, and computational efficiency.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Differentiable Spike-Time Solvers.