Delay-Spike Approach in Neural Networks

• Delay-Spike Approach is a design principle that treats temporal delays and spike times as active control variables, enhancing timing alignment in neural computations.
• Various architectures implement delay strategies through explicit skip delays, learnable axonal shifts, and state-space delay vectors to balance accuracy and latency.
• Learnable delay parameters are optimized via event-based gradients and local plasticity, yielding improved performance metrics and more efficient spike-based inference.

The “Delay-Spike Approach” (Editor’s term) denotes a family of methods in which temporal delays, spike times, or latency offsets are explicit control variables rather than passive side effects of computation. In the sources associated with this term, delays are learned to align branches in Time-To-First-Spike (TTFS) networks, attached to synapses or axons to increase temporal expressivity, used to decide when spike-based inference should stop, compensated when comparing sparse spike trains, and imposed as Spike Trains of Uneven Duration and Delay in laser–plasma control (Kim et al., 2023, Chen et al., 2023, Kreuz et al., 2022, Afeyan et al., 2012). This suggests that the phrase is best understood as a design principle centered on shaping timing itself.

1. Temporal coding as the substrate

In TTFS coding each neuron is allowed to fire at most one spike during a forward pass, and all information is carried by when that spike occurs, not by how many spikes are emitted (Kim et al., 2023). In the non-leaky integrate-and-fire formulation used for TTFS skip-connection analysis, the membrane potential is

$$V(t) = \sum_{k=1}^N U(t - t_k)\, w_k \left(1 - e^{-(t - t_k)}\right),$$

and the output spike time can be written in $z$-transform notation as

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$

Because the causal set $C$ contains only spikes that arrive before firing, any architectural change that shifts spikes later in time directly changes which inputs are even counted. In this setting, delay is not auxiliary; it is part of the code itself.

A related but more radical formulation appears in Delay Neural Networks, where the information of a neuron is computed based on the sum of its input synaptic delays and on the spike times of the electrical potentials received from other neurons, rather than on the sum of amplitudes of electrical potentials (Gattepaille et al., 10 Jan 2025). In the form emphasized there, the neuron output is built from delayed spike times through a temporal kernel,

$$t_{j} = \sum_i \operatorname{sign}(d_{ij}^{s}) \Big[ \kappa(z_{i} + d_{ij}) - \kappa(z_{i} + 1) \Big],$$

with a temporal softmin used for classification. This suggests a continuum of delay-spike formalisms ranging from threshold-based TTFS neurons to analytic delay-coded networks in which time is the primary state variable.

2. Architectural forms of delay shaping

The most direct architectural use of the approach is to place explicit delays on computational branches. In TTFS skip connections, addition-based skips and concatenation-based skips behave differently because they operate on spike times rather than on static activations (Kim et al., 2023). Addition-based skips introduce an additional delay; the paper states that adding a spike from a skip connection at time $t_A$ and a spike after convolutional operation at time $t_B$ results in the output at time $t_A + t_B$. Concatenation-based skips avoid that additional delay, but create a time gap between the convolutional branch and the skip branch. The proposed remedy is a learnable delay block,

$D(X \mid \theta_l) = X + \theta_l,$

inserted on the skip path so that channel-wise delays $\theta_{l,c}$ can align branch timings.

Other architectures place delay at different granularities. The Rectified Axonal Delay module models a learnable, layer-wise temporal shift on spike trains before they reach the next layer,

$z$0

with rectification constraining the effective delay to $z$1 (Sun et al., 2022). In delay learning with Dilated Convolutions with Learnable Spacings, each synapse is represented by a sparse temporal kernel with one non-zero entry whose position encodes delay and whose amplitude encodes weight; Gaussian interpolation is used during training and the kernels are collapsed to discrete delays at inference (Hammouamri et al., 2023). TSkips move the same idea to a macro-architectural scale by connecting layer $z$2 at time $z$3 to layer $z$4 at time $z$5,

$z$6

thereby adding explicit temporal delay connections across both depth and time (Malettira et al., 2024).

A further generalization is the state-space view of delays, where each neuron carries an additional delay state vector that stores a finite history of its inputs (Karilanova et al., 1 Dec 2025). With $z$7, the delay state becomes

$z$8

Delays are then not attached to a single synapse or skip path, but embedded as structured temporal memory inside the neuron state. This suggests that the approach includes at least three architectural families: explicit path delays, learnable transmission delays, and state-embedded delay memories.

3. Learning rules and delay-adaptive inference

One branch of the literature treats delays as trainable parameters in the same optimization loop as weights. DelGrad derives exact event-based gradients for both weights and delays using only spike times, not membrane traces (Göltz et al., 2024). Delays enter as additive shifts $z$9, and the delay gradient takes the form

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$0

The paper emphasizes that the inclusion of delays in the training process emerges naturally within the formalism, enriching the model’s search space with a temporal dimension. A related event-based method for recurrent SNNs reports exact gradients with respect to weights and delays, supports multiple spikes per neuron, uses less than half the memory of the current state-of-the-art delay-learning method, and is up to 26x faster (Mészáros et al., 13 Jan 2025).

A second branch uses local delay plasticity rather than end-to-end backpropagation. In propagation-delay learning, the adaptive variable is the axonal or dendritic delay $$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$1, not the synaptic weight $$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$2 (Farner et al., 2022). For presynaptic spikes arriving within a 10 ms window before a postsynaptic spike, the update rule is

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$3

which shifts earlier-than-average arrivals later and later-than-average arrivals earlier. The stated objective is to align the arrival times of all presynaptic spikes that causally contribute to a postsynaptic spike so that they cluster in time just before firing.

A third branch uses delays to decide when inference should terminate. SpikeCP wraps any pre-trained SNN classifier and constructs prediction sets $$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$4 at multiple checkpoints using nonconformity scores such as

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$5

then stops at

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$6

With Bonferroni correction, the method guarantees

$$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$7

under exchangeability (Chen et al., 2023). Here the delay variable is not a synaptic parameter but the stopping time itself, adapted per input from spike-derived evidence.

4. Reported performance across tasks

On TTFS image recognition and scientific machine-learning tasks, learnable delay on concatenation-based skips produced the best reported accuracy–latency tradeoff in the cited study (Kim et al., 2023). On MNIST, the baseline achieved 97.9% with latency 3.53, addition-based skip achieved 98.4% with latency 4.57, concatenation-based skip achieved 98.4% with latency 2.16, and concatenation + Delay achieved 98.5% with latency 1.88. On Fashion-MNIST, the corresponding figures were 90.7% and 3.71 for the baseline, 91.1% and 4.89 for addition-based skip, 90.6% and 2.28 for concatenation, and 91.4% and 2.29 for concatenation + Delay. On the wave-equation source-localization task, concatenation + Delay reached 99.74% with latency 2.83 for $$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$8 labeling and 98.19% with latency 3.45 for $$z_{\text{out}} = \frac{\sum_{k\in C} w_k z_k}{\sum_{k\in C} w_k - 1}.$$9 labeling.

On event-based vision and audio, explicit temporal delay skips similarly improved performance (Malettira et al., 2024). For DSEC-flow, the Base model moved from Average Endpoint Error 1.35 to 1.12 with a forward TSkip and to 1.13 with a backward TSkip. For DVS128 Gesture, the baseline SNN reported 88.75% accuracy, while forward-only TSkips reported 94.82%, backward-only 95.97%, and forward + backward 97.52%. On SHD, a deeper baseline at 86.42% increased to 94.73% with forward + backward TSkips; on SSC, a deeper baseline at 67.54% increased to 80.23%. The paper’s abstract summarizes these gains as up to 18% reduction in AEE on DSEC-flow, 8% increase in classification accuracy on DVS128 Gesture, and up to 8% and 16% higher classification accuracy on SHD and SSC, respectively.

Layer-wise delay modulation also shows task dependence (Sun et al., 2022). On NMNIST, introducing rectified axonal delay changed 99.26 ± 0.02% to 99.33 ± 0.03%. On DVS Gesture, it changed 95.64 ± 0.65% to 96.21 ± 0.63%. On N-TIDIGITS18, the contrast was much larger: 78.86 ± 0.47% with $C$0, 93.83 ± 0.10% with $C$1, and 94.19 ± 0.18% with $C$2. This suggests that delay learning is especially consequential when the task’s discriminative structure is strongly temporal.

Delay-centric models also report favorable parameter and energy profiles. Delay Neural Networks obtained 97.57% on DVS Gesture with 19,216 parameters and 7,895 active, and 97.73% on GSC with 175,467 parameters (Gattepaille et al., 10 Jan 2025). In a different exact event-based framework, recurrent delay learning was reported to enhance classification accuracy compared to architectures without delays, and recurrent delays were found to be particularly beneficial in small networks (Mészáros et al., 13 Jan 2025).

5. Delay-spike methods beyond neural classifiers

The same temporal principle appears in spike-train analysis. In sparse neuronal data, latency is treated as a global temporal shift $C$3 acting on the spike times of train $C$4, and multivariate latency correction estimates these shifts directly from matched spikes rather than from rate profiles (Kreuz et al., 2022). Spike matching is defined with an adaptive coincidence window, matched spikes yield pairwise time differences $C$5, and the global objective is to minimize

$C$6

over all train delays by simulated annealing. The method is explicitly designed to correct systematic delays while maintaining all other kinds of noisy disturbances. This suggests a distinct but related use of the approach: delay as a nuisance variable to be estimated and removed so that timing-based structure becomes comparable across trials or channels.

A still broader manifestation appears in laser–plasma control. Spike Trains of Uneven Duration and Delay (STUD pulses) are laser pulses composed of on and off sequences with at least one or two orders of magnitude contrast between them, with spatial hot-spot scrambling between spikes (Afeyan et al., 2012). The off times allow damping of driven waves, the scrambling of the hot spots allows tens of damping times to elapse before hot spot locations experience recurring high intensity spikes, and crossing beams can be interlaced or staggered so that their interactions are controlled with an on-off switch and a dimmer. In simulations of stimulated Brillouin scattering, the efficiency of the technique was attributed to minimizing successive re-amplification in space and time of parametrically excited plasma waves inside laser hot spots, with instability growth reduced by orders of magnitude and ion acoustic wave fluctuation levels staying orders of magnitude below those of RPP and SSD beams (Hüller et al., 2012). In the strongly nonlinear kinetic regime of stimulated Raman scattering, STUD pulses were reported to decrease SRS reflectivity by more than an order of magnitude over random-phase-plate or induced-spatial-incoherence beams of the same average intensity and comparable bandwidth (Albright et al., 2013).

6. Limitations, misconceptions, and open directions

A recurrent misconception is that delay is only an implementation detail. The cited work argues the opposite in several ways: addition-based skips in TTFS can be harmful precisely because they introduce additional delay; fixed or random delays can help, and learned delays can help more; and exact event-based training treats delay as a first-class parameter alongside weight (Kim et al., 2023, Hammouamri et al., 2023, Göltz et al., 2024). Another misconception is that delay-based methods are synonymous with arbitrary recurrent memory. The literature instead shows multiple granularities—channel-wise skip delays, layer-wise axonal delay, per-synapse delays, state-space delay vectors, and adaptive stopping times—with different expressive and computational trade-offs (Sun et al., 2022, Karilanova et al., 1 Dec 2025, Chen et al., 2023).

The limitations are equally explicit. Delay initialization experiments in TTFS skip learning show a trade-off between latency and accuracy, and pixel-wise delays add complexity for only marginal improvement over channel-wise delays (Kim et al., 2023). TSkips enlarge the search space over skip positions and delay values, requiring training-free NAS to remain tractable (Malettira et al., 2024). Bonferroni correction in SpikeCP is reliable but conservative, so more checkpoints can increase set sizes and delay (Chen et al., 2023). Local delay plasticity can collapse patterns into homogeneous responses if training is not controlled, and the cited proof-of-concept is feedforward and small-scale (Farner et al., 2022). State-space delay models add state variables and therefore architectural trade-offs, even when their parameter counts scale more favorably than per-synapse delay schemes (Karilanova et al., 1 Dec 2025).

Taken together, these results suggest that the “Delay-Spike Approach” is not a single algorithmic family but a cross-cutting temporal strategy. It includes learning when spikes arrive, deciding when spike-based inference should stop, compensating for when spikes were observed, and scheduling when energetic spikes should be delivered to a physical medium. Across these settings, the common claim is precise: temporal alignment, temporal separation, and temporal reset are themselves optimizable resources.