Input-aware Multi-Level Spike (IMLS)

• IMLS is a spiking neuron firing mechanism that compresses multi-timestep spiking into a single timestep using adaptive, input-aware thresholding.
• IMLS improves training efficiency by reducing memory and computational overhead, ensuring stable firing on variable-length speech sequences.
• Integrated in IML-Spikeformer, IMLS preserves multi-timestep spike coding semantics while achieving lower error rates and reduced energy consumption in speech tasks.

Searching arXiv for the cited IMLS and related multi-level spiking papers to ground the article. Input-aware Multi-Level Spike (IMLS) is a spiking-neuron firing mechanism introduced in the spiking Transformer architecture IML-Spikeformer for large-scale speech processing. Its defining objective is to simulate multi-timestep spike firing within a single timestep using an adaptive, input-aware thresholding scheme. In this formulation, a neuron emits an integer-valued spike magnitude that represents the number of effective spikes that would have been produced across a temporal window, while the firing threshold is scaled from current input statistics. The mechanism therefore couples multi-level spike coding with input-dependent threshold adaptation, and is presented as a way to improve training efficiency, stabilize firing on variable-length speech sequences, and retain the representational advantages of multi-timestep spiking without explicit temporal unrolling (Song et al., 10 Jul 2025). Related work on valued spikes in spiking neural networks uses mechanistically input-dependent multi-level firing to reduce quantization error and achieve 1-timestep inference, but does not frame the method as IMLS; this distinction is central to the taxonomy of recent multi-level spiking methods (Castagnetti et al., 28 Oct 2025).

1. Definition and problem setting

IMLS was proposed in response to two difficulties identified for spiking neural networks in speech processing: the high computational overhead during training caused by multi-timestep spike firing, and the absence of large-scale SNN architectures tailored to speech processing tasks. Within IML-Spikeformer, IMLS is the central firing rule used to compress the behavior of repeated Integrate-and-Fire activity into a single forward-pass timestep while preserving a discrete spike-count interpretation (Song et al., 10 Jul 2025).

The mechanism is defined by two coupled properties. First, it compresses multi-timestep firing into one timestep using multi-level spikes. Second, it adapts the firing threshold to the input distribution to preserve stable spiking activity on speech data. The paper presents this as particularly important for variable-length speech sequences, arguing that BatchNorm is problematic because batch statistics become inconsistent when sequence lengths vary, which destabilizes training. In that sense, IMLS is not only a quantization rule but also an implicit normalization or adaptive thresholding mechanism.

A common point of confusion is whether “input-aware” denotes a learned policy or merely input dependence. In IMLS, the threshold is explicitly input-conditioned through pre-synaptic statistics, so the input-aware characterization is part of the formal mechanism rather than a loose description. This distinguishes IMLS from fixed-threshold multi-level spike formulations, even when those formulations also produce spike magnitudes that increase with activation strength.

2. Neuron model and mathematical formulation

The IMLS formulation begins from a discrete-time Leaky Integrate-and-Fire baseline:

$v^l[t] = \beta \, v^l[t - 1] + x^{l-1}[t],$

$$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$

$v^l[t] = v^l[t] - \theta s^l[t].$

Here, $t$ is the timestep, $\theta$ is the threshold, $x^{l-1}[t]$ is pre-synaptic input, $v^l[t]$ is membrane potential, and $s^l[t]$ is the output spike (Song et al., 10 Jul 2025).

The precursor to IMLS is Multi-Level Spike (MLS). For the case where input arrives only at the first timestep, $X=\{x[1]\}$, MLS replaces the explicit spike train $\{s[t]\}_{t=1}^T$ with a single integer-valued spike:

$$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$0

The paper gives the equivalent closed form

$$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$1

The output levels are therefore

$$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$2

The magnitude directly represents the count of equivalent spikes over the time window: $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$3 means no spikes, $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$4 means one effective spike, and $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$5 means that the neuron would have fired at every timestep in the original multi-step IF simulation.

Because the floor operation is not differentiable, the paper uses the Generalized Straight-Through Estimator (G-STE), with surrogate derivative

$$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$6

This makes the multi-level spike train trainable by passing gradients through a clipped linear surrogate. The paper also notes that a multi-level spike of value $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$7 can be implemented on neuromorphic hardware as $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$8 consecutive binary spikes, similar to burst coding or graded spike firing.

3. Input-aware threshold adaptation

IMLS extends MLS by replacing the fixed threshold $$s^l[t] = \mathcal{F}(v^l[t]) = \begin{cases} 0, & v^l[t] < \theta \ 1, & v^l[t] \geq \theta \end{cases}$$9 with an input-dependent threshold $v^l[t] = v^l[t] - \theta s^l[t].$0. The scaling factor is channel-wise and is computed from the maximum pre-synaptic input in each channel. For the $v^l[t] = v^l[t] - \theta s^l[t].$1-th input batch $v^l[t] = v^l[t] - \theta s^l[t].$2, the scaling is

$v^l[t] = v^l[t] - \theta s^l[t].$3

To avoid computing a max operation at inference time, the training procedure maintains a running average:

$v^l[t] = v^l[t] - \theta s^l[t].$4

The threshold is thus dynamic during training and fixed after calibration during inference (Song et al., 10 Jul 2025).

The intended effect is to make thresholding track the input range. The paper explicitly presents this as preventing saturation when inputs are too large and silence or near-zero firing when inputs are too small. It further argues that this behaves like an implicit normalization mechanism replacing the role of BatchNorm in spiking transformers for speech. The reported consequence is more stable firing-rate distributions across layers, fewer dead or silent neurons, and better information propagation through depth.

In conceptual terms, IMLS preserves the spike-count semantics of MLS while inserting an adaptive scale that depends on current input statistics. This makes the multi-level coding rule sensitive not only to membrane magnitude but also to the distributional regime of the incoming sequence, which is especially relevant when sequence length and signal amplitude vary across speech batches.

4. Architectural role in IML-Spikeformer

Within IML-Spikeformer, IMLS is used as the spiking neuron layer $v^l[t] = v^l[t] - \theta s^l[t].$5 throughout the model. The architecture combines HD-RepSSA for token mixing and Spiking ChannelMLP for channel mixing, and the paper states that “The IMLS firing spiking neurons are used for the spiking neuron layers $v^l[t] = v^l[t] - \theta s^l[t].$6 in both modules.” IMLS is therefore not confined to a single subcomponent but serves as the firing rule across the architecture (Song et al., 10 Jul 2025).

The attention block itself is described as a Reparameterized Spiking Self-Attention module with a Hierarchical Decay Mask, forming the HD-RepSSA module. The abstract states that this module enhances the precision of attention maps and enables modeling of multi-scale temporal dependencies in speech signals. IMLS supplies the spike-generation mechanism that allows these blocks to operate with a single-timestep training representation while retaining an interpretation in terms of multi-timestep firing.

The model uses IMLS in both training and inference, but not in the same numerical form. During training, the network propagates multi-level spike values. During inference, those multi-level spikes are converted into equivalent binary spike trains. The paper gives the identity

$v^l[t] = v^l[t] - \theta s^l[t].$7

which formalizes the equivalence between dense training-time processing with $v^l[t] = v^l[t] - \theta s^l[t].$8 and sparse accumulation of binary spikes during deployment. This equivalence is central to the claim that IMLS reduces training cost without abandoning a spike-based inference interpretation.

5. Computational properties and empirical evidence

The computational argument for IMLS is that it eliminates explicit temporal unrolling during training. Conventional multi-timestep SNN training requires explicit unrolling across $v^l[t] = v^l[t] - \theta s^l[t].$9 timesteps, storing intermediate states for BPTT, memory complexity $t$0, and time complexity that grows with $t$1. IMLS instead computes the equivalent multi-level spike directly from the initial membrane potential and reduces training and memory complexity to $t$2, as stated in the paper (Song et al., 10 Jul 2025).

The reported implementation results were obtained on 4 NVIDIA GeForce RTX 3090 Ti GPUs. Relative to iterative multi-timestep firing, IMLS reduces per-epoch training time by 2.24× and cuts memory cost by 3.31×. The authors also stress that IMLS keeps constant training cost regardless of $t$3, with examples at $t$4 and $t$5. At the full-model level, the abstract reports word error rates of 6.0\% on AiShell-1 and 3.4\% on Librispeech-960, while reducing theoretical inference energy consumption by 4.64$t$6 and 4.32$t$7 respectively.

The ablation study isolates the effect of the firing rule on AiShell-1. The full IML-Spikeformer reports 5.9 CER on test. Replacing IMLS with MLS increases test CER to 8.3, a degradation of +2.4. Replacing IMLS with multi-timestep firing increases test CER to 9.1, a degradation of +3.2. The spike-rate visualization is presented as complementary evidence: vanilla multi-timestep firing shows many near-zero-firing neurons, especially in deep layers; MLS with fixed threshold improves activation but still has inconsistent layer-wise firing; IMLS maintains stable firing-rate distributions across layers. The paper interprets these results as evidence that input-aware thresholding adds substantial benefit beyond multi-level firing alone.

6. Relation to multi-level spiking research and residual architectures

IMLS belongs to a broader family of methods that replace binary instantaneous spikes with richer discrete signaling, but the literature distinguishes between explicit input-aware threshold adaptation and deterministic multi-level firing without such adaptation (Song et al., 10 Jul 2025, Castagnetti et al., 28 Oct 2025).

Mechanism	Threshold rule	Output form
MLS	Fixed $t$8	$t$9
IMLS	Input-dependent $\theta$0 during training; fixed calibrated $\theta$1 during inference	Multi-level spike with adaptive thresholding
Multi-level spiking neuron	Deterministic threshold-based repeated discharge within a timestep	$\theta$2 with $\theta$3

A closely related formulation appears in the paper “All in one timestep: Enhancing Sparsity and Energy efficiency in Multi-level Spiking Neural Networks,” which replaces the binary spike $\theta$4 with a valued spike $\theta$5. In that model, the neuron is charged and discharged across $\theta$6 micro-timesteps inside one normal timestep, and only the summed valued spike is passed to the next layer. For constant input, the quantization function has exactly $\theta$7 quantization levels; binary IF is recovered as the special case $\theta$8, yielding $\theta$9 levels. The paper explicitly notes that $x^{l-1}[t]$0 and $x^{l-1}[t]$1 provide the same number of quantization intervals, but the latter communicates the information in one timestep rather than four. This work is mechanistically input-aware in the sense that larger membrane input leads to more internal firings and thus a larger transmitted spike value, but it is not described as an adaptive IMLS rule with an explicit input-conditioned thresholding module.

That paper also makes the architectural issue of residual spike accumulation explicit. It introduces Sparse-ResNet and identifies a spike avalanche effect in residual networks, where spikes from the main path and shortcut path are added and can amplify activity in deeper layers. Sparse-ResNet uses multi-level spiking neurons in the direct path together with a barrier neuron after residual summation to control the spike avalanche, preserve low quantization error at the summation point, and reduce the number of spikes passed to the next layer. Because the barrier neuron can cause vanishing gradients if trained with a standard surrogate derivative, the method applies a Straight-Through Estimator for the barrier neuron. The reported evidence includes reductions at individual summation points from 45,665 to 37,150 spikes and from 67,848 to 35,819, characterized as up to about 47\% less activity where spike accumulation is severe. For the full network on CIFAR-10, with $x^{l-1}[t]$2 Sparse-ResNet reduces activity by 25\% versus SEW-ResNet, and with $x^{l-1}[t]$3 the reduction reaches 30\%, with only a small accuracy drop.

This comparison clarifies the status of IMLS within current spiking-neuron design. IMLS is an explicit input-aware multi-level mechanism aimed at compressing multi-timestep firing and stabilizing training for speech transformers. The valued-spike neuron of the related image and neuromorphic vision work is conceptually close because output magnitude depends on input magnitude and enables 1-timestep inference, but its framing is multi-level firing, valued spikes, and micro-timestep discharge rather than an explicit input-conditioned thresholding policy. The distinction is methodological rather than superficial, and it marks a boundary between adaptive threshold-based IMLS and deterministic multi-level IF generalizations.