Spiking ConvNeXt for Efficient Neural Vocoding

• Spiking ConvNeXt is a ConvNeXt block modified with PLIF neurons to shift heavy computations onto sparse, binary spikes for optimized energy efficiency.
• It incorporates an amplitude shortcut path that compensates for the binary nature of spikes, preserving essential signal dynamics during pointwise convolutions.
• Embedded within Spiking Vocos, the module leverages temporal shift and self-architectural distillation to closely match ANN performance while reducing energy consumption to 14.7%.

Spiking ConvNeXt denotes a ConvNeXt-derived building block adapted to a spiking neural network (SNN) vocoder, introduced within "Spiking Vocos: An Energy-Efficient Neural Vocoder" (Chen et al., 16 Sep 2025). The module preserves the overall ConvNeXt-style block structure while inserting Parametric Leaky Integrate-and-Fire (PLIF) neurons before the computationally intensive pointwise convolutions and adding an amplitude shortcut path to compensate for information loss caused by binary spikes. Its stated objective is to retain ConvNeXt’s modeling capacity while shifting most expensive computation onto sparse spike activations, so that heavy operations are executed as cheaper accumulations rather than full-precision multiply-accumulate operations. In the reported configuration, the resulting system achieves performance comparable to its artificial neural network (ANN) counterpart, with UTMOS and PESQ scores of 3.74 and 3.45 respectively, while consuming only 14.7% of the energy (Chen et al., 16 Sep 2025).

1. ConvNeXt lineage and the emergence of a spiking variant

ConvNeXt originated as a modernized convolutional architecture built around large-kernel depthwise convolution, inverted bottlenecks, and residual structure. In audio classification on AudioSet, ConvNeXt was adapted from vision by using log-mel spectrograms, a task-specific 527-way classifier, and strong regularization; the best reported ConvNeXt-Tiny reached 0.471 mean-average precision on AudioSet while using three times fewer parameters than recent large audio transformers (Pellegrini et al., 2023). In conditional diffusion modeling, ConvNeXt-style design was revived in the fully convolutional diffusion model (FCDM), which retained large-kernel depthwise convolution and inverted bottlenecks while adding diffusion conditioning and a U-shaped hierarchy (Kwon et al., 10 Mar 2026).

These antecedents are significant because they show that ConvNeXt design choices transfer beyond image classification. The audio-classification study explicitly states that it does not discuss spiking neural networks directly, but identifies depthwise separable convolutions, inverted bottlenecks, residual blocks, strided stem and downsampling, and strong regularization as the architectural elements that matter for audio adaptation (Pellegrini et al., 2023). A plausible implication is that Spiking ConvNeXt should be understood not as an isolated invention, but as a task-specific SNN reformulation of a ConvNeXt block that had already demonstrated cross-domain portability in ANN settings.

2. Block structure and architectural principle

The standard ConvNeXt block is described in the surrounding literature as a sequence built around a large-kernel depthwise convolution, normalization, two $1 \times 1$ pointwise convolutions for channel expansion and reduction, and a residual connection; ConvNeXt V2 additionally uses Global Response Normalization (GRN) to mitigate channel redundancy (Kwon et al., 10 Mar 2026). In the AudioSet adaptation, the key block is summarized as a depthwise $7 \times 7$ convolution, an inverted bottleneck consisting of pointwise expansion by 4, GELU, pointwise projection back to the original channel dimension, and a residual connection (Pellegrini et al., 2023).

Spiking ConvNeXt keeps this overall ConvNeXt-style organization but changes where spiking is introduced. The central design decision is to place PLIF neurons directly before the pointwise convolutions, because those pointwise convolutions account for the majority of the computational load (Chen et al., 16 Sep 2025). The paper states:

“Since the two pointwise convolutions account for the majority of the computational load, we choose to insert spiking neurons directly before them. This ensures that these computationally intensive operations are performed on sparse, binary spikes, maximizing the energy savings of the SNN.” (Chen et al., 16 Sep 2025)

This placement is the defining architectural move. Rather than converting every component into a generic spiking analogue, the module selectively targets the compute-dominant sublayers. The resulting contrast can be stated succinctly. A standard ConvNeXt block uses dense ANN operations with conventional activations and residual paths, whereas Spiking ConvNeXt preserves the block skeleton but replaces dense nonlinear processing with PLIF neurons before the pointwise convolutions and supplements the spiking path with an amplitude shortcut (Chen et al., 16 Sep 2025).

A common misconception is that Spiking ConvNeXt is merely a conventional spiking convolutional block in which every layer is replaced by a spike neuron. The paper explicitly distinguishes its design from that pattern: it is a targeted conversion strategy centered on the pointwise convolutions rather than a blanket substitution of all layers (Chen et al., 16 Sep 2025).

3. PLIF dynamics and the amplitude shortcut path

The neuronal primitive used in the block is the Parametric Leaky Integrate-and-Fire neuron, chosen “for a higher diversity and expressiveness” (Chen et al., 16 Sep 2025). Unlike a standard LIF neuron, PLIF has a learnable time constant $\tau$, allowing adaptive control over the balance between current input and membrane memory. The paper presents its dynamics in charging, firing, and resetting stages:

$$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$

$S_t = \Theta \left( H_t - V^{\text{th}} \right)$

$V_t = V^{\text{re}} S_t + H_t(1-S_t)$

Here, $H_t$ is the charged membrane potential, $S_t$ is the binary spike output via a Heaviside step function, and $V_t$ is the post-reset membrane potential; $V^{\text{th}}$ is the firing threshold and $7 \times 7$0 is the reset potential (Chen et al., 16 Sep 2025).

The necessity of the amplitude shortcut follows from a stated limitation of SNNs. The paper characterizes the all-or-none nature of spikes as an information bottleneck and describes a “saturation phenomenon,” in which supra-threshold inputs collapse to the same spike response (Chen et al., 16 Sep 2025). To mitigate this, Spiking ConvNeXt introduces an amplitude shortcut path that re-injects magnitude information into the block output:

$7 \times 7$1

This mechanism is intended to let the sparse spiking path provide efficient computation while a parallel multiplicative recovery path preserves signal dynamics that would otherwise be lost under binarization (Chen et al., 16 Sep 2025). The combination of PLIF-based sparse coding and amplitude recovery is therefore the module’s primary answer to the representational compression imposed by spiking.

4. Role within Spiking Vocos

Spiking ConvNeXt is not presented as a standalone benchmarked backbone; it is the fundamental building block of Spiking Vocos, which is built on the Vocos frequency-domain vocoder backbone (Chen et al., 16 Sep 2025). The paper states that Vocos already avoids heavy upsampling layers by maintaining constant temporal resolution, and the generator is composed of an input mel-spectrogram, a stack of Spiking ConvNeXt blocks, Temporal Shift Module (TSM) operations applied in each block, and an output spectral representation (Chen et al., 16 Sep 2025).

This embedding inside Vocos matters for two reasons. First, the macro-architecture of the student and teacher is aligned: ANN Vocos serves as the teacher and Spiking Vocos serves as the SNN student, with the same overall architecture (Chen et al., 16 Sep 2025). Second, the constant-resolution frequency-domain formulation is compatible with the block’s objective of reducing expensive dense computation without introducing additional structural overhead from aggressive temporal upsampling or decoder complexity.

In this setting, Spiking ConvNeXt functions as the local feature-transformation unit of the generator. Its depthwise convolution remains in place, the pointwise convolutions become spike-driven after PLIF gating, and residual connectivity is preserved (Chen et al., 16 Sep 2025). The paper’s framing therefore ties the module tightly to neural vocoding rather than positioning it as a general-purpose spiking replacement for all ConvNeXt deployments.

5. Distillation and temporal context augmentation

The paper couples Spiking ConvNeXt with two additional mechanisms designed to compensate for the representational constraints of spiking computation: self-architectural distillation and the Temporal Shift Module (Chen et al., 16 Sep 2025).

Self-architectural distillation aligns the SNN student with the ANN teacher at both intermediate and output levels. For intermediate features, the paper defines a layer-wise feature alignment loss:

$7 \times 7$2

where $7 \times 7$3 and $7 \times 7$4 are student and teacher intermediate features, $7 \times 7$5 is an adapter function, and $7 \times 7$6 is the number of distilled blocks (Chen et al., 16 Sep 2025). At the output level, separate losses are applied for magnitude and phase. The magnitude loss is

$7 \times 7$7

and the anti-wrapping phase error is

$7 \times 7$8

with phase losses

$7 \times 7$9

$\tau$0

$\tau$1

$\tau$2

and total distillation loss

$\tau$3

(Chen et al., 16 Sep 2025).

The Temporal Shift Module is introduced because SNNs can suffer from “partial-time dependency” (Chen et al., 16 Sep 2025). The input feature $\tau$4 is split into channel groups $\tau$5, shifted across time, and then blended residually:

$\tau$6

$\tau$7

where $\tau$8 controls shift intensity (Chen et al., 16 Sep 2025). The paper states that TSM allows each block to “see” past and future simultaneously, and also notes an implementation detail: when both TSM and distillation are used, the intermediate distillation points are shifted to the subsequent ConvNeXt block so that feature alignment is not disrupted by the temporal shift operation (Chen et al., 16 Sep 2025).

6. Compute, energy, empirical behavior, and scope

The energy rationale of Spiking ConvNeXt is explicitly tied to the asymmetry between depthwise and pointwise computation. In the reported formulation, the depthwise convolution still uses MACs, whereas the pointwise convolutions operate on sparse spikes and are therefore modeled with cheaper accumulations (Chen et al., 16 Sep 2025). The paper expresses the energy of these components as

$\tau$9

$$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$0

where $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$1 and $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$2 are kernel sizes, $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$3 and $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$4 are channel counts, $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$5 is sequence length, $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$6 is timesteps, $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$7 is firing rate, $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$8 pJ, and $$H_t = V_{t-1} + \frac{1}{\tau}\left( X_t - (V_{t-1} - V^{\text{re}}) \right)$$9 pJ (Chen et al., 16 Sep 2025).

The reported system-level numbers quantify that tradeoff. ANN Vocos is estimated at $S_t = \Theta \left( H_t - V^{\text{th}} \right)$0 pJ, whereas Spiking Vocos with 4-step + TSM + distillation is estimated at $S_t = \Theta \left( H_t - V^{\text{th}} \right)$1 pJ (Chen et al., 16 Sep 2025). The paper also states that the final 4-step model consumes 14.7% of the energy of ANN Vocos, corresponding to more than $S_t = \Theta \left( H_t - V^{\text{th}} \right)$2 improvement in energy efficiency (Chen et al., 16 Sep 2025). On perceptual and objective measures, the final spiking model with TSM and distillation reaches UTMOS 3.74 and PESQ 3.45, compared with the ANN baseline Vocos at UTMOS 3.82 and PESQ 3.65 (Chen et al., 16 Sep 2025). The ablation sequence is likewise explicit: 4-step vanilla Spiking Vocos obtains UTMOS 3.46; adding TSM gives 3.71; adding distillation gives 3.70; and combining TSM with distillation gives 3.74 and PESQ 3.45 (Chen et al., 16 Sep 2025).

These results delimit the present scope of the term. Spiking ConvNeXt is supported here by evidence from neural vocoding rather than by broad evaluation across image classification, audio tagging, or diffusion. By contrast, the other ConvNeXt adaptations in the cited literature are ANN systems: the AudioSet model emphasizes regularization against overfitting in spectrogram classification (Pellegrini et al., 2023), and FCDM emphasizes training efficiency and diffusion-specific conditioning in generative modeling (Kwon et al., 10 Mar 2026). This suggests that the distinctive contribution of Spiking ConvNeXt is not the generic reuse of ConvNeXt alone, but the specific combination of pointwise spike gating, amplitude recovery, temporal shifting, and self-architectural distillation required to make a ConvNeXt-style block viable in an SNN vocoder (Chen et al., 16 Sep 2025).