--- title: Spiking Direct Feedback Alignment (SDFA) url: https://www.emergentmind.com/topics/spiking-direct-feedback-alignment-sdfa type: topic --- # Spiking Direct Feedback Alignment (SDFA) Searching arXiv for recent and foundational papers on Spiking Direct Feedback Alignment and adjacent direct-feedback methods. Spiking Direct Feedback Alignment (SDFA) denotes a family of training methods for spiking neural networks (SNNs) that replace exact layer-by-layer error backpropagation with direct error delivery from the output layer to hidden layers through auxiliary feedback pathways. Across the literature, the term covers several related but non-identical constructions: learned feedback via spike-based causal estimation of backward weights [1910.01689], fixed-random direct feedback adapted to time-unrolled SNNs for hardware-efficient training [2507.15603], gradient-free augmented DFA with relaxed backward nonlinearities for leaky integrate-and-fire (LIF) networks [2409.07776], and the closely related Spiking Forward Direct Feedback Alignment (SFDFA), which learns feedback connections online from spike-dependent directional derivatives [2403.08804]. What unifies these approaches is the abandonment of exact reverse-mode differentiation through transposed forward weights in favor of local updates driven by a broadcast output error and a layer-specific feedback transform, a design that is repeatedly motivated by the non-differentiability of spikes, the cost of temporal credit assignment, and the requirements of neuromorphic and in-memory hardware. ## 1. Conceptual definition and scope In conventional backpropagation for multilayer networks, the error at layer \(l\) is computed recursively from the error at layer \(l+1\) using the transpose of the forward weight matrix. For SNNs, this is especially burdensome because spike generation is discontinuous and the network dynamics unfold over many timesteps. Several SDFA formulations therefore replace this recursion by a direct mapping from output error to each hidden layer. A representative non-spiking DFA formulation, used to motivate SDFA, writes the hidden-layer update as \[ \delta \mathbf{W}_i = -\big[(\mathbf{B}_i \delta \mathbf{a}_y) \odot f_i'(\mathbf{a}_i)\big]\mathbf{h}_{i-1}^{T}, \qquad \delta \mathbf{a}_y = \frac{\partial \mathcal{L}}{\partial \mathbf{a}_y}, \] where \(\mathbf{B}_i\) is a fixed random feedback matrix and \(\delta \mathbf{a}_y\) is the output-layer error [2012.06373]. In a direct spiking adaptation for time-unrolled SNNs, the error signal at timestep \(t\) is written \[ \delta_{l}^t \gets B_{l} \delta_L \odot f'_{l}(W_lh_{l-1}+b_l), \] with the same projected output error \(B_l\delta_L\) reused across timesteps [2507.15603]. In a spiking wearable adaptation setting, the same core idea appears in the simpler layerwise form \[ \delta_l = e \cdot B_l, \] contrasted with backpropagation through \(W_l^T\) [2502.06736]. The term SDFA is not fully standardized. In [1910.01689], the method labeled in subsequent summaries as SDFA is a spike-based feedback-learning procedure that uses regression discontinuity design (RDD) to make backward weights approximate forward weights. In [2409.07776], the spiking method is termed augmented direct feedback alignment (aDFA) for SNNs and is described as a gradient-free random-projection method. In [2403.08804], the paper introduces Spiking Forward Direct Feedback Alignment (SFDFA), a spiking adaptation of Forward Direct Feedback Alignment rather than a fixed-random DFA rule. A plausible implication is that “SDFA” functions as an umbrella label for spiking direct-feedback training schemes rather than a single canonical algorithm. ## 2. Relation to backpropagation and standard DFA The defining contrast is with backpropagation. For SNNs, one reported backpropagation layer error is \[ \delta_l = \left( W_l^T \delta_{l+1} \right) \odot f^{\prime}(h_{l}) \quad l \in \{1, 2, \dots, L-1\}, \] which requires transposed weights, sequential layerwise error propagation, and dependency across timesteps in spiking settings [2507.15603]. This is repeatedly described as problematic because spiking dynamics unfold over time, backward passes are sequential, and training suffers from backward locking [2507.15603]. Closely related work on forward-gradient alternatives also frames backpropagation as limited by backward locking, weight transport, low-power implementation constraints, and online-learning incompatibility [2212.07282]. Standard direct feedback alignment replaces the recursive chain with a direct projection from output error: \[ \delta_l = \left( B_l \delta_{L} \right) \odot f^{\prime}(h_{l}) \quad l \in \{1, 2, \dots, L-1\}, \] where \(B_l\) is fixed and random [2507.15603]. The same architectural asymmetry is central in photonic and privacy-oriented DFA systems, which emphasize that the backward signal is not propagated through exact transposed weights but through a fixed random projection of the top-layer error [2012.06373], [2111.06862], [2010.03701], [2106.03645]. This asymmetry reduces inter-layer backward communication and permits local, parallelizable learning updates [2012.06373]. For spiking systems, the appeal is stronger because exact reverse-mode differentiation through the spike nonlinearity is difficult. One summary explicitly states that in SDFA a “spike-based analogue” can project the output spike-train error or membrane mismatch to hidden layers through fixed synapses or a random-mixing module and then use that signal to gate local spike-timing-dependent-plasticity (STDP)-like or surrogate-gradient updates [2012.06373]. Another adjacent work states directly that direct-feedback methods could be adapted to SNNs by combining feedback learning with surrogate gradients or STDP [2212.07282]. ## 3. Principal algorithmic variants The literature contains at least four technically distinct variants associated with spiking direct feedback training. | Variant | Core feedback mechanism | Distinctive feature | |---|---|---| | RDD-based spike feedback learning | Learned \(Y_{ik}\) from causal effect \(\beta_{ik}\) | Solves weight transport by spike-threshold discontinuity [1910.01689] | | Time-reused fixed-random SDFA | Fixed random \(B_l\) | Reuses \(B_l\delta_L\) across timesteps [2507.15603] | | aDFA-SNN | Fixed random \(B_n\) with arbitrary nonlinear \(g\) | Replaces \(f'\) by relaxed backward function \(g\) [2409.07776] | | SFDFA | Learned feedback from spike grades | Estimates output-hidden weights online [2403.08804] | The RDD-based method of [1910.01689] differs most sharply from fixed-random DFA. During a special feedback-training phase, a network is simulated as a LIF system, and spike-threshold discontinuities are used as a quasi-experimental signal to estimate the causal influence of neuron \(i\) on downstream neuron \(k\). The feedback synaptic weight \(Y_{ik}\) is set proportional to an estimated discontinuity \(\beta_{ik}\), so that backward weights approximate forward weights rather than remaining fixed random matrices. The hardware-oriented SDFA of [2507.15603] remains closer to conventional DFA. Its key innovation is that the feedback matrix size is decoupled from the number of timesteps: the same projected output error \(B_l\delta_L\) is reused across timesteps, reducing temporal feedback complexity and memory footprint. This formulation is explicitly proposed to eliminate sequential error propagation over time, eliminate weight transposition, and remove backward locking. The aDFA-SNN method of [2409.07776] takes a more radical step. Instead of retaining the conventional derivative term \(f'(a_n)\), it defines \[ e_{n}=\left[ B_{n}e\right] \odot g\left( a_{n}\right), \] where \(g\) is an arbitrary nonlinear function. This is presented as a gradient-free training rule based on random projection, meant to be easier to implement physically and more biologically plausible than backpropagation or standard DFA. SFDFA [2403.08804] is explicitly a spiking adaptation of Forward Direct Feedback Alignment. It replaces fixed random feedback by learned or estimated feedback weights between output and hidden neurons, using spike-dependent directional derivatives or “spike grades.” The method is described as computing exact local gradients of spikes online, accounting for intra-neuron dependencies between multiple post-synaptic spikes, and then reformulating the dynamics for neuromorphic hardware compatibility. ## 4. Spike-based mechanisms, local dynamics, and update rules ### 4.1 LIF dynamics in spiking direct-feedback training Several formulations use LIF neurons. In the RDD-based feedback-learning phase, the voltage and input drive obey \[ \frac{dv_i}{dt} = -g_L v_i + g_D(I_i - v_i), \qquad \frac{du_i}{dt} = -g_L u_i + g_D(I_i - u_i), \] with voltage reset to \[ v_{\text{reset}} = -1 \] after threshold crossing, while \(u_i\) does not reset [1910.01689]. In aDFA-SNN, the input current, membrane state, spike output, and reset are \[ v\left( t\right)_{i} =\sum_{j} W_{ij}a\left( t\right)_{j} +b_{i}, \] \[ h\left( t\right)_{i} =\left( 1-\frac{\Delta t}{\tau } \right) h\left( t-1\right)_{i} +\frac{\Delta t}{\tau } v\left( t\right)_{i} +\eta \left( t\right), \] \[ a\left( t\right)_{i} = \begin{cases} 1&h\left( t\right)_{i} \geqq h_{th}\\ 0&h\left( t\right)_{i} \theta,\\ 0, & \text{otherwise,} \end{cases} \] with reset to zero after firing [2502.06736]. ### 4.2 Local feedback signals The fixed-random SDFA formulation for temporal SNNs propagates \[ \delta_{l}^t \gets B_{l} \delta_L \odot f'_{l}(W_lh_{l-1}+b_l), \] followed by temporal gradient accumulation \[ dW_l^t \gets \delta_{l}^t \cdot (h_{l-1}^t)^T + dW_l^{t-1}, \qquad db_l^t \gets \delta_l^t + db_l^{t-1}, \] and weight updates \[ W_l \gets W_l - \eta \cdot dW_l^T, \qquad b_l \gets b_l - \eta \cdot db_l^T \] for all layers [2507.15603]. The paper emphasizes that the feedback path is effectively timestep-invariant, allowing reuse across time. In aDFA-SNN, the hidden-layer training signal is \[ e_{n}=\left[ B_{n}e\right] \odot g\left( a_{n}\right), \] and the weight update is \[ \delta W_{n}=-e_{n}\cdot x^{T}_{n}. \] The feedback matrix initialization is also specified: \[ B_{n}=\gamma^{D}\prod \limits_{i=n}^{n+D}\left[\bar{W}_{n+1}+2\sqrt{3}\sigma_{W_{n+1}}\left(rand-0.5\right)\right], \] with \(rand \sim U[0,1]\) [2409.07776]. ### 4.3 Feedback learning rather than fixed randomness The RDD-based method learns feedback weights by fitting a piecewise linear regression near threshold. When \(|v_i-\theta|<\alpha\), an RDD window of duration \(T=30\) ms is opened, and samples \((u_i^{\max},\Delta q_k^{\text{avg}})\) are collected. The regression model is \[ f_{ik}(x) \coloneqq \begin{cases} c^1_{ik} x + c^2_{ik} & \text{if } x < \theta \\ c^3_{ik} x + c^4_{ik} & \text{if } x \geq \theta \end{cases} \] and the causal effect estimate is the threshold discontinuity \[ \beta_{ik} \coloneqq \lim_{x \rightarrow \theta^+} f_{ik}(x) - \lim_{x \rightarrow \theta^-} f_{ik}(x). \] The feedback weight is then set as \[ Y_{ik} = \beta_{ik}\frac{\gamma}{\sigma^2_\beta}. \] This is presented as a mechanism by which learned feedback weights approximate forward weights and thereby address the weight transport problem [1910.01689]. SFDFA likewise abandons fixed random feedback. The feedback matrix update is \[ b_{ji} \leftarrow (1-\alpha)b_{ji} + \alpha \, d_j \, p_i^1, \] where \(d_j\) is a grade-like directional signal and \(p_i^1\) a perturbation [2403.08804]. The paper derives, for a single hidden layer, \[ \mathbb{E}[d_o \, p_i^1] = w_{o,i}, \] under the stated independence assumptions, which is used to argue that the learned feedback weights asymptotically approximate forward weights [2403.08804]. ### 4.4 Exact and modified spike-time derivatives SFDFA is distinctive in deriving explicit local spike-time derivatives. Under \(\tau = 2\tau_s\), spike timing can be written \[ t_i^x = \tau \ln \left(\frac{2 a_i^x}{b_i^x + \chi_i^x}\right), \] with \[ a_i^x := \sum_{j \in \mathcal{P}_i} w_{ij} \sum_{z=1}^{x} \exp\left(\frac{t_j^z}{\tau_s}\right), \] \[ b_i^x := \sum_{j \in \mathcal{P}_i} w_{ij} \sum_{z=1}^{x} \exp\left(\frac{t_j^z}{\tau}\right) - \vartheta \sum_{z=1}^{x} \exp\left(\frac{t_i^z}{\tau}\right), \] \[ \chi_i^x := \sqrt{\left(b_i^x\right)^2 - 4 a_i^x \vartheta}. \] The total derivative accounts for intra-neuron dependencies between successive spikes: \[ \frac{d t_i^x}{d w_{ij}} = \frac{\partial t_i^x}{\partial w_{ij}} + \frac{\partial t_i^x}{\partial t_i^{x-1}}\frac{d t_i^{x-1}}{d w_{ij}} +\cdots+ \frac{\partial t_i^x}{\partial t_i^1}\frac{d t_i^1}{d w_{ij}}. \] Because the exact local gradient diverges when \(\chi_i^x \to 0\), the method uses a modified derivative, \[ \frac{d t_i^x}{d w_{ij}} = \frac{\tau}{a_i^x \exp\left(\frac{-t_i^x}{\tau_s}\right)} \left[ f_{ij}^x \exp\left(\frac{-t_i^x}{\tau_s}\right) - h_{ij}^x \exp\left(\frac{-t_i^x}{\tau}\right) \right], \] which is described as an ad hoc stabilization that drops the threshold term from the unstable denominator [2403.08804]. ## 5. Hardware-oriented interpretations and system co-design A persistent theme in direct-feedback work is that the algorithmic asymmetry of DFA changes the communication/computation trade-off in favor of hardware efficiency. The photonic DFA literature argues that standard backpropagation is limited by memory movement and inter-device communication as models scale, whereas DFA removes the need for inter-layer backward communication and requires only the final output error to be shared [2012.06373]. This is explicitly tied to the relevance of SDFA, because spiking systems also benefit from fixed random feedback channels that drive local plasticity without precise reverse-mode differentiation through spike nonlinearities [2012.06373]. The same paper presents a photonic co-processor that performs the random projection \(\mathbf{B}\delta \mathbf{a}_y\) optically through random light scattering and holographic recovery of the full complex optical field, with up to 1 million input components and 2 million output components and a matrix with trillions of parameters computed in milliseconds [2012.06373]. A practical detail is that the optical input must be binary, so the output error is ternarized into \(\{-1,0,1\}\); the projection is obtained by subtracting the projection of the negative component from the positive component. This preserves the direction of the error rather than its magnitude. The discussion explicitly notes that such binary or ternary feedback codes are suggestive for spiking systems because they resemble discrete event-based communication [2012.06373]. Related silicon photonic work computes the DFA gradient vector \[ \boldsymbol{\delta}^{(k)} = \mathbf{B}^{(k)} \mathbf{e} \odot g'\left(\mathbf{a}^{(k)}\right) \] using microring resonator arrays and wavelength-division multiplexing, with the same output error \(\mathbf{e}\) broadcast optically to all hidden layers [2111.06862]. Although this is not a spiking system, its direct error broadcast, fixed random feedback, and non-accumulating error paths are explicitly identified as relevant to SDFA-style designs [2111.06862]. The most explicit spiking hardware co-design is PipeSDFA [2507.15603], an RRAM-based in-memory computing accelerator. It contains a Forward Core, Error Propagation Core, and Backward Core, and is organized around a three-level pipeline: timestep-level, data-level, and batch-level. The algorithmic structure of SDFA is described as hardware-friendly because it requires no weight transposition, no sequential error propagation, and reduced temporal complexity due to reusing the same \(B_l\delta_L\) across timesteps. The paper additionally reports that the intrinsic stochasticity of RRAM programming is used to generate the fixed random matrices \(B_l\): 1000 Monte Carlo simulations produced resistance distributions close to Gaussian; the targeted ideal write resistance was 10 k\(\Omega\); each RRAM cell encodes 2-bit weights, so 4-bit weights use 2 cells per synapse [2507.15603]. A related RRAM-crossbar study for wearable analytics does not formally introduce the acronym SDFA, but it effectively applies spiking DFA to online adaptation of pre-trained SNNs under hardware noise [2502.06736]. The central hardware claim is that DFA avoids transposable crossbars and the associated row-side peripheral circuitry required by backpropagation, even though it introduces additional feedback arrays [2502.06736]. ## 6. Empirical performance across formulations The empirical record is heterogeneous because the methods grouped under SDFA differ substantially. ### 6.1 RDD-based spike causal learning The spike-based causal inference method of [1910.01689] reports that RDD feedback training substantially improves training and test error over fixed random feedback alignment on Fashion-MNIST, SVHN, CIFAR-10, and VOC, and that performance approaches backprop-level accuracy. The paper further states that RDD significantly increases sign alignment between forward and feedback weights, improves magnitude matching, and more strongly decreases \(\mathcal{R}_{\text{self}}\) than standard feedback alignment, especially in earlier layers. A precise benchmark table is not given in the supplied data, so the quantitative comparison is reported qualitatively. ### 6.2 Time-reused fixed-random SDFA on SNN benchmarks PipeSDFA evaluates SDFA on N-MNIST, SHD, Braille letter, DVS-Gesture, and N-Caltech101 [2507.15603]. The reported accuracy is within less than 2% loss compared to backpropagation overall. Concrete numbers include: - SHD: 73.29% for SDFA versus 74.09% for BP. - Braille letter: 99.26% versus 99.62%. - DVS-Gesture, ConvNet: 93.75% versus 95.14%. - DVS-Gesture, VGG11: 92.36% versus 88.92%. - N-Caltech101, VGG11: 64.38% versus 59.53%. The paper also reports that SDFA converges slightly more slowly than BP on DVS-Gesture but still reaches comparable accuracy around 150 epochs [2507.15603]. ### 6.3 aDFA-SNN on fully connected LIF networks The aDFA-SNN study uses a 784–1000–10 fully connected three-layer SNN with LIF neurons on MNIST and Fashion-MNIST for 20 epochs [2409.07776]. Its table reports: | Framework | MNIST Best | MNIST Avg | F-MNIST Best | F-MNIST Avg | |---|---:|---:|---:|---:| | BP | 97.78% | 87.46% | 72.71% | 66.17% | | DFA | 96.75% | 92.09% | 84.48% | 82.54% | | aDFA | 98.01% | 97.91% | 87.43% | 87.20% | The paper further reports a range around 97.66%–98.10% on MNIST and 87.20%–87.49% on Fashion-MNIST for aDFA variants, and describes the method as more stable than BP and DFA in these experiments [2409.07776]. Since the supplied data also notes instability of BP and DFA across trials, this should be read as a claim specific to that experimental setup. ### 6.4 SFDFA on spike-time-based SNNs SFDFA is evaluated on MNIST, EMNIST, Fashion MNIST, and SHD with fully connected SNNs [2403.08804]. Reported average best test performance is: | Dataset | BP | DFA | SFDFA | |---|---:|---:|---:| | MNIST | 98.88 ± 0.02% | 98.42 ± 0.06% | 98.56 ± 0.04% | | EMNIST | 85.75 ± 0.06% | 79.48 ± 0.11% | 82.33 ± 0.10% | | Fashion MNIST | 90.19 ± 0.12% | 89.41 ± 0.12% | 89.73 ± 0.17% | | SHD | 66.79 ± 0.66% | 52.70 ± 2.30% | 54.63 ± 1.16% | The reported conclusions are that SFDFA consistently outperforms DFA, is closer to BP on image datasets, and that on SHD both DFA and SFDFA remain far below BP [2403.08804]. The same study also reports better gradient alignment and stronger weight alignment than DFA, especially near the output layer [2403.08804]. ### 6.5 Online adaptation on noisy RRAM hardware The RRAM wearable analytics study applies DFA-based online adaptation to pre-trained SNNs for Fashion MNIST, UCI-HAR, and HHAR [2502.06736]. It reports that DFA achieves approximately 64.1% lower energy, 10.1% lower area, 2.1× lower latency, and up to 7.55% higher inference accuracy than BP for online hardware adaptation. Task-specific post-adaptation accuracies are: - Fashion MNIST: BP 83.55%, DFA 85.12%. - UCI-HAR: BP 84.93%, DFA 85.61%. - HHAR: BP 75.0%, DFA 82.55%. The study attributes these gains to elimination of sequential gradient error accumulation, lower hardware overhead, and better concurrency in layer updates [2502.06736]. ## 7. Alignment, biological plausibility, and open tensions The phrase “feedback alignment” originally referred to the empirical fact that forward weights can adapt so that random feedback pathways become useful learning signals. Spiking direct-feedback methods complicate that picture because some seek to retain fixed randomness while others attempt to learn the feedback pathway itself. The RDD-based approach of [1910.01689] is the most explicit attempt to solve the weight transport problem in a biologically motivated way. It frames weight symmetry as a causal inference problem and uses threshold discontinuities in spiking neurons to estimate downstream causal effects, thereby learning backward weights without copying forward weights. The paper argues that this avoids several unrealistic assumptions of earlier biological learning proposals, although it also notes limitations: the implementation does not strictly follow Dale’s principle, the task network is still a standard convolutional network rather than an end-to-end spiking model, and spiking simulation is expensive enough that spikes are used only for feedback training [1910.01689]. The aDFA-SNN literature pushes a different notion of biological plausibility: rather than learning accurate backward weights, it relaxes the derivative itself. The replacement \[ e_{n}=\left[ B_{n}e\right] \odot g\left( a_{n}\right) \] is explicitly presented as removing the need for exact transposed weights and exact derivatives of discontinuous spike functions [2409.07776]. The paper reports that aDFA works stably when the correlation coefficient \(\eta\) between \(g\) and \(f'\) satisfies \(\eta > -0.2\), but becomes unstable or poor when \(\eta < -0.2\). It also reports that genetic algorithm search over positive random Fourier series tends to evolve backward functions toward a bell-curve-like shape near the peak of \(f'\) [2409.07776]. This suggests that the backward nonlinearity cannot be arbitrary in practice, even within a gradient-free framing. SFDFA [2403.08804] represents yet another resolution of the same tension. It does not use generic surrogate derivatives in the final method; instead, it derives an exact spike-time derivative and then replaces it with a modified version to avoid singularities. The paper explicitly acknowledges that this stabilization introduces bias. A plausible implication is that spiking direct-feedback methods often trade exactness for stability at different points in the pipeline: fixed random feedback introduces backward bias, learned feedback reduces that bias but may require additional estimation machinery, and derivative relaxations or modified exact derivatives stabilize learning at the cost of further approximation. From a systems perspective, the motivation is consistent across this literature. Non-spiking DFA studies repeatedly argue that direct feedback is attractive because it removes inter-layer backward communication, supports asynchronous and parallelizable updates, and better matches hardware constrained by bandwidth rather than arithmetic throughput [2012.06373], [2111.06862]. Spiking papers inherit the same argument but add temporal constraints: avoiding time-expanded backward chains, transposed weights, and backward locking is especially valuable for SNNs [2507.15603], [2212.07282]. The main open tension is performance versus implementability. Some SDFA-like methods approach backpropagation on selected benchmarks or hardware-adaptation tasks [1910.01689], [2507.15603], [2502.06736], whereas others remain consistently below BP, particularly on temporally rich datasets such as SHD [2403.08804]. The evidence therefore supports a narrower conclusion: direct-feedback training is a viable and technically diverse alternative for SNNs, especially when local computation, online learning, or hardware co-design is the primary objective, but its empirical behavior depends strongly on how the feedback pathway, spike derivative, and temporal structure are handled. Source: https://www.emergentmind.com/topics/spiking-direct-feedback-alignment-sdfa