Surrogate Gradient Methods

• Surrogate gradient methods are optimization techniques that replace non-differentiable or intractable true gradients with smooth proxy gradients, enabling gradient-based learning in complex models.
• They facilitate training in discrete, stochastic, or black-box models by substituting non-differentiable components with tractable approximations, which enhances convergence and efficiency.
• Key applications include spiking neural networks, variational inference, and evolutionary strategies, demonstrating robust empirical performance across diverse learning tasks.

Surrogate gradient methods are a class of optimization techniques that enable gradient-based learning and parameter updates in models or scenarios where the true gradient is unavailable, undefined, intractable, or non-informative. These approaches are core to the trainability of spiking neural networks (SNNs), discrete stochastic models, black-box optimizers, and various meta-learning or zeroth-order domains where non-differentiabilities or non-transparent computational graphs prevail.

1. Fundamental Principles and Problem Scope

Surrogate gradient methods replace an inaccessible or ill-defined true gradient—arising from non-differentiable or discrete event-based mechanisms, stochastic nodes, or black-box systems—with a tractable, often smooth, proxy gradient. In spiking neural networks, for example, the Heaviside step function used for spike emission has a derivative that is zero almost everywhere and undefined at threshold, precluding standard backpropagation. Surrogate gradients resolve this by substituting a differentiable “pseudo-derivative” in the backward pass, allowing the application of gradient descent and its variants (Neftci et al., 2019, Gygax et al., 2024).

Zeroth-order and evolutionary optimization methods, variational inference over intractable distributions, and offline black-box optimization also employ surrogate gradients to incorporate structural, local, or approximate gradient information where direct computation is inaccessible (Maheswaranathan et al., 2018, So et al., 2023, Hoang et al., 26 Feb 2025).

2. Mathematical Formulations and Proxy Gradient Construction

The surrogate gradient construction depends on the context:

• SNNs and Discrete Neural Models: The core strategy is to replace the problematic derivative of a non-differentiable activation (e.g., step, sign, quantization) with a smooth function, such as:
  • Rectangular: $$\sigma'_{\text{rect}}(x) = \frac{1}{2\delta} \mathbb{1}_{|x|<\delta}$$
  • Fast sigmoid: $$\sigma'_{\text{fs}}(x) = \frac{\alpha}{(1+\alpha|x|)^2}$$
  • Logistic: $h(x) = \beta \sigma_\beta(x)(1-\sigma_\beta(x))$
  • Arctan: $h(x) = \frac{\alpha}{1+(\alpha x)^2}$
  • These surrogates provide non-zero, bounded gradients around threshold regions, restoring chain rule connectivity for backpropagation (Neftci et al., 2019, Gygax et al., 2024, Jiang et al., 2023).
• Stochastic and Probabilistic Models: In variational inference or maximum likelihood over intractable distributions, surrogate gradients are implemented by constructing a surrogate distribution $q(\theta; \phi)$ such that the natural gradient with respect to $q$ is tractable, and parameters are mapped back to the original problem (So et al., 2023).
• Black-Box and Evolutionary Optimization: Here, surrogate gradient directions may be constructed from biased estimators, historical descent steps, or learned predictive models. These are fused via principled projections or by elongating the random search distribution along surrogate-informed subspaces (Maheswaranathan et al., 2018, Meier et al., 2019, Taminiau et al., 11 Feb 2025, Hoang et al., 26 Feb 2025).
• Surrogate Construction via General Cost Functions: In a broader optimization framework, a surrogate is established via a systematic cost $c(x, y)$, resulting in alternation between surrogate minimization and cost-induced update steps, generalizing and unifying gradient, mirror, and natural-gradient descent (Léger et al., 2023).

3. Algorithmic Methods and Variations

Surrogate gradient methods manifest through diverse algorithmic forms, including:

• Straight-Through Surrogate Backpropagation (in SNNs): Explicit substitution of the derivative in the backward pass, often via an injected differentiable function or, more flexibly, via forward gradient injection (FGI), which manipulates computational graphs to produce the desired backward signal (Otte, 2024).
• Parametric and Learnable Surrogates: Adaptive methods where the surrogate shape (e.g., width, slope) is treated as a learnable parameter, updated via backpropagation, as in parametric surrogate gradients (PSG) and KLIF neurons (Wang et al., 2023, Jiang et al., 2023).
• Sparse or Masked Surrogates: Approaches such as Masked Surrogate Gradient (MSG), where the gradient update is applied only to a subset of parameters or connections at each step, achieving a trade-off between learning power and network sparsity (Li et al., 2024).
• Surrogate-Aided Black-Box Optimization: Techniques that augment derivative-free optimization (DFO) with surrogate models, whose gradients—fit via both observed values and finite-difference approximations—guide parameter updates, often with Armijo-type line search validation on the true function (Taminiau et al., 11 Feb 2025).
• Surrogate-Aided ES and Meta-Learning: Use of surrogate-informed search directions or subspaces to inform evolutionary strategies or meta-learning updates, notably via principled fusion with random search or past directions (Maheswaranathan et al., 2018, Meier et al., 2019).

4. Theoretical Justification and Limitations

Recent work has provided a rigorous theoretical foundation for surrogate gradient methods in several contexts:

• Stochastic Automatic Differentiation (stochAD): Demonstrates that the surrogate gradient equals the expected derivative of the escape-noise model in stochastic SNNs, matching empirically used surrogates, and justifying their use even when no explicit surrogate loss exists. Notably, for typical surrogates the induced vector field is not conservative; i.e., it does not correspond to an underlying scalar loss function (Gygax et al., 2024).
• Neural Tangent Kernel Generalization: The classical NTK analysis is ill-posed for jump non-linearities, but a surrogate gradient NTK (SG-NTK) provides a well-defined, predictive theory of SGL dynamics in the infinite-width regime, demonstrating that surrogate gradient descent converges as expected to kernel regression solutions with the induced SG-NTK (Eilers et al., 2024).
• Convergence and Bias-Variance Trade-Offs: In black-box and evolutionary optimization, analytic bias and variance formulas quantify the trade-offs in blending surrogate and random search directions, with closed-form solutions controlling the safe incorporation of surrogate information (Maheswaranathan et al., 2018, Meier et al., 2019).
• Offline Optimization Risk Bound: Explicit performance bounds relate the sub-optimality gap of surrogate-guided optimization to the worst-case gradient discrepancy. Mechanisms such as gradient matching directly minimize this gap (Hoang et al., 26 Feb 2025).

Limitations of surrogate gradient frameworks include:

• Surrogate field integrability: most surrogate gradients in deep SNNs do not integrate to a unique loss.
• Surrogate model selection remains an empirical or structural heuristic in many cases.
• Potential distortion of geometry or loss of Fisher efficiency in probabilistic surrogate descent (So et al., 2023).
• Computational overhead in managing parametric or learned surrogates, especially in online or hardware-constrained environments.

5. Applications and Empirical Impact

Surrogate gradient methods have broad empirical impact:

• Spiking Neural Networks: Enable state-of-the-art performance in both static and event-based vision tasks, closing the gap with non-spiking ANNs, supporting training with both BPTT and local online rules, and mapping efficiently onto neuromorphic chips (Neftci et al., 2019, Li et al., 2024, Jiang et al., 2023, Wang et al., 2023).
• Black-Box and DFO Optimization: Accelerate convergence in derivative-free settings (e.g., chemistry, design) by enabling gradient-based surrogate-guided search, both in low and high dimensions (Hoang et al., 26 Feb 2025, Taminiau et al., 11 Feb 2025).
• Variational Inference and Probabilistic Models: Address intractable natural-gradient updates via surrogate distributions, enabling faster and more stable parameter estimation in mixture models, copulas, and skewed distributions (So et al., 2023).
• Meta-Learning and Hyperparameter Optimization: Allow meta-gradients or hypergradients to be stably computed despite truncations or biased gradient estimates, via inclusion of surrogate corrections in both inner and outer learning loops (Maheswaranathan et al., 2018).

Empirical benchmarks demonstrate:

• Substantial speedup in SNN training/inference, especially with frameworks like FGI and TorchScript/torch.compile (Otte, 2024).
• Surrogates with learnable shape parameters improve both accuracy and robustness across datasets.
• Sparse surrogate schemes (e.g., MSG) help manage the regularization-generalization trade-off and preserve event-based hardware efficiency (Li et al., 2024).

6. Practical Considerations and Best Practices

Several guidelines arise for deploying surrogate gradient methods:

• Select surrogate functions that best reflect the stochastic or dynamical noise model of the system (e.g., match logistic escape noise with the sigmoid derivative) (Gygax et al., 2024).
• For hardware or large-scale systems, use piecewise-linear surrogates or table lookups for maximal efficiency.
• Tune or learn surrogate width/slope parameters to each layer or architecture for improved convergence and signal propagation (Jiang et al., 2023, Wang et al., 2023).
• In training pipelines, leverage frameworks that support flexible or injected surrogate gradients (FGI) to minimize code complexity and maximize deployment speed (Otte, 2024).
• In sparse or event-driven systems, utilize masked or sparse surrogate updates (e.g., MSG) to maintain computational and energy efficiency (Li et al., 2024).

A summary table consolidating methods:

Approach	Target Problem	Key Mechanism / Surrogate
Straight-Through/SG	SNNs, discrete activations	Pseudo-derivative
PSG, KLIF, Adaptive	Deep SNNs, hardware noise	Learnable slope/shape
Masked SG (MSG)	Sparse SNNs, neuromorphic	Masked pseudo-gradient
Guided ES, DFO+SG	Black-box optimization	Subspace elongation, Sobolev
SNGD, Surrogate Natural	Probabilistic/VI/MLE	Surrogate Fisher geometry
SG-NTK	Infinite-width neural kernel	Chain-rule w/ surrogate

These methods collectively constitute the state-of-the-art toolkit for overcoming non-differentiabilities and intractabilities in gradient-based learning and optimization, with tight theoretical and empirical backing across multiple problem classes (Neftci et al., 2019, So et al., 2023, Gygax et al., 2024, Eilers et al., 2024, Hoang et al., 26 Feb 2025).