Differentiating through Stochastic Differential Equations: A Primer

Abstract: Dynamical systems are essential to model various phenomena in physics, finance, economics, and are also of current interest in machine learning. A central modeling task is investigating parameter sensitivity, whether tuning atmospheric coefficients, computing financial Greeks, or optimizing neural networks. These sensitivities are mathematically expressed as derivatives of an objective function with respect to parameters of interest and are rarely available analytically, necessitating numerical methods for approximating them. While the literature for differentiation of deterministic systems is well-covered, the treatment of stochastic systems, such as stochastic differential equations (SDEs), in most curricula is less comprehensive than the subtleties arising from the interplay of noise and discretization require. This paper provides a primer on numerical differentiation of SDEs organized as a two-tale narrative. Tale 1 demonstrates differentiating through discretized SDEs, known the discretize-optimize approach, is reliable for both Itô and Stratonovich calculus. Tale 2 examines the optimize-discretize approach, investigating the continuous limit of backward equations from Tale 1 corresponding to the desired gradients. Our aim is to equip readers with a clear guide on the numerical differentiation of SDEs: computing gradients correctly in both Itô and Stratonovich settings, understanding when discretize-optimize and optimize-discretize agree or diverge, and developing intuition for reasoning about stochastic differentiation beyond the cases explicitly covered.

• The paper introduces and rigorously analyzes two primary methods for computing gradients in SDEs: discretize‐optimize and optimize‐discretize.
• It details the use of Itô and Stratonovich calculus, highlighting how drift corrections and integrator consistency impact gradient bias and convergence.
• Empirical results, including the Black–Scholes and CEV models, validate the approaches and offer practical recommendations for finance and machine learning applications.

Differentiating through Stochastic Differential Equations: Theory, Methods, and Practical Implications

Introduction and Motivation

This primer provides a comprehensive and technically rigorous treatment of gradient computation for functionals of stochastic differential equations (SDEs), a foundational task in physics, finance, computational science, and machine learning. Whereas sensitivities of ODE-driven systems (parameter gradients, adjoints) are well-understood and reliably computed, analogous operations on SDEs present intricacies due to the interplay of stochasticity, discretization, and chain rule application. The paper supplies a dual-narrative pedagogical structure: (1) Discretize-Optimize, differentiating through discrete SDE integrators; (2) Optimize-Discretize, deriving and discretizing the backward (adjoint) SDE. Central themes include when these approaches yield consistent gradients, the impact of Itô vs Stratonovich calculus, and practical recommendations for gradient computation in SDE-based models.

Core Problem Formulation

Let $X(s; x_0)$ be an SDE solution starting from $x_0 \in \mathbb{R}^d$, described in either Itô or Stratonovich form. We study objectives of the form

$J(x_0) = \mathbb{E}\left[ \Phi(X(T; x_0)) \right],$

with $\Phi$ a suitably regular terminal cost, and seek to compute (or estimate) $\nabla_{x_0} J(x_0)$.

This challenge pervades applications:

• In finance, Greeks are sensitivities of option prices to parameters in a stochastic asset model.
• In control and machine learning, optimizing neural or parametric SDEs requires backpropagating loss gradients through SDE trajectories.

Discretize-Optimize for SDEs

Itô SDEs: Direct Differentiation through Discrete Integrators

Under an Itô SDE

$$dX(s) = f(s, X(s)) ds + g(s, X(s)) dW(s), \quad X(0) = x_0,$$

the Euler–Maruyama scheme discretizes the forward solution. Automatic differentiation through the simulation (be it forward-mode or reverse-mode/adjoint) yields valid unbiased estimators of gradients, provided the chain rule is followed for the discrete system.

Reverse-mode differentiation leads to a backward recursion for the discrete adjoint $p_n$, updating via pathwise Jacobians involving drift and diffusion, and ultimately returns gradients by averaging over Monte Carlo trajectories. This is summarized by the recursion:

$$p_n = \left( \mathbf{I} + \Delta t \frac{\partial f}{\partial X_n}^\top + \Delta W_n^\top \frac{\partial g}{\partial X_n}^\top \right) p_{n+1}$$

Empirically, as shown in experiments using the Black–Scholes SDE, the computed pathwise gradients converge to analytic option Greeks as the step size $\Delta t \to 0$, up to MC error.

Figure 1: Error in discrete adjoint gradient compared to the analytical $\Delta$ versus time step size $\Delta t$ for Black–Scholes model on log-log scale.

Stratonovich SDEs: Differentiation through Heun Schemes

For Stratonovich SDEs, the Euler–Maruyama method is inconsistent and thus the Heun (or stochastic trapezoidal) method is used. The chain rule propagates through both predictor and corrector steps, generating a more complex adjoint recursion due to the two-stage update. Nevertheless, automatic differentiation handles these expressions efficiently.

The essential claim is that for both calculi, discretize-optimize is robust: differentiating through a consistent numerical SDE integrator yields correct gradients for discrete problems, and as the mesh refines, those gradients converge.

Parameter Sensitivities and Running Costs

The adjoint recursions extend to additional parameters in drift or diffusion by augmentation (i.e., treating parameters as constant state variables), resulting in an augmented adjoint or coupled recursion. Similarly, objectives with running costs (integrals along trajectories) are handled via state augmentation. These reductions enable the same automatic differentiation workflows.

When Discretize-Optimize and Optimize-Discretize Diverge

A crucial finding is that unlike ODEs, discretize-optimize and optimize-discretize may yield fundamentally different limits for SDEs, particularly for Itô SDEs with state-dependent diffusion.

A naive optimize–discretize procedure—deriving the continuous Itô adjoint SDE, then discretizing—is generally ill-posed: the backward equation is non-adapted, does not correspond to a valid Itô SDE, and, as shown by explicit experiments with the CEV model, results in significantly biased gradients.

Figure 2: Histograms of pathwise gradients $\Delta = \partial C / \partial S_0$ for the CEV model with $\beta = 1.33$, comparing discrete-adjoint and naive-adjoint estimators.

Contradictory to naïve expectations and classical ODE adjoint logic, this bias can be severe and is only avoided in rare cases (e.g., additive or affine noise) where the diffusion Jacobian is state-independent.

Optimize-Discretize via Stratonovich Calculus

Optimize-discretize can be made consistent and unbiased only for Stratonovich SDEs. Here, backward continuous-time adjoint SDEs exist:

$$dp(s) = -\frac{\partial f}{\partial x}^\top p(s) ds - \sum_j \frac{\partial g_{:j}}{\partial x}^\top p(s) \circ dW^j(s)$$

and their discretizations (e.g., Heun's scheme) converge to the correct continuous object. This result is well-supported by rough path theory.

When the underlying modeling or application context requires Itô SDEs (e.g., finance), one must convert to equivalent Stratonovich form (via drift correction), compute the continuous adjoint, and convert back if necessary. Empirical evidence confirms that this procedure yields agreement between approaches for pathwise gradients.

Figure 3: Histograms of pathwise gradients $\Delta = \partial C / \partial S_0$ for the CEV model with $\beta = 1.33$, comparing Itô discrete adjoint and Stratonovich-corrected optimize-discretize estimators.

Practical Implementation Considerations

For both approaches, reusing the same random seed (Brownian path) in the forward and backward passes is essential for gradient unbiasedness. Discretize-optimize requires storing or regenerating forward state trajectories (possibly via checkpointing). Optimize-discretize trades reduced memory demands for potential approximation error in reconstructing forward trajectories via time reversal—benefiting most when Stratonovich calculus is admissible.

Implications and Outlook

Theoretical Implications

• For general SDEs, discretize-optimize using automatic differentiation through the chosen SDE integrator is the most reliable and robust method for unbiased gradient estimation, especially for Itô SDEs with state-dependent diffusion.
• Optimize-discretize is justified only for Stratonovich SDEs or for Itô SDEs after drift conversion to Stratonovich form.
• The developed adjoint methodology is non-adapted, tractable for simulation, and does not require second-order derivatives or the simultaneous estimation of auxiliary backward processes, in contrast to the adapted stochastic adjoint SDEs prevalent in the optimal control literature.

Practical Recommendations

• Finance: Discretize-optimize with Itô Euler–Maruyama for gradient estimation; if continuous adjoints are required, perform drift correction and use the Stratonovich framework.
• Machine learning (Neural SDEs/score-based generative models): Prefer Stratonovich formulations for continuous-time models; for simulation, use automatic differentiation through compatible integrators.
• Control/theory: For analytic insight or backward equations, use Stratonovich adjoint SDEs; but for numerics, always verify that discretization matches the calculus.
• Software: Write SDE solvers in AD-enabled frameworks (e.g., PyTorch, JAX)—differentiate directly through simulation for reliability.

Conclusion

This tutorial synthesizes and clarifies the theoretical and practical aspects of differentiating through SDEs. Through precise analysis, illustrative numerical results, and concrete recommendations, it demonstrates:

• Discretize-optimize is a universally robust approach for SDE gradient computation.
• Optimize-discretize is valid when exploiting the symmetry of Stratonovich calculus (or by explicit Itô-to-Stratonovich correction).
• Naive transfer of deterministic adjoint logic to SDEs yields incorrect gradients when diffusion depends on state.

The work serves as both a technical reference and practical guide for researchers and practitioners in scientific computing, quantitative finance, and machine learning. Ongoing and future research may address advanced SDE integrators, efficient checkpointing and adjoint storage, and algorithmic differentiation through SDEs on pathspace, as well as extensions to high-dimensional PDEs via stochastic representation formulas.