Stochastic Generating Function NN (SGFNN)

• SGFNN is a neural network model that learns stochastic Hamiltonian systems by embedding the symplectic geometry through a stochastic generating function.
• It employs an autoencoder to extract latent Gaussian variables that capture the randomness driving the system and enforce structure in the learned one-step map.
• Empirical results show that SGFNN offers improved long-term prediction accuracy and better preservation of physical invariants compared to standard stochastic flow map methods.

The stochastic generating function neural network (SGFNN) is a neural network model for learning stochastic Hamiltonian systems from observational data. SGFNN preserves symplectic structure of the underlying stochastic Hamiltonian system and produces symplectic predictions. The model uses an autoencoder framework to identify the randomness of the latent system by the encoder network, and detects the stochastic generating function of the system through the decoder network based on the random variables extracted from the encoder. Symplectic predictions are then generated by the stochastic generating function (Chen et al., 19 Jul 2025).

1. Stochastic Hamiltonian setting

SGFNN is formulated for $2d$-dimensional Stratonovich stochastic Hamiltonian systems of the form

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$

Here $p,q \in \mathbb{R}^d$, and $W_k$, $k=1,\dots,r$, are independent standard Brownian motions. The system is Hamiltonian in the stochastic sense if there exist smooth functions $H(p,q)$ and $H_k(p,q)$, $k=1,\dots,r$, such that

$$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$

This formulation covers additive and multiplicative noise, separable and non-separable Hamiltonians, and systems with single or multiple noises (Chen et al., 19 Jul 2025).

The solution flow $\varphi_t(x_0;\omega)$, with $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$0, is symplectic almost surely. If $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$1, then

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$2

A learned or numerical one-step map $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$3 is symplectic if it satisfies the same condition. In the SGFNN construction, this property is imposed by design rather than recovered only empirically (Chen et al., 19 Jul 2025).

The examples used to instantiate the framework include a linear stochastic oscillator with additive noise, the Kubo oscillator with multiplicative noise, a non-separable stochastic Hamiltonian system with

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$4

and synchrotron oscillations with two multiplicative noises. These examples are chosen to span distinct structural regimes of stochastic Hamiltonian dynamics (Chen et al., 19 Jul 2025).

2. Stochastic generating functions and random symplectic maps

In symplectic geometry, a generating function is a scalar function whose partial derivatives define a symplectic map. For the first kind, a function $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$5 defines the map $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$6 through

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$7

In the stochastic setting, the generating function becomes a random function $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$8. If $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$9 is a smooth random map, then $p,q \in \mathbb{R}^d$0 is symplectic if there exists locally a smooth random map $p,q \in \mathbb{R}^d$1 such that $p,q \in \mathbb{R}^d$2 is invertible almost surely and

$p,q \in \mathbb{R}^d$3

This is the structural device used by SGFNN to ensure that the learned one-step map is symplectic (Chen et al., 19 Jul 2025).

For a time step $p,q \in \mathbb{R}^d$4, the exact one-step solution map of an SHS is a random symplectic map $p,q \in \mathbb{R}^d$5. SGFNN represents it locally by a stochastic generating function

$p,q \in \mathbb{R}^d$6

where $p,q \in \mathbb{R}^d$7 is a finite-dimensional latent representation of the stochastic forcing over $p,q \in \mathbb{R}^d$8. The paper does not derive an explicit analytic formula for $p,q \in \mathbb{R}^d$9; instead, $W_k$0 is treated as an unknown function to be learned (Chen et al., 19 Jul 2025).

A common terminological confusion is to read “generating function” here as a probabilistic generating function. In SGFNN, the term refers to the symplectic-geometry generating function of a random map. This suggests that the model is fundamentally a structure-preserving discretization of stochastic Hamiltonian flow rather than a generic stochastic latent-variable network.

3. Autoencoder architecture and learning objective

For a fixed time step $W_k$1, the training data are one-step pairs

$W_k$2

SGFNN assumes that for each pair there exists a latent Gaussian variable $W_k$3 such that the pair is generated by an implicit symplectic map induced by a stochastic generating function (Chen et al., 19 Jul 2025).

The encoder is

$W_k$4

The latent variable is trained to satisfy three conditions: it should be independent of $W_k$5, distributed as standard Gaussian, and have dimension $W_k$6 matching the number of effective stochastic drivers. For a batch $W_k$7, the distribution-matching loss is

$W_k$8

with

$W_k$9

The moment term matches central moments up to order $k=1,\dots,r$0 and penalizes cross-correlations among latent dimensions (Chen et al., 19 Jul 2025).

The decoder is the learned stochastic generating function

$k=1,\dots,r$1

For a one-step pair $k=1,\dots,r$2, the generating-function relations are

$k=1,\dots,r$3

The data-fitting loss over a batch $k=1,\dots,r$4 is

$k=1,\dots,r$5

where the partial derivatives are computed by automatic differentiation at $k=1,\dots,r$6 (Chen et al., 19 Jul 2025).

The total objective is

$k=1,\dots,r$7

Minimizing this objective yields an encoder that outputs approximate $k=1,\dots,r$8, a decoder that approximates the stochastic generating function $k=1,\dots,r$9, and a learned symplectic one-step map $H(p,q)$0 (Chen et al., 19 Jul 2025).

4. Training data, optimization, and prediction

SGFNN requires only state observations sampled at equal time step $H(p,q)$1. In the numerical study, the training data are discrete trajectories

$H(p,q)$2

and the effective training set is the collection of one-step pairs $H(p,q)$3. The reported experiments use $H(p,q)$4 trajectories generated numerically by the midpoint scheme with time step $H(p,q)$5. Both encoder and decoder are fully connected networks with $H(p,q)$6 hidden layers, $H(p,q)$7 neurons each, and ELU activation. Optimization uses Adam with learning rate $H(p,q)$8. The training configuration is $H(p,q)$9 batches of size $H_k(p,q)$0 (Chen et al., 19 Jul 2025).

The method adopts the sub-sampling strategy from sFML: for each batch, the initial states $H_k(p,q)$1 are selected as nearest neighbors to a randomly chosen center, so that the empirical distribution of $H_k(p,q)$2 in the batch is non-Gaussian and the independence of $H_k(p,q)$3 from $H_k(p,q)$4 becomes more identifiable. This batching rule is part of the latent-noise identification mechanism (Chen et al., 19 Jul 2025).

After training, prediction uses only the decoder. Given $H_k(p,q)$5 and a new sample $H_k(p,q)$6, the one-step map is implicit in $H_k(p,q)$7: $H_k(p,q)$8 The paper solves this with a fixed-point iteration: $H_k(p,q)$9 until

$k=1,\dots,r$0

with $k=1,\dots,r$1. The predicted step is then

$k=1,\dots,r$2

Multi-step prediction is obtained by iteration in time, drawing independent $k=1,\dots,r$3 at each step (Chen et al., 19 Jul 2025).

The symplecticity guarantee is architectural. For any fixed $k=1,\dots,r$4, the map defined by the generating-function relations is symplectic provided $k=1,\dots,r$5 is invertible. Because the decoder parameterizes the stochastic generating function rather than an unconstrained flow map, the learned discrete-time dynamics are symplectic by construction (Chen et al., 19 Jul 2025).

5. Empirical behavior and comparison with stochastic flow map learning

The benchmark in the study is stochastic Flow Map Learning (sFML). sFML also uses an autoencoder, but its decoder learns a direct flow map

$k=1,\dots,r$6

with no structural constraint such as symplecticity. SGFNN instead learns the generating function $k=1,\dots,r$7, and the one-step map is recovered through symplectic relations. The reported prediction metrics at final time $k=1,\dots,r$8 are

$k=1,\dots,r$9

Across all reported systems, SGFNN achieves smaller errors than sFML, with the difference especially pronounced in long-term predictions (Chen et al., 19 Jul 2025).

Example	$$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$0	$$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$1
Linear oscillator, $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$2	sFML $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$3 → SGFNN $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$4	sFML $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$5 → SGFNN $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$6
Kubo oscillator, $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$7	sFML $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$8 → SGFNN $$\begin{aligned} f(p,q) &= -\frac{\partial H}{\partial q}(p,q)^T, &\quad \sigma_k(p,q) &= -\frac{\partial H_k}{\partial q}(p,q)^T, \ g(p,q) &= \phantom{-}\frac{\partial H}{\partial p}(p,q)^T, &\quad \gamma_k(p,q) &= \phantom{-}\frac{\partial H_k}{\partial p}(p,q)^T. \end{aligned}$$9	sFML $\varphi_t(x_0;\omega)$0 → SGFNN $\varphi_t(x_0;\omega)$1
Non-separable SHS, $\varphi_t(x_0;\omega)$2	sFML $\varphi_t(x_0;\omega)$3 → SGFNN $\varphi_t(x_0;\omega)$4	sFML $\varphi_t(x_0;\omega)$5 → SGFNN $\varphi_t(x_0;\omega)$6
Synchrotron oscillations, $\varphi_t(x_0;\omega)$7	sFML $\varphi_t(x_0;\omega)$8 → SGFNN $\varphi_t(x_0;\omega)$9	sFML $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$00 → SGFNN $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$01

The qualitative comparisons align with these metrics. For the linear stochastic oscillator,

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$02

and SGFNN reproduces the linear growth of the second moment up to $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$03, whereas sFML deviates. For the Kubo oscillator,

$$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$04

and SGFNN phase plots remain circular and radius-conserving, while sFML trajectories drift away from circles. In the non-separable and synchrotron examples, SGFNN tracks mean and standard deviation more accurately over time and yields probability density functions closer to the reference solution. In all cases, the histograms of the learned latent variables $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$05 match standard normal densities, which is consistent with successful extraction of the stochastic drivers (Chen et al., 19 Jul 2025).

6. Relation to adjacent stochastic neural models

SGFNN belongs to a broader family of neural models that embed structural priors from stochastic dynamics, but its defining feature is the use of a learned stochastic generating function to enforce symplecticity. This distinguishes it from several nearby lines of work.

“Stochastic Generative Flow Networks” extend GFlowNets to stochastic environments by decomposing each transition into a policy-controlled deterministic step and an environment step, together with a learned dynamics model $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$06 (Pan et al., 2023). That framework addresses reward-proportional sampling in stochastic DAG environments rather than learning stochastic Hamiltonian flows. “Embedding stochastic differential equations into neural networks via dual processes” constructs networks for predicting expectations of SDEs by matching neural-network Taylor coefficients to the coefficients of a dual process, and explicitly avoids input-output datasets (Sugishita et al., 2023). GFINNs encode the GENERIC formalism and obey symmetric degeneracy conditions for deterministic and stochastic dynamical systems (Zhang et al., 2021). Stochastic feedforward networks and Gaussian-process neurons study stochastic mappings, stochastic activations, or stochastic latent units, but not learned stochastic generating functions for random symplectic maps (Merkh et al., 2019, Lee et al., 2017, Urban et al., 2017).

A second recurring misconception concerns the phrase “generating function.” In SGFNN it is not a probability-generating or moment-generating function. It is the first-kind generating function of symplectic geometry, made stochastic through latent random variables. This suggests that SGFNN is best interpreted as a structure-preserving model class specialized to stochastic Hamiltonian systems, not as a general-purpose stochastic sequence model.

7. Assumptions, limitations, and extensions

The method assumes that the underlying dynamics are genuinely stochastic Hamiltonian and that the effective stochastic driving can be summarized by a finite-dimensional Gaussian latent variable $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$07. If the true system is non-Hamiltonian, dissipative, or driven by noise that is strongly non-Gaussian or highly path-dependent, the structural prior is mis-specified. Training is based on pairwise one-step data and depends on the sub-sampling strategy inherited from sFML. At inference time, every step requires solving an implicit equation for $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$08 by fixed-point iteration, which may be heavier than an explicit flow-map evaluation. The reported experiments are confined to two-dimensional phase spaces (Chen et al., 19 Jul 2025).

The paper identifies several natural directions for extension. These include more advanced parameterizations of the generating function $$\begin{aligned} d p &= f(p,q)\,dt + \sum_{k=1}^r \sigma_k(p,q) \circ dW_k(t), \quad p(t_0)=p_0, \ d q &= g(p,q)\,dt + \sum_{k=1}^r \gamma_k(p,q) \circ dW_k(t), \quad q(t_0)=q_0. \end{aligned}$$09, such as SympNets-like constructions, theoretical analysis of generalization, extensions beyond Brownian-driven SHSs to Lévy noise or colored noise, integration with density-based approaches such as Fokker–Planck or normalizing-flow methods, and incorporation of additional physical structures such as symmetries and invariants (Chen et al., 19 Jul 2025). A plausible implication is that future SGFNN variants may be organized around richer geometric priors rather than around larger unconstrained decoders.

Within its stated domain, SGFNN provides a direct synthesis of latent-noise identification, generating-function learning, and symplectic structure preservation. Its central claim is not merely that stochastic Hamiltonian systems can be approximated by neural networks, but that the approximation can be carried out through a learned random symplectic map whose discrete-time predictions remain symplectic by construction (Chen et al., 19 Jul 2025).