---
title: Stochastic PushForward (SPF) Overview
url: https://www.emergentmind.com/topics/stochastic-pushforward-spf
type: topic
---

# Stochastic PushForward (SPF) Overview

Searching arXiv for recent papers on "Stochastic PushForward" and related "push-forward" formulations.
arxiv_search.query({"search_query":"all:\"Stochastic PushForward\" OR all:\"Stochastic Push-Forward\" OR ti:\"Stochastic PushForward\" OR ti:\"Stochastic Push-Forward\"","start":0,"max_results":10,"sort_by":"submittedDate","sort_order":"descending"})
Querying arXiv for exact title matches and closely related terms.
arxiv_search.search({"query":"\"Improving Long-term Autoregressive Spatiotemporal Predictions: A Proof of Concept with Fluid Dynamics\" OR \"Uncertainty Propagation in Stochastic Hybrid Systems with Dimension-Varying Resets\" OR \"Push-forward method for piecewise deterministic biochemical simulations\"","max_results":10})
Looking up arXiv ids directly to anchor the terminology.
arxiv_search.lookup({"id_list":["2508.18565","2604.06708","2009.06577"]})
Stochastic PushForward (SPF) denotes a push-forward-based construction that has acquired at least two distinct technical meanings in recent arXiv literature. In one usage, SPF is a training framework for long-term autoregressive spatiotemporal prediction that retains one-step-ahead training while enabling multi-step learning by building a supplementary dataset from model predictions and combining it with ground truth via a stochastic acquisition strategy [2508.18565]. In another usage, Stochastic Push-Forward is a weak-form formulation for stochastic hybrid systems in which outgoing probability flux through a guard set is pushed forward under a reset map, producing measure-valued source terms in the target mode [2604.06708]. A related antecedent is the push-forward method for piecewise deterministic biochemical simulations, which computes how the probability measure is spread by the deterministic ODE flow of PDMPs through analytic expressions of the corresponding semigroup and was later generalized to non-integrable systems [2009.06577].

## 1. Terminological scope and lineage

Current arXiv usage suggests that SPF is not a field-wide standardized term but rather a family resemblance centered on push-forward transport. In the machine-learning formulation, the transported object is a model-generated latent-state dataset. In the stochastic-hybrid formulation, the transported object is probability flux across guards. A plausible implication is that the commonality lies less in application domain than in a shared operator-level viewpoint: a distribution, measure, or dataset is mapped forward through a deterministic transformation and then reinserted into a learning or evolution equation.

| Usage | Setting | Core mechanism |
|---|---|---|
| Stochastic PushForward | Long-term autoregressive spatiotemporal prediction | Build \(\mathcal D_\delta\) from \(f^\delta(\boldsymbol\eta_t)\) and mix it with \(\mathcal D_1\) by Bernoulli sampling |
| Stochastic Push-Forward | Stochastic hybrid systems with reset maps | Push forward outgoing probability flux through the guard under the reset map |
| Push-forward method | Piecewise deterministic biochemical simulations | Spread probability measure by deterministic ODE flow using analytic semigroup expressions |

The biochemical PDMP work provides a historical anchor for the terminology. It treats biochemical networks for which smooth ODE dynamics are punctuated by discrete stochastic events, notes that PDMP models are well adapted for such situations, and describes a more general simulation algorithm that works also for non-integrable systems, with applications in fundamental biology, biotechnology and biocomputing; it is also identified as an extended version of work presented at CMSB2019 [2009.06577].

## 2. Formal construction of SPF for autoregressive prediction

In the spatiotemporal prediction setting, SPF is formulated in latent space. Let
\[
\mathcal D_1=\{\boldsymbol\eta_1,\dots,\boldsymbol\eta_T\}
\]
be the original one-step training dataset of latent states, let \(f(\cdot;\theta_f)\) be the one-step-ahead surrogate model, and let \(f^\delta\) denote its \(\delta\)-fold composition. SPF constructs a supplementary dataset by rolling the model forward:
\[
\tilde{\boldsymbol\eta}_{t+\delta}=f^\delta\bigl(\boldsymbol\eta_t\bigr), \qquad t=1,\dots,T-\delta.
\]
It then defines
\[
\mathcal D_\delta=\{\boldsymbol\eta_1,\dots,\boldsymbol\eta_\delta,\,
\tilde{\boldsymbol\eta}_{\delta+1},\dots,\tilde{\boldsymbol\eta}_T\},
\qquad
\mathcal D=\mathcal D_1\oplus \mathcal D_\delta.
\]

The stochastic acquisition step samples, for each training index \(t\),
\[
I_t\sim\mathrm{Bernoulli}(p),
\qquad
\boldsymbol\eta^I_t
=
I_t\,\boldsymbol\eta^{\mathcal D_1}_{t}
+
(1-I_t)\,\boldsymbol\eta^{\mathcal D_\delta}_{t},
\qquad
\boldsymbol\eta_{t+1}=\boldsymbol\eta^{\mathcal D_1}_{t+1}.
\]
The training target is always the true next latent state from \(\mathcal D_1\). The corresponding weighted single-step loss is
\[
\tilde{\boldsymbol\eta}_{t+1}=f\bigl(\boldsymbol\eta^I_t\bigr),
\qquad
\mathcal L_t=\bigl\lVert \tilde{\boldsymbol\eta}_{t+1}-\boldsymbol\eta_{t+1}\bigr\rVert_2^2,
\]
with weight
\[
\gamma_t=
\begin{cases}
1, & \text{if }I_t=1,\\
\alpha\in(0,1], & \text{if }I_t=0,
\end{cases}
\qquad
\mathcal L(\theta_f)=\frac1{|\mathcal D|}\sum_{t\in\mathcal D}\gamma_t\,\mathcal L_t.
\]
Optimization is by standard gradient descent such as Adam. This construction retains one-step-ahead training while enabling multi-step learning [2508.18565].

## 3. Sequential training procedure and optimization logic

The reported workflow begins with a fully trained one-step model \(f\) on \(\mathcal D_1\). During SPF sequential training, every \(N_{\rm UI}\) epochs the model is run forward \(\delta\) steps on each \(\boldsymbol\eta_t\in\mathcal D_1\) to form \(\tilde{\boldsymbol\eta}_{t+\delta}\), after which \(\mathcal D_\delta\) is assembled by replacing the last \(T-\delta\) ground truths with these \(\delta\)-step predictions. Training iterations then sample inputs from \(\mathcal D_1\) or \(\mathcal D_\delta\) via \(I_t\sim\mathrm{Bernoulli}(p)\), while always keeping the target as the true next latent state \(\boldsymbol\eta_{t+1}\in\mathcal D_1\).

This design balances short-term and long-term behavior by mixing ground-truth inputs with probability \(p\) and model-rolled-out inputs with probability \(1-p\). The auxiliary dataset teaches the model to correct its own future states without incurring the memory cost of backpropagating through the unroll. The paper further states that stochastically drawing from both distributions reduces overfitting to either extreme and combats distribution shift in long rollouts. Its theoretical intuition parallels “scheduled sampling” but preserves one-step gradient flow, guaranteeing stable optimization. Because multi-step predictions are precomputed between epochs, memory usage remains stable without storing full unrolled sequences [2508.18565].

## 4. Empirical performance on canonical fluid benchmarks

The experimental evaluation is reported on Burgers’ equation and a Shallow Water benchmark, with accumulated MSE over rollout and SSIM as primary image-like fidelity metrics, and with energy conservation additionally reported for Shallow Water. The qualitative pattern is consistent across settings: basic one-step LSTM training exhibits rapid long-horizon deterioration, ATF and PF improve over the naive baseline, PF can become unstable at larger \(\delta\), and SPF2/SPF3 produce the strongest long-term fidelity.

| Benchmark | Metrics | Reported SPF behavior |
|---|---|---|
| Burgers’ equation, \(128\times128\), \(Re=1/V\), 300 time-steps | accumulated MSE; SSIM | SPF2/SPF3: 50–70 % lower accumulated error; SSIM sustained above 0.9 for \(>270\) steps; SPF3 achieves the best long-term fidelity |
| Shallow Water, \(64\times64\), \(u,v,h\), 300 steps | accumulated MSE; SSIM; energy conservation | SPF2/3 consistently outperforms ATF and PF in MSE/SSIM; SPF3 preserves total energy within 1–2 % error |
| Noisy inputs, Matern noise with correlation length \(L=4\) | SSIM under perturbation | SPF3 retains SSIM\(\ge 0.8\) for \(\sim 30\%\) more steps than ATF/PF |

For Burgers’ equation, the baseline LSTM is reported to show rapid error growth, with SSIM dropping below \(0.8\) after \(\sim 100\) steps, whereas SPF3 with \(\delta=3\) achieves the best long-term fidelity. For Shallow Water, ATF2/3 and PF2/3 improve over naive LSTM, but PF suffers from unstable rollouts at larger \(\delta\). Under limited data, the contrast is sharper: with only \(5\%\) of training data, ATF collapses, with SSIM \(<0.8\) in \(<10\) steps, while SPF retains SSIM \(>0.8\) over \(\sim 30\) steps. In extrapolation beyond the training horizon, over steps \(351\)–\(450\), SPF models maintain structural fidelity, with SSIM \(\approx 0.85\) at step \(50\), outperforming ATF/PF [2508.18565].

## 5. Memory profile, hyperparameters, and implementation considerations

A central claim of SPF is that multi-step learning can be obtained without the GPU-memory growth associated with stored autoregressive unrolls. The reported complexity is \(O(T)\) for storing the datasets on CPU and \(O(1)\) on GPU per batch, independently of \(\delta\), because no multi-step backpropagation graph is stored.

On a Tesla T4 (Google Colab), training with a 512-dim CAE + LSTM yields the following approximate GPU RAM usage:

| Configuration | GPU RAM |
|---|---:|
| Basic LSTM (\(\delta=1\)) | 567 MB |
| ATF2 (\(\delta=2\)) | 959 MB |
| ATF3 (\(\delta=3\)) | 1 159 MB |
| PF2 (\(\delta=2\)) | 873 MB |
| PF3 (\(\delta=3\)) | 1 065 MB |
| SPF2 & SPF3 | \(\sim 689\) MB |

These values motivate the statement that SPF reduces GPU memory by \(\sim 40\%\) at \(\delta=3\) versus ATF while retaining multi-step learning benefits. Hyperparameter sensitivity for \(\delta=2\) is also reported explicitly: short-term one-step MSE lies in \([1.2\mathrm e{-5},3.9\mathrm e{-5}]\) and SSIM is \(>0.98\) for all \((p,\alpha)\), whereas long-term 150-step MSE lies in \([0.0011,0.0028]\) and SSIM in \([0.8161,0.8959]\). The best MSE occurs at \((p=0.5,\alpha=1.0)\), and the best SSIM at \((p=0.5,\alpha=0.75)\).

The stated limitations are added complexity in maintaining and updating \(\mathcal D_1\) and \(\mathcal D_\delta\), CPU–GPU communication overhead when loading large datasets each batch, and the introduction of the extra hyperparameters \((p,\alpha)\) and \(N_{\rm UI}\). The stated best practices are to pretrain the one-step model thoroughly before building \(\mathcal D_\delta\), choose \(p\approx0.5\) as a default trade-off, tune \(\alpha\) in \([0.5,1]\), set \(N_{\rm UI}\) to \(5\)–\(10\) epochs, store datasets on fast CPU RAM or NVMe, batch-load to GPU, and monitor both one-step loss and rollout metrics such as MSE, SSIM, and physical invariants. The stated potential extensions are a progressive curriculum of \(\delta\), integration of physics-based penalties as in SPF-PC, asynchronous or streamed dataset loading, and application to other sequence architectures including Transformers and Graph-Nets [2508.18565].

## 6. Push-forward source terms in stochastic hybrid systems

In the stochastic-hybrid usage, SPF is a measure-theoretic formulation of probability-density evolution for systems with reset maps that may change the dimension of the continuous state. The key claim is that classical boundary-condition-based Frobenius–Perron formulations are insufficient when resets map guard sets into the interior of another mode or onto lower-dimensional subsets. SPF addresses this by representing reset-induced transfer as the pushforward of probability flux across the guard.

Let \(G\) be a codimension-one guard manifold in the source space \(X\subset\mathbb R^n\), with surface measure \(dS(x)\) and outward unit normal \(\hat n(x)\), and let
\[
R:G\to Y\subset\mathbb R^m
\]
be a measurable reset map. If \(\mu\) is a measure on \(G\), its push-forward \(R_\#\mu\) is defined by
\[
(R_\#\mu)(A)=\mu\bigl(\{x\in G:R(x)\in A\}\bigr)
\qquad
\forall\text{ measurable }A\subset Y.
\]
When \(\mu\) has density \(\gamma(x)\) with respect to \(dS\), one may write
\[
R_\#(\gamma\,dS)(B)
=
\int_{R^{-1}(B)}\gamma(x)\,dS(x)
=
\int_B\Bigl[\sum_{x\in R^{-1}(y)}\frac{\gamma(x)}{|\det DR(x)|}\Bigr]\,dy
\]
whenever \(R\) is a local diffeomorphism onto its image.

For a mode density \(\rho(x,t)\) satisfying the deterministic Liouville or Fokker–Planck equation
\[
\partial_t\rho(x,t)+\nabla_x\!\cdot\bigl(f(x)\rho(x,t)\bigr)=0
\qquad
\text{for }x\in X\setminus G,
\]
the weak form introduces the guard contribution
\[
\int_G\phi(x)\,\bigl(f\rho\cdot\hat n\bigr)(x)\,dS(x).
\]
Under reset \(R\), this outgoing flux is instantaneously injected into the target domain \(Y\), so the right-hand side is rewritten as
\[
\int_Y \phi(y)\;d\Bigl[R_\#\bigl((f\rho\cdot\hat n)\,dS\bigr)\Bigr](y).
\]
Hence, in the target mode,
\[
\partial_t\rho(y,t)+\nabla_y\!\cdot\bigl(f(y)\rho(y,t)\bigr)=d\eta(y,t),
\]
where
\[
d\eta(t)=R_\#\bigl((f\rho\cdot\hat n)\,dS\bigr).
\]
With multiple guards and resets,
\[
\eta(t)=\sum_{\substack{\text{all source}\\\text{guards }G_r}}
R_{r\#}\Bigl(\bigl(f_r\rho_r\cdot \hat n_r\bigr)\,dS_r\Bigr).
\]
This produces a unified description in which guard-to-mode transitions appear as nonlocal source or sink terms rather than only as boundary conditions [2604.06708].

## 7. Dimension-varying resets and the merge/split example

The most distinctive feature of the stochastic-hybrid SPF formulation is its treatment of dimension-changing resets. When the reset decreases dimension, the pushed-forward guard flux becomes an absolutely continuous measure in the target mode. Concretely, if
\[
R:G\to Y\subset\mathbb R^{n-1}
\]
is a smooth bijection from an \((n-1)\)-dimensional guard, then
\[
d\eta(y,t)=s(y,t)\,dy,
\qquad
s(y,t)=\sum_{x\in R^{-1}(y)}
\frac{\bigl(f(x)\rho(x,t)\cdot\hat n(x)\bigr)}{|\det DR(x)|},
\]
and the strong PDE becomes
\[
\partial_t\rho(y,t)+\nabla\cdot\bigl(f(y)\rho(y,t)\bigr)=s(y,t).
\]
When the reset increases dimension, the push-forward measure is singular on the image manifold \(M=R(G)\). If
\[
R:G\to Y\subset\mathbb R^{n+1},
\]
then
\[
d\eta(y,t)
=
\int_G\delta\bigl(y-R(x)\bigr)\,
\bigl(f(x)\rho(x,t)\cdot\hat n(x)\bigr)\,dS(x),
\]
so the source is supported on a lower-dimensional subset of the target space.

The merge/split model illustrates both cases within a single system. In Mode 1, two particles \(x_A,x_B\in[0,1]\) evolve in \(X_1\subset\mathbb R^2\), with guard
\[
G_1=\{|x_A-x_B|=\varepsilon\},
\]
and probability current
\[
Y_1=(Y_A,Y_B)
=\bigl(X_{1}(x)\rho_1-\tfrac{\sigma_1^2}{2}\nabla\rho_1\bigr).
\]
The reset
\[
\Phi_1:G_1\to X_2\subset\mathbb R^1,
\qquad
\Phi_1(x_A,x_B)=x_C=(x_A+x_B)/2,
\]
maps the one-dimensional guard diffeomorphically onto an interval in \(X_2\). The source in Mode 2 is therefore absolutely continuous:
\[
d\eta_2(t)=s_2(x_C,t)\,dx_C,
\]
with
\[
s_2(x_C,t)
=\Bigl[-Y_A\bigl(x_C+\tfrac\varepsilon2,x_C-\tfrac\varepsilon2\bigr)
+Y_B\bigl(x_C+\tfrac\varepsilon2,x_C-\tfrac\varepsilon2\bigr)\Bigr]
+
\Bigl[\,Y_A\bigl(x_C-\tfrac\varepsilon2,x_C+\tfrac\varepsilon2\bigr)
-Y_B\bigl(x_C-\tfrac\varepsilon2,x_C+\tfrac\varepsilon2\bigr)\Bigr].
\]
Thus Mode 2 satisfies
\[
\partial_t\rho_2+\partial_{x_C}(Y_2)=s_2(x_C,t).
\]

In Mode 2, the single particle \(x_C\in[0,1]\) has guard
\[
G_2=\{x_C=1\},
\]
with current \(Y_2(1,t)\). The reset
\[
\Phi_2:G_2\to X_1\subset\mathbb R^2
\]
sends \(1\mapsto(x_A^\ast,x_B^\ast)\), a single point in Mode 1. The source into Mode 1 is therefore a Dirac measure,
\[
d\eta_1(t)=Y_2(1,t)\,\delta_{(x_A^\ast,x_B^\ast)}(x_A,x_B),
\]
and the hybrid Fokker–Planck equation in Mode 1 becomes
\[
\partial_t\rho_1+\nabla\cdot Y_1
=
Y_2(1,t)\,\delta_{(x_A^\ast,x_B^\ast)}.
\]

This example makes the conceptual distinction precise. When the reset decreases dimension, SPF yields a smooth interior source density; when it increases dimension, SPF yields a lower-dimensional singular source, such as a Dirac measure. A common misconception would be to treat both situations as ordinary boundary-condition problems. The weak-form push-forward formulation shows that they are more naturally understood as reset-induced source terms whose regularity is determined by the geometry of the reset map [2604.06708].

Source: https://www.emergentmind.com/topics/stochastic-pushforward-spf