Probabilistic Taylor Series Expansion

• Probabilistic Taylor series expansion is a generalization of classical Taylor series that replaces deterministic variables with random elements to account for uncertainty.
• It quantifies the propagation of uncertainty through numerical measures, providing rigorous variance estimates and remainder terms in stochastic settings.
• Applications span probabilistic numerics, Gaussian process modeling, stochastic differential equations, and programming semantics for enhanced uncertainty analysis.

A probabilistic Taylor series expansion generalizes the classical Taylor series by replacing deterministic variables, functions, or operators with random variables, stochastic processes, or probability distributions. This expansion addresses the propagation of uncertainty, enables uncertainty quantification in numerical analysis, defines measurement update flows in sequential Monte Carlo methods, underpins probabilistic program semantics, and establishes rigorous statistical foundations for analyses involving random variables or estimators. The term subsumes several distinct developments across applied probability, numerical analysis, stochastic filtering, Gaussian process modeling, statistics, and the semantics of probabilistic programming languages.

1. Conceptual Foundations

The probabilistic Taylor series expansion replaces the deterministic Taylor expansion of a function $f(x)$ about a fixed point with an expansion involving random variables, stochastic inputs, or functions with inherently random behavior. This generalization is essential when the arguments of $f$ are random vectors or when $f$ itself is a random process. The expansion serves several purposes:

• Propagation of input uncertainty: For $f(X)$, where $X$ is a random vector with known mean and variance, the expansion quantifies how uncertainty in $X$ maps to the mean and variance of $f(X)$ (Wang, 2024).
• Stochastic process approximation: In stochastic differential equations (SDEs), expansions such as the Taylor–Ito and Taylor–Stratonovich series enable high-order strong approximation schemes, representing $X_{t+h}$ in terms of iterated stochastic integrals (Kuznetsov, 2017).
• Statistical analysis rigor: Classical Taylor's theorem is extended to random functions and random vectors by ensuring the remainder's intermediate point is a measurable random variable, critical for statistical theory and limit theorems (Yang et al., 2021).
• Probabilistic numerics: Gaussian process models with Taylor kernels reproduce truncated Taylor polynomials in the posterior mean and provide exact remainder quantification via the posterior variance (Karvonen et al., 2021).
• Resource-based program semantics: In lambda calculi with probabilistic choice, the Taylor expansion lifts to probabilistic resource terms, yielding a semantics that matches observable behavior (probabilistic Böhm trees) (Lago et al., 2019).
• Nonlinear Bayesian filtering: In particle flow filters, drift and diffusion terms are obtained from high-order multivariate Taylor expansions of the log-posterior, capturing nonlinearity and uncertainty in sequential inference (Servadio, 2 May 2025).

2. Statistical Taylor Expansions and Variance Propagation

When $f$ takes random inputs $X=(X_1,\dots,X_n)$ with known means $\mu_i$ and variances $\sigma_i^2$, and all $X_i$ are independent (uncorrelated uncertainty assumption), the Taylor expansion of $f(X)$ yields:

• Mean:

$$E[f(X)] = f(\mu) + \frac{1}{2} \sum_{i,j} f_{ij}(\mu) \operatorname{Cov}(X_i, X_j) + O(\|\sigma\|^3)$$

• Variance:

$$\operatorname{Var}[f(X)] \approx \nabla f(\mu)^T \Sigma \nabla f(\mu) + \frac{1}{2} \operatorname{tr}(H_f(\mu) \Sigma H_f(\mu) \Sigma)$$

where $$\Sigma=\operatorname{diag}(\sigma_1^2,\dots,\sigma_n^2)$$ and $H_f(\mu)$ is the Hessian (Wang, 2024).

Variance arithmetic implements these rules with exact symbolic derivatives and keeps mean and variance synchronized through all operations, ensuring path-independence. Requirements include series convergence over the distribution’s support and careful handling of possible function singularities.

3. Probabilistic Taylor Expansions in Filtering and Control

In particle filtering, measurement updates are often performed by moving particles via stochastic differential equations whose drift and diffusion are derived from Taylor polynomial representations:

• The log-posterior is expressed as $$\mathcal{P}(x,\lambda) = \mathcal{T}(x) + \lambda \mathcal{L}(x)$$ where $\mathcal{T}$ is prior log-density and $\mathcal{L}$ is the log-likelihood.
• Drift and diffusion are given in closed-form on the polynomial representation using Differential Algebra to compute all derivatives exactly up to order $c$:

$$\mathcal{F}(x,\lambda) = \mathbf{H}(x,\lambda)^{-1} \mathbf{G}_{\mathcal{L}}(x)$$

$$\mathcal{Q}(x,\lambda) = \mathbf{H}(x,\lambda)^{-1} [\nabla_x \mathcal{F}(x,\lambda)]^T$$

where $\mathbf{G}$ is the gradient and $\mathbf{H}$ the Hessian (Servadio, 2 May 2025).

Two strategies for the Taylor expansion center are used: mean-centered expansion (DAPFF v1-c) and per-particle expansion (DAPFF v2-c). At first order ($c=1$), the method recovers the classical Gromov flow. Incorporating higher-order terms results in significantly improved posterior particle alignment and uncertainty calibration.

In dynamic programming, the Bellman operator of controlled Markov chains is approximated by a second-order Taylor expansion, replacing the discrete Bellman equation with a continuous-space PDE, called the Taylored Control Problem (TCP), involving only the first two moments of the transition kernel. Explicit error bounds on the optimality gap and aggregation algorithms are derived (Braverman et al., 2018).

4. Probabilistic Taylor Expansions and Stochastic Analysis

In SDE numerics, the stochastic Taylor–Ito and Taylor–Stratonovich expansions express the solution $X_{t+h}$ as a sum over iterated stochastic integrals:

$$f(X_{t+h}) = f(X_t) + \sum_{s=1}^M \sum_{\alpha} G_\alpha(X_t, t) I_\alpha^{(\text{Ito})}(t, t+h) + O(h^{(M+1)/2})$$

Approximating the required stochastic integrals (of multiplicities up to 6) via multiple Fourier-Legendre series yields expansion coefficients and explicit convergence rates in mean-square and almost sure senses (Kuznetsov, 2017).

This formalism enables high-order strong numerical methods for SDEs where every expansion coefficient is determined symbolically, and the randomness is introduced via sampled independent Gaussian random variables.

5. Gaussian Process and Probabilistic Numerics Perspective

A Gaussian process (GP) prior with a Taylor kernel parameterization yields a posterior mean that exactly matches the classical $N$th-order Taylor polynomial at a chosen expansion point, provided the data comprises derivatives of $f$ at that point:

$$m_N(x) = \sum_{n=0}^N \frac{f^{(n)}(\xi)}{n!}(x-\xi)^n$$

The corresponding posterior variance is

$$P_N(x, x) = \sigma^2 \sum_{n=N+1}^\infty \frac{c_n \lambda^n}{(n!)^2} (x-\xi)^{2n}$$

which serves as a principled, probabilistic remainder term. The kernel hyperparameters can be estimated by maximum likelihood on derivatives, encoding prior beliefs about the function’s smoothness and scale. The expansion converges uniformly over compact sets when $f$ belongs to the reproducing kernel Hilbert space (RKHS) of the Taylor kernel (Karvonen et al., 2021).

6. Rigorous Random-variable Extensions and Statistical Applications

Classical Taylor’s theorem is insufficient in probabilistic and statistical contexts because the intermediate point in the remainder is not guaranteed to be measurable or random-variable-valued. A probabilistic Taylor theorem states that for suitably regular random variables $\mathbf{X}$ and increments $\mathbf{Y}$,

$$f(\mathbf{X}+\mathbf{Y}) = \sum_{m=0}^{k} \frac{1}{m!} (D_{\mathbf{Y}})^m f(\mathbf{X}) + \frac{1}{(k+1)!} (D_{\mathbf{Y}})^{k+1} f(\boldsymbol{\xi})$$

where $\boldsymbol{\xi}$ is a measurable random vector lying on the segment between $\mathbf{X}$ and $\mathbf{X}+\mathbf{Y}$ (Yang et al., 2021). This result underpins the delta method, maximum likelihood asymptotics, and profile likelihood theory, filling a technical gap in classical proofs by ensuring that the “remainder point” is a properly defined random vector.

7. Probabilistic Taylor Expansions in Programming Semantics

Within probabilistic λ-calculus, Ehrhard–Regnier Taylor expansion extends to a semantics for probabilistic resource terms. For a probabilistic program $M$ with probabilistic choice, the Taylor expansion $T(M)$ is a weighted sum over resource terms with explicit coefficients corresponding to the program’s probabilistic path structure:

$$T(M) = \sum_{n=0}^{\infty} \frac{1}{n!} \sum_{t \in \Delta_n} \alpha_{M,t} t$$

This expansion provides a compositional and adequate interpretation of programs, reflecting the induced probability distribution over head normal forms and corresponding to the probability weights in the probabilistic Böhm tree semantics (Lago et al., 2019).

---

References:

• (Servadio, 2 May 2025): Particle flow filtering by high-order Taylor expansions using differential algebra.
• (Wang, 2024): Statistical Taylor expansions and path-independent variance arithmetic.
• (Karvonen et al., 2021): Gaussian process Taylor expansion with Taylor kernels and remainder quantification.
• (Yang et al., 2021): Measurable intermediate points in random Taylor’s theorem in statistics.
• (Braverman et al., 2018): Taylored control problem (TCP) and Bellman operator expansion in Markov decision processes.
• (Kuznetsov, 2017): Taylor–Ito and Taylor–Stratonovich expansions for SDEs and mean-square Legendre approximation.
• (Lago et al., 2019): Taylor expansion semantics for probabilistic λ-calculus.