Higher-Order Taylor Expansions

• Higher-Order Taylor Expansions are systematic generalizations that approximate smooth functions with polynomials using derivatives of arbitrary order.
• They enable robust error estimation and convergence guarantees in numerical analysis, stochastic calculus, and optimization.
• Advanced methods include regularization, fractional expansions, and manifold adaptations to address nonconvexity and non-Euclidean domains.

Higher-order Taylor expansions are systematic generalizations of the classical Taylor series, providing polynomial approximations of smooth functions (or functionals) using derivatives of arbitrary order. In advanced applications spanning numerical analysis, stochastic calculus, convex optimization, geometric analysis, and functional analysis on measure spaces, higher-order Taylor expansions and their regularized or generalized forms play a central role in algorithm design, error control, and theoretical insight. The methodology involves quantifying local approximation error and leveraging higher derivatives for optimal convergence and robustness under various structural conditions.

1. Mathematical Formulation and Remainder Analysis

The $k$-th order Taylor expansion of a sufficiently smooth function $f : \mathbb{R}^d \to \mathbb{R}$ about point $x_0$ is

$$f(x) = f(x_0) + \sum_{j=1}^{k} \frac{1}{j!} D^j f(x_0)[x-x_0]^j + R_k(x),$$

where $D^j f(x_0)$ denotes the symmetric $j$-linear form of the $j$-th derivative, and $R_k(x)$ is the remainder term. If $D^k f$ is $L$-Lipschitz, then

$|R_k(x)| \leq \frac{L}{(k+1)!} \|x-x_0\|^{k+1}.$

Generalizations include multi-index notation for the expansion, as well as expansions involving functionals on measure spaces and non-Euclidean domains (Nabou, 3 Mar 2025, Salkeld, 2023, Ahmadi et al., 2023, Kamzolov et al., 2022, Nabou et al., 4 Mar 2025).

2. Regularization and Convexification of Higher-Order Models

Higher-order Taylor models can become nonconvex and non-coercive for $k>2$. To ensure global solvability and descent properties in optimization, one adds a $(k+1)$-st power regularization: $$T_k(x; x_0) = f(x_0) + \sum_{i=1}^k \frac{1}{i!} D^i f(x_0)[x-x_0]^i + \frac{H}{(k+1)!}\|x-x_0\|^{k+1},$$ with $H$ chosen (typically $H \geq (k+1) L_{k, f}$) so that $T_k$ is convex (Kamzolov et al., 2022, Ahmadi et al., 2023, Nabou, 3 Mar 2025, Nabou et al., 4 Mar 2025). Sum-of-squares convexification is used for arbitrary-order expansions: $$p(x) = T_{x_0, k}(x) + t\|x-x_0\|^{d'}, \quad d' = \text{next even integer},$$ where $t$ is chosen via semidefinite programming to guarantee SOS-convexity (Ahmadi et al., 2023).

3. Higher-Order Expansions in Advanced Domains

3.1. Lions–Taylor Expansions on Measure Spaces

For real-valued functionals $F$ on $\mathcal{P}_2(\mathbb{R}^d)$ (probability measures with finite second moment, with the Wasserstein distance), higher-order Lions derivatives $\partial_\mu^k F(\mu)(x_1,...,x_k)$ quantify $k$-th order sensitivity to measure perturbations. The expansion is

$$F(\nu) = F(\mu) + \sum_{\ell=1}^k \frac{1}{\ell!} \int_{(\mathbb{R}^d)^\ell} \partial_\mu^\ell F(\mu)(x_1,...,x_\ell) \, d(\nu-\mu)^{\otimes \ell}(x_1,...,x_\ell) + R_k(\mu, \nu),$$

and the remainder is controlled by $|R_k| \leq C W_2(\mu, \nu)^{k+1}$ under $C^k_b$ regularity (Salkeld, 2023).

3.2. Fractional-Order Expansions and Regularized Taylor Series

For Hölderian functions, ordinary derivatives may fail or become singular. Fractional Taylor expansions use the concept of fractional velocity: $$v_{\beta}^+ f(x) = \lim_{\epsilon\rightarrow0^+} \frac{f(x+\epsilon) - f(x)}{\epsilon^\beta}$$ yielding expansions

$$f(x+\epsilon) = f(x) + v_{\beta}^+ f(x) \epsilon^\beta + o(\epsilon^\beta).$$

Regularized derivatives remove the leading singularity and provide consistent generalizations at nondifferentiable points (Prodanov, 2015).

3.3. Expansions for Manifold-Valued Maps

For submanifolds $M \subset \mathbb{R}^n$, the Taylor expansion of the exponential map up to order three incorporates the second fundamental form and its covariant derivative, revealing geometric invariants such as lateral and frontal deviation directions: $$\exp_m(x) = m + x + \tfrac{1}{2}\alpha_m(x,x) + \tfrac{1}{6}[(\nabla_x \alpha_m)(x,x) - \alpha^\sharp_m(x)\cdot \alpha_m(x,x)] + O(|x|^4)$$ (Monera et al., 2012).

4. Numerical Schemes: Stochastic and Deterministic Approximations

4.1. Stochastic Differential Equations

Itô–Taylor expansions of order $k$ for the SDE $dX_t = a(X_t)\,dt + b(X_t)\,dW_t$ are crucial for constructing high-order strong and weak schemes: $$X_{t+\Delta t} = X_t + \sum_{|\alpha| \leq k} L^\alpha \text{Id}(X_t) I_\alpha[0, \Delta t] + R_{k+1}$$ with Milstein ($k=1$) and simplified weak Taylor ($k=2$) schemes achieving strong order 1 and weak order 2, respectively (Negyesi et al., 19 Jan 2025, Kuznetsov, 2017). Mean-square Fourier–Legendre expansions allow practical simulation of iterated stochastic integrals up to multiplicity 6 (Kuznetsov, 2017).

4.2. Stochastic PDEs

Stochastic Taylor expansions for SPDEs with additive noise utilize the mild solution framework and combinatorial tree-wood expansions, enabling arbitrarily high-order expansion of the solution in the Hilbert space: $$U_t = U_{t_0} + \sum_{\text{trees in wood}} \Phi(\text{tree})(t) + \text{remainder},$$ with the error controlled by the order of the expansion and robust to the noise process (Jentzen et al., 2010).

5. Convergence Rates, Remainder Estimation, and Global Guarantees

Higher-order models enable optimal convergence rates under suitable regularity. In optimization, regularized $p$-th order methods achieve $\mathcal{O}(k^{-(3p+1)/2})$ rates for convex functions and accelerated second-order rates ($\mathcal{O}(k^{-5})$) under additional smoothness (Kamzolov et al., 2022). The remainder estimates are explicit: $$|F(y) - T_p^F(y;x)| \leq \frac{L_{p,X}}{(p+1)!}\|y-x\|^{p+1},$$ and ensure step control and global descent in composite settings (Nabou, 3 Mar 2025, Nabou et al., 4 Mar 2025). Nonmonotone algorithms relax the need for global Lipschitz constants via local adjustment and reference sequence averaging (Nabou, 3 Mar 2025).

6. High-Order Expansions in Quantum Field Theory and Statistical Physics

In lattice QCD and effective field theories, high-order Taylor expansions of thermodynamic observables (e.g. pressure, density, susceptibilities) with respect to chemical potential provide access to phase structure and critical phenomena. Algorithmic differentiation propagates high-order derivatives through implicit equations, enabling computation of Taylor coefficients up to order 20+ (Karsch et al., 2010). The radius of convergence, estimated via ratio or Mercer-Roberts tests,

$r_{2n} = \big|c_{2n}/c_{2n+2}\big|^{1/2},$

diagnoses proximity to critical endpoints or crossover regions. Padé approximants accelerate convergence and facilitate analytic continuation beyond the Taylor radius (Bollweg et al., 2022, Karsch et al., 2010).

Expansion Domain	Remainder/Convergence Bound	Applications
Euclidean $\mathbb{R}^d$	$\leq \frac{L}{(k+1)!}\|x-x_0\|^{k+1}$	Optimization, ODE/SDE numerics
Wasserstein Space	$\leq C\,W_2(\mu, \nu)^{k+1}$	Mean-field games, measure perturbations
Hölderian Functions	$O(\epsilon^{n+\alpha})$	Analysis of non-differentiable regularity
Manifolds	$O(|x|^4)$ (3rd-order)	Submanifold geometry, curvature invariants
SPDEs	$O((t-t_0)^r)$ (tree order $r$)	Strong/weak global numerics
QFT/QCD	$r_{2n} \to \text{CEP radius}$	Phase structure, critical point estimation

7. Practical Implementation and Guidelines

• Polynomial-time minimization of higher-order regularized models is feasible, with subproblem complexity determined by polynomial degree and ambient dimension (Kamzolov et al., 2022, Ahmadi et al., 2023).
• For stochastic and SPDE applications, Fourier and combinatorial expansions provide systematic simulation and error control (Kuznetsov, 2017, Jentzen et al., 2010).
• Algorithmic differentiation is essential for high-order coefficient computation in QFT (Karsch et al., 2010).
• In numerical optimization and composite settings, nonmonotone acceptance and local regularization yield global guarantees without restrictive smoothness assumptions (Nabou, 3 Mar 2025, Nabou et al., 4 Mar 2025).
• Padé resummation is critical for extending physical predictions beyond the strict Taylor radius in statistical physics (Bollweg et al., 2022).

Higher-order Taylor expansions thus provide a rigorous and versatile toolset for precision approximation, algorithmic design, and theoretical analysis across mathematics, statistics, and applied science.