Infinitesimal Generator of Controlled Diffusions

• Infinitesimal Generator of Controlled Diffusions is a key operator that formalizes the impact of control inputs on the evolution of stochastic systems in SDEs.
• It encodes both drift and diffusion components, enabling spectral analysis, ergodic assessment, and optimal control evaluations using resolvent methods.
• Data-driven approaches leverage RKHS and kernel regression, providing nonparametric estimates with convergence guarantees even under biased sampling.

The infinitesimal generator of controlled diffusion processes is a central operator in stochastic analysis and mathematical control theory, formalizing the action of a stochastic process—including external controls—on functionals of the state. The generator encodes both the drift and diffusion terms in stochastic differential equations (SDEs), providing the analytic backbone for spectral estimation, ergodic properties, data-driven modeling, and the solution of associated control problems. Its definition, estimation, and use in nonparametric, biased, or path-dependent regimes represent current research directions at the intersection of probability, dynamical systems, control, and machine learning.

1. Formal Definition for Controlled Diffusion

Consider a controlled diffusion process in $\mathbb{R}^{n_x}$,

$$\mathrm{d}x_t = b(x_t, u_t)\,\mathrm{d}t + \sigma(x_t)\,\mathrm{d}W_t,$$

where $b(x, u) = f(x) + G(x)u$ is the control-affine drift, $\sigma(x)$ is the diffusion matrix (often taken as isotropic, $\sigma(x)=\sqrt{2\epsilon} I_{n_x}$), $u_t$ is a control input, and $W_t$ is a standard Wiener process. For any twice differentiable function $\phi \in C^2(\mathbb{R}^{n_x})$, the controlled infinitesimal generator $\mathcal{L}^u$ acts as

$$\mathcal{L}^u \phi(x) = b(x, u) \cdot \nabla\phi(x) + \frac{1}{2}\mathrm{trace}[\sigma(x)\sigma(x)^{T}\nabla^2\phi(x)].$$

For isotropic diffusions, this reduces to

$$\mathcal{L}^u \phi(x) = [f(x) + G(x)u]\cdot \nabla \phi(x) + \epsilon \Delta \phi(x),$$

with $\Delta$ denoting the Laplacian. This operator extends the classical Kolmogorov backward generator to the controlled setting and underpins the analysis and computation of value functions, transition rate spectra, and more (Bevanda et al., 2 Dec 2024).

2. Infinitesimal Generators in Biased (Controlled) Diffusion

In applications such as rare event simulation or molecular dynamics, the system is often simulated under a biased or controlled potential to facilitate exploration. Consider overdamped Langevin dynamics,

$$\mathrm{d}X_t = -\nabla U(X_t)\,\mathrm{d}t + \sqrt{2\beta^{-1}}\,\mathrm{d}W_t,$$

where $\beta$ is the inverse temperature and $U$ is the potential. The generator is

$$\mathcal{L} f(x) = -\nabla U(x) \cdot \nabla f(x) + \beta^{-1} \Delta f(x).$$

If a bias potential $V(x)$ is added, the new generator becomes

$$\mathcal{L}_\text{bias} f(x) = -\nabla (U+V)(x) \cdot \nabla f(x) + \beta^{-1} \Delta f(x) = \mathcal{L} f(x) + [-\nabla V(x)\cdot \nabla f(x)],$$

where the perturbation $Bf = -\nabla V(x)\cdot \nabla f(x)$ is a first-order differential operator representing the influence of control or bias (Devergne et al., 13 Jun 2024).

3. Operator-Theoretic and Kernel-Based Estimation

The generator is an unbounded operator and estimating it directly from data is non-trivial. A tractable approach is to estimate its resolvent $(\alpha I - \mathcal{L})^{-1}$ for some $\alpha>0$, since the resolvent is bounded and compact under mild assumptions. Specifically,

$$(R_\alpha z)(x) = \int_0^\infty e^{-\alpha t} \mathbb{E}[z(X_t) | X_0 = x]\,dt,$$

integrates the process semigroup against $e^{-\alpha t}$. The resolvent's Hermitian structure enables spectral analysis: if $R_\alpha \psi = \nu \psi$, then $\mathcal{L}\psi = (\alpha - 1/\nu)\psi$.

For data-driven estimation, kernel methods embed the generator in reproducing kernel Hilbert spaces (RKHS) with smooth kernels (e.g., Gaussian). The generator is approximated via operator-valued kernel regression: $$\hat{A} = (C_\gamma)^{-1} T_\pi, \qquad C_\gamma = C + \gamma I,$$ where $C$ and $T_\pi$ are Gramian and cross-covariance operators, both estimated empirically from observed trajectories, and $\gamma$ is a regularization parameter (Bevanda et al., 2 Dec 2024).

In the biased sampling context, empirical averages are computed with explicit Radon–Nikodym weights to account for the sampling bias, and transition rates and eigenfunctions for the unbiased dynamics are recovered via a finite-dimensional eigenproblem involving reweighted covariance matrices.

4. Weak Generators Beyond the Markovian Setting

Functional Itô calculus provides a weak, pathwise notion of the infinitesimal generator on spaces of Wiener functionals. Given a process $X$ and a filtration-discrete skeleton (a sequence of pure jump approximators $X^k$ indexed by mesh size), one analyzes the limiting behavior of discrete “generators” $U^{k,j}X$ via dual projections on the skeleton. Under finite-energy and stability assumptions, these discrete generators converge to a weak limit $U X \in L^1(P\times dt)$,

$$X(t) = X(0) + \sum_{j=1}^d \int_0^t D^j X(s) dB^j(s) + \int_0^t U X(s) ds,$$

where $D^j X$ is the weak derivative. In the classical Markovian (Itô) case, $U X$ coincides with the second-order elliptic Kolmogorov generator; in non-Markovian or singular settings, $U X$ captures non-Markovian “orthogonal” drift (Leão et al., 2017).

5. Spectral and Control Applications

The eigenfunctions and eigenvalues of the infinitesimal generator (or its resolvent) encode slow transition modes and relaxation timescales vital for metastability analysis in physical and chemical systems. Concretely, for the unbiased generator $\mathcal{L}$: $$\mathcal{L} \psi_i = \lambda_i \psi_i, \quad \text{ with } 0 = \lambda_1 > \lambda_2 > \ldots$$ Spectral estimation proceeds by projecting the resolvent onto a chosen set of basis functions and solving a reweighted, regularized eigenproblem. In the control context, the learned infinitesimal generator is incorporated into data-driven Hamilton–Jacobi–Bellman (HJB) recursion, yielding policies and value functions via finite-dimensional backward ODEs projected onto the RKHS sample subspace: $$-\dot{v}_S = \widehat A^T v_S + q_S + \mathcal{D}_r^*( \widehat B^T v_S ), \quad v_S(T) = 0,$$ where the Fenchel conjugate term $\mathcal{D}_r^*$ encodes control constraints and costs (Bevanda et al., 2 Dec 2024). Empirical results demonstrate superior performance to discretized operator methods and nonlinear programming-based controllers across benchmark control tasks.

6. Estimation from Biased Data and Convergence Guarantees

When data are collected under a control or bias, recovery of properties for the unbiased dynamics requires careful reweighting. For samples drawn from the biased invariant measure $\pi' \propto e^{-\beta(U+V)}\,dx$, the Radon–Nikodym weight is

$$w(x) = \frac{d\pi}{d\pi'}(x) = \frac{e^{\beta V(x)}}{\int e^{\beta V(u)} \pi'(du)}.$$

Estimators for unbiased spectral quantities are then constructed as weighted empirical averages: $$C_{ij} = \mathbb{E}_{x' \sim \pi'} [ w(x') z_i(x') z_j(x') ], \quad W_{ij} = \mathbb{E}_{x' \sim \pi'} [ w(x') \langle \nabla z_i(x'), \nabla z_j(x')\rangle / \beta ],$$ yielding a regularized resolvent estimator $G = (W + \gamma I)^{-1} C$. Theorem 4.1 of (Devergne et al., 13 Jun 2024) establishes consistency of these eigenpair estimators: as sample size $n\to\infty$, regularization $\gamma\to 0$, and basis size $m$ increases such that the span of the basis becomes dense in the appropriate Sobolev space, the learned spectral quantities converge to those of the true (uncontrolled) generator with high probability.

7. Outlook and Limitations

The analytic and data-driven theory of infinitesimal generators for controlled diffusions is well developed for Markovian and elliptic operators, but less so for high-dimensional, singular, or path-dependent regimes. Kernel-based methods provide nonparametric consistency and finite-sample convergence, but suffer from scalability limitations due to Gram matrix inversion; the cube scaling in sample size precludes naive extension to very large datasets (Bevanda et al., 2 Dec 2024). In the weak generator context, filtration-discrete frameworks extend stochastic calculus beyond semimartingale settings, suggesting applicability to rough paths and non-Markovian noise (Leão et al., 2017). Current and future research focuses on efficient high-dimensional approximations, adaptive basis selection, and robust estimation under highly biased or incomplete sampling.

---

Paper	Generator Context	Key Application Area
(Devergne et al., 13 Jun 2024)	Biased Langevin diffusion, resolvent	Spectral estimation from biased data
(Leão et al., 2017)	Path-dependent/weak Itô generator	Functional calculus, non-Markovian SDE
(Bevanda et al., 2 Dec 2024)	RKHS operator of controlled diffusion	Data-driven optimal control

Each approach highlights distinct analytic, statistical, and computational aspects of the infinitesimal generator concept, with ongoing developments accessing broader classes of stochastic processes, control regimes, and high-dimensional function spaces.