Continuous Time Latent Processes

• Continuous time latent processes are mathematical models that capture unobserved state trajectories evolving continuously using SDEs, ODEs, and Gaussian process priors.
• Inference techniques such as particle filtering, variational inference, and EM enable robust estimation of latent paths even from irregular or sparse observations.
• These models are applied in time-series analysis, dynamic networks, and survival analysis, offering enhanced interpretability and uncertainty quantification in complex systems.

A continuous time latent process is a stochastic or deterministic process evolving over a continuous time domain (typically $t \in [0,T]$), whose state is unobserved (latent) and whose structure underlies the observed data or drives the behavior of a larger system. Such processes are central in modern probabilistic modeling, especially in contexts where continuous-time trajectories, irregular observation times, and non-Markovian or high-dimensional dynamics must be accommodated. Models built around continuous time latent processes include stochastic differential equations (SDEs) with latent state, neural ODEs/SDEs with variational or particle-based inference, continuous-time hidden Markov models, and nonparametric latent functional representations such as Gaussian or piecewise-linear latent curves. The combination of continuous-time latent state and structural or statistical modeling enables flexible, data-driven inference and prediction across domains such as dynamical systems, time-series analysis, dynamic networks, survival analysis, and event-history modeling.

1. Mathematical Formulation of Continuous-Time Latent Processes

A continuous time latent process is most generally specified by:

• A latent state trajectory $\{x_t\}_{t\in[0,T]}$ or $\{z(t)\}_{t\in[0,T]}$, with state space in $\mathbb{R}^d$, $\mathbb{R}^{n\times d}$, or a discrete set for Markov models.
• Evolution according to a stochastic differential equation (SDE), deterministic ODE, Gaussian process, or piecewise model:
  • SDE form:

  $$dx_t = \mu_{\theta}(x_t, t)\,dt + \sigma_{\theta}(x_t, t)\,dW_t,$$

  where $W_t$ is a Wiener process, $\mu$ the drift, and $\sigma$ the diffusion (Deng et al., 2022, Deng et al., 2021, Duncker et al., 2019). - ODE or GP-ODE form:

  $$\frac{d}{dt} h^a(t) = f_{\mathrm{ind}}(h^a(t), c^a) + \sum_{a'\in \mathcal{N}(a)} m(h^a(t), c^a, h^{a'}(t), c^{a'}),$$

  with vector-field terms parameterized by GPs (Yıldız et al., 2022). - Nonparametric Gaussian process prior:

  $\theta_i(t) \sim \mathrm{GP}(m(t), K(t,t')),$

  often for individual- or object-level latent trajectories (Chen et al., 2019).

• Observational process, e.g., at discrete (possibly irregular) times $\{t_i\}$, with likelihood terms conditioned on the current or trajectory of the latent process:

  $$y_{t_i} \sim p(y_{t_i}|x_{t_i}),\quad y_n \sim p(y_n|h(t_n)),\quad Y_{ij}(t_{is}) \sim g_j(y|\theta_i(t_{is})),$$

  and for network or event times, via marked point processes or hazard models (Celikkanat et al., 2023, Rastelli et al., 2021).

The formulation directly accommodates missingness (latent initial conditions, missing dynamic/static features), non-regular sampling, and high-dimensional state and observation spaces.

2. Posterior Inference and Filtering Techniques

Inference over continuous time latent processes is complicated by intractable integrals over paths and the necessity to condition on possibly sparse and irregular observations. Key methodologies include:

• Continuous-Time Particle Filtering: A Monte Carlo approximation to the posterior over sample paths, using importance weights derived via Girsanov's theorem:

  $$w_t(\omega) = \exp\left(-\frac{1}{2}\int_0^t\|u(x_s, s)\|^2 ds - \int_0^t u(x_s, s)^\top dW_s\right),$$

  where the proposal SDE is parameterized with its own drift (Deng et al., 2022). Between observation times, continuous SDE simulation is maintained, and resampling occurs at observed data points.

• De-Biased Particle Filters with Poisson Estimators: For hidden Markov models with intractable integral weights, unbiased Poisson series estimators replace Naive discretization, minimizing bias super-exponentially in the grid size (Jin et al., 2022).

• Variational Inference Schemes: Variational posteriors over the latent SDE (e.g., parameterized ODE-RNN, ODE-LSTM or full GP) are optimized using reparameterizations, with or without inducing-point GPs and pathwise smoothing for efficiency (Deng et al., 2021, Yıldız et al., 2022, Duncker et al., 2019).

• Maximum Marginal Likelihood / EM: For models where latent curves are GPs, backbone EM or stochastic EM methods sample or optimize latent path variables, treating hyperparameters as parameters to be optimized by maximizing marginal likelihood (Chen et al., 2019, Chen, 2019).

• Kinematic or Event-History State-Space Methods: Extended Kalman filtering and expectation-maximization for models with latent event histories driving the observed process, enabling stable parameter identification even at fine time resolution (Framba et al., 2023).

These approaches accommodate both continuous-time latent state and discrete/irregular observation regimes and enable estimation of both path-wise posteriors and marginal likelihoods.

3. Model Classes and Application Domains

Continuous time latent processes are foundational to diverse statistical and machine learning settings:

Model Class	Latent Evolution	Observation/Emission
Latent SDEs	Nonlinear SDE for $x_t$	Noisy time series, counts, events
Latent ODEs/ODE-LSTM	Deterministic ODE for $z_t$	Arbitrary irregular time series
Latent GPs	GP prior for $\theta(t)$	Multivariate longitudinal data
Multistate Markov	CTMC on finite state space	Discrete emissions, possibly misclassification
Dynamic Networks	Node latent curves $r_i(t)$	Time-varying network edges, event sequences
Point Process Models	Piecewise trajectories, rate/hazard linked to latent state	Event times, marked events
Interacting Systems	Coupled ODE/SDE/CTMCs	Partial/noisy observations of system state

Applications include forecasting and density estimation for physical and biological systems (Deng et al., 2022, Deng et al., 2021), interaction and social/contact dynamics (Rastelli et al., 2021, Celikkanat et al., 2023), functional network analysis (MacDonald et al., 2022), survival modeling from medical data (Zeng et al., 1 Aug 2025), reaction system identification (Framba et al., 2023), and educational process data analysis (Chen, 2019).

4. Theoretical Properties and Inference Guarantees

Rigorous analysis has established key theoretical results for continuous time latent process models and their inference:

• Consistency: Particle filters in continuous time converge to the true posterior as particle number $N \to \infty$, with $O(1/N)$ bias and variance decay for log-likelihood estimates under appropriate conditions (Deng et al., 2022).

• Unbiasedness: Poisson-series-based de-biased estimators achieve nearly unbiased likelihood and filtering distribution estimation, with truncation bias decaying super-exponentially in grid step (Jin et al., 2022).

• Variance Reduction: Resampling dead particles at each observation time significantly reduces variance versus importance sampling only (“IWAE”) (Deng et al., 2022).

• Identifiability and Empirical Bayes: GP-based latent curve and continuous-time IRT models are identified only up to affine transform; constraints or anchor parameters are needed for uniqueness (Chen et al., 2019, Chen, 2019).

• Consistency in Functional Network Recovery: Under basis-smoothness and design regularity, theoretical error bounds in recovering the true latent process are available, with scaling governed by signal-to-noise and basis approximation error (MacDonald et al., 2022).

These properties inform model design and practical parameterization choices, especially concerning trade-offs in computation and statistical fidelity for large $d$, dense observations, or long timespans.

5. Extensions and Limitations

Continuous time latent process models present both rich opportunities for methodological expansion and distinct computational/statistical challenges:

• Extensions: Adaptive resampling policies, control variates for variance reduction, Rao–Blackwellization, differentiable particle filters, and extensions to jump-diffusions or Lévy-driven systems are well-motivated in SDE-based frameworks (Deng et al., 2022).

• Limitations:

  • High-dimensional latent state requires large N for PF to avoid degeneracy (Deng et al., 2022).
  • Resampling and importance weighting introduce non-trivial dependencies, complicating GPU acceleration and gradient-based optimization.
  • Differentiation through resampling is nontrivial, requiring surrogates such as Gumbel-softmax or REINFORCE (Deng et al., 2022).
  • Time-inhomogeneity in generator matrices for CTMCs incurs bias if forced onto a piecewise-homogeneous grid (Kendall et al., 2022).
  • Numerical instability at small $\Delta t$ in classical local-linear approximations for event-history settings, which can be addressed by latent count augmentation (Framba et al., 2023).
  • Model selection for mean/covariance structure in latent GPs remains crucial for robust inference (Chen et al., 2019).
  • Extensions to semi-observed, graph-coupled, or long-range interacting systems complicate both model specification and inference (learning twist potentials, scalable SMC) (Migliorini et al., 14 Oct 2025).

A plausible implication is that successful application of continuous time latent process models requires careful balancing of model class, inference framework, computational resources, and domain-specific knowledge of observation structure and data quality.

6. Empirical Benchmarks and Interpretability

Experimental studies demonstrate that continuous time latent process models robustly outperform discrete-time or naive variational baselines on a range of benchmarks:

• Negative log-likelihood and Forecast Error: Continuous-time latent particle filtering yields lower NLL and L2 forecast error than standard latent SDE or ODE-based variational inference, especially as observation intensity increases (Deng et al., 2022, Deng et al., 2021).
• Interpretability: Fixed-point–Jacobian conditioning on sparse Gaussian process drift fields provides readily interpretable dynamical portraits (identifying attractors, limit cycles) (Duncker et al., 2019).
• Latent Trajectory Analysis: In event-history and network settings, clustering and vector-field decomposition of latent trajectories yields insight into group-level difference (e.g., risk strata in survival, clusters in dynamic networks) (Zeng et al., 1 Aug 2025, Celikkanat et al., 2023).
• Uncertainty Quantification: Variational approximations with explicit Monte Carlo uncertainty propagation provide calibrated forecast intervals, strongly improving over deterministic or over-smoothed approaches (Yıldız et al., 2022, MacDonald et al., 2022).

Across task domains, continuous time latent process frameworks provide a statistically principled, computationally efficient, and theoretically grounded approach to learning, inference, and interpretation for dynamic systems with unobserved or partially observed state evolution.