Extending Path-Dependent NJ-ODEs to Noisy Observations and a Dependent Observation Framework

Abstract: The Path-Dependent Neural Jump Ordinary Differential Equation (PD-NJ-ODE) is a model for predicting continuous-time stochastic processes with irregular and incomplete observations. In particular, the method learns optimal forecasts given irregularly sampled time series of incomplete past observations. So far the process itself and the coordinate-wise observation times were assumed to be independent and observations were assumed to be noiseless. In this work we discuss two extensions to lift these restrictions and provide theoretical guarantees as well as empirical examples for them. In particular, we can lift the assumption of independence by extending the theory to much more realistic settings of conditional independence without any need to change the algorithm. Moreover, we introduce a new loss function, which allows us to deal with noisy observations and explain why the previously used loss function did not lead to a consistent estimator.

• The paper extends PD-NJ-ODEs by adapting its loss function to handle noisy data while accurately estimating conditional expectations.
• It removes previous independence assumptions by jointly modeling the observation process with stochastic dynamics.
• The enhanced framework demonstrates robust performance in forecasting irregular, incomplete data, notably in medical monitoring applications.

Extending Path-Dependent NJ-ODEs to Noisy Observations and a Dependent Observation Framework

Introduction

The paper, "Extending Path-Dependent NJ-ODEs to Noisy Observations and a Dependent Observation Framework" (2307.13147), develops extensions to the Path-Dependent Neural Jump Ordinary Differential Equations (PD-NJ-ODEs). The extensions address scenarios with noisy observations and dependency between the stochastic process and the observation framework, lifting previous restrictions that assumed independence and noiseless observations. The model is particularly useful for predicting continuous-time stochastic processes sampled irregularly and incompletely, with applications in fields such as medical data analysis.

PD-NJ-ODEs with Noisy Observations

The introduction of noisy observations to PD-NJ-ODEs requires modifying the original loss function to exclude terms that inappropriately force the model to fit noise rather than the underlying process. Instead of forcing the model to jump to noisy observations, the noise-adapted loss function ensures the model approximates the conditional expectation, taking the independence and zero-mean nature of the noise into account.

The adjusted noise-adapted loss function is formulated as follows:

$$\Phi_{\text{noisy}}(\theta) := E\left[ \frac{1}{N} \sum_{j=1}^N \frac{1}{n^{(j)}} \sum_{i=1}^{n^{(j)}} \left\lvert M_i^{(j)} \odot \left( O_{t_i^{(j)}}^{(j)} - \hat{Y}_{t_i^{(j)}-} \right) \right\rvert_2^2 \right]$$

where $$O_{t_i^{(j)}}^{(j)} = X_{t_i^{(j)}} + \epsilon_i^{(j)}$$ represents the noisy observations.

Figure 1: Evaluation MSE of the PD-NJ-ODE trained on the Physionet dataset with the original and the noise-adapted loss with different sizes for the standard deviation factor of the synthetically added observation noise.

Addressing Dependency in Observation Framework

Crucially, this work removes previous independence assumptions between the process $\gX{}$ and elements of the observation framework such as observation times $t_i$ and observation masks $M$. The model now incorporates joint probability space definitions, allowing for observations to be dependent on past observations of the process.

A key assumption ensuring tractability and consistency of the model is the conditional independence of the process given all past observable information:

• For every $1 \leq i \leq n$, the process $\gX{}_{t_i-}$ is conditionally independent of the observation framework $(n, \gM{t})$ given the available information $\mathcal{A}_{t_i-}$.

Practical Implications and Evaluation

Applying the PD-NJ-ODEs in settings characterized by incomplete and noisy data requires careful consideration of computational efficiency and robustness against noise distortion. In practical scenarios such as medical monitoring with irregular sampling, the presented framework excels in forecasting and filtering applications.

Figure 2: A test sample of a Brownian motion with noisy observations.

Conclusion

This research extends the applicability of PD-NJ-ODEs to more realistic scenarios with noise and observation dependencies, broadening their potential impact and usability in practical applications. Future work should aim to explore the nuances of the conditional independence assumption and further validate the model's performance across a broader range of datasets and domains.