Time-dependent density estimation using binary classifiers (2506.15505v1)

Abstract: We propose a data-driven method to learn the time-dependent probability density of a multivariate stochastic process from sample paths, assuming that the initial probability density is known and can be evaluated. Our method uses a novel time-dependent binary classifier trained using a contrastive estimation-based objective that trains the classifier to discriminate between realizations of the stochastic process at two nearby time instants. Significantly, the proposed method explicitly models the time-dependent probability distribution, which means that it is possible to obtain the value of the probability density within the time horizon of interest. Additionally, the input before the final activation in the time-dependent classifier is a second-order approximation to the partial derivative, with respect to time, of the logarithm of the density. We apply the proposed approach to approximate the time-dependent probability density functions for systems driven by stochastic excitations. We also use the proposed approach to synthesize new samples of a random vector from a given set of its realizations. In such applications, we generate sample paths necessary for training using stochastic interpolants. Subsequently, new samples are generated using gradient-based Markov chain Monte Carlo methods because automatic differentiation can efficiently provide the necessary gradient. Further, we demonstrate the utility of an explicit approximation to the time-dependent probability density function through applications in unsupervised outlier detection. Through several numerical experiments, we show that the proposed method accurately reconstructs complex time-dependent, multi-modal, and near-degenerate densities, scales effectively to moderately high-dimensional problems, and reliably detects rare events among real-world data.

Time-Dependent Density Estimation Using Binary Classifiers

The paper "Time-dependent density estimation using binary classifiers" introduces a novel approach to dynamically estimate the probability density of a multivariate stochastic process. This methodology assumes the initial probability density function is known, enabling the authors to leverage a binary classifier to approximate the time-dependent evolution of densities in systems driven by random processes. The primary innovation lies in using classifiers to learn dynamic density changes, which can be particularly useful for unsupervised outlier detection and generative modeling.

Methodology Overview

The researchers propose a data-driven technique wherein a binary classifier approximates the partial time derivative of the logarithm of the probability density. The classifier is trained using a contrastive estimation-based objective. This formulation sets it apart from traditional methods as the classifier is designed to distinguish between realizations of the process at two closely spaced time points. This approach offers an explicit modeling of the time-dependent probability distribution, facilitating the exact computation of density values within a specified time horizon.

In practice, sample paths are generated using stochastic interpolants, ensuring the classifier efficiently captures the density dynamics. Automatic differentiation enhances the utility of these learned densities, facilitating effective sample generation via gradient-based Markov chain Monte Carlo methods.

Numerical Results

The paper demonstrates the approach through several numerical experiments, emphasizing its flexibility and efficiency in reconstructing complex time-dependent densities:

1. Stochastic Dynamic Systems: The authors apply their method to systems driven by stochastic excitations, showcasing its ability to accurately capture evolving, multi-modal, and near-degenerate densities.
2. Generative Modeling: By synthesizing random vectors from given samples, the method exhibits strong generative capabilities. The classifier-based density estimation is notably robust even in moderately high-dimensional spaces.
3. Outlier Detection: The approach is validated through unsupervised outlier detection tasks. It reliably identifies rare events, a critical application in anomaly detection contexts.

Implications and Future Work

The proposed methodology has several practical and theoretical implications. Practically, it offers a scalable and computationally efficient alternative for modeling complex stochastic processes with high-dimensionality, potentially influencing applications in physics-based simulations, generative modeling, and uncertainty quantification. Theoretically, it underlines the utility of time-dependent classifiers in dynamic density estimation, presenting opportunities for further research into classifier-based density estimation approaches.

Future research could explore extending this technique to more sophisticated interpolants or hybrid models that combine data-driven insights with physical principles. This exploration could enhance its applicability across different domains, including advanced machine learning tasks and real-time dynamic systems analysis.