Learning from Irregularly-Sampled Time Series: A Missing Data Perspective

Abstract: Irregularly-sampled time series occur in many domains including healthcare. They can be challenging to model because they do not naturally yield a fixed-dimensional representation as required by many standard machine learning models. In this paper, we consider irregular sampling from the perspective of missing data. We model observed irregularly-sampled time series data as a sequence of index-value pairs sampled from a continuous but unobserved function. We introduce an encoder-decoder framework for learning from such generic indexed sequences. We propose learning methods for this framework based on variational autoencoders and generative adversarial networks. For continuous irregularly-sampled time series, we introduce continuous convolutional layers that can efficiently interface with existing neural network architectures. Experiments show that our models are able to achieve competitive or better classification results on irregularly-sampled multivariate time series compared to recent RNN models while offering significantly faster training times.

• The paper proposes a novel encoder-decoder framework that transforms irregularly-sampled time series into a structured missing data problem using VAEs and GANs.
• It introduces continuous convolutional layers to efficiently capture local temporal patterns and maintain permutation invariance with fewer parameters.
• Experiments on healthcare and image datasets demonstrate faster training and higher accuracy compared to traditional RNN-based methods.

Learning from Irregularly-Sampled Time Series: A Missing Data Perspective

Introduction

Irregularly-sampled time series present substantial challenges for traditional machine learning models, which often demand fixed-dimensional data representations. This study approaches modeling such data through the lens of missing data, proposing a general encoder-decoder framework. The central contribution involves translating irregularly-sampled time series into a generic missing data problem, leveraging methods rooted in variational autoencoders (VAEs) and generative adversarial networks (GANs). Furthermore, the introduction of continuous convolutional layers facilitates the integration with existing neural network architectures, enabling competitive performance and faster training, particularly evident in classification tasks involving these complex domains.

Encoder-Decoder Framework

The proposed framework transforms the irregularly-sampled time series into a missing data problem through a structured generative process. The framework models complete data as an indexed sequence sampled from a latent continuous function, encapsulated using an encoder-decoder architecture. The encoder is tasked with learning a permutation-invariant representation, critical to handling data of arbitrary sizes, while the decoder reconstructs complete data samples from latent representations.

For finite index settings, the model uses VAEs, optimizing a conditional evidence lower bound (ELBO) that assumes independence between data indices and latent variables. This independence simplifies training by focusing on maximizing the conditional log-likelihood of observed data. Alternatively, the P-BiGAN model capitalizes on adversarial learning principles, matching joint distributions between real and generated samples to improve data inference capability without explicit density specification, beneficial for the high-uncertainty settings prevalent in irregularly-sampled domains.

Continuous Index Set Models

Addressing continuous index sets typical in time series, the study develops computationally efficient neural architectures. The decoder applies a kernel smoother for temporal function interpolation, facilitating the generation of smooth, time-representative outputs from encoded latent variables. The encoder adapts standard convolutional layers to irregular samples through continuous convolutional operations, computing correlations between indexed observations and continuous filter functions.

By harnessing piecewise-linear functions for filter parameterization, the continuous convolutional layers capture local patterns with fewer parameters than alternative architectures while maintaining permutation invariance. These enhancements cater specifically to the deployment in multivariate irregulary-sampled datasets, such as those in healthcare, where the framework demonstrates substantial improvements over existing RNN-based methods, as evidenced by real-world medical dataset applications.

Applications and Experiments

Extending to applications like imputation and supervised learning, the proposed models exhibit strong performance on standard benchmarks. Image datasets with controlled missingness demonstrate the models' proficiency in generating and completing samples, with P-BiGAN showing superior capability in environments characterized by high data variability due to its implicit modeling nature. Meanwhile, in supervised learning scenarios, classification tasks on datasets like MIMIC-III validate the framework's efficiency and accuracy, with CUDA-driven convolutional operations significantly reducing training epochs compared to ODE-based approaches.

Figure 1: At the top we plot the 2D latent codes drawn from the encoder $q_\phi(\mathbf{z}|\mathbf{x},\mathbf{t})$ with three different incomplete MNIST examples. The resulting incomplete images are shown as the second image in each row.

Conclusion

This study provides a comprehensive framework for managing irregularly-sampled time series by framing them as missing data problems, enabling model usage across domains characterized by sampling irregularity. The encoder-decoder architecture, supported by continuous convolutional enhancements, achieves comparable or superior predictive power over recent advancements, with significant training efficiencies. These findings elucidate potential pathways to more effective modeling strategies for complex temporal datasets, promising broader applicability and future refinement in machine learning methodologies for handling incomplete multi-dimensional data streams.