Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Tripletformer for Probabilistic Interpolation of Irregularly sampled Time Series (2210.02091v2)

Published 5 Oct 2022 in cs.LG

Abstract: Irregularly sampled time series data with missing values is observed in many fields like healthcare, astronomy, and climate science. Interpolation of these types of time series is crucial for tasks such as root cause analysis and medical diagnosis, as well as for smoothing out irregular or noisy data. To address this challenge, we present a novel encoder-decoder architecture called "Tripletformer" for probabilistic interpolation of irregularly sampled time series with missing values. This attention-based model operates on sets of observations, where each element is composed of a triple of time, channel, and value. The encoder and decoder of the Tripletformer are designed with attention layers and fully connected layers, enabling the model to effectively process the presented set elements. We evaluate the Tripletformer against a range of baselines on multiple real-world and synthetic datasets and show that it produces more accurate and certain interpolations. Results indicate an improvement in negative loglikelihood error by up to 32% on real-world datasets and 85% on synthetic datasets when using the Tripletformer compared to the next best model.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. P. Yadav, M. Steinbach, V. Kumar, and G. Simon, “Mining electronic health records (ehrs) a survey,” ACM Computing Surveys (CSUR), vol. 50, no. 6, pp. 1–40, 2018.
  2. M. Lepot, J.-B. Aubin, and F. H. Clemens, “Interpolation in time series: An introductive overview of existing methods, their performance criteria and uncertainty assessment,” Water, vol. 9, no. 10, p. 796, 2017.
  3. S. N. Shukla and B. Marlin, “Multi-time attention networks for irregularly sampled time series,” in International Conference on Learning Representations, 2021.
  4. S. C.-X. Li and B. Marlin, “A scalable end-to-end gaussian process adapter for irregularly sampled time series classification,” Advances in Neural Information Processing Systems 29, 2016.
  5. Y. Rubanova, R. T. Chen, and D. Duvenaud, “Latent odes for irregularly-sampled time series,” Advances in Neural Information Processing Systems 32, 2019.
  6. P. Kidger, J. Morrill, J. Foster, and T. Lyons, “Neural controlled differential equations for irregular time series,” arXiv preprint arXiv:2005.08926, 2020.
  7. M. Horn, M. Moor, C. Bock, B. Rieck, and K. Borgwardt, “Set functions for time series,” in International Conference on Machine Learning, pp. 4353–4363, PMLR, 2020.
  8. S. N. Shukla and B. M. Marlin, “Interpolation-prediction networks for irregularly sampled time series,” International Conference on Learning Representations, 2019.
  9. V. K. Yalavarthi, J. Burchert, and L. Schmidt-Thieme, “DCSF: deep convolutional set functions for classification of asynchronous time series,” in 9th IEEE International Conference on Data Science and Advanced Analytics, DSAA 2022, Shenzhen, China, October 13-16, 2022, pp. 1–10, IEEE, 2022.
  10. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud, “Neural ordinary differential equations,” arXiv preprint arXiv:1806.07366, 2018.
  11. E. V. Bonilla, K. Chai, and C. Williams, “Multi-task gaussian process prediction,” Advances in neural information processing systems, vol. 20, 2007.
  12. C. K. Williams and C. E. Rasmussen, Gaussian processes for machine learning. MIT press Cambridge, MA, 2006.
  13. R. Dürichen, M. A. Pimentel, L. Clifton, A. Schweikard, and D. A. Clifton, “Multitask gaussian processes for multivariate physiological time-series analysis,” IEEE Transactions on Biomedical Engineering, vol. 62, no. 1, pp. 314–322, 2014.
  14. Y. Tashiro, J. Song, Y. Song, and S. Ermon, “Csdi: Conditional score-based diffusion models for probabilistic time series imputation,” Advances in Neural Information Processing Systems, vol. 34, pp. 24804–24816, 2021.
  15. S. N. Shukla and B. M. Marlin, “Heteroscedastic temporal variational autoencoder for irregularly sampled time series,” International Conference on Learning Representation, 2022.
  16. J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh, “Set transformer: A framework for attention-based permutation-invariant neural networks,” in International conference on machine learning, pp. 3744–3753, PMLR, 2019.
  17. J. Yoon, W. R. Zame, and M. Van Der Schaar, “Deep sensing: Active sensing using multi-directional recurrent neural networks,” in International Conference on Learning Representations, 2018.
  18. W. Cao, D. Wang, J. Li, H. Zhou, L. Li, and Y. Li, “Brits: Bidirectional recurrent imputation for time series,” Advances in neural information processing systems, vol. 31, 2018.
  19. Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu, “Recurrent neural networks for multivariate time series with missing values,” Scientific reports, vol. 8, no. 1, p. 6085, 2018.
  20. X. Li, T.-K. L. Wong, R. T. Chen, and D. Duvenaud, “Scalable gradients for stochastic differential equations,” in International Conference on Artificial Intelligence and Statistics, pp. 3870–3882, PMLR, 2020.
  21. A. Norcliffe, C. Bodnar, B. Day, J. Moss, and P. Liò, “Neural ode processes,” 9th International Conference on Learning Representations, ICLR, 2021.
  22. M. Garnelo, J. Schwarz, D. Rosenbaum, F. Viola, D. J. Rezende, S. Eslami, and Y. W. Teh, “Neural processes,” arXiv preprint arXiv:1807.01622, 2018.
  23. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  24. H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 11106–11115, 2021.
  25. I. Silva, G. Moody, D. J. Scott, L. A. Celi, and R. G. Mark, “Predicting in-hospital mortality of icu patients: The physionet/computing in cardiology challenge 2012,” in 2012 Computing in Cardiology, pp. 245–248, IEEE, 2012.
  26. A. L. Goldberger, L. A. Amaral, L. Glass, J. M. Hausdorff, P. C. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley, “Physiobank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals,” circulation, vol. 101, no. 23, pp. e215–e220, 2000.
  27. A. E. Johnson, T. J. Pollard, L. Shen, H. L. Li-Wei, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark, “Mimic-iii, a freely accessible critical care database,” Scientific data, vol. 3, no. 1, pp. 1–9, 2016.
  28. H. Harutyunyan, H. Khachatrian, D. C. Kale, G. Ver Steeg, and A. Galstyan, “Multitask learning and benchmarking with clinical time series data,” Scientific data, vol. 6, no. 1, pp. 1–18, 2019.
  29. M. A. Reyna, C. Josef, S. Seyedi, R. Jeter, S. P. Shashikumar, M. B. Westover, A. Sharma, S. Nemati, and G. D. Clifford, “Early prediction of sepsis from clinical data: the physionet/computing in cardiology challenge 2019,” in 2019 Computing in Cardiology (CinC), pp. Page–1, IEEE, 2019.
  30. A. P. Ruiz, M. Flynn, J. Large, M. Middlehurst, and A. Bagnall, “The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances,” Data Mining and Knowledge Discovery, vol. 35, no. 2, pp. 401–449, 2021.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now