Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Continuous-time Autoencoders for Regular and Irregular Time Series Imputation (2312.16581v3)

Published 27 Dec 2023 in cs.LG and cs.IR

Abstract: Time series imputation is one of the most fundamental tasks for time series. Real-world time series datasets are frequently incomplete (or irregular with missing observations), in which case imputation is strongly required. Many different time series imputation methods have been proposed. Recent self-attention-based methods show the state-of-the-art imputation performance. However, it has been overlooked for a long time to design an imputation method based on continuous-time recurrent neural networks (RNNs), i.e., neural controlled differential equations (NCDEs). To this end, we redesign time series (variational) autoencoders based on NCDEs. Our method, called continuous-time autoencoder (CTA), encodes an input time series sample into a continuous hidden path (rather than a hidden vector) and decodes it to reconstruct and impute the input. In our experiments with 4 datasets and 19 baselines, our method shows the best imputation performance in almost all cases.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Justin Bayer and Christian Osendorfer. 2014. Learning stochastic recurrent networks. arXiv preprint arXiv:1411.7610 (2014).
  2. Brits: Bidirectional recurrent imputation for time series. Advances in neural information processing systems 31 (2018).
  3. Recurrent neural networks for multivariate time series with missing values. Scientific reports 8, 1 (2018), 1–12.
  4. Freeway performance measurement system: mining loop detector data. Transportation Research Record 1748, 1 (2001), 96–102.
  5. Neural ordinary differential equations. Advances in neural information processing systems 31 (2018).
  6. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).
  7. Climate Modeling with Neural Advection–Diffusion Equation. Knowledge and Information Systems (2023).
  8. Graph Neural Controlled Differential Equations for Traffic Forecasting. In AAAI.
  9. Jeongwhan Choi and Noseong Park. 2023. Graph Neural Rough Differential Equations for Traffic Forecasting. ACM Transactions on Intelligent Systems and Technology (2023).
  10. A recurrent latent variable model for sequential data. Advances in neural information processing systems 28 (2015).
  11. Imre Csiszár. 1975. I-divergence geometry of probability distributions and minimization problems. The annals of probability (1975), 146–158.
  12. Alexiei Dingli and Karl Sant Fournier. 2017. Financial time series forecasting-a deep learning approach. International Journal of Machine Learning and Computing 7, 5 (2017), 118–122.
  13. Saits: Self-attention-based imputation for time series. Expert Systems with Applications 219 (2023), 119619.
  14. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks 107 (2018), 3–11.
  15. Spatial-temporal graph ode networks for traffic flow forecasting. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining. 364–373.
  16. Gp-vae: Deep probabilistic time series imputation. In AISTATS.
  17. Sequential neural models with stochastic layers. Advances in neural information processing systems 29 (2016).
  18. Dynamical Variational Autoencoders: A Comprehensive Review. Foundations and Trends® in Machine Learning 15, 1-2 (2021), 1–175. https://doi.org/10.1561/2200000089
  19. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing 24 (2020), 16453–16482.
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780.
  21. Climate Modeling with Neural Diffusion Equations. In ICDM. 230–239.
  22. GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networks. In Advances in Neural Information Processing Systems.
  23. Weiwei Jiang. 2021. Applications of deep learning in stock market prediction: recent progress. Expert Systems with Applications 184 (2021), 115537.
  24. Weiwei Jiang and Jiayun Luo. 2022. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications 207 (2022).
  25. Zahra Karevan and Johan AK Suykens. 2020. Transductive LSTM for time-series prediction: An application to weather forecasting. Neural Networks 125 (2020), 1–9.
  26. Short term stock price prediction using deep learning. In 2017 2nd IEEE international conference on recent trends in electronics, information & communication technology (RTEICT). IEEE, 482–486.
  27. Neural controlled differential equations for irregular time series. Advances in neural information processing systems 33 (2020), 6696–6707.
  28. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
  29. Mengzhang Li and Zhanxing Zhu. 2021. Spatial-temporal fusion graph neural networks for traffic flow forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4189–4196.
  30. Multivariate time series imputation with generative adversarial networks. Advances in neural information processing systems 31 (2018).
  31. E2gan: End-to-end generative adversarial network for multivariate time series imputation. In Proceedings of the 28th international joint conference on artificial intelligence. AAAI Press, 3094–3100.
  32. CDSA: cross-dimensional self-attention for multivariate, geo-tagged time series imputation. arXiv preprint arXiv:1905.09904 (2019).
  33. Sky McKinley and Megan Levine. 1998. Cubic spline interpolation. College of the Redwoods 45, 1 (1998), 1049–1060.
  34. Handling incomplete heterogeneous data using vaes. Pattern Recognition 107 (2020), 107501.
  35. Latent ordinary differential equations for irregularly-sampled time series. Advances in neural information processing systems 32 (2019).
  36. Jaydip Sen and Sidra Mehtab. 2021. Accurate stock price forecasting using robust and optimized deep learning models. In 2021 International Conference on Intelligent Technologies (CONIT). IEEE, 1–9.
  37. Nrtsi: Non-recurrent time series imputation. arXiv preprint arXiv:2102.03340 (2021).
  38. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in neural information processing systems 28 (2015).
  39. Satya Narayan Shukla and Benjamin Marlin. 2020. Multi-Time Attention Networks for Irregularly Sampled Time Series. In International Conference on Learning Representations.
  40. Satya Narayan Shukla and Benjamin Marlin. 2021. Heteroscedastic Temporal Variational Autoencoder For Irregularly Sampled Time Series. In International Conference on Learning Representations.
  41. Predicting in-hospital mortality of icu patients: The physionet/computing in cardiology challenge 2012. In 2012 Computing in Cardiology. IEEE, 245–248.
  42. T-J Stieltjes. 1894. Recherches sur les fractions continues. In Annales de la Faculté des sciences de Toulouse: Mathématiques, Vol. 8. J1–J122.
  43. Recurrent imputation for multivariate time series with missing values. In 2019 IEEE international conference on healthcare informatics (ICHI). IEEE, 1–3.
  44. Efficient Transformers: A Survey. ACM Comput. Surv. 55, 6 (2022).
  45. Spatio-temporal weather forecasting and attention mechanism on convolutional lstms. arXiv preprint arXiv:2102.00696 (2021).
  46. Deep learning for stock market prediction from financial news articles. In 2017 IEEE international conference on computational intelligence and virtual environments for measurement systems and applications (CIVEMSA). IEEE, 60–65.
  47. Attention is all you need. Advances in neural information processing systems 30 (2017).
  48. Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121 (2019).
  49. Time-series generative adversarial networks. Advances in neural information processing systems 32 (2019).
  50. Gain: Missing data imputation using generative adversarial nets. In International conference on machine learning. PMLR, 5689–5698.
  51. Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Transactions on Biomedical Engineering 66, 5 (2018), 1477–1490.
  52. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875 (2017).
  53. Are transformers effective for time series forecasting? arXiv preprint arXiv:2205.13504 (2022).
  54. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 11106–11115.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now