Missing Value Imputation on Multidimensional Time Series
Abstract: We present DeepMVI, a deep learning method for missing value imputation in multidimensional time-series datasets. Missing values are commonplace in decision support platforms that aggregate data over long time stretches from disparate sources, and reliable data analytics calls for careful handling of missing data. One strategy is imputing the missing values, and a wide variety of algorithms exist spanning simple interpolation, matrix factorization methods like SVD, statistical models like Kalman filters, and recent deep learning methods. We show that often these provide worse results on aggregate analytics compared to just excluding the missing data. DeepMVI uses a neural network to combine fine-grained and coarse-grained patterns along a time series, and trends from related series across categorical dimensions. After failing with off-the-shelf neural architectures, we design our own network that includes a temporal transformer with a novel convolutional window feature, and kernel regression with learned embeddings. The parameters and their training are designed carefully to generalize across different placements of missing blocks and data characteristics. Experiments across nine real datasets, four different missing scenarios, comparing seven existing methods show that DeepMVI is significantly more accurate, reducing error by more than 50% in more than half the cases, compared to the best existing method. Although slower than simpler matrix factorization methods, we justify the increased time overheads by showing that DeepMVI is the only option that provided overall more accurate analytics than dropping missing values.
- Optuna: A Next-generation Hyperparameter Optimization Framework. In Proceedings of the 25rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
- A singular value thresholding algorithm for matrix completion. SIAM Journal on optimization 20, 4 (2010), 1956–1982.
- Query Optimization for Dynamic Imputation. Proc. VLDB Endow. 10, 11 (2017).
- Brits: Bidirectional recurrent imputation for time series. arXiv preprint arXiv:1805.10572 (2018).
- Prathamesh Deshpande and Sunita Sarawagi. 2019. Streaming adaptation of deep forecasting models using adaptive recurrent units. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1560–1568.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (2018).
- DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks. CoRR abs/1704.04110 (2017).
- Gp-vae: Deep probabilistic time series imputation. In International Conference on Artificial Intelligence and Statistics. PMLR, 1651–1661.
- Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural networks 18, 5-6 (2005), 602–610.
- Profiler: Integrated statistical analysis and visualization for data quality assessment. In Proceedings of the International Working Conference on Advanced Visual Interfaces. 547–554.
- Scalable recovery of missing blocks in time series with high and low cross-correlations. Knowledge and Information Systems (2019), 1–24.
- Mind the gap: an experimental evaluation of imputation of missing values techniques in time series. Proceedings of the VLDB Endowment 13, 5 (2020), 768–782.
- Jasper: An end-to-end convolutional neural acoustic model. arXiv preprint arXiv:1904.03288 (2019).
- Dynammo: Mining and summarization of coevolving sequences with missing values. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 507–516.
- Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in Neural Information Processing Systems. 5243–5253.
- Roderick JA Little and Donald B Rubin. 2002. Single imputation methods. Statistical analysis with missing data (2002), 59–74.
- NAOMI: Non-autoregressive multiresolution sequence imputation. In Advances in Neural Information Processing Systems. 11238–11248.
- ERACER: A Database Approach for Statistical Inference and Data Cleaning. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data.
- Spectral regularization algorithms for learning large incomplete matrices. The Journal of Machine Learning Research 11 (2010), 2287–2322.
- Nonnegative matrix factorization for time series recovery from a few temporal aggregates. In International Conference on Machine Learning. PMLR, 2382–2390.
- Tova Milo and Amit Somech. 2020. Automating exploratory data analysis via machine learning: An overview. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 2617–2622.
- High-dimensional multivariate forecasting with low-rank Gaussian Copula Processes. In Advances in Neural Information Processing Systems. 6827–6837.
- Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. arXiv preprint arXiv:1905.03806 (2019).
- Missing value estimation methods for DNA microarrays. Bioinformatics 17, 6 (2001), 520–525.
- Attention is All you Need. In NIPS.
- Continuous imputation of missing values in streams of pattern-determining time series. (2017).
- Estimating missing data in temporal data streams using multi-directional recurrent neural networks. IEEE Transactions on Biomedical Engineering 66, 5 (2018), 1477–1490.
- Temporal regularized matrix factorization for high-dimensional time series prediction. In Advances in neural information processing systems. 847–855.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
