Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
121 tokens/sec
GPT-4o
9 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Probabilistic Reduced-Dimensional Vector Autoregressive Modeling with Oblique Projections (2401.07206v1)

Published 14 Jan 2024 in stat.ML, cs.LG, cs.SY, and eess.SY

Abstract: In this paper, we propose a probabilistic reduced-dimensional vector autoregressive (PredVAR) model to extract low-dimensional dynamics from high-dimensional noisy data. The model utilizes an oblique projection to partition the measurement space into a subspace that accommodates the reduced-dimensional dynamics and a complementary static subspace. An optimal oblique decomposition is derived for the best predictability regarding prediction error covariance. Building on this, we develop an iterative PredVAR algorithm using maximum likelihood and the expectation-maximization (EM) framework. This algorithm alternately updates the estimates of the latent dynamics and optimal oblique projection, yielding dynamic latent variables with rank-ordered predictability and an explicit latent VAR model that is consistent with the outer projection model. The superior performance and efficiency of the proposed approach are demonstrated using data sets from a synthesized Lorenz system and an industrial process from Eastman Chemical.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. H. Akaike. Autoregressive model fitting for control. Ann. Inst. Stat. Math., 23:163–180, 1971.
  2. J. Bai and K. Li. Statistical analysis of factor models of high dimension. Ann. Stat., 40:436–465, 2012.
  3. A canonical analysis of multiple time series. Biometrika, 64(2):355–365, 1977.
  4. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge university press, 2004.
  5. Identification of low rank vector processes. Automatica, 151:110938, 2023.
  6. A. Chiuso and G. Picci. Consistency analysis of some closed-loop subspace identification methods. Automatica, 41:377–391, 2005.
  7. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Methodol., 39:1–22, 1977.
  8. Efficient dynamic latent variable analysis for high-dimensional time series data. IEEE Trans. Ind. Informat., 16(6):4068–4076, 2020.
  9. Y. Dong and S. J. Qin. Dynamic latent variable analytics for process operations and control. Comput. Chem. Eng., 114:69–80, 2018.
  10. Y. Dong and S. J. Qin. A novel dynamic PCA algorithm for dynamic data modeling and process monitoring. J. Process Control, 67:1–11, 2018.
  11. Semi-supervised dynamic latent variable modeling: I/o probabilistic slow feature analysis approach. AIChE J., 65(3), 2019.
  12. Dynamic probabilistic predictable feature analysis for multivariate temporal process monitoring. IEEE Trans. Control Syst. Technol., page e17609, 2022.
  13. LaSDI: Parametric latent space dynamics identification. Comput. Methods Appl. Mech. Eng., 399:115436, 2022.
  14. Z. Gao and R. S. Tsay. Modeling high-dimensional time series: A factor model with dynamically dependent factors and diverging eigenvalues. J. Am. Stat. Assoc., pages 1–17, 2021.
  15. P. Geladi and B. R. Kowalski. Partial least-squares regression: A tutorial. Anal. Chim. Acta, 185:1–17, 1986.
  16. A. J. Izenman. Reduced-rank regression for the multivariate linear model. J. Multivar. Anal., 5(2):248–264, 1975.
  17. M. Khosravi and R. S. Smith. The existence and uniqueness of solutions for kernel-based system identification. Automatica, 148:110728, 2023.
  18. C. Lam and Q. Yao. Factor modeling for high-dimensional time series: Inference for the number of factors. Ann. Stat., pages 694–726, 2012.
  19. A new method of dynamic latent-variable modeling for process monitoring. IEEE Trans. Ind. Electron., 61:6438–6445, 2014.
  20. L. Ljung. System Identification: Theory for the User. Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1999. Second Edition.
  21. A novel multivariate statistical process monitoring algorithm: Orthonormal subspace analysis. Automatica, 138:110148, 2022.
  22. Probabilistic reduced-dimensional vector autoregressive modeling for dynamics prediction and reconstruction with oblique projections. In IEEE Conf. Decis. Control (CDC), 2023.
  23. Subspace Identification for Linear Systems. Kluwer Academic Publishers, 1996.
  24. J. Pan and Q. Yao. Modelling multiple time series via common factors. Biometrika, 95(2):365–379, 2008.
  25. D. Peña and G. E. P. Box. Identifying a simplifying structure in time series. J. Am. Stat. Assoc., 82:836–843, 1987.
  26. Forecasting multiple time series with one-sided dynamic principal components. J. Am. Stat. Assoc., 2019.
  27. Kernel methods in system identification, machine learning and function estimation: A survey. Automatica, 50:657–682, 2014.
  28. S. J. Qin. Latent vector autoregressive modeling for reduced dimensional dynamic feature extraction and prediction. In IEEE Conf. Decis. Control (CDC), pages 3689–3694, 2021.
  29. S. J. Qin. Latent vector autoregressive modeling and feature analysis of high dimensional and noisy data from dynamic systems. AIChE J., page e17703, 2022.
  30. Bridging systems theory and data science: A unifying review of dynamic latent variable analytics and process monitoring. Annu Rev Control, 50:29–48, 2020.
  31. Partial least squares, steepest descent, and conjugate gradient for regularized predictive modeling. AIChE J., 69(4):e17992, 2023.
  32. G. Reinsel and R. Velu. Multivariate Reduced-Rank Regression, volume 136 of Lecture Notes in Statistics. Springer, New York, NY, 1998.
  33. Recursive slow feature analysis for adaptive monitoring of industrial processes. IEEE Trans. Ind. Electron., 65(11):8895–8905, 2018.
  34. Forecasting using principal components from a large number of predictors. J. Am. Stat. Assoc., 97:1167–1179, 2002.
  35. M. Sznaier. Control oriented learning in the era of big data. IEEE Control Syst. Lett., 5:1855–1867, 2020.
  36. Prediction error identification of linear dynamic networks with rank-reduced noise. Automatica, 98:256–268, 2018.
  37. Data-based linear Gaussian state-space model for dynamic process monitoring. AIChE J., 58(12):3763–3776, 2012.
  38. J. Yu and S. J. Qin. Latent state space modeling of high-dimensional time series with a canonical correlation objective. IEEE Control Syst. Lett., 6:3469–3474, 2022.
  39. A generalized probabilistic monitoring model with both random and sequential data. Automatica, 144:110468, 2022.
  40. C. Zhao and B. Huang. A full-condition monitoring method for nonstationary dynamic chemical processes with cointegration and slow feature analysis. AIChE J., 64(5):1662–1681, 2018.
  41. Autoregressive dynamic latent variable models for process monitoring. IEEE Trans. Control Syst. Technol., 25:366–373, 2016.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now