Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Information Theoretically Optimal Sample Complexity of Learning Dynamical Directed Acyclic Graphs (2308.16859v2)

Published 31 Aug 2023 in stat.ML, cs.LG, cs.SY, eess.SY, and math.OC

Abstract: In this article, the optimal sample complexity of learning the underlying interactions or dependencies of a Linear Dynamical System (LDS) over a Directed Acyclic Graph (DAG) is studied. We call such a DAG underlying an LDS as dynamical DAG (DDAG). In particular, we consider a DDAG where the nodal dynamics are driven by unobserved exogenous noise sources that are wide-sense stationary (WSS) in time but are mutually uncorrelated, and have the same {power spectral density (PSD)}. Inspired by the static DAG setting, a metric and an algorithm based on the PSD matrix of the observed time series are proposed to reconstruct the DDAG. It is shown that the optimal sample complexity (or length of state trajectory) needed to learn the DDAG is $n=\Theta(q\log(p/q))$, where $p$ is the number of nodes and $q$ is the maximum number of parents per node. To prove the sample complexity upper bound, a concentration bound for the PSD estimation is derived, under two different sampling strategies. A matching min-max lower bound using generalized Fano's inequality also is provided, thus showing the order optimality of the proposed algorithm.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. S. Basu and G. Michailidis. Regularized estimation in sparse high-dimensional time series models. The Annals of Statistics, pages 1535–1567, 2015.
  2. J. M. Bower and D. Beeman. The book of GENESIS: exploring realistic neural models with the GEneral NEural SImulation System. Springer Science & Business Media, 2012.
  3. On causal discovery with an equal-variance assumption. Biometrika, 106(4):973–980, 2019.
  4. Covariance matrix estimation from linearly-correlated gaussian samples. IEEE Transactions on Signal Processing, 67(8):2187–2195, 2019.
  5. R. Dahlhaus. Graphical interaction models for multivariate time series. Metrika, 51:157–172, 2000.
  6. A. Dallakyan. On learning time series summary dags: A frequency domain approach. arXiv preprint arXiv:2304.08482, 2023.
  7. Efficient and passive learning of networked dynamical systems driven by non-white exogenous inputs. In International Conference on Artificial Intelligence and Statistics, pages 9982–9997. PMLR, 2022.
  8. Finite time identification in unstable linear systems. Automatica, 96:342–353, 2018.
  9. Spectral analysis of high-dimensional time series. Electronic Journal of Statistics, 13(2):4079–4101, 2019.
  10. N. Friedman and Z. Yakhini. On the sample complexity of learning bayesian networks. In Proceedings of the Twelfth international conference on Uncertainty in artificial intelligence, pages 274–282, 1996.
  11. Optimal estimation of gaussian dag models. In International Conference on Artificial Intelligence and Statistics, pages 8738–8757. PMLR, 2022.
  12. Advanced spectral methods for climatic time series. Reviews of Geophysics, 40(1):3–1–3–41, 2002. doi: 10.1029/2000RG000092. URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2000RG000092.
  13. A. Ghoshal and J. Honorio. Information-theoretic limits of bayesian network structure learning. In Artificial Intelligence and Statistics, pages 767–775. PMLR, 2017a.
  14. A. Ghoshal and J. Honorio. Learning identifiable gaussian bayesian networks in polynomial time and sample complexity. Advances in Neural Information Processing Systems, 30, 2017b.
  15. A. Ghoshal and J. Honorio. Learning linear structural equation models in polynomial time and sample complexity. In A. Storkey and F. Perez-Cruz, editors, Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 1466–1475. PMLR, 09–11 Apr 2018. URL https://proceedings.mlr.press/v84/ghoshal18a.html.
  16. Matrix Analysis. Cambridge University Press, USA, 2nd edition, 2012. ISBN 0521548233.
  17. Time series analysis for financial market meltdowns. Journal of Banking & Finance, 35(8):1879–1891, 2011.
  18. A. Lamperski. Non-asymptotic pointwise and worst-case bounds for classical spectrum estimators. arXiv preprint arXiv:2303.11908, 2023.
  19. High-dimensional regression with noisy and missing data: Provable guarantees with non-convexity. Advances in neural information processing systems, 24, 2011.
  20. D. Materassi and G. Innocenti. Unveiling the connectivity structure of financial networks via high-frequency analysis. Physica A: Statistical Mechanics and its Applications, 388(18):3866–3878, 2009. ISSN 0378-4371. doi: https://doi.org/10.1016/j.physa.2009.06.003. URL https://www.sciencedirect.com/science/article/pii/S0378437109004324.
  21. D. Materassi and M. V. Salapaka. On the problem of reconstructing an unknown topology via locality properties of the wiener filter. IEEE Transactions on Automatic Control, 57(7):1765–1777, July 2012. ISSN 0018-9286. doi: 10.1109/TAC.2012.2183170.
  22. D. Materassi and M. V. Salapaka. Reconstruction of directed acyclic networks of dynamical systems. In 2013 American Control Conference, pages 4687–4692. IEEE, 2013.
  23. G. Park. Identifiability of additive noise models using conditional variances. The Journal of Machine Learning Research, 21(1):2896–2929, 2020.
  24. G. Park and G. Raskutti. Learning quadratic variance function (qvf) dag models via overdispersion scoring (ods). J. Mach. Learn. Res., 18(224):1–44, 2017.
  25. J. Peters and P. Bühlmann. Identifiability of gaussian structural equation models with equal error variances. Biometrika, 101(1):219–228, 2014.
  26. Directed information graphs. IEEE Transactions on Information Theory, 61(12):6887–6909, 2015. doi: 10.1109/TIT.2015.2478440.
  27. Inferring causation from time series in earth system sciences. Nature communications, 10(1):2553, 2019.
  28. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
  29. Learning without mixing: Towards a sharp analysis of linear system identification. In Conference On Learning Theory, pages 439–473. PMLR, 2018.
  30. J. Songsiri and L. Vandenberghe. Topology selection in graphical models of autoregressive processes. The Journal of Machine Learning Research, 11:2671–2705, 2010.
  31. P. Spirtes and K. Zhang. Causal discovery and inference: concepts and recent methodological advances. In Applied informatics, volume 3, pages 1–28. SpringerOpen, 2016.
  32. Spectral analysis of signals, volume 452. Pearson Prentice Hall Upper Saddle River, NJ, 2005.
  33. Detecting causality in complex ecosystems. science, 338(6106):496–500, 2012.
  34. Topology learning of radial dynamical systems with latent nodes. In 2018 Annual American Control Conference (ACC), pages 1096–1101. IEEE, 2018.
  35. Physics informed topology learning in networks of linear dynamical systems. Automatica, 112:108705, 2020. ISSN 0005-1098. doi: https://doi.org/10.1016/j.automatica.2019.108705.
  36. Bayesian structure learning for stationary time series. In Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, pages 872–881, 2015.
  37. Topology learning of linear dynamical systems with latent nodes using matrix decomposition. IEEE Transactions on Automatic Control, Early Access:1–1, 2021. doi: 10.1109/TAC.2021.3124979.
  38. M. J. Wainwright. Random matrices and covariance estimation, page 159–193. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019. doi: 10.1017/9781108627771.006.
  39. Transportation, germs, culture: a dynamic graph model of covid-19 outbreak. Quantitative Biology, 8:238–244, 2020.
  40. D. Zhang and W. B. Wu. Convergence of covariance and spectral density estimates for high-dimensional locally stationary processes. The Annals of Statistics, 49(1):233–254, 2021.
  41. On the number of samples needed to learn the correct structure of a bayesian network. In Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence, pages 560–567, 2006.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now