Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Recovering the Graph Underlying Networked Dynamical Systems under Partial Observability: A Deep Learning Approach (2208.04405v3)

Published 8 Aug 2022 in cs.LG, cs.MA, and stat.ME

Abstract: We study the problem of graph structure identification, i.e., of recovering the graph of dependencies among time series. We model these time series data as components of the state of linear stochastic networked dynamical systems. We assume partial observability, where the state evolution of only a subset of nodes comprising the network is observed. We devise a new feature vector computed from the observed time series and prove that these features are linearly separable, i.e., there exists a hyperplane that separates the cluster of features associated with connected pairs of nodes from those associated with disconnected pairs. This renders the features amenable to train a variety of classifiers to perform causal inference. In particular, we use these features to train Convolutional Neural Networks (CNNs). The resulting causal inference mechanism outperforms state-of-the-art counterparts w.r.t. sample-complexity. The trained CNNs generalize well over structurally distinct networks (dense or sparse) and noise-level profiles. Remarkably, they also generalize well to real-world networks while trained over a synthetic network (realization of a random graph). Finally, the proposed method consistently reconstructs the graph in a pairwise manner, that is, by deciding if an edge or arrow is present or absent in each pair of nodes, from the corresponding time series of each pair. This fits the framework of large-scale systems, where observation or processing of all nodes in the network is prohibitive.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (65)
  1. Identification of Partially Observed Causal Models: Graphical Conditions for the Linear Non-Gaussian and Heterogeneous Cases. In Advances in Neural Information Processing Systems 34 pre-proceedings (NeurIPS 2021), NeurIPS ’21.
  2. Learning Linear Bayesian Networks with Latent Variables. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, 249–257. Atlanta, Georgia, USA: PMLR.
  3. High-dimensional Gaussian Graphical Model Selection: Walk Summability and Local Separation Criterion. J. Mach. Learn. Res., 13(1): 2293–2337.
  4. Learning loopy graphical models with latent variables: Efficient methods and guarantees. Ann. Statist., 41(2): 401–435.
  5. Dynamical Processes on Complex Networks. London, UK: Cambridge University Press. ISBN 9781107626256.
  6. Community Detection in Temporal Multilayer Networks, with an Application to Correlation Networks. Multiscale Modeling & Simulation, 14(1): 1–41.
  7. The Complexity of Distinguishing Markov Random Fields. In Approximation, Randomization and Combinatorial Optimization. Algorithms and Techniques, 331–342. Berlin, Heidelberg: Springer Berlin Heidelberg. ISBN 978-3-540-85363-3.
  8. Network dismantling. Proceedings of the National Academy of Sciences, 113(44): 12368–12373.
  9. Hardness of Parameter Estimation in Graphical Models. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1, NIPS’14, 1062–1070. Cambridge, MA, USA: MIT Press.
  10. Latent variable graphical model selection via convex optimization. The Annals of Statistics, 40(4): 1935–1967.
  11. An Unbiased Symmetric Matrix Estimator for Topology Inference under Partial Observability. IEEE Signal Processing Letters, 29(02): 1257–1261.
  12. Chickering, D. M. 2003. Optimal Structure Identification with Greedy Search. J. Mach. Learn. Res., 3(null): 507–554.
  13. Reconstructing links in directed networks from noisy dynamics. Phys. Rev. E, 95: 010301.
  14. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 14(3): 462–467.
  15. Learning High-dimensional Directed Acyclic Graphs with Latent and Selection Variables. The Annals of Statistics, 40(1): 294–321.
  16. Dax, A. 2006. The distance between two convex sets. Linear Algebra and its Applications, 416(1): 184–213. Special Issue devoted to the Haifa 2005 conference on matrix theory.
  17. ‘Functional Connectivity’ is a Sensitive Predictor of Epilepsy Diagnosis after the First Seizure. PloS one, 5.
  18. Dynamic communities in multichannel data: An application to the foreign exchange market during the 2007–2008 credit crisis. Chaos: An Interdisciplinary Journal of Nonlinear Science, 19(3): 033119.
  19. Dynamical clustering of exchange rates. Quantitative Finance, 12(10): 1493–1520.
  20. The effect of network topology on the spread of epidemics. In Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies., volume 2, 1455–1466.
  21. Causal Inference by Identification of Vector Autoregressive Processes with Hidden Components. In Proc. International Conference on Machine Learning, volume 37, 1917–1925.
  22. Granger, C. W. J. 1969. Investigating Causal Relations by Econometric Models and Cross-spectral Methods. Econometrica, 37(3): 424–438.
  23. Markov fields on finite graphs and lattices. University of Oxford.
  24. Fundamentals of Convex Analysis. Grundlehren Text Editions. Springer-Verlag Berlin Heidelberg. ISBN 978-3-540-42205-1.
  25. Learning the Dependence Graph of Time Series with Latent Factors. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML’12, 619–626. Madison, WI, USA: Omnipress. ISBN 9781450312851.
  26. Identification of diffusively coupled linear networks through structured polynomial models. IEEE Transactions on Automatic Control, 1–16.
  27. A Deterministic Model for Gonorrhea in a Nonhomogeneous Population. Mathematical Biosciences, 28: 221–236.
  28. The Human Organism as an Integrated Interaction Network: Recent Conceptual and Methodological Challenges. Frontiers in Physiology, 11.
  29. Revisiting correlation-based functional connectivity and its relationship with structural connectivity. Network Neuroscience, 4(4): 1235–1251.
  30. Liggett, T. 2005. Interacting Particle Systems. Springer-Verlag Berlin Heidelberg, first edition. ISBN 978-3-540-26962-5.
  31. Operator-valued kernel-based vector autoregressive models for network inference. Machine Learning, 99(3): 489–513.
  32. Necessary and sufficient conditions for causal feature selection in time series with latent common causes. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, 7502–7511. PMLR.
  33. Connecting the Dots: Identifying Network Structure via Graph Signal Processing. IEEE Signal Processing Magazine, 36(3): 16–43.
  34. Network reconstruction of dynamical polytrees with unobserved nodes. In Proc. IEEE Conference on Decision and Control (CDC), 4629–4634. Maui, Hawaii.
  35. On the problem of reconstructing an unknown topology via locality properties of the Wiener filter. IEEE Transactions on Automatic Control, 57(7): 1765–1777.
  36. Identification of network components in presence of unobserved nodes. In Proc. IEEE Conference on Decision and Control (CDC), 1563–1568. Osaka, Japan.
  37. Graph Learning under Partial Observability. Proceedings of the IEEE, 108: 2049 – 2066.
  38. Graph Learning over Partially Observed Diffusion Networks: Role of Degree Concentration. IEEE Open Journal of Signal Processing, 335–371.
  39. Linear identification of nonlinear systems: A lifting technique based on the Koopman operator. In 2016 IEEE 55th Conference on Decision and Control (CDC), 6500–6505. Las Vegas, USA.
  40. Signal Processing on Graphs: Causal Modeling of Unstructured Data. IEEE Transactions on Signal Processing, 65(8): 2077–2092.
  41. SILVar: Single Index Latent Variable Models. IEEE Transactions on Signal Processing, 66(11): 2790–2803.
  42. An Informed Multitask Diffusion Adaptation Approach to Study Tremor in Parkinson’s Disease. IEEE Journal of Selected Topics in Signal Processing, 10(7): 1306–1314.
  43. Causal Search in Structural Vector Autoregressive Models. In Proceedings of the 12th International Conference on Neural Information Processing Systems (NIPS) Mini-Symposium on Causality in Time Series, 95–118. Vancouver, Canada.
  44. Reconstructing the topology of sparsely connected dynamical networks. Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, 77: 026103.
  45. Disrupted functional connectivity in PD with probable RBD and its cognitive correlates. Scientific Reports, 11.
  46. Pearl, J. 2009. Causality. Cambridge University Press, 2 edition.
  47. Learning Networks of Stochastic Differential Equations. In Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc.
  48. Dynamical Systems on Networks: A Tutorial. Springer International Publishing. ISBN 9783319266411.
  49. A million variables and more: the Fast Greedy Equivalence Search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. International Journal of Data Science and Analytics, 3: 121–129.
  50. Regional functional connectivity predicts distinct cognitive impairments in Alzheimer’s disease spectrum. NeuroImage: Clinical, 5: 385–395.
  51. Generalized network dismantling. Proceedings of the National Academy of Sciences, 116(14): 6554–6559.
  52. Robert, P. 2003. Stochastic Networks and Queues. Springer-Verlag. ISBN 978-3-540-00657-2.
  53. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI.
  54. Discrete Signal Processing on Graphs. IEEE Transactions on Signal Processing, 61(7): 1644–1656.
  55. Local Tomography of Large Networks under the Low-Observability Regime. IEEE Transactions on Information Theory, 66: 587 – 613.
  56. Bi-Virus SIS Epidemics over Networks: Qualitative Analysis. IEEE Transactions on Network Science and Engineering, 2(1): 17–29.
  57. Sayed, A. H. 2014. Adaptation, Learning, and Optimization over Networks. Found. Trends Mach. Learn., 7(4-5): 311–801.
  58. Network Topology Inference from Spectral Templates. IEEE Transactions on Signal and Information Processing over Networks, 3(3): 467–483.
  59. Network inference from consensus dynamics. In 2017 IEEE 56th Annual Conference on Decision and Control (CDC), 3212–3217.
  60. An Algorithm for Fast Recovery of Sparse Causal Graphs. Social Science Computer Review, 9(1): 62–72.
  61. Causation, Prediction, and Search. MIT press, 2nd edition.
  62. Causal Inference in the Presence of Latent Variables and Selection Bias. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, UAI’95, 499–506. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc. ISBN 1558603859.
  63. Small-World Networks and Functional Connectivity in Alzheimer’s Disease. Cerebral Cortex, 17(1): 92–99.
  64. Network Perspectives on Epilepsy Using EEG/MEG Source Connectivity. Frontiers in Neurology, 10.
  65. Identifiability and Estimation of Partially-observed Influence Models. IEEE Control Systems Letters, 1–1.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now