Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

NeuroKoopman Dynamic Causal Discovery (2404.16326v1)

Published 25 Apr 2024 in cs.LG

Abstract: In many real-world applications where the system dynamics has an underlying interdependency among its variables (such as power grid, economics, neuroscience, omics networks, environmental ecosystems, and others), one is often interested in knowing whether the past values of one time series influences the future of another, known as Granger causality, and the associated underlying dynamics. This paper introduces a Koopman-inspired framework that leverages neural networks for data-driven learning of the Koopman bases, termed NeuroKoopman Dynamic Causal Discovery (NKDCD), for reliably inferring the Granger causality along with the underlying nonlinear dynamics. NKDCD employs an autoencoder architecture that lifts the nonlinear dynamics to a higher dimension using data-learned bases, where the lifted time series can be reliably modeled linearly. The lifting function, the linear Granger causality lag matrices, and the projection function (from lifted space to base space) are all represented as multilayer perceptrons and are all learned simultaneously in one go. NKDCD also utilizes sparsity-inducing penalties on the weights of the lag matrices, encouraging the model to select only the needed causal dependencies within the data. Through extensive testing on practically applicable datasets, it is shown that the NKDCD outperforms the existing nonlinear Granger causality discovery approaches.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. R. Guo, L. Cheng, J. Li, P. R. Hahn, and H. Liu, “A survey of learning causality with data: Problems and methods,” ACM Computing Surveys (CSUR), vol. 53, no. 4, pp. 1–37, 2020.
  2. C. W. Granger, “Investigating causal relations by econometric models and cross-spectral methods,” Econometrica: Journal of the Econometric Society, pp. 424–438, 1969.
  3. Y. Hong, Y. Liu, and S. Wang, “Granger causality in risk and detection of extreme risk spillover between financial markets,” Journal of Econometrics, vol. 150, no. 2, pp. 271–287, 2009.
  4. A. T. Reid, D. B. Headley, R. D. Mill, R. Sanchez-Romero, L. Q. Uddin, D. Marinazzo, D. J. Lurie, P. A. Valdés-Sosa, S. J. Hanson, B. B. Biswal et al., “Advancing functional connectivity research from association to causation,” Nature neuroscience, vol. 22, no. 11, pp. 1751–1760, 2019.
  5. T. J. Mosedale, D. B. Stephenson, M. Collins, and T. C. Mills, “Granger causality of coupled climate processes: Ocean feedback on the north atlantic oscillation,” Journal of climate, vol. 19, no. 7, pp. 1182–1194, 2006.
  6. S. Z. Chiou-Wei, C.-F. Chen, and Z. Zhu, “Economic growth and energy consumption revisited—evidence from linear and nonlinear granger causality,” Energy economics, vol. 30, no. 6, pp. 3063–3076, 2008.
  7. P. A. Stokes and P. L. Purdon, “A study of problems encountered in Granger causality analysis from a neuroscience perspective,” Proceedings of the National Academy of Sciences, vol. 114, no. 34, pp. E7063–E7072, 2017.
  8. A. Sheikhattar, S. Miran, J. Liu, J. B. Fritz, S. A. Shamma, P. O. Kanold, and B. Babadi, “Extracting neuronal functional network dynamics via adaptive Granger causality analysis,” Proceedings of the National Academy of Sciences, vol. 115, no. 17, pp. E3869–E3878, 2018.
  9. A. Fujita, P. Severino, J. R. Sato, and S. Miyano, “Granger causality in systems biology: modeling gene networks in time series microarray data using vector autoregressive models,” in Brazilian Symposium on Bioinformatics.   Springer, 2010, pp. 13–24.
  10. A. C. Lozano, N. Abe, Y. Liu, and S. Rosset, “Grouped graphical granger modeling methods for temporal causal modeling,” in Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.   ACM, 2009, pp. 577–586.
  11. C. A. Sims, “Money, income, and causality,” The American economic review, vol. 62, no. 4, pp. 540–552, 1972.
  12. G. Chamberlain, “The general equivalence of granger and sims causality,” Econometrica: Journal of the Econometric Society, pp. 569–581, 1982.
  13. R. Tibshirani, “Regression shrinkage and selection via the lasso,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, 1996.
  14. M. Yuan and Y. Lin, “Model selection and estimation in regression with grouped variables,” Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol. 68, no. 1, pp. 49–67, 2006.
  15. W. B. Nicholson, J. Bien, and D. S. Matteson, “Hierarchical vector autoregression,” arXiv preprint arXiv:1412.5250, 2014.
  16. A. Shojaie and G. Michailidis, “Discovering graphical Granger causality using the truncating lasso penalty,” Bioinformatics, vol. 26, no. 18, pp. i517–i523, 2010.
  17. B. Lusch, P. D. Maia, and J. N. Kutz, “Inferring connectivity in networked dynamical systems: Challenges using Granger causality,” Physical Review E, vol. 94, no. 3, p. 032220, 2016.
  18. R. Vicente, M. Wibral, M. Lindner, and G. Pipa, “Transfer entropy—a model-free measure of effective connectivity for the neurosciences,” Journal of Computational Neuroscience, vol. 30, no. 1, pp. 45–67, 2011.
  19. P.-O. Amblard and O. J. Michel, “On directed information theory and Granger causality graphs,” Journal of Computational Neuroscience, vol. 30, no. 1, pp. 7–16, 2011.
  20. J. Runge, J. Heitzig, V. Petoukhov, and J. Kurths, “Escaping the curse of dimensionality in estimating multivariate transfer entropy,” Physical Review Letters, vol. 108, no. 25, p. 258701, 2012.
  21. M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Multistep neural networks for data-driven discovery of nonlinear dynamical systems,” arXiv preprint arXiv:1801.01236, 2018.
  22. Ö. Kişi, “River flow modeling using artificial neural networks,” Journal of Hydrologic Engineering, vol. 9, no. 1, pp. 60–63, 2004.
  23. A. Graves, “Supervised sequence labelling,” in Supervised Sequence Labelling with Recurrent Neural Networks.   Springer, 2012, pp. 5–13.
  24. Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Graph convolutional recurrent neural network: Data-driven traffic forecasting,” arXiv preprint arXiv:1707.01926, 2017.
  25. A. Tank, I. Covert, N. Foti, A. Shojaie, and E. B. Fox, “Neural granger causality,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 8, pp. 4267–4279, 2021.
  26. B. O. Koopman, “Hamiltonian systems and transformation in hilbert space,” Proceedings of the National Academy of Sciences, vol. 17, no. 5, pp. 315–318, 1931.
  27. M. O. Williams, I. G. Kevrekidis, and C. W. Rowley, “A data–driven approximation of the koopman operator: Extending dynamic mode decomposition,” Journal of Nonlinear Science, vol. 25, pp. 1307–1346, 2015.
  28. P. J. Schmid, “Dynamic mode decomposition and its variants,” Annual Review of Fluid Mechanics, vol. 54, pp. 225–254, 2022.
  29. M. Korda and I. Mezić, “Linear predictors for nonlinear dynamical systems: Koopman operator meets model predictive control,” Automatica, vol. 93, pp. 149–160, 2018.
  30. S. Sinha and U. Vaidya, “On data-driven computation of information transfer for causal inference in discrete-time dynamical systems,” Journal of Nonlinear Science, vol. 30, no. 4, pp. 1651–1676, 2020.
  31. R. Gunjal, S. S. Nayyer, S. Wagh, A. Stankovic, and N. Singh, “Granger causality for prediction in dynamic mode decomposition: Application to power systems,” Electric Power Systems Research, vol. 225, p. 109865, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0378779623007538
  32. C. Wehmeyer and F. Noé, “Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics,” The Journal of chemical physics, vol. 148, no. 24, 2018.
  33. B. Lusch, J. N. Kutz, and S. L. Brunton, “Deep learning for universal linear embeddings of nonlinear dynamics,” Nature communications, vol. 9, no. 1, p. 4950, 2018.
  34. R. R. Hossain, R. Adesunkanmi, and R. Kumar, “Data-driven linear koopman embedding for networked systems: Model-predictive grid control,” IEEE Systems Journal, 2023.
  35. M. Nauta, D. Bucur, and C. Seifert, “Causal discovery with attention-based convolutional neural networks,” Machine Learning and Knowledge Extraction, vol. 1, no. 1, pp. 312–340, 2019.
  36. E. N. Lorenz, “Predictability: A problem partly solved,” in Proc. Seminar on predictability, vol. 1, no. 1.   Reading, 1996.
  37. S. M. Smith, K. L. Miller, G. Salimi-Khorshidi, M. Webster, C. F. Beckmann, T. E. Nichols, J. D. Ramsey, and M. W. Woolrich, “Network modelling methods for fmri,” Neuroimage, vol. 54, no. 2, pp. 875–891, 2011.
  38. R. J. Prill, D. Marbach, J. Saez-Rodriguez, P. K. Sorger, L. G. Alexopoulos, X. Xue, N. D. Clarke, G. Altan-Bonnet, and G. Stolovitzky, “Towards a rigorous assessment of systems biology models: the dream3 challenges,” PloS One, vol. 5, no. 2, p. e9202, 2010.
  39. A. C. Lozano, N. Abe, Y. Liu, and S. Rosset, “Grouped graphical granger modeling for gene expression regulatory networks discovery,” Bioinformatics, vol. 25, no. 12, pp. i110–i118, 2009.
  40. A. Tank, I. Covert, N. Foti, A. Shojaie, and E. Fox, “Neural granger causality,” arXiv preprint arXiv:1802.05842, 2018.
  41. N. Parikh, S. Boyd et al., “Proximal algorithms,” Foundations and Trends in Optimization, vol. 1, no. 3, pp. 127–239, 2014.
  42. E. F. Fama and K. R. French, “The cross-section of expected stock returns,” the Journal of Finance, vol. 47, no. 2, pp. 427–465, 1992.
  43. R. B. Buxton, E. C. Wong, and L. R. Frank, “Dynamics of blood flow and oxygenation changes during brain activation: the balloon model,” Magnetic resonance in medicine, vol. 39, no. 6, pp. 855–864, 1998.
  44. N. Lim, F. d’Alché Buc, C. Auliac, and G. Michailidis, “Operator-valued kernel-based vector autoregressive models for network inference,” Machine learning, vol. 99, pp. 489–513, 2015.
  45. S. Lèbre, “Inferring dynamic genetic networks with low order independencies,” Statistical applications in genetics and molecular biology, vol. 8, no. 1, 2009.
  46. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Summary

We haven't generated a summary for this paper yet.

X Twitter Logo Streamline Icon: https://streamlinehq.com